C - Program fails to get file descriptor only when running with GDB

Question

I'm not an expert C programmer. I'm having trouble debugging a program using GDB. (The bug I am trying to fix is unrelated to the problem I am asking about here.) My problem is that the program runs fine when I run the binary directly from a shell, but the program crashes when I run it using GDB.

Here is some information about the program which may be useful: it is a 20+ year old piece of database software, originally written for Solaris (I think) but since ported to Linux, which is setuid (but not to root, thank god).

The program crashes in GDB when trying to open a file for writing. Using GDB, I was able to determine that crash occurs because the following system call fails:

fd = open(path, O_WRONLY|O_CREAT|O_TRUNC, 0644);

For clarification: path is the path to a lockfile which should not exist. If the lock file exists, then the program shuts down cleanly before it even reaches this system call.

I do not understand why this system call would fail, since 1) The user this program runs as has rwx permissions on the directory containing path (I have verified this by examining the value of the variable stored in path), and 2) the program successfully opens the file for writing when I am not using GDB to debug it.

Are there any reasons why I cannot

Is there any chance this is a race condition? Could another thread be creating the lock file? — steveha
– steveha, Commented Nov 4, 2013 at 23:27
If you stop the program with the debugger and alter its internal data and then resume it, you can defeat the program's internal security measures. So, to sidestep this, the kernel disables setuid-ness whenever a process is run with debug enabled. (Consider a trivial example, a game with a "high scores" file. You run the game under the debugger, die immediately, but stop at the "register score" function. You change your score to ONE BILLION POINTS!!! and resume.) — torek
– torek, Commented Nov 4, 2013 at 23:52
@torek - that was the problem, thanks. I sudo su -ed to the user that this program runs as, and now GDB works. add it as an answer and I'll accept it — jayhendren
– jayhendren, Commented Nov 4, 2013 at 23:56

torek · Accepted Answer · 2013-11-05 00:05:50Z

8

The key turns out to be this bit:

... is setuid (but not to root, thank god).

When you run a program under (any) debugger (using any of the stop-and-inspect/modify program facilities), the kernel disables setuid-ness, even for non-root setuid.

If you think about this a bit it makes sense. Consider a game that keeps a "high scores" file, and uses "setuid games" to do this, with:

fd = open(GAME_SCORE_FILE, open_mode, file_mode);
score_data = read_scores(fd);
/* set breakpoint here or so */
if (check_for_new_high_score(current_score, score_data)) {
    printf("congratulations, you've entered the High Scores records!\n");
    save_scores(fd, score_data);
}
close(fd);

Access to the "high scores" file is protected by file permissions: only the "games" user can write to it.

If you run the game under a debugger, though, you can set a breakpoint at the marked line, and set the current_score data to some super-high value and then resume the program.

To avoid allowing debuggers to corrupt the internal data of setuid programs, the kernel simply disables setuid-ness when running code with debug facilities enabled. If you can su (or sudo or whatever) to the user, indicating that you have permission regardless of any debugging, you can then run gdb itself as that user, so that the program runs as the user it "would have" setuid-ed to.

answered Nov 5, 2013 at 0:05

torek

499k71 gold badges763 silver badges888 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

jayhendren Over a year ago

Unfortunately, this means I need to find a new method for debugging my problem, as the bug I am trying to investigate using GDB is something that only occurs for one user of the program, so it's important that I run this program as her user. Thanks for the help though!

torek Over a year ago

Figures. :-) Setuid is a pain, it's no wonder they dropped the whole concept from Plan 9 (client/server with proper authentication is a much simpler model). The first thing to look at with her settings is any special environmental setups, such as different env variables, user limits (ulimit output), and sometimes even weird stuff like tty modes.

jayhendren Over a year ago

I think it may be her Windows PuTTY client. She's having weird keybinding issues. Nobody (including myself) in a Linux environment can replicate the problem, and I can't replicate it when I su - to her either. I was hoping to run GDB as her from her Windows box and examine the value of the variables which store her keystrokes, but alas, I don't think that approach will work.

steveha Over a year ago

@jayhendren, at this point your best bet is to sprinkle logging statements into the code, and let her run it and see what you get in the log output. When all else fails, debug with printf().

jayhendren Over a year ago

I'm thinking I can get gdb to work by attaching to somebody else's running instance of this program. First, she'll fire up the program, then I'll su and gdb <program name> -p <pid>. I've tried it a bit but I can't get it to work right. GDB keeps claiming ptrace: Operation not permitted. after it attaches to the process.

|

Collectives™ on Stack Overflow

C - Program fails to get file descriptor only when running with GDB

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related