Following information is derived from runs on my Ubuntu-14.04 system using gcc version 4.8.4 as my compiler and gdb version 7.7.1 as my debugger
First, the buffer overflow happens as a result of the strcpy function, and if you overflow buf so that it overwrites the memory location of auth, but the following if-else block will overwrite your changes.
Secondly you can see what is happening by looking at the stack in a debugger. I made a slight modification to you code, by initializing auth to 0xbbbbbbbb (just so I can find here auth is located on the stack).
Setting a break point on main and stepping into the function we can examine the values of the various registers:
(gdb) info reg
rax 0x0 0
rbx 0x0 0
rcx 0x0 0
rdx 0x7fffffffdf30 140737488346928
rsi 0x7fffffffdf18 140737488346904
rdi 0x2 2
rbp 0x7fffffffde30 0x7fffffffde30
rsp 0x7fffffffddf0 0x7fffffffddf0
[... some lines removed ...]
rip 0x400652 0x400652 <main+37>
eflags 0x246 [ PF ZF IF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
From this we can see that the stack extends from 0x7fffffffddf0 to 0x7fffffffde30. Now stopping right before the call to strcpy, we can take a look at the stack:
(gdb) x/76xb $rsp
0x7fffffffddf0: 0x18 0xdf 0xff 0xff 0xff 0x7f 0x00 0x00
0x7fffffffddf8: 0x1d 0x07 0x40 0x00 0x02 0x00 0x00 0x00
0x7fffffffde00: 0x30 0xde 0xff 0xff 0xff 0x7f 0x00 0x00
0x7fffffffde08: 0x00 0x00 0x00 0x00 0xbb 0xbb 0xbb 0xbb
0x7fffffffde10: 0xd0 0x06 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffde18: 0x40 0x05 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffde20: 0x10 0xdf 0xff 0xff 0xff 0x7f 0x00 0x00
0x7fffffffde28: 0x00 0x2b 0x25 0x07 0xdd 0x7a 0xc0 0x6d
0x7fffffffde30: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffde38: 0x45 0x6f 0xa3 0xf7
Looking at this, we can see that auth is located at a memory address of 0x7fffffffde0c.
I set as a command line argument passwordAAAAAAAA111, and now we can single step across the strcpy call and look at memory again:
(gdb) x/76xb $rsp
0x7fffffffddf0: 0x18 0xdf 0xff 0xff 0xff 0x7f 0x00 0x00
0x7fffffffddf8: 0x1d 0x07 0x40 0x00 0x02 0x00 0x00 0x00
0x7fffffffde00: 0x30 0xde 0xff 0xff 0xff 0x7f 0x00 0x00
0x7fffffffde08: 0x00 0x00 0x00 0x00 0xbb 0xbb 0xbb 0xbb
0x7fffffffde10: 0x70 0x61 0x73 0x73 0x77 0x6f 0x72 0x64
0x7fffffffde18: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41
0x7fffffffde20: 0x31 0x31 0x31 0x31 0x00 0x7f 0x00 0x00
0x7fffffffde28: 0x00 0x2b 0x25 0x07 0xdd 0x7a 0xc0 0x6d
0x7fffffffde30: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffde38: 0x45 0x6f 0xa3 0xf7
(gdb)
From this, we can see that the value of auth has not be touched (notice the four 0xbb still in memory starting at 0x7fffffffde0c). Also we can now see where the password is stored in memory, it starts at 0x7fffffffde10. The four 'A's that I used are where the four 0x41s are and the four '1's that I used are where the four 0x31s are
So, on my system I do not see a way that you would be able to overflow into the auth variable.
Finally, the question that you originally raised, remember that the command line arguments are treated as a character array, so passing in something line AAAA1 on the command line will result in the array [0x41 0x41 0x41 0x41 0x31] being passed to your program. What you want your program to receive is actually [0x41 0x41 0x41 0x41 0x01 0x00 0x00 0x00] (assuming 32-bit, little endian architecture). There are two issues that you will face,
1. 0x01 is a non-printable character
2. 0x00 being the null terminator will stop the string input at the first null.
There is not alot you can do about the issue 2, with just a simple input; however as others have suggested the solution around issue 1 is to create a driver program that builds the input buffer the way that you want and then passes that to the program.
authvariable? Also, you do know that in C anything non-zero is considered "true", and since the character'1'with ASCII encoding is equal to the integer49I don't really see any problem (since you don't actually check explicitly forauth == 1, what you have know is equivalent toauth != 0).printf("%x %x\n",&buffer,&auth);gave me28feb0 28feac: bummer :) damn padding optimization? order of variables in the stack is not guaranteed to be the same as declaration.