What is the debugger doing under the hood when I ask it to set a breakpoint? Let's watch how GDB is changing the machine code of a program at runtime, from within the program itself.

Here's a somewhat sketchy¹ C program that prints out the first 20 bytes of its own machine code, starting at the first instruction of the main function.

#include <stdio.h>

int main(void) {
  unsigned char *addr = (void *) &main;
  for (int i = 0; i < 20; i++) {
    printf("%02x ", *(addr + i));
  }
  printf("\n");
  return 0;
}

At this stage, it's not terribly important to understand exactly how the code works. For now, it's enough to know that it does the job: it prints out the first 20 bytes of its own machine code.

Now let's see it in action. First, we compile this on an x86 Linux machine using GCC. Here's its output when I run it directly:

55 48 89 e5 48 83 ec 10 48 8d 05 f1 ff ff ff 48 89 45 f8 c7

We can verify that these are indeed the first few instructions of the main function by looking at the disassembly of the program, either using objdump -d or the GDB command disassemble (the option /r shows the raw bytes alongside the decoded instructions):

(gdb) disassemble /r main
Dump of assembler code for function main:
   0x0000000000001149 <+0>:   55                     push   %rbp
   0x000000000000114a <+1>:   48 89 e5               mov    %rsp,%rbp
   0x000000000000114d <+4>:   48 83 ec 10            sub    $0x10,%rsp
   0x0000000000001151 <+8>:   48 8d 05 f1 ff ff ff   lea    -0xf(%rip),%rax
   0x0000000000001158 <+15>:  48 89 45 f8            mov    %rax,-0x8(%rbp)
   0x000000000000115c <+19>:  c7 45 f4 00 00 00 00   movl   $0x0,-0xc(%rbp)
   ...

Yep, this matches the program's output.

Now, I'll run the program under GDB with a breakpoint set on the main function, i.e. (gdb) break main. Upon running the program, it pauses at the beginning of the main function as expected. After continuing, the program prints the following:

55 48 89 e5 48 83 ec 10 cc 8d 05 f1 ff ff ff 48 89 45 f8 c7

Let's look at the two outputs side-by-side:

55 48 89 e5 48 83 ec 10 48 8d 05 f1 ff ff ff 48 89 45 f8 c7
55 48 89 e5 48 83 ec 10 cc 8d 05 f1 ff ff ff 48 89 45 f8 c7

Notice that all of the output is the same, except for the ninth byte. Looking at the disassembly, this is the first byte of the fourth instruction LEA. Instead of 0x48, it's been changed to 0xCC. Neat! GDB must have put that there.

In the x86 instruction set, 0xCC is a single-byte instruction known as INT3². Basically, all INT3 does is tell the OS to stop executing the process. Once execution is paused, GDB can inspect the execution state of the process, continue execution, etc.

This was an empirical approach to understanding how x86 GDB uses INT3. I approach this same topic from a theoretical standpoint in another post.

We can apply this same technique to see how debuggers implement breakpoints across different architectures. Let's try this experiment again, but this time on an M4 MacBook Pro. So instead of x86, GCC, and GDB, we will be using ARM, Clang and LLDB.

Here are the outputs from two runs on my M4 MacBook Pro, first without a debugger and second under LLDB after (lldb) breakpoint set -n main:

ff c3 00 d1 fd 7b 02 a9 fd 83 00 91 bf c3 1f b8 08 00 00 90
00 00 20 d4 fd 7b 02 a9 fd 83 00 91 bf c3 1f b8 08 00 00 90

Again, we can verify that this is the correct output by checking the disassembly. Note that the instructions are printed as 32-bit little-endian values.

(lldb) breakpoint set --name main
Breakpoint 1: where = a.out`main, address = 0x0000000100000460
(lldb) run
...
(lldb) disassemble --bytes
a.out`main:
->  0x100000460 <+0>:   0xd100c3ff   sub    sp, sp, #0x30
    0x100000464 <+4>:   0xa9027bfd   stp    x29, x30, [sp, #0x20]
    0x100000468 <+8>:   0x910083fd   add    x29, sp, #0x20
    0x10000046c <+12>:  0xb81fc3bf   stur   wzr, [x29, #-0x4]
    0x100000470 <+16>:  0x90000008   adrp   x8, 0
...

This time, we can see that the entire first instruction has been replaced. Instead of 0xD100C3FF, we have 0xD4200000. This is the BRK #0 instruction in 64-bit ARM.

So now we know that, roughly speaking, BRK #0 is to ARM as INT3 is to x86.

I found this to be a fun way to peek under the hood of existing debuggers and see an implementation detail that the debugger usually hides from the user. Do try this at home!

I say this is sketchy because if you try to compile this with GCC with the -Wpedantic flag, you'll get a warning like this:
```
$ gcc -Wpedantic main.c
main.c: In function ‘main’:
main.c:4:25: warning: ISO C forbids conversion of function pointer to object pointer type [-Wpedantic]
    4 |   unsigned char *addr = (void *) &main;
      |
```
Fair enough, there probably isn't a practical use case for treating a function pointer like a data pointer. That being said, I needed to do this to enable pointer arithmetic so I could print out the raw instruction bytes at runtime. My intent here is to explore my computer, not to write robust software.

Fortunately, both GCC and Clang allow me to compile this program regardless. ↩
For reference, Intel's x86 manual has this to say about INT3:

INT n/INTO/INT3/INT1 — Call to Interrupt Procedure

...

The INT3 instruction uses a one-byte opcode (CC) and is intended for calling the debug exception handler with a breakpoint exception (#BP). (This one-byte form is useful because it can replace the first byte of any instruction at which a breakpoint is desired, including other one-byte instructions, without overwriting other instructions.)

↩

Alex Yang

Who watches the debugger?

INT n/INTO/INT3/INT1 — Call to Interrupt Procedure