Anti-Debugging Techniques

Anti-debugging techniques detect debugger attachment or dynamic analysis, preventing you from inspecting the binary at runtime. CTF challenges use these to force you into static analysis or to make reversing harder. The most common technique on Linux is the ptrace check — but there are several others you should recognize.

Detection and Bypass Flow

Bypass Techniques by Method

if (ptrace(PTRACE_TRACEME, 0, 1, 0) == -1) {
    puts("Debugger detected!");
    exit(1);
}

gdb ./binary
(gdb) catch syscall ptrace
(gdb) commands
  > set $rax = 0   # force ptrace to return 0 (success)
  > continue
  > end
(gdb) run

// ptrace_bypass.c
#include <sys/ptrace.h>
long ptrace(enum __ptrace_request req, ...) { return 0; }

gcc -shared -fPIC -o ptrace_bypass.so ptrace_bypass.c
LD_PRELOAD=./ptrace_bypass.so ./binary

Obfuscation Bypass

file ./binary                    # UPX compressed?
strings ./binary | grep -i upx   # UPX strings
strings ./binary                  # very few readable strings = encrypted
strace ./binary 2>&1 | grep -E "(ptrace|time|open.*proc)"

# UPX (most common):
upx -d packed_binary -o unpacked_binary

# Custom packer - dump from memory after unpack:
gdb packed_binary
gef> catch syscall mprotect
gef> run
gef> vmmap                     # find new RWX regions
gef> dump binary memory clean.bin 0x400000 0x500000

gdb ./binary
(gdb) break *0x401234    # address after decrypt function call
(gdb) run
(gdb) x/s $rax           # read decrypted string from return value

Common Anti-Debug Patterns

Checklist

• strings → check for "debugger", "ptrace", "TracerPid" references
• strace → look for ptrace syscall at startup
• GDB: catch syscall ptrace → force return 0
• Packed? → upx -d or dump from memory
• Encrypted strings? → breakpoint after decrypt, read memory
• Timing check? → intercept gettimeofday
• angr for path exploration when manual analysis is stuck

Last updated on Aug 1, 2025