Debugger Detection Techniques: The Summary of a Summary

Section 2 of The Art of Unpacking, by Mark Vincent Yason, explains the most commonly used techniques for debugger detection. I highly recommend reading it, as it includes examples of code that are very helpful for understanding each technique in depth. This post briefly outlines the techniques discussed in the mentioned section.

2.1 Is DebuggerPresent() #

The packer checks the BeingDebugged flag in the Process Environment Block (PEB) manually or by calling kernel32!IsDebuggerPresent().

Solution #

This technique can be easily bypassed by manually patching the PEB.BeingDebugged flag with the byte value of 0x00.

2.2 PEB.NtGlobalFlag, Heap Flags #

There are other fields from PEB that packers can use to detect if a program had been loaded by a debugger.

Solution #

Patch with the corresponding non-debugging values.

2.3 DebugPort: CheckRemoteDebuggerPresent() / NtQueryInformationProcess() #

Kernel32!CheckRemoteDebuggerPresent() calls NtQueryInformationProcess() that queries the DebugPort field of the EPROCESS kernel structure. A non-zero value in the DebugPort field indicates that the process is being debugged by user-mode debugger.

Solution #

Patch the return value of NtQueryInformationProcess() to 0x00.

2.4 Debugger Interrupts #

The exception handler will not be invoked in debuggers when interrupting with INT3 and INT1. Some packers set flags inside the exception handler, so if the flag is not set, the process is being debugged. kernel32!DebugBreak() function is used by some packers due to internally invoking INT3.

Solution #

1. The packer creates the interruption to check if it is being debugged. The debugger is paused.
2. We find the exception handler address and set a breakpoint.
3. Use exception pass (Shift+F9 in OllyDbg) so the breakpoint in the exception handler is hit and you can trace the exception handler. Here you can patch whatever necessary so the packer does not realize is being debugged.

• Another option is allowing single-step/breakpoint exceptions. “INT 3 breaks” and “Single-step break” in OllyDbg.

2.5 Timing Checks #

The packer calculates the time spent in between different instructions, the time will be higher when the application is being debugged.

Solution #

Identify where the time checks are and avoid stepping thru. The time might be checked using:

• kernel32!GetTickCount()
• RDTSC (Read Time-Stamp Counter) instruction.
• Checking the value of the TickCountLow and TickCountMultiplier fields of the SharedUserData data structure.

2.6 SeDebugPrivilege #

A process being debugged inherits the SeDebugPrivilege enabled from the deugger. The packer can attempt to open CSRSS.EXE process in order to check if SeDebugPrivilege is enabled.

Solution #

The packer can find the address of CSRSS.EXE by using ntdll!CsrGetProcessId() or manually via process enumeration. After that it will try to open the process, so we will set a breakpoint at ntdll!NtOpenProcess() and if the passed PID is that of CSRSS.EXE we will set the return value to EAX 0XC0000022 (STATUS_ACCESS_DENIED).

2.7 Parent Process #

Some packers check if the Parent PID equals the PID of explorer.exe. Typically a process has explorer.exe as its parent process, if it does not, it could mean that the process is being debugged. It also may mean execution via command prompt or that the default shell is different.

Solution #

Patch kernel32!Process32NextW() to fail (i.e., return EAX = 0). Packer process enumeration will fail and may skip the PID checks.

2.8 DebugObject: NtQueryObject() #

ntdll!NtQueryObject() is used to obtain all the objects in the kernel that are of type DebugObject. Every time an application is being debugged, an object of this type is created.

Solution #

ntdll!NtQueryObject() returns OBJECT_ALL_INFORMATION structure, which can be patched to set NumberOfObjectsTypes field to 0 so packers do not iterate thru the ObjectTypeInformation array. ObjectTypeInformation array stores the field TypeName which could be the string “DebugObject”.

2.9 Debugger Window #

The packer uses user32!FindWindow() or user32!FindWindowEx() to look for windows with specific class names (e.g., OLLYDBG of rOllyDbg, or WinDBGFrameClass for WinDbg).

Solution #

Set a breakpoint in the mentioned functions and change the return value.

2.10 Debugger Process #

Similar to previous technique but checking the name of the running processes with Process32First() and Process32Next(). Or checking the process memory using kernel32!ReadProcessMemory() searching for debugger related strings.

Solution #

Patch kernel32!Process32NextW() to always fail. Hence, preventing process enumeration.

2.11 Device Drivers #

Detecting kernel mode debuggers by calling kernel32!CreateFile() against well-known device names used by kernel mode debuggers.

Solution #

Set a breakpoint to kernel32!CreateFileW() and manipulate the parameter FileName or the return value to INVALID_HANDLE_VALUE (0XFFFFFFFF)

2.12 OllyDbg: Guard Pages #

When setting an on-access/write memory in OllyDbg, the *PAGE_GUARD * page protection is set to that memory. When that memory is accessed while debugging no exception will be thrown, but if not debugged with OllyDbg, STATUS_GUARD_PAGE_VIOLATION (0X80000001) exception will be raised. In the exception handler the packer will set a marker.

Solution #

When guard pages accessed, trigger an exception so the exception handler will be called.