Using x64dbg for Dynamic Analysis

Using x64dbg for Dynamic Analysis

Overview and Learning Objectives

x64dbg is the primary debugger used in modern malware analysis. This lesson provides a comprehensive workflow for using x64dbg effectively. By the end, you will:

• Navigate the x64dbg interface confidently and use essential shortcuts
• Set strategic breakpoints to capture malware behavior efficiently
• Use memory breakpoints to track unpacking and injection
• Work with ScyllaHide to bypass anti-debug protections
• Dump unpacked code from memory using the Scylla plugin
• Build complete behavioral timelines from debug sessions

What is x64dbg?

x64dbg is a free, open-source debugger for Windows. It supports both x86 (x32dbg) and x64 (x64dbg) binaries. It has largely replaced OllyDbg as the go-to debugger for malware analysis due to its active development, modern interface, and extensive plugin ecosystem.

Interface Overview

Essential Keyboard Shortcuts

Complete Debugging Workflow

Step 1: Pre-Analysis Checks

Before loading in x64dbg, perform these checks:

# Check if DynamicBase (ASLR) is enabled using CFF Explorer or pestudio
# If enabled, disable it to get consistent addresses:
setdllcharacteristics.exe -d sample.exe

# Check architecture (32-bit vs 64-bit) to use correct debugger
# x32dbg for PE32, x64dbg for PE32+

Step 2: Load the Sample

1. File > Open -- select the malware executable
2. x64dbg breaks at the system entry point (in ntdll.dll)
3. Press F9 to run to the program entry point (PE entry / CRT startup)
4. You are now at the beginning of the sample's code

Important: Some malware uses TLS callbacks that execute before the entry point. Enable Options > Preferences > Events > TLS Callbacks to catch these.

Step 3: Set Strategic Breakpoints

The fastest way to understand malware behavior is to set breakpoints on key Windows APIs. Use the command window at the bottom of the CPU tab:

; Set breakpoints using the command window (SetBPX command):
SetBPX CreateFileA
SetBPX CreateFileW
SetBPX WriteFile
SetBPX RegSetValueExA
SetBPX RegSetValueExW
SetBPX InternetConnectA
SetBPX HttpSendRequestA
SetBPX connect
SetBPX send
SetBPX VirtualAllocEx
SetBPX WriteProcessMemory
SetBPX CreateRemoteThread
SetBPX CreateProcessA
SetBPX CreateProcessW

API breakpoint categories for malware analysis:

Step 4: Run and Observe

1. Press F9 to run
2. When a breakpoint hits, examine:
   • Parameters: Check registers (x64: RCX, RDX, R8, R9) or stack (x86: [ESP+4], [ESP+8]...)
   • Return address: Check the Call Stack tab to see which function made this call
   • Context: Note the timestamp and sequence of events
3. Press F9 to continue to next breakpoint
4. Build a timeline of observed API calls

Step 5: Examine Memory

When the malware allocates or writes memory:

1. Note the address from VirtualAlloc return value (RAX/EAX)
2. In the Dump panel, press Ctrl+G and navigate to that address
3. Watch memory contents change after WriteProcessMemory
4. Right-click > Follow in Disassembler if the memory contains executable code
5. To dump the memory region to a file: right-click in Dump > Follow in Memory Map > right-click the region > Dump Memory to File

Memory Breakpoints for Unpacking

Memory breakpoints are essential for analyzing packers and self-modifying code:

1. Identify the target region: After VirtualAlloc, note the allocated address and size
2. Set memory breakpoint: In the Dump panel, select the memory range, right-click > Breakpoint > Memory, Write
3. Execution breaks whenever that memory is written to
4. After the write completes, change to Execute breakpoint: right-click > Breakpoint > Memory, Execute
5. The next break reveals when the unpacked code starts running

Practical unpacking pattern:

1. Break on VirtualAlloc --> note returned address (e.g., 0x02750000)
2. Set Memory Write BP on 0x02750000
3. Run --> breaks when packer writes decrypted code
4. Remove write BP, set Memory Execute BP on same region
5. Run --> breaks when execution transfers to unpacked code
6. You are now at the Original Entry Point (OEP)

Working with ScyllaHide

ScyllaHide is an essential x64dbg plugin that hides the debugger from anti-debug checks by hooking OS functions at the kernel level.

Configuration

1. Plugins > ScyllaHide > Options
2. Enable all checkboxes in the first column for maximum protection:
   • Hide from PEB (BeingDebugged, HeapFlags, NtGlobalFlag, StartupInfo, OsBuildNumber)
   • NtSetInformationThread, NtSetInformationProcess
   • NtQuerySystemInformation, NtQueryInformationProcess
   • NtQueryObject, NtYieldExecution
   • NtCreateThreadEx, OutputDebugStringA
   • BlockInput, NtUserFindWindowEx
   • NtUserBuildHwndList, NtUserQueryWindow
   • NtSetDebugFilterState, NtClose
   • Remove Debug Privileges
3. Click OK (no restart needed)

Caution: Enabling all ScyllaHide options is generally safe, but in rare cases it may break aspects of the malware unrelated to debugger detection. If the sample behaves oddly with ScyllaHide enabled, try disabling options one at a time.

Configuring Exception Handling

Some malware uses exceptions for anti-debug. Configure x64dbg to pass them through:

1. Options > Preferences > Exceptions
2. Select the 00000000-FFFFFFFF filter
3. Click "Do not break"
4. Click Save

Note: x64dbg may not persist this setting across restarts. Re-apply before each debugging session.

Dumping Unpacked Code with Scylla

When malware unpacks itself in memory, use Scylla to dump the unpacked PE:

1. Let the unpacking stub run until execution reaches the OEP
2. Open Scylla: Plugins > Scylla
3. Set OEP to the current instruction address (the original entry point)
4. Click IAT Autosearch -- Scylla finds the Import Address Table
5. Click Get Imports -- Scylla resolves all imported functions
6. Review the imports -- remove any marked as invalid
7. Click Dump -- save the unpacked binary to disk
8. Click Fix Dump -- select the dumped file to repair its import table
9. The resulting file can be analyzed statically in Ghidra, PEStudio, etc.

Building a Behavioral Timeline

As you debug, build a timeline documenting each significant action:

[T+0.0s]  Entry point reached at 0x00401000
[T+0.1s]  IsDebuggerPresent called -- returned 0 (ScyllaHide active)
[T+0.3s]  VirtualAlloc(0, 0x10000, MEM_COMMIT, PAGE_EXECUTE_READWRITE) = 0x02750000
[T+0.5s]  Decryption loop at 0x00401250 writes to 0x02750000
[T+0.8s]  CreateFileA("C:\\Users\\Public\\updater.exe", GENERIC_WRITE)
[T+0.9s]  WriteFile -- writes 45,056 bytes (dropped payload)
[T+1.1s]  RegSetValueExA(HKCU\\...\\Run, "WindowsUpdate", "C:\\Users\\Public\\updater.exe")
[T+1.3s]  InternetConnectA("1.234.27.146", 80)
[T+1.5s]  HttpSendRequestA(POST /api/gate.php) -- C2 beacon

Practical Exercise

1. Load a malware sample in x64dbg (use x32dbg for 32-bit samples)
2. Enable ScyllaHide with all first-column checkboxes
3. Configure exceptions to "Do not break"
4. Set breakpoints on file, registry, network, and injection APIs
5. Run the sample and document each API call with parameters
6. When memory allocation is detected, set memory breakpoints to track writes
7. If injection is found, dump the injected code from memory
8. Build a complete behavioral timeline from your debug session
9. Export patches if any were applied: Ctrl+P > Export