Analysis Methodology Overview

Analysis Methodology Overview

A structured analysis methodology is what separates a random exploration of a binary from a professional investigation. Without a methodology, you waste time, miss critical findings, and produce inconsistent results. This lesson covers the layered analysis approach used by professional malware analysts, when to use each technique, and how to build a repeatable workflow.

The Analysis Pyramid

Professional malware analysis follows a structured, layered approach. You start at the bottom (safest, fastest, least effort) and work your way up (riskiest, slowest, deepest understanding). Each layer answers different questions and produces different types of intelligence.

         /\
        /  \        REVERSE ENGINEERING
       / RE \       Disassembly, decompilation, protocol RE
      /──────\      Deepest understanding; highest effort
     /Advanced\
    / Dynamic  \    Debugging, API tracing, memory forensics
   /────────────\   Targeted investigation of specific behaviors
  / Basic Dynamic\
 /  Analysis      \ Execute in VM, observe with Procmon/Wireshark
/──────────────────\ Quick behavioral overview
 Basic Static       Strings, PE structure, imports, packer detection
────────────────────  No execution; safe on any system
 Automated Analysis  Sandbox submissions (VirusTotal, Any.Run)
────────────────────  Outsource execution; get quick reports
 OSINT / Triage      Hash lookup, reputation check, AV detection
────────────────────  Fastest; often done before touching the file

When to Stop

Not every sample requires full reverse engineering. The pyramid is a guide, not a checklist:

• SOC Analyst / Triage: Layers 1-2 (OSINT + Automated) may be sufficient
• Incident Responder: Layers 1-4 (through Basic Dynamic) for IOC extraction
• Malware Analyst: All layers as needed based on the sample complexity
• Threat Intelligence: Focus on layers that reveal TTPs and campaign connections

Layer-by-Layer Breakdown

Layer 1: OSINT and Triage

Goal: Determine if the file is known-malicious, already analyzed, or benign.

Time required: 2-5 minutes

Tools: VirusTotal, MalwareBazaar, Hybrid Analysis, OSINT platforms

Workflow:

# Step 1: Compute the hash
sha256sum sample.exe
# Output: e47060d0f7de5ee651878... sample.exe

# Step 2: Check VirusTotal
# Search the hash at virustotal.com or via API:
vt file HASH_HERE

# Step 3: Check MalwareBazaar
curl -s -X POST "https://mb-api.abuse.ch/api/v1/" \
  -d "query=get_info&hash=HASH_HERE"

# Step 4: Check if already analyzed
# Search on Hybrid Analysis, Any.Run, Joe Sandbox

Decision point:

• If hash matches known-good software -> likely false positive, verify and close
• If hash has extensive existing analysis -> review reports, extract relevant IOCs
• If hash is unknown or has minimal detection -> proceed to next layer

Pro Tip: Even if VirusTotal shows 0 detections, the file may still be malicious. Zero detection means the sample is either clean or too new / targeted for AV signatures. Continue analysis.

Layer 2: Automated Sandbox Analysis

Goal: Get a quick behavioral overview without setting up your own environment.

Time required: 5-15 minutes (plus sandbox processing time)

Tools: Any.Run, Hybrid Analysis, Joe Sandbox, CAPE Sandbox

What you get: Process trees, network connections, dropped files, registry changes, MITRE ATT&CK mapping -- all automated.

OPSEC Warning: Public sandboxes share your sample with the world. For sensitive investigations (targeted attacks, client-specific malware), use a private sandbox or skip to local analysis.

Layer 3: Basic Static Analysis

Goal: Understand the file's structure and capabilities without executing it. Form hypotheses that dynamic analysis will test.

Time required: 15-60 minutes

Tools and techniques:

Example static analysis session:

# On REMnux - a structured static analysis workflow:

# 1. File identification
file sample.exe
# PE32 executable (GUI) Intel 80386, for MS Windows

# 2. Hashing
sha256sum sample.exe
md5sum sample.exe

# 3. String extraction
pestr sample.exe | less
# Look for: URLs, IPs, registry paths, mutexes, API names

# 4. Packer detection
capa sample.exe
# Look for: "packed with UPX", "anti-analysis" capabilities

# 5. PE structure review
python3 -c "
import pefile
pe = pefile.PE('sample.exe')
print('Compile time:', pe.FILE_HEADER.TimeDateStamp)
print('Entry point:', hex(pe.OPTIONAL_HEADER.AddressOfEntryPoint))
print('Imphash:', pe.get_imphash())
for s in pe.sections:
    name = s.Name.decode().rstrip('\\x00')
    print(f'  Section {name}: entropy={s.get_entropy():.2f}, size={s.SizeOfRawData}')
"

What to document after static analysis:

• File type, size, hashes (SHA256, MD5, imphash)
• Compilation timestamp and whether it appears plausible
• Whether the binary is packed (and what packer)
• Key strings that suggest functionality (C2 URLs, registry paths, API calls)
• Hypotheses to test in dynamic analysis

Layer 4: Basic Dynamic Analysis

Goal: Observe what the malware actually does when executed. Confirm or disprove hypotheses from static analysis.

Time required: 30-90 minutes

Pre-requisites: Isolated VM environment, monitoring tools running, clean snapshot

Workflow:

BEFORE EXECUTION:
  1. Revert Windows VM to clean snapshot
  2. Start INetSim on REMnux
  3. Start Wireshark on REMnux
  4. Start Procmon on Windows VM (with filters)
  5. Start Process Explorer on Windows VM
  6. Take Regshot "1st shot" (optional)
  7. Open a command prompt for manual observation

EXECUTE THE SAMPLE:
  8. Run the malware (as administrator if testing full capability)
  9. Wait 3-5 minutes minimum (some malware has sleep timers)
  10. Interact with the system (open browser, move mouse)
       -- some malware waits for user activity

AFTER EXECUTION:
  11. Take Regshot "2nd shot" and compare
  12. Stop Wireshark capture and save PCAP
  13. Export Procmon logs (PML and CSV)
  14. Screenshot Process Explorer process tree
  15. Check INetSim logs on REMnux
  16. Revert to clean snapshot

Layer 5: Advanced Dynamic Analysis

Goal: Investigate specific behaviors in detail using debugging and memory analysis.

Time required: Hours to days

Tools: x64dbg/x32dbg (debugger), API Monitor, Volatility (memory forensics), Frida (dynamic instrumentation)

When to use:

• Malware detects the sandbox and refuses to run
• You need to unpack the binary at runtime
• You need to understand the C2 protocol in detail
• You need to extract encryption keys from memory
• The malware uses anti-analysis techniques that basic dynamic analysis cannot bypass

Layer 6: Reverse Engineering

Goal: Fully understand the malware's code, algorithms, and logic at the assembly or decompiled level.

Time required: Days to weeks

Tools: Ghidra (free, NSA), IDA Pro (commercial), Binary Ninja, Radare2/Cutter

When to use:

• You need to understand the full C2 protocol to build a decoder
• You need to write a decryptor for ransomware
• You need to understand a zero-day exploit
• You are building comprehensive detection signatures
• You are doing attribution analysis

The Analyst's Decision Tree

When a new sample arrives, use this decision tree to determine how deep to go:

NEW SAMPLE ARRIVES
       |
       v
[Hash known on VT?]──YES──> [Extensive reports exist?]──YES──> Review existing
       |                              |                         reports, extract
       NO                             NO                        IOCs, DONE
       |                              |
       v                              v
[File type supported         [Submit to sandbox]
 by your tools?]                    |
       |                            v
       YES                  [Sandbox results
       |                    sufficient?]──YES──> Extract IOCs, DONE
       v                          |
[Perform basic static             NO
 analysis]                        |
       |                          v
       v                  [Perform local basic
[Is it packed?]──YES──>   dynamic analysis]
       |                          |
       NO                         v
       |                  [Behavior clear?]──YES──> Extract IOCs,
       v                          |                  write report, DONE
[Static analysis                  NO
 sufficient?]──YES──>             |
       |              Extract      v
       NO             IOCs   [Advanced dynamic /
       |              DONE    reverse engineering]
       v                          |
[Perform basic                    v
 dynamic analysis]          [Full analysis report]

Analysis Documentation: The Report

Every analysis session should produce a structured report. Here is a standard template:

MALWARE ANALYSIS REPORT
═══════════════════════
1. EXECUTIVE SUMMARY
   - One paragraph: what is this malware and what does it do?

2. FILE INFORMATION
   - Filename, size, hashes (SHA256, MD5, SHA1, imphash)
   - File type, compilation timestamp
   - Packer/compiler identification

3. STATIC ANALYSIS FINDINGS
   - Key strings discovered
   - Import table highlights
   - PE structure anomalies
   - CAPA/YARA matches

4. DYNAMIC ANALYSIS FINDINGS
   - Behavioral timeline (ordered list of actions)
   - Files created/modified/deleted
   - Registry modifications
   - Network activity (DNS, HTTP, raw connections)
   - Process activity (spawned processes, injection)

5. INDICATORS OF COMPROMISE
   - Table of all IOCs with type, value, and confidence
   - MITRE ATT&CK technique mapping

6. DETECTION RECOMMENDATIONS
   - YARA rules, Sigma rules, SNORT signatures
   - Network-based detection (domains, IPs, JA3)
   - Host-based detection (registry, file paths, mutexes)

7. ANALYST NOTES
   - Hypotheses not yet confirmed
   - Suggested follow-up analysis
   - Related samples or campaigns

OSINT Investigation and Pivoting

A powerful analysis technique involves using OSINT to expand your understanding of a malware sample by pivoting from known attributes to discover new ones. This is the process of examining publicly available data sources to look for associations between known characteristics and new ones.

Pivoting workflow:

Start: Known IOC (e.g., a C2 domain)
   |
   v
Search SecurityTrails, PassiveTotal for:
   - Historical DNS resolutions (what IPs did this domain resolve to?)
   - Reverse DNS (what other domains resolve to those IPs?)
   |
   v
Search Open Threat Exchange, VirusTotal for:
   - Other files that communicate with this domain
   - Related malware samples (same C2, same mutex, same imphash)
   |
   v
Cross-reference findings:
   - Do the related samples share mutexes, registry keys, or code patterns?
   - Can you identify the malware family?
   - Can you map the attacker's infrastructure?

OPSEC Warning: When conducting OSINT research that involves connecting to potentially malicious infrastructure (visiting suspicious domains, downloading related samples), use a VPN or TOR from a lab system. Adversaries may monitor who investigates their infrastructure. Do not use your normal internet connection, and never interact with malicious sites from a production system.

Common Pitfalls and Pro Tips

Pitfall: Skipping static analysis. Jumping straight to dynamic analysis means you miss clues that guide your observation. Static analysis tells you what to look for during execution.

Pitfall: Not waiting long enough. Some malware has sleep timers of minutes or hours before activating. If you only watch for 30 seconds, you miss the real behavior.

Pitfall: Trusting sandbox results blindly. Sandboxes can be detected by malware. If a sandbox report shows no malicious behavior, it does not mean the sample is clean -- the malware may have detected the sandbox and stayed dormant.

Pitfall: Inconsistent documentation. If you do not document your findings as you go, you will forget details and produce incomplete reports.

Pro Tip: Use AI tools like ChatGPT or Claude as analysis companions. You can paste code snippets, API call sequences, or assembly excerpts and ask for explanations. But always validate AI output with your own analysis -- AI can be confidently incorrect.

Pro Tip: Keep a personal "analysis playbook" with your preferred tool configurations, Procmon filters, Wireshark display filters, and common YARA rules. This makes your workflow repeatable and efficient.