Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
96 lines (74 loc) · 5.58 KB

ARCHITECTURE.md

File metadata and controls

96 lines (74 loc) · 5.58 KB
Raw

Architecture

Problem

Detect security vulnerabilities (Reentrancy, Access Control, Token Handling, etc.) in Solidity smart contracts from real Code4rena audit contests. Predict each finding's vulnerability_type, severity, and affected file. Scored on precision, recall, and F1 against auditor-labeled findings.

Hard parts:

  • Noisy static analysis. ~80 pattern-based detectors fire hundreds of low-signal flags per contest. Most are false positives.
  • Scope. Contests ship many .sol files but only a subset is in scope. Analyzing out-of-scope files wastes API budget and adds false positives.
  • Taxonomy drift. LLM emits free-form vulnerability labels, but the scorer matches against a fixed taxonomy.
  • Duplicates. Per-file and project-level passes both surface the same bug, often with different wording.

Pipeline

A 9-stage pipeline that maximizes filtering before LLM calls:

  1. Scope filtering (loader/scope.rs) — reads scope.txt, restricts to in-scope .sol files
  2. Static analysis triage (loader/sa.rs + config.rs) — drops ~50 noise detectors, remaps surviving slugs to canonical types
  3. SA engine (engines/sa_engine.rs) — classifies SA findings into predictions and candidates with line numbers
  4. IR engine (engines/ir_engine.rs) — detects patterns from contract structure: missing access control, unprotected selfdestruct, state mutation after external calls, unchecked return values, etc.
  5. Call graph (engines/call_graph.rs) — builds inter-function call graph from IR, detects cross-function reentrancy and external call chains
  6. Cross-contract (engines/cross_contract.rs) — analyzes multi-contract interactions, shared state, delegatecall patterns
  7. Taint analysis (engines/taint.rs) — tracks data flow from taint sources (msg.sender, msg.value, tx.origin, block.timestamp) to sensitive sinks (selfdestruct, delegatecall, storage writes, transfers)
  8. Symbolic verification (engines/symbolic_engine.rs) — validates/kills candidates using pre-computed symbolic execution paths
  9. LLM analysis (pipeline.rs + llm.rs) — per-file and project-level prompts with full context from prior stages
  10. Post-processing (postprocess/) — JSON parsing, validation, taxonomy normalization, Jaccard deduplication, confidence scoring

Concurrency

  • Tokio async runtime with semaphore-bounded concurrency for LLM calls (16 per-file, 4 per-contest in batch)
  • Engine stages run synchronously per contest (fast, no I/O)
  • Batch mode runs contests in parallel via JoinSet
  • File-level analysis cache (SHA256-keyed) skips redundant LLM calls
  • Engine cache skips stages 1-8 when input data hasn't changed

Why two LLM passes

Per-file gives depth (full source in context, line-level reasoning). Project-level gives breadth (inheritance chains, shared state, multi-contract flows). They overlap on purpose: dedup collapses duplicates, and the union catches more true positives than either alone.

Confidence scoring

confidence.rs normalizes and weights engine scores:

  • Each engine output carries a raw confidence score
  • Configurable per-engine weights (SA: 0.3, IR: 0.35, taint: 0.4, symbolic: 0.5, call_graph: 0.25, cross_contract: 0.3, LLM: 0.7)
  • Multi-engine corroboration boosts confidence
  • Findings below min_confidence threshold get filtered

Report generation

report/ module produces three output formats:

  • Legacy — flat JSON array of predictions (backward-compatible)
  • JSON — structured report with metadata, summary statistics, and findings
  • HTML — dark-theme dashboard with severity cards, engine breakdown table, vulnerability type table, finding cards with confidence bars, evidence chains, and source code snippets

Diagram

flowchart LR
    A[main.rs]
    L[loader/<br/>scope, SA, IR,<br/>symbolic data]
    E[engines/<br/>SA, IR, call graph,<br/>cross-contract,<br/>taint, symbolic]
    LLM[LLM<br/>per-file +<br/>project pass]
    P[postprocess/<br/>parse, validate,<br/>dedup, confidence]
    R[report/<br/>legacy/json/html]
    S[score.rs]

    A -->|1| L
    L -->|2| E
    E -->|3| LLM
    LLM -->|4| P
    P -->|5| R
    R -->|6| S
Loading

File map

Module Files Role
main.rs — CLI entry point, arg parsing, orchestration
pipeline.rs — Contest/batch runner, stage coordination, caching
loader/ mod.rs, context.rs, ir.rs, sa.rs, symbolic.rs, scope.rs, labels.rs, raw_types.rs Load and parse contest data (IR, SA, symbolic, source files, labels)
engines/ sa_engine.rs, ir_engine.rs, call_graph.rs, cross_contract.rs, taint.rs, symbolic_engine.rs, constants.rs Pattern-based detection engines
llm.rs — Anthropic API client with retry and rate limiting
prompts.rs — System + per-file + project-level prompt builders
postprocess/ mod.rs, similarity.rs JSON parsing, validation, taxonomy remap, Jaccard dedup
confidence.rs — Multi-engine confidence scoring and normalization
config.rs — Slug-to-type mapping, noise lists, detector config
score.rs — Heuristic scorer (file + type matching), per-detector/severity breakdown
report/ mod.rs, stats.rs, json.rs, html.rs Report generation (legacy, JSON, HTML formats)
types.rs — Core types: Prediction, Candidate, VulnType, Severity, SAFinding, etc.
cache.rs — File-based caching for engine and LLM results
error.rs — Error types (LoadError, LlmError, AnalysisError)
path_utils.rs — Path normalization and matching utilities