Tradecraft Lookup

The Tradecraft Lookup tool gives the agent a personal, user-curated catalog of trusted offensive-security knowledge URLs — HackTricks, PayloadsAllTheThings, CVE PoC repos, vendor research blogs — and lets it pull the exact page it needs mid attack chain.

Two things make it different from the agent's other reference tools:

• You curate the sources. The agent only consults sites you explicitly trusted. There is no open-web crawl at runtime.
• The catalog itself is the routing intelligence. When you add a resource, the system writes a 250-350 word summary of what it covers. That summary is what the model reads to decide whether the resource is the right place to look. No vector database, no learned router.

The catalog is configured per-user in Global Settings → Tradecraft. This page covers what the tool does at runtime, how the cache works, and how it differs from web_search and the FAISS Knowledge Base. For full implementation details and sequence diagrams, see README.TRADECRAFT.md.

When the agent uses it

The tool is wired into the exploitation and post-exploitation phases only. The agent is told to prefer it after query_graph and web_search when it needs a specific exploitation page or PoC, not general background.

Typical calls:

If zero resources are enabled, the tool is removed from the registry entirely — the agent will not see it as an option.

Two phases: Verify (once) vs Agent session (every call)

The tool's design separates a slow, expensive one-time setup from a fast, cheap per-call lookup:

flowchart LR
    subgraph "Verify phase (once per resource)"
        V1[Detect type from homepage]
        V2[Build sitemap with type-specific strategy]
        V3[LLM writes 250-350 word summary]
        V4[Persist sitemap + summary]
    end

    subgraph "Agent session (every tool call)"
        A1[Agent picks resource_id]
        A2[Lexical rank sitemap, top 30]
        A3[Small LLM picks best page]
        A4[Tier 1 HTTP fetch / cache hit]
        A5[Wrap in untrusted-content envelope]
    end

    V1 --> V2 --> V3 --> V4
    V4 -.JSON in DB.-> A2

• Verify phase runs once when you click Add Resource (or Refresh). Slow (seconds to minutes), expensive (one or many LLM calls), produces durable data.
• Agent session runs every time the agent calls the tool. Fast (sub-second on cache hit), cheap (zero or one small LLM call), reads the data verify wrote.

Resource types

The tool understands six resource types. Each one drives a different sitemap-extraction strategy and a different default cache TTL. Detection runs at verify time on the homepage HTML — there is no manual override.

For the in-depth comparison of each type — what triggers the detector, how the sitemap is built, and what the agent sees at runtime — see the Resource type differences section in Global Settings.

Tool signature

tradecraft_lookup(
    resource_id:    str,            # slug from your enabled resources catalog
    query:          str = "",       # free-text technique/topic (used for section pick)
    cve_id:         str = "",       # required when resource_id points to a cve-poc-db
    section_path:   str = "",       # skip auto-pick, force a specific URL or repo path
    force_refresh:  bool = False,   # bypass cache for this call
)

The tool's docstring is rebuilt on every project load from your enabled resources. The agent sees a catalog like:

TOOL: tradecraft_lookup

Use this tool to fetch specific exploitation tradecraft from a known
knowledge resource.

Available resources (pick the best resource_id for your query):

  hacktricks   (mkdocs-wiki) https://book.hacktricks.wiki
    Comprehensive offensive security wiki. Covers web (XSS, SSRF,
    SSTI, deserialization), Active Directory (Kerberoasting, DCSync,
    golden ticket, ADCS), Linux/Windows privesc, cloud, container
    escapes... [your saved summary]

  payloads     (github-repo) https://github.com/swisskyrepo/PayloadsAllTheThings
    Payload library organized per vulnerability class. Each folder
    has Intruder lists, bypass cheatsheets... [your saved summary]

  trickest_cve (cve-poc-db) https://github.com/trickest/cve
    CVE -> public PoC mapping. Pass cve_id="CVE-YYYY-NNNNN".
    [your saved summary]

Toggling a resource off in the UI removes it from this catalog on the next agent invocation.

What happens on each call

sequenceDiagram
    participant Agent as Claude (main agent)
    participant Tool as tradecraft_lookup
    participant Cache as sqlite + disk
    participant Web as Target site
    participant LLM as Claude Haiku (section picker)

    Agent->>Tool: resource_id, query
    Tool->>Tool: Look up resource by slug
    Tool->>Tool: Lexical rank sitemap (Jaccard + substring overlap, top 30)
    alt sitemap small or top-1 score >= 0.6
        Tool->>Tool: Use top lexical match
    else
        Tool->>LLM: Pick best of 30 titles
        LLM-->>Tool: index 1..30
    end
    Tool->>Cache: Lookup URL
    alt Cache hit & fresh
        Cache-->>Tool: cached markdown (~ms)
    else Cache miss or stale
        Tool->>Web: Tier 1 HTTP GET
        alt Tier 1 thin (< 800 bytes)
            Tool->>Web: Tier 2 Playwright render
        end
        Web-->>Tool: HTML / PDF / JSON
        Tool->>Tool: Strip nav/footer, extract article, convert to markdown
        Tool->>Cache: Save to disk + sqlite row
    end
    Tool-->>Agent: [BEGIN UNTRUSTED TRADECRAFT RESULT] envelope

Section picker

Two stages:

1. Cheap lexical filter — score every sitemap entry against the query using Jaccard similarity over tokens, with a substring-overlap bonus that catches near-misses like kerberoasting vs kerberoast. Keep top 30.
2. One small LLM call (Claude Haiku by default, configurable via TRADECRAFT_SECTION_PICKER_MODEL) — show the model the 30 candidates and ask "which one best answers the query?". Short-circuited when the sitemap has ≤5 entries or the top-1 lexical score is already ≥0.6.

If the LLM call fails for any reason, the tool falls back to the top-1 lexical match. The cve-poc-db type skips this stage entirely and uses the deterministic CVE-ID path.

Two-tier fetch

GitHub blob/ URLs are auto-rewritten to raw.githubusercontent.com/.../{branch}/{path} so Tier 1 hits the raw markdown directly instead of the rendered GitHub HTML page.

PDF sub-extractor

If the homepage or a fetched page comes back with Content-Type: application/pdf (or with the %PDF- magic bytes), the tool runs the PDF sub-extractor instead of HTML parsing. It extracts up to 200 pages, stores each page as page-N.md under <resource>/<sha256(url)>/, and writes a single SQLite row keyed by URL. Per-page lookups become deterministic on subsequent calls.

Output envelope

Every tool response is wrapped in an untrusted-content envelope so the agent's existing prompt-injection scrubbing applies:

[BEGIN UNTRUSTED TRADECRAFT RESULT]
resource: hacktricks
url: https://book.hacktricks.wiki/.../kerberoast.html
section_title: Kerberoast
fetched_at: 2026-04-26T14:02:11Z (cache miss, tier 1)
---
<the markdown body of the page, capped by the agent's TOOL_OUTPUT_MAX_CHARS>

Code blocks:
- bash:
    impacket-GetUserSPNs domain.local/user:password ...
- powershell:
    Invoke-Kerberoast | Out-File hashes.txt
[END UNTRUSTED TRADECRAFT RESULT]

No tradecraft-specific cap is applied — the agent's global TOOL_OUTPUT_MAX_CHARS setting is the single source of truth.

CVE PoC special path

The cve-poc-db type is special-cased because the repos involved hold hundreds of thousands of files organized deterministically by ID:

trickest/cve/
  2021/
    CVE-2021-41773.md
    CVE-2021-44228.md
  2022/
    ...

flowchart TD
    A[Agent calls<br>resource_id='trickest_cve'<br>cve_id='CVE-2021-41773'] --> B[Tool sees type=cve-poc-db]
    B --> C[Skip section picker entirely]
    C --> D[Build path: contents/2021/CVE-2021-41773.md]
    D --> E[GitHub Contents API direct hit]
    E --> F{200 OK?}
    F -->|yes| G[Return markdown]
    F -->|no| H[Fallback: list /contents/2021<br>+ substring match]
    H --> I{found?}
    I -->|yes| G
    I -->|no| J[Return 'PoC not found']

The tool docstring tells the agent "You MUST pass cve_id='CVE-YYYY-NNNNN'" for cve-poc-db resources, so Claude knows to provide the ID when calling.

The cache layer

All fetched pages live on disk under /app/tradecraft_cache/<resource_id>/<sha256(url)>.md, indexed by a SQLite database at /app/tradecraft_cache/index.sqlite. Each row tracks {url, fetched_at, ttl, resource_id, file_path, tier}.

flowchart LR
    Q[Tool query] --> CC{Cache lookup}
    CC -->|hit & fresh| H[Return cached file<br>~ms]
    CC -->|miss or stale| F[Fetch live]
    F --> S[Save to disk + sqlite row]
    S --> R[Return content<br>~seconds]

The cache is per-resource and per-URL. Disabling a resource does not delete cached pages; deleting the resource does.

SSRF guard

Both the verify endpoint and the at-query-time fetcher run the same URL validation:

The verify endpoint runs SSRF validation before building the LLM client, so a private-IP probe never wakes the LLM at all.

Comparison with other reference tools

The agent's prompt is updated to tell it: prefer tradecraft_lookup after query_graph and web_search when you need a specific exploitation page or PoC, not general background.

Lifecycle of one resource

stateDiagram-v2
    [*] --> Adding: User clicks Add Resource
    Adding --> Verifying: Webapp creates row,<br>calls /tradecraft/verify
    Verifying --> Active: Summary + type + sitemap saved
    Verifying --> ActiveDegraded: Verify failed,<br>resource saved with lastError

    Active --> ToolCallable: Agent loads project
    ActiveDegraded --> ToolCallable: Same, with warning chip

    ToolCallable --> CacheMiss: Agent calls tool, no cache
    ToolCallable --> CacheHit: Agent calls tool, cache hit
    CacheMiss --> ToolCallable: Page fetched + cached
    CacheHit --> ToolCallable: Page returned ~ms

    Active --> Refreshing: User clicks Refresh
    Refreshing --> Active: Sitemap rebuilt

    Active --> Disabled: User toggles off
    Disabled --> Active: User toggles on
    Disabled --> [*]: User deletes
    Active --> [*]: User deletes

When a resource is disabled, it is filtered out before the agent's tool docstring is built, so the agent literally cannot see it as an option. There is no risk of the agent trying to call a disabled resource.

If the agent has a stale conversation and tries to call a slug that no longer exists (e.g. you deleted the resource), the tool returns an error envelope: Resource '<slug>' not configured. The agent recovers and tries something else.

Configuration

All knobs live under the project's Project Settings → agent block, prefixed TRADECRAFT_*:

Per-user knobs live on each resource card (see Global Settings → Tradecraft):

Glossary

• Tradecraft — applied operator knowledge for offensive security, the "how to actually do it" between theory and tools. Originally a CIA term for spy operational skills.
• Resource — one URL you added to the catalog (e.g. HackTricks).
• Resource type — one of mkdocs-wiki, github-repo, cve-poc-db, sphinx-docs, gitbook, agentic-crawl. Determines how the system extracts a sitemap.
• Slug — server-generated stable identifier (hacktricks, hacktricks-2, ...) the agent passes as resource_id. Stable across renames so in-flight conversations and cache rows stay valid.
• Sitemap — the per-resource map of pages, stored as JSON in the database. Built once at verify time.
• Catalog — the dynamic tool description built from all enabled resources, shown to the agent at runtime.
• Verify — the one-time process of fetching, typing, sitemap-building, and summarizing a newly-added resource.
• Section picker — the at-query-time logic that narrows a sitemap to one page URL using lexical ranking + an optional small Claude call.
• Tier 1 / Tier 2 — two-step content fetch. Tier 1 is plain HTTPS; Tier 2 is Playwright. Used both at verify time and query time.
• Untrusted-content envelope — the [BEGIN UNTRUSTED TRADECRAFT RESULT] / [END ...] wrapper around fetched content; engages the agent's existing prompt-injection scrubbing.