AI in the Recon Pipeline

The idea: AI in small doses, not a black box

The loudest promise in AI offensive security is the fully autonomous agent: hand it a target, it reasons its way to a shell on its own. RedAmon ships that agent too, but it is not where most of the leverage lives. The reconnaissance pipeline is deterministic by design. The same target produces the same graph, the same scans fire in the same order, and every step is one you can read, test, and explain.

What this page documents is the other half of the strategy: injecting the LLM in small, well-scoped doses inside that deterministic pipeline rather than letting it drive the whole run. Each injection replaces one brittle heuristic (a static extension list, a hardcoded keyword match) with a narrow, tech-aware decision, then hands control straight back to the pipeline. The scan stays predictable and repeatable; only the point that needed judgement gets smarter.

The payoff is trust. An autonomous agent that guesses wrong leaves you without a thread to pull. A deterministic scanner with an AI-sharpened step gives you a result you can point at, reproduce, and improve one helper at a time.

How it differs from Adversarial AI Recon

Two things in RedAmon carry "AI" in their name and it is worth keeping them apart:

	AI in the Recon Pipeline (this page)	Adversarial AI Recon
Question it answers	"How do I make my scanners smarter?"	"Is the target running AI?"
What the LLM decides	ffuf extensions, nuclei tags, WAF / takeover / false-positive calls	no LLM decision at scan time; static signatures and active probes tag AI services on graph nodes
Where it lives	`recon/helpers/ai_planner/` decision helpers	thin AI-aware hooks inside existing recon modules
Effect	better scan inputs and fewer false positives	`ai_` / `is_ai_` properties + `Technology(category=ai-*)` nodes

Same philosophy (AI in small doses inside deterministic modules), opposite direction: this page is the LLM improving the scan, Adversarial AI Recon is the scan discovering the target's AI attack surface.

The three contracts every helper obeys

Every helper under recon/helpers/ai_planner/ follows the same three rules. They are what make it safe to drop an LLM call into the middle of a live scan.

1. Never raise, always fall back

An AI helper is never allowed to break a scan. Every failure path (no API key, network timeout, malformed response, schema mismatch) returns the value the pipeline would have used without AI, never an empty result. Empty would silently turn a step off; falling back to the user's static configuration keeps the deterministic behaviour intact. Recon stdout also tails into the webapp's live recon drawer, so a raised exception would blank the stream as well as break the run.

2. Keys never enter the scan container

The recon image carries no LLM SDKs and no per-user API keys. Every helper POSTs its decision to an endpoint on the agent container (for example /llm/ffuf-extensions, /llm/nuclei-tags, /llm/waf-classify) which holds the credentials and talks to the provider. One credential boundary, one place to rotate keys, and a scan container that stays safe to spawn against a target. See AI Model Providers for where those keys are configured.

3. Cache by fingerprint, not by target

Where a decision depends on the tech stack, the helper caches on a fingerprint (the Server, X-Powered-By and similar headers) rather than the hostname. A hundred hosts behind one stack collapse to a single LLM call. A helper that only runs once per scan (the nuclei tag selector) needs no cache at all.

The five helpers

Helper	Replaces	Fires during	Cache
FFuf extension planner	static `FFUF_EXTENSIONS` list	resource enumeration	per-target (by header fingerprint)
Nuclei tag selector	static `NUCLEI_TAGS` list	vuln scanning setup	once per scan
Nuclei false-positive response filter	keyword `is_false_positive()`	nuclei result triage	per response
WAF classifier	header-token WAF detection	HTTP probing / security checks	per response
Takeover false-positive classifier	signature matching	subdomain takeover triage	per candidate

FFuf extension planner

The static extension list is a compromise: .php, .bak, .zip and a handful more, the same for every target. The planner reads the target's response headers (Server, X-Powered-By, X-AspNet-Version, ...) and asks the LLM for the extensions that actually match the stack, so an ASP.NET host gets .aspx / .asmx and a PHP host gets .php / .phtml instead of both getting the generic set. Falls back to a safe default list on any failure. Inspired by ffufai, reworked to keep keys out of the scan container.

Nuclei tag selector

Nuclei ships 9,000+ templates. Running all of them is slow; running a fixed tag subset misses stack-specific checks. The selector aggregates the http_probe fingerprint for the target (detected technologies plus Server headers) and asks the LLM for the tag set worth running, validated against the installed templates' TEMPLATES-STATS.json so it never selects a tag with no templates behind it. Runs once per scan, falls back to a broad candidate tag list.

Nuclei false-positive response filter

The static is_false_positive() check keys off a keyword list, which fails two ways: it fires on legitimate pages that merely contain the word "WAF" or "Access Denied" (admin panels, docs, API payloads), and it misses rebranded block pages that contain none of its keywords (custom Imperva themes, AWS WAF JSON, empty 406 bodies). When the keyword path is ambiguous, this helper hands the response body to the LLM to decide whether it is a genuine finding or a block page. Falls back to "not blocked" so a failed AI call never suppresses a real result.

WAF classifier

Modern WAFs strip or rebrand the headers the static _has_cdn_markers / check_waf_bypass logic looks for (cf-ray, Server: cloudflare, ...). The classifier scores WAF presence from the full response — status, headers, body sample, and response time — so it catches challenge-page body fingerprints, WAF cookie shapes (__cf_bm, incap_ses_), latency outliers, and status/body mismatches that no fixed token list can. Falls back to "no WAF detected".

Takeover false-positive classifier

A subdomain takeover candidate and a WAF block page can look identical: both can be a 404/403 whose body reads "resource not found", which also matches Heroku's "nothing here yet" and generic unclaimed-SaaS fingerprints. This classifier is invoked only in that ambiguous middle case — a candidate already flagged, with no clear vendor cookie or header — and asks the LLM whether the page is a real third-party unclaimed page or a WAF block, so a gated subdomain doesn't become a false takeover. Falls back to "not a WAF block" (keeps the candidate for review).

Turning it on

There is one master switch, AI in Pipeline (aiInPipeline), and one per-tool toggle for each helper (ffufAiExtensions, nucleiAiTags, nucleiAiResponseFilter, wafAiClassifier, takeoverAiClassifier). The master switch is a hard cascade, not just a convenience: turn it on and all five helpers are forced on, turn it off and all five are forced off. This is deliberate defence against drift between the master panel and the individual tool sections, which both bind to the same fields. Practically: flip AI in Pipeline in the project's Target section to control the whole set, and use each tool's own section (Nuclei, FFuf, and so on) to see the same state.

Every helper shares one model, chosen by the AI Pipeline Model picker (aiPipelineModel). The recon container never holds the key: it delegates each call to the agent's /llm/* endpoints, so all five helpers use whichever provider that model id routes to. At least one AI Model Provider must be configured in Global Settings; with no provider (or the model left blank), the helpers fall back to their static behaviour and the scan runs exactly as it would without AI. See Project Settings Reference for every parameter.

These flags are also part of the AI-in-pipeline cascade, so an AI-generated Recon Preset can set them for you from a natural-language description of your goals.

Partial recon too. The helpers are wired into the shared entry functions, so they also fire when you re-run a single phase from the workflow graph rather than a full scan. The FFuf extension planner, for example, runs in both the full resource-enumeration phase and the standalone web-crawling partial-recon run.

Watching it work

Every helper logs to recon stdout with a bracketed tag so you can see the decisions in the live recon drawer:

[*][FFuf-AI] / [!][FFuf-AI]
[*][Nuclei-AI] / [!][Nuclei-AI]
[*][Nuclei-FP-AI] / [!][Nuclei-FP-AI]
[*][WAF-AI] / [!][WAF-AI]
[*][Takeover-AI] / [!][Takeover-AI]

[*] marks a decision the LLM made; [!] marks a fallback to the static value. Because every failure falls back, you can stop the agent container mid-scan and the recon run still completes on its deterministic path — the [!] lines simply tell you which steps ran without AI.