excavate: add AIApplicationExtractor for LLM endpoints, SDKs, and leaked AI provider keys

[DevamShah]

Summary

Adds a passive excavate submodule (AIApplicationExtractor) that fingerprints AI/LLM application surface — provider API endpoints, embedded client SDKs (LangChain / LlamaIndex / OpenAI) — and flags leaked AI provider API keys (OpenAI, Anthropic, Google Gemini) directly from HTTP_RESPONSE bodies and JavaScript, emitting TECHNOLOGY and FINDING events.

Problem / motivation

BBOT already excels at passively surfacing attack surface from HTTP responses, but it has no native recognition of the AI/LLM layer that now ships in most modern web front-ends. Two concrete gaps:

1. No AI attack-surface fingerprinting. Client-side bundles routinely hard-code calls to api.openai.com/v1/chat/completions, api.anthropic.com, generativelanguage.googleapis.com, Azure OpenAI deployments, and SSE chat streams, plus @langchain/* / llama_index / openai SDK markers. These are high-value pivots for an assessor (prompt-injection sinks, server-side proxy endpoints, model/billing abuse) yet are invisible to current modules.
2. No detection of leaked AI provider credentials. OpenAI (sk-…, sk-proj-…), Anthropic (sk-ant-api03-…), and Google (AIza…) keys are regularly committed into front-end JS and inline <script> blobs. A leaked inference key is a direct financial-loss and data-exfiltration primitive.

This reuses excavate's existing YARA pipeline (excavate already ships ~140 YARA references), so there is no new dependency or architectural change.

Change

• New AIApplicationExtractor(ExcavateRule) in bbot/modules/internal/excavate.py, placed alongside the other detection extractors (FunctionalityExtractor, SerializationExtractor, ErrorExtractor).
• Four YARA rules, compiled into excavate's existing combined ruleset:
  • ai_provider_endpoint — OpenAI / Anthropic / Google Generative AI / Azure OpenAI hosts + /v1/chat/completions.
  • ai_chat_streaming — SSE chat stream, gated on both text/event-stream and an LLM-shaped data: {... "delta"/"choices" ...} frame to avoid matching generic event streams.
  • ai_client_library — LangChain (@langchain/…), LlamaIndex (llama[_-]?index), and OpenAI SDK import/instantiation markers.
  • ai_provider_apikey — anchored, length-bounded key formats for OpenAI, OpenAI project, Anthropic, and Google keys.
• process() maps each matched YARA string identifier to either a TECHNOLOGY event (endpoint/SDK fingerprint, requires a host) or a FINDING event (leaked key). Secrets are redacted (prefix…suffix) before they enter scan output rather than echoed in full.
• Two new test classes in bbot/test/test_step_2/module_tests/test_module_excavate.py following the existing TestExcavateCSP / TestExcavateSerialization* conventions: a positive case (all techs + all three providers' keys) and a negative case (benign AI marketing copy + a Stripe sk_live_ key + "blockchain language" decoy) asserting zero false positives and that full secrets are never emitted.

produced_events on the parent module is intentionally left unchanged, consistent with the existing extractors (SerializationExtractor/CSPExtractor already emit FINDING/DNS_NAME without enumerating them there).

Security rationale

• OWASP LLM Top 10 (2025): maps to LLM02 — Sensitive Information Disclosure (leaked provider keys) and LLM10 — Unbounded Consumption (exposed inference endpoints enabling model/billing abuse). Surfacing the endpoints also locates the LLM01 — Prompt Injection sinks for follow-on testing.
• CWE-798 (Use of Hard-coded Credentials) and CWE-312 (Cleartext Storage of Sensitive Information): leaked sk-/sk-ant-/AIza keys in client-delivered JS are a textbook instance; detecting them passively closes a common, high-impact reconnaissance gap.
• MITRE ATT&CK T1552.001 (Unsecured Credentials: Credentials in Files): client bundles are exactly the kind of artifact where these credentials end up.
• False-positive discipline is treated as a first-class requirement: every key pattern is \b-anchored and length-bounded, provider key namespaces are mutually exclusive (hyphenated sk-ant-/sk-proj- cannot match the 48-char legacy OpenAI pattern), the SSE rule requires two independent signals, and detected secrets are redacted before emission.

Testing / validation

Validated against a fresh clone with yara-python 4.5.2 (the version BBOT pins) and the full bbot/test harness on Python 3.12:

1. In-harness tests pass. Both new classes pass under the standard bbot/test pytest run:

   pytest bbot/test/test_step_2/module_tests/test_module_excavate.py::TestExcavateAIApplicationPositive \
          bbot/test/test_step_2/module_tests/test_module_excavate.py::TestExcavateAIApplicationNegative
   # 2 passed
   

   The positive case asserts the expected TECHNOLOGY events (OpenAI/Anthropic/Gemini API, chat-completion endpoint, LangChain, OpenAI SDK) and FINDING events for the three leaked provider keys, and that full secrets are never echoed. The negative case asserts zero AI FINDING/TECHNOLOGY on benign content (Stripe sk_live_, AI marketing prose, "blockchain language model").
2. Lint/format: ruff check → "All checks passed!"; ruff format --check → clean (line-length 119).

The submodule follows the same shape as the existing SerializationExtractor/FunctionalityExtractor extractors, so it carries no standalone doc file (module docs are generated from metadata).

[liquidsec]

Hi, thanks for the PR! This looks really cool, I think we definitely want this. Just a couple notes:

• We are weeks away from 3.0 release, currently on dev. Can you retarget to dev? There is a VERY significant difference between 2.8 and 3.0, i'd anticipate conflicts.
• Also as a result of being so close to release, I am not sure if it will make it in 3.0 since we are essentially freezing new capabilities until after that. Most likely it would go in dev after 3.0 release, and ultimately be released in 3.1.

[DevamShah]

Thanks @liquidsec — glad it's a fit! Retargeted to dev and rebased the branch onto current dev (3.0).

The ExcavateRule contract was unchanged between 2.8 and 3.0, so the module itself applied cleanly with no logic changes. One 3.0 behavior change did surface in testing: the TECHNOLOGY event now lowercases its technology field in _sanitize_data, so I adapted the positive/negative test assertions to compare case-insensitively (the module still emits the same readable labels — core just normalizes the case). No assertions were weakened.

Validated on the dev base: the full test_module_excavate.py suite passes (52 passed, including the two new AIApplication positive/negative tests), and ruff check/ruff format are clean.

Totally fine if this lands in dev post-3.0 / ships in 3.1 — no rush on my end. Happy to make any further changes you'd like.

[liquidsec]

Had a chance to do a cursory review, there are a few things:

The redactions:

BBOT doesn't have any other cases where credentials harvested from a target are redacted in output. There are some instances where we redact our own api key secrets etc. But the philosophy here is, this is an offensive tool, many users will want to go on to actually use the harvested secrets. They'd also still be in output.json, so the potential usecase is definitely narrow at best and it adds complexity. The URL is also in the output, and re-fetch is trivial anyway.

Missing Severity And Confidence in yara metadata:

The custom severity and confidence fields get passed on to the eventual finding. In this case, we hope the regexes are good enough we can call them "confirmed". The severity we could debate about. However, in the absence of a severity it will be emitted as informational which is probably not what we want.

Regex

I suspect the provider endpoint stuff (looking for "api.openai.com") but not be specific enough, and i believe would just fire on any mentions of them

Im not totally sure that the api key patterns wouldn't occasionally false positive but im not sure

Process() override

understand why you did this, but i think rather than do that override i'd almost rather split it into two separate excavate signatures, and then do some modifications to the base to support technology events. I'd need to think a little deeper into it first, but I can definitely help with that part.

overall

The only other thing that makes me pause a little bit with this is the potential overlap with the trufflehog module. I'd want to do some more testing to see if trufflehog would pick this up...

There's a larger question here which is the degree with which we want to outsource secret detection to trufflehog vs do it ourselves.

I think what im leaning towards right now, pending actually getting my hands dirty w/this and testing, is leaving the API key detection to trufflehog and handle the detection of actual LLM endpoints with an excavate submodule.