AI Threat Detection for LLM Traffic: The Signals Live in the Request and the Response

A gateway on the decrypted AI request path sees four fields about every call: which identity made it, which model and provider host it targeted, what the prompt carried, and what the response returned. Those four fields are the entire raw material for detecting a threat that lives inside AI traffic. Everything a detection rule can fire on comes from reading them at the moment the request passes through, before it reaches api.openai.com or any other provider endpoint.

"AI threat detection" names two different problems, and search results blur them. One uses machine-learning models to score threats across a whole estate, the domain SOC vendors have sold for a decade. The other detects threats carried by AI traffic itself, when your own employees and service agents call an LLM and something in that call is hostile or leaking. I want to walk through the second one, because it is the newer gap and the one an identity-aware proxy actually addresses.

The four signals observable at the request boundary

Threats inside AI traffic show up as four distinct patterns on the decrypted HTTP path.

Prompt injection attempts appear in the request body as instructions aimed at the model rather than data for it: an override of the system prompt, a "disregard prior instructions" string, or an indirect payload smuggled inside retrieved content. OWASP ranks prompt injection as LLM01 in its 2025 Top 10 for LLM applications, which is why detecting it at the request layer matters more than any single model guardrail. Signal-level detection of these strings is covered in prompt injection detection.

An unrecognized identity making model calls is the second signal, and it is the one most teams miss. A service account named svc-invoice-parser that normally issues 40 calls an hour to gpt-4o suddenly issues 900 to a model it has never touched, at 02:14 on a Sunday. That anomaly is only legible if every call carries an attributable principal, which requires binding the request to identity at the proxy. Non-human identity attribution is the harder half of this problem, and I covered the mechanics in AI agent identity.

Anomalous egress destinations are the third: a workload that should only reach two sanctioned providers starts calling a model host nobody approved. Destination-level detection sits closer to the network layer and I treat it separately in AI egress monitoring, because the destination decision and the payload decision are different controls.

Data-exfiltration patterns in responses are the fourth. A response that returns a block of what looks like customer PII, or a prompt that carries source code and secrets outbound to an external provider, is a leak in progress. Zscaler's 2026 ThreatLabz reporting quantified how much sensitive data already moves through enterprise AI traffic, which I summarized in the enterprise AI data exposure report. The exfiltration mechanics themselves are in data exfiltration via LLM.

Why the signals require the decrypted path

Network-layer tools see the destination host and the byte count of a TLS session. They read none of the four signals above, because the prompt and the response are inside encrypted HTTP bodies. A firewall log that says a workload sent 48 KB to api.openai.com cannot tell you whether those bytes were a benign summarization request or an SSN dump with an injection string appended.

Detecting a threat inside AI traffic therefore requires terminating the AI request and reading the decrypted body. That is the same architectural line that separates destination control from payload governance. Once a proxy sits on the decrypted path, the four signals become fields it can classify and score per request, which is the substrate every downstream detection rule depends on.

The audit log is the detection substrate

A per-decision audit record is what makes detection possible after the fact and repeatable in real time. Each record captures who called, what model, what the request contained, what the response returned, and whether policy allowed it. A useful event shape looks like this:

Detection then reduces to writing rules over these records. An anomaly rule for the unrecognized-identity case reads directly off the same fields:

The record only carries weight if it cannot be edited after the decision. Tamper-evidence and signing are what let a record stand as evidence rather than as an application's own account of itself, and I detailed that in signed audit logs for AI requests. Detection built on a mutable log inherits the log's weakness.

Where a gateway stops, and what takes over

A gateway detects threats in AI traffic. It does not detect host-layer or network-layer intrusion, and pretending otherwise would oversell it. A proxy on the AI request path has no view of a compromised laptop running malware, a lateral-movement attempt across internal subnets, a stolen credential used against a non-AI system, or a process injecting code on a host. Those belong to EDR and network detection, and no amount of prompt classification changes that boundary. I would rather lose a deal than let a buyer believe a proxy replaces their endpoint sensors.

The correct posture treats these as complementary. The gateway is the sensor for the AI request and response, and it forwards its per-decision records into the SIEM where they correlate with everything else. When the audit record ships in a format a SIEM can parse, a spike in blocked SSN-egress decisions from one service account can be joined against an EDR alert on the same host, which neither tool detects alone. The formats and field mappings for that handoff are in AI audit log SIEM formats, and the broader monitoring picture across signals is in LLM security monitoring.

Detection latency is the other reason to catch these signals inline. Mandiant's M-Trends 2026 report, based on 500,000+ hours of frontline incident response, found the median time between initial access and handoff to a secondary threat group collapsed from over eight hours in 2022 to 22 seconds in 2025 (Help Net Security summary). A detection rule that fires a day later, after logs batch into a warehouse, misses the window entirely. Shadow AI breaches already take 247 days on average to detect per the IBM Cost of a Data Breach report, six days longer than standard breaches, and most of that gap comes from AI traffic that no sensor was reading.

DeepInspect

This is the gap DeepInspect closes. DeepInspect is a stateless, identity-aware proxy on the AI request path. It terminates the AI request, so it reads the decrypted prompt and response rather than an encrypted tunnel, binds each call to the identity the application supplies, classifies the request and response content across PII, PHI, MNPI, source code, and jurisdictional categories, and detects injection signals before the request reaches its provider.

Every decision produces a signed record with the identity, the model, the classified content, and the policy outcome. Those records are the detection substrate: real-time anomaly rules run against them inline, and the same records forward into your SIEM to correlate with the host and network signals a gateway cannot see. DeepInspect covers the AI request and response boundary and hands the rest to the tools built for it. Book a demo today.