LLM Agent Incident Response: Containment, Forensics, Recovery

An LLM-agent incident needs four decisions in order: preserve the request evidence, contain the routed model traffic, eradicate the compromised component with the owning team, then prove recovery with a clean request sample. The Microsoft Prompts Become Shells disclosure from May 7, 2026 and the Marimo CVE-2026-39987 incident from May 10 show why the SOC needs that sequence. Prompt injection, agent escalation, model-extraction attempts, LLM-operated post-exploitation and prompt-based data exfiltration produce evidence at the request boundary alongside the host and application evidence used for eradication.

I want to walk through the AI-layer incident classes the SOC should recognize, the detection signals each class produces, the containment actions that work at the request boundary, the forensic evidence the post-mortem needs, and the integration points with the rest of the security operations stack.

TL;DR

Preserve the identity, route, policy version, classifier result and decision for each suspect model request before containment changes the evidence. Use the gateway for routed HTTP model-traffic controls, then hand tool authorization, host remediation, credential rotation and recovery testing to the teams that own those layers.

The AI-layer incident classes

The classes recur across enterprise deployments. A playbook that names them and assigns the response is more useful than a generic "AI incident" category.

Prompt injection in production starts when a user-facing AI feature receives a prompt that overrides the system instructions. The model executes the overridden instructions, which often involve exfiltrating context data, revealing system prompts, or steering an agent toward an attacker-controlled tool. The detection signal can be an unexpected tool selection, response content inconsistent with the approved task, or output length far outside the input pattern.

Indirect prompt injection occurs when the agent reads attacker-controlled content, such as a document, email or webpage, and embedded instructions reshape the agent's behavior. The user did not type the instruction; the agent encountered it in its retrieval pipeline. The detection signal is a divergence between the user's stated request and the agent's actions, plus the presence of attacker-controllable content in the agent's context.

Agent tool-call escalation happens when an agent selects a tool with attacker-controlled arguments and the host executes the resulting action. The detection signal is an unexpected tool invocation, unusual argument shape, or invocation of a high-privilege tool such as a shell, code interpreter or file system outside the normal usage pattern.

Model extraction and reconstruction uses repeated, carefully crafted inputs to extract training data, system prompts or proprietary fine-tuning. The detection signal is high-volume request patterns with unusual prompt structures, often from a single identity or a coordinated set of identities.

LLM-driven post-exploitation begins after an attacker compromises a system through a non-AI vector, such as a service CVE, stolen credential or supply-chain attack, then uses an LLM agent inside the victim environment to operate. The Marimo CVE-2026-39987 incident is the canonical example: pre-auth RCE in a service, harvested AWS keys, then LLM-driven Secrets Manager calls. The detection signal is unusual LLM activity from inside the environment, especially LLM calls that drive cloud provider APIs.

Prompt-based data exfiltration uses an AI feature to move sensitive data outside its approved path. The pattern is "summarize the customer list," "translate this contract into [language the attacker controls]," or "encode this PHI in base64 and return it." The detection signal is the pairing of sensitive data classification in the prompt with an unusual output channel.

Detection signals the AI request boundary produces

The detection signals that work for AI-layer incidents differ from network or endpoint signals. They emerge at the AI request boundary.

Identity-anchored anomaly detection uses the user or agent identity plus the policy in force for the route. An anomaly detector at the request boundary that baselines per-identity request patterns surfaces requests that fall outside the baseline. The signal is more precise than a network-layer or endpoint-layer signal because the identity is known at the request layer.

Policy-decision tracking treats the gateway's pass-or-block decision as a high-value signal. A sudden spike in policy blocks for a given identity, route, or data classification can indicate an attack in progress or a policy misconfiguration. The incident queue needs to investigate either condition.

Content classification signals identify PII, PHI, injection patterns and credentials for the SOC event stream. A prompt that contains a credential pattern and is being sent to an external model from a non-secrets-management identity is a high-priority signal.

Cross-tier correlation

The AI request boundary signal combined with the endpoint or network signal produces a stronger correlation than either alone. An anomalous LLM call from a host that also shows credential-theft indicators in the endpoint telemetry is a different incident than either signal alone.

Containment actions at the request boundary

When an incident is detected, containment has to happen at the AI request boundary, because the AI request is the active attack surface.

Identity-level policy escalation

Revoke or downgrade the affected identity's AI access at the gateway. The identity may retain access to other systems while AI access is held until the investigation completes. The granularity matters: a per-route, per-role policy escalation is less disruptive than a wholesale identity disable.

Route-level fail-closed posture

For an AI route under active attack, escalate the route to fail-closed at the gateway. All requests on the route are denied until the incident is resolved. The fail-closed posture is deterministic; it does not depend on the application's behavior or the model's reasoning.

Tool-binding suspension

For an agent under tool-call escalation, suspend the affected tool binding in the agent framework or its tool authorization layer. Preserve the gateway's correlated model-request records while the owning team reviews that binding. Tool authorization and local execution sit outside an HTTP proxy's enforcement boundary.

Data classification escalation

For a route showing sensitive-data exfiltration patterns, escalate the data classification policy. Prompts that previously passed because the classification was at one tier are now blocked because the classification threshold for the route is tightened.

Forensic evidence the post-mortem needs

The post-mortem requires evidence at a granularity the classical incident response often does not capture.

Per-decision audit records at the request boundary

For each request involved in the incident, the audit record needs to show the identity, the policy version in force, the data classification, the decision outcome, and the timestamp. Without per-decision records, the post-mortem operates on aggregated counts, which obscure the specific attack pattern.

Policy-state history

The post-mortem needs the policy state at the moment of each decision, not just the current policy state. A policy that was changed during the incident produces different decisions before and after the change. The post-mortem has to reconstruct the policy timeline.

Content classification history

The classifier outputs at the moment of each decision matter for the post-mortem. A prompt that the classifier flagged but the policy passed is a different signal than a prompt that the classifier missed. Both indicate different remediation paths.

Identity context

The identity context for each request (user, role, session, source IP, authentication method) anchors the forensic timeline. Without identity context, the post-mortem cannot distinguish coordinated activity across identities from concentrated activity from a single identity.

Integration with the rest of the security operations stack

The AI-layer playbook fits inside the classical playbook and contributes request-level evidence to it.

SIEM ingestion of AI request boundary events

The per-decision audit records and the detection signals from the AI request boundary feed the SIEM in the same way the network and endpoint events do. The SIEM correlation rules then operate across AI and non-AI signals.

SOAR playbook triggers

The detection signals at the AI request boundary trigger the SOAR playbooks the SOC already runs. An identity-anomaly signal triggers the user-investigation playbook. A policy-block spike triggers the policy-review playbook. A tool-binding anomaly triggers the agent-investigation playbook.

EDR cross-correlation

For the LLM-driven post-exploitation class (Marimo CVE-2026-39987 pattern), the AI request boundary signal correlates with EDR signals on the host running the agent. The correlation tightens the investigation timeline. The Marimo incident reported earlier surfaced because the security team correlated unusual LLM activity with the host telemetry, not because either signal alone triggered an alarm.

Compliance evidence

The per-decision audit records can support the request-level evidence needed for EU AI Act Article 12 and adjacent regimes. Retention, legal applicability and the rest of the evidence package remain deployment-specific.

For the request-level containment design, see AI gateway architecture. The OWASP LLM01 prompt-injection guide covers the injection path, while AI audit-log chain of custody explains how to retain the resulting evidence.

DeepInspect

This is the AI request boundary the playbook above operates against. DeepInspect sits as a stateless proxy between authenticated users or agents and the LLM endpoints, enforces identity-bound policy on every request, and writes per-decision audit records with policy version, identity context, data classification, and decision outcome.

For the SOC's incident response, the gateway produces detection signals such as identity anomalies, policy blocks and content-classifier matches; it supports per-identity and per-route policy escalation; and it retains per-decision audit records with full context. The records integrate with the SIEM and SOAR pipelines the SOC already runs. Tool suspension, host cleanup and credential rotation remain with their respective control layers.

If you are standing up an AI incident response playbook and the AI request boundary in your environment does not produce per-decision records, the next incident will run without the evidence the post-mortem needs. Book a demo today.

Beyond AI-specific response

The AI-layer playbook fits inside the broader incident response framework. NIST SP 800-61 sets the four-phase incident response model: preparation, detection and analysis, containment-eradication-recovery, and post-incident activity. The AI-layer playbook is a specialization of each phase for AI-layer incidents. The integration with the SIEM, SOAR, EDR, and compliance evidence pipelines means the playbook does not stand outside the classical security operations practice.