Is every denied prompt an incident?

No. A denial may show the control operated as designed. Apply the organization's incident criteria using context and data. Account for identity and destination. Also consider repetition and bypass evidence. Assess the impact separately. Keep the event available for trend analysis even when it remains below the incident threshold.

Can the SOC detect every prompt injection?

No. NIST AI 100-2e2025 warns that detection systems can be attacked and that current indirect-prompt-injection mitigations do not cover every technique. Use layered signals and least privilege. Add constrained interfaces and route policy. Human review and downstream authorization provide further checks. Record detector coverage and known failure modes.

What should the analyst preserve from a routed LLM event?

Preserve the original identifier and principal reference. Add the application and data classification. Record the model destination and policy version. Keep the action and event time. Include the response status and correlation value. Add the repeatable query. Keep the related identity and endpoint records used in triage, along with the network records. Include the provider and retrieval records. Add downstream records. Apply retention and access rules to sensitive content.

Where does an HTTP gateway stop helping?

It covers model traffic deliberately routed through the control point. A personal browser session or direct provider call can sit outside that path. A local model or STDIO tool can also sit outside it. Endpoint compromise and stolen credentials may be out of view. Vendor-internal inference may be as well. The incident record should name the sensor and owner for each excluded route.

SOC Analyst AI Risk Checklist for Incident Triage

At 02:13 UTC, a detector reports that a finance user sent pre-announcement earnings text toward an unknown model endpoint. The alert has a policy name but no caller ID or application. It also lacks a response status and correlation key. A SOC analyst AI risk checklist should turn that thin event into a defensible triage decision with preserved records. It should also expose what remains unknown. I would close an AI product launch before accepting an alert stream that cannot identify the principal behind a blocked request.

TL;DR

Define the incident threshold before alerts arrive. Specify the evidence needed to distinguish a policy event from a security incident.
Join model-route telemetry with identity and application records. Add endpoint and destination records to estimate scope and impact.
Preserve the request decision and related events. Keep analyst actions and provenance under the incident record.
Test containment and recovery on the affected path. Hand off unresolved questions about endpoints or vendor and local-execution paths to their owners.

Check 1: apply the incident threshold

Record the observed event and the rule that produced it. Determine if the event meets the organization's incident criteria or remains an alert for analysis. An expected denial by a working control can belong in either category. A blocked request can prove useful enforcement without proving a broader compromise.

NIST SP 800-61 Revision 3 supersedes Revision 2 and integrates incident response across the six NIST Cybersecurity Framework 2.0 Functions. Its DE.AE-08 outcome calls for applying incident criteria to analyzed activity while considering known false positives. Incident work runs through Detect and Respond. Recover follows, supported by preparation and improvement through the other Functions.

Write the threshold in the playbook before the 02:13 shift sees the alert. Include the risk condition and minimum evidence. Name the escalation owner and time target. AI incident response planning covers the wider operating plan.

Pass condition: the analyst records the applicable criterion and assigns the event a documented status. The record must identify it as an alert or an incident. It may instead identify an expected control outcome or another organization-defined status.

Check 2: establish the route from identity to destination

Start with the principal and source application. Add the route and provider account. Record the model endpoint and policy version. Add the request time and response disposition, followed by the correlation identifier. Verify the identity assertion against the application or identity-provider event rather than trusting a display label.

NIST SP 800-61r3 DE.CM covers monitoring across networks and computing assets. It also covers runtime environments and data, along with personnel activity and external providers. DE.AE-02 recommends continuous log monitoring with SIEM or SOAR support. Manual review is required when automation falls short. DE.AE-03 calls for correlation across sources.

An HTTP policy event supplies one part of that join. Endpoint telemetry may show browser use or a direct provider client. DNS events can reveal another route. Secure web gateway events can do the same. A vendor-hosted feature may produce only its own audit export.

Pass condition: the incident record identifies the principal and application. It also identifies the request route and destination. The record includes the effective policy and the source system for each fact.

Check 3: test AI-specific indicators

NIST AI 100-2e2025 defines direct prompt injection as lower-trust user instructions appended to higher-trust instructions. It describes possible misuse and privacy invasion. It also describes integrity violations that include manipulated tool or API calls. Indirect prompt injection begins in a resource consumed by the model, allowing an attacker to affect integrity or privacy. In some cases, availability can also be affected without the attacker being the primary user.

Look for attempted system-prompt extraction and unusual retrieval access. Check for manipulated tool calls and attacker-controlled outbound URLs. Also check for poisoned documents and hidden instructions. Unexpected capability suppression is another indicator. Keep the test tied to the application design. An outbound URL may be expected in one research agent and forbidden in a claims assistant.

NIST says monitoring and logging user activity can support identification and response. It also warns that detection systems can be attacked. Those systems may fail in correlation with the protected model. Record the detector's limits instead of treating one clean score as proof of safety.

Pass condition: the analyst tests relevant direct and indirect injection indicators against the known application path. The record includes detector coverage and gaps.

Check 4: estimate scope and impact

NIST SP 800-61r3 DE.AE-04 calls for estimating impact and scope. DE.AE-06 covers supplying alerts and log-analysis findings to the SOC and authorized responders. Start with the earliest known event. Search related activity by principal and application. Then search by model destination and retrieved source. Extend the search with the policy rule and correlation identifier.

Determine which data classes crossed the route and if the gateway forwarded the request. Then establish if a response reached the caller. Check for retries through another endpoint. Identify any downstream tool or business action. A deny result shows that the governed point withheld that request when enforcement happened before transmission. It says nothing about an unobserved bypass.

Triage should reflect asset criticality and functional impact. Account for data impact and attack stage. Threat context and recoverability also matter. Arrival order is a poor substitute for risk. A denied public-data prompt and a permitted restricted-data request deserve different queues.

Pass condition: the record states the affected period and principals. It identifies the systems and data. The record also states destinations and downstream actions. Material unknowns remain explicit.

Check 5: preserve evidence and analyst actions

NIST SP 800-61r3 RS.AN-06 calls for recording investigative actions and safeguarding confidentiality and integrity. RS.AN-07 calls for preserving incident-data integrity and provenance under organizational procedures and retention policy.

Preserve the original decision identifier and policy version. Keep the event time and principal reference. Preserve the application and prompt classification. Keep the destination and action. Include the response status. Keep the query used to select related events. Record each analyst action with time and operator. Full prompt retention can create a sensitive archive, so use the organization's classification and legal rules when deciding if content or a controlled reference belongs in the case. A fingerprint may be the appropriate alternative.

A screenshot of a dashboard can support orientation. It cannot replace the source event and repeatable query. AI audit log chain of custody covers the narrower handling sequence.

Pass condition: another authorized analyst can reproduce the event set and verify its provenance. The record also shows every material investigative action.

Check 6: contain the affected path

Containment should match the path. A compromised application route may need a policy denial or credential rotation. Connector disablement or a temporary destination block may also be required. Poisoned retrieval content may require quarantine and index rebuild. A tool-call issue may require reduced permissions in the destination system.

NIST SP 800-61r3 RS.MI covers containment and eradication. NIST AI 100-2e2025 says current indirect-prompt-injection mitigations do not protect against every technique. It points toward well-defined interfaces and components with different permissions. That supports least-privilege containment rather than relying on model refusal alone.

Record who authorized the action and what evidence confirms it took effect. Keep the gateway boundary honest. It can stop routed HTTP model traffic. Endpoint compromise and local models require other owners. So do stolen credentials and vendor-internal inference.

Pass condition: the incident record names the containment action and owner. It states the effective time and verification result. Remaining routes and the rollback condition are recorded.

Check 7: verify recovery and handoff

NIST SP 800-61r3 RC.RP covers secure recovery actions and verification of restored assets. It also covers confirmation of normal operation and completion criteria. An after-action report is also covered. Test the affected model route after remediation. Confirm the intended policy version and identity mapping. Verify the destination and evidence write. Then confirm the downstream permission.

The handoff should retain a concise summary and assigned actions. Include expected time frames and indicators. State the current scope and next steps. Keep unresolved questions attached to named owners. Endpoint teams may need to examine local copies. Vendor management obtains provider records, while application owners validate retrieved content.

The analyst should also record what changed in the detector or playbook. A new rule without a test case becomes another unverified assumption. Preserve one synthetic replay that would have produced the original alert and one normal request that should pass.

Pass condition: recovery tests pass on the affected route and completion criteria are signed. Unresolved work has an owner plus evidence requirement.

DeepInspect

DeepInspect sits inline between authenticated enterprise users or agents and HTTP-based LLM endpoints. It evaluates application-supplied identity and request context. It then classifies prompts and responses. It applies per-role and per-route policy and writes a signed decision record outside the calling application's write path.

That event can give the SOC the principal and route needed for governed-path triage. It can also supply the destination and classification. The record can provide the policy version and outcome. It includes time and correlation data. DeepInspect cannot prove endpoint state or inspect local execution. It also cannot recover vendor-internal logs or stop traffic that bypasses its route. Those limits belong in the incident record beside the controls that cover them. Book a technical deep dive at deepinspect.ai.