AI Red Teaming Checklist for Release Acceptance

A red-team report can contain fifty successful attacks and still leave the release decision unanswered. This AI red teaming checklist turns one bounded exercise into a signed acceptance record. It verifies the target and rules of engagement. It also verifies identity coverage and attack paths. Reviewers check evidence and remediation, then regression results and residual risk for a named release. The release gate needs reproducible records behind every screenshot.

TL;DR

• Freeze the release and deployed components before testing. Also freeze identities and data sources, then tools and rules of engagement.
• Cover direct and indirect injection plus cross-identity behavior. Test retrieval and tool calls, along with multi-turn state and response handling.
• Require reproducible findings with request and response evidence. Include policy and identity evidence, plus downstream-effect evidence.
• Retest closed findings and record residual risk. Obtain a signed go or no-go decision for the exact release.

Check 1: freeze the release and test boundary

Name the application build. Record the model and version, then the system prompt and retrieval index snapshot. Identify the tool set and policy bundle. Add the identity source and environment. Record every HTTP model route in scope. Include third-party components the deployment invokes and the supplier behavior the team can observe.

Map excluded surfaces to another owner. Local shell execution and host credentials sit outside an LLM HTTP proxy unless their calls traverse a governed request boundary. The same boundary applies to plugin loading and endpoint copy actions. Downstream API authorization also sits outside the proxy unless its calls cross that boundary. The AI agent red teaming guide covers agent loops and delegated authority in depth.

Pass when the reviewer can identify the exact release under test and every exclusion has a named control owner.

Check 2: approve rules of engagement

Document authorized testers and test identities. Record dates and environments. Define prohibited actions and data-handling rules. Set stop conditions and escalation contacts. Assign cleanup duties. Define allowed payloads and downstream effects. Use synthetic records where a test might otherwise expose customer or employee data.

NIST SP 800-53 CA-8 describes pretest analysis and agreed rules of engagement. It also requires testing within stated constraints. The guidance notes that testing can expose protected information, which makes handling expectations part of the exercise design.

A midnight stop contact printed at the top of the runbook is a useful physical detail. Pass when the red team and system owner approve the same version before execution.

Check 3: map identities and expected authority

List human roles and workload identities. Add agent identities and tenant boundaries. Record delegated principals and provider credentials. Define the expected policy outcome for each identity against sensitive data and model routes. Include missing and malformed claims. Test expired and replayed claims, along with excessive claims.

Run the same request under at least two roles with different permissions. Test identity claims embedded inside prompts and retrieved content. The application should supply verified context to the policy layer. The resulting record should bind the decision to one subject and policy version.

Pass when effective model access follows enterprise identity and approved delegation. Failed identity cases must reach the documented outcome.

Check 4: cover direct and indirect injection

Run direct instruction overrides and role manipulation. Test encoded payloads and long-context placement. Add system-prompt extraction and output-format abuse. Then plant instructions in retrieval documents and web content. Repeat the test through support tickets and email, then through tool output that the target can ingest.

OWASP's LLM01 Prompt Injection distinguishes direct and indirect injection and says foolproof prevention remains unclear because of generative models' stochastic behavior. Its mitigation guidance combines constrained behavior and output validation. It also calls for filtering and least privilege, with human approval for high-risk actions. The acceptance record should therefore score system outcomes and avoid promises of perfect prompt detection.

The prompt injection test cases provide reusable payload families. Pass when the suite covers each applicable ingestion path and captures actual impact.

Check 5: exercise tools, retrieval, and response paths

For each tool, list permitted operations and arguments. Record target systems and the approving identity. Test requests that attempt excess scope or argument injection. Repeat the test for cross-tenant access and composition of individually permitted tools. For retrieval, test access filters and poisoned documents. Check stale indexes and source attribution.

Inspect the model response before delivery or execution. Attempt sensitive-data disclosure and system-prompt recovery. Test structured output that could become an unsafe downstream command. Record whether each action was denied or allowed. Include any modification in that record.

Pass when the evidence shows the model call and downstream effect separately. A blocked response proves response enforcement. It cannot prove a local tool was prevented unless the tool system supplies that evidence.

Check 6: run multi-turn and persistence tests

Use sessions long enough to test goal drift and gradual authority claims. Check conversation-state confusion and memory poisoning. Test cross-session effects. Restart components and open a new user session where persistence is in scope. Test one user's content against another user's later interaction.

NIST AI 600-1 calls for AI red teaming against prompt injection and adversarial prompts. It also covers data poisoning and model extraction, along with related attacks. The profile recommends regular adversarial testing to identify vulnerabilities and misuse scenarios. Testing should also identify unintended outputs.

Pass when the test duration matches the deployed state model and every persistent effect can be traced to its source and cleanup action.

Check 7: require reconstructable findings

Each finding should contain a stable ID and target version. Record the identity and preconditions. Include the input sequence and any retrieved or tool-supplied content. Attach the model request and observed response. Include policy decisions and downstream effects. Add timestamps and the severity rationale. Attach original records. A slide can link to them while presenting the decision summary.

The test should distinguish model behavior from application behavior. It should also separate request-path enforcement from downstream authorization. That separation routes remediation correctly. A screenshot of a harmful answer may show impact while hiding the exact request that produced it. It may also hide the identity and policy.

Pass when an independent reviewer can reproduce the result or explain why safety constraints prohibit reproduction.

Check 8: verify remediation and regression evidence

Assign each finding to an owner and control point. Record the fix version and expected result. Then rerun the original sequence against the release candidate. Add the successful attack to a regression suite. Include negative tests that protect permitted business behavior.

NIST SP 800-218A lists red-team and adversarial exercises among AI model security methods. Its examples also include penetration and use-case tests. The profile recommends development-pipeline automation where possible. It also recommends documentation and triage of discovered issues. Retesting should follow retraining or new data sources.

Pass when closed findings have fresh evidence from the release candidate. A ticket marked fixed without a retest remains open for this gate.

Check 9: sign the AI red teaming checklist decision

Summarize open findings by affected use case and reachable identity. Record the data class and exploit reliability. Add the downstream consequence. Record compensating controls and expiry dates. The accepting authority should see which scenarios were tested or skipped. It should identify inconclusive scenarios. The record must also distinguish fixed scenarios from accepted ones.

The final artifact names the release and evidence package. It records the decision and approvers. It also states the conditions and retest triggers. High-severity findings with reachable impact should block release unless the named risk authority signs a time-bounded exception.

A product deadline provides zero risk reduction. I would pass the release only when the authorized engineering and security roles sign the same decision record. The business-risk role must sign that record too.

DeepInspect

DeepInspect can provide identity-bound request and response decisions for authenticated HTTP traffic between users or agents and LLM endpoints. Its records can show the supplied caller context and destination. They can also show the classification and policy version, along with the outcome for test execution. Those records strengthen reconstruction and regression evidence for request-path findings.

DeepInspect supplies request-path evidence while the red team conducts the test. Host controls and retrieval authorization retain their duties. Tool endpoints and application owners retain theirs. The checklist keeps each finding attached to the enforcement point that can close it. Book a technical deep dive at deepinspect.ai.