Azure AI Red Teaming Agent: Deployment and Acceptance Runbook

The Azure AI Red Teaming Agent in Microsoft Foundry runs PyRIT-based adversarial probes, evaluates each response with Foundry risk and safety evaluators, and produces attack success rates and scorecards. It supports larger predeployment exercises and scheduled postdeployment runs. A useful deployment starts with prerequisites and role assignment, then checks target and region support before anyone launches a scan. I would block the first run until a human has reviewed the prohibited-action taxonomy and named the owner for every result category.

TL;DR

• Confirm the Foundry project, current Foundry User role, supported region, eligible target, and test data before configuration.
• Generate and review the prohibited-action taxonomy with a human before the run.
• Treat the attack success rate and scorecard as evaluation evidence, then preserve the exact case used for remediation replay.
• Keep production enforcement separate: the agent scans a target, while an inline control governs live traffic on its configured route.

The agent combines PyRIT with Foundry evaluators

Microsoft's AI Red Teaming Agent concept page describes a managed workflow built on PyRIT and Foundry Risk and Safety Evaluators. The system generates adversarial attacks, sends them to the selected target, evaluates the resulting responses, and summarizes outcomes through an Attack Success Rate and scorecards.

The scanner generates test traffic, judges responses against selected risk categories, and logs the work. Its output is evaluation evidence rather than proof that production traffic will be blocked, because the scan does not become the mandatory request path for the deployed service.

The automated red-teaming guide covers the testing-versus-enforcement distinction and the production acceptance handoff.

Prerequisites decide whether the run is valid

The Microsoft deployment guide requires a Foundry project and the current Foundry User role. Microsoft documents a role-name change, so copied setup notes can become stale. Verify the live role name and assignment at the project used for the scan.

Next, check the target because agentic scenarios require an existing Foundry Agent target, and confirm that its region is supported. Record the exact target and region in the run sheet instead of relying on a workspace nickname. A green portal tile does not prove that the intended deployment is the one being tested.

Use synthetic or approved test data. Define the test identity and the routes the target can reach. Add stop conditions and incident contacts before the scan. An adversarial prompt can cause a tool-enabled agent to touch a real downstream system if the test target retains production permissions.

Human review belongs before prohibited-action testing

Microsoft's deployment guide describes generating a prohibited-action taxonomy, followed by human review and confirmation before the run. Keep that review as a release gate. The taxonomy defines the actions the evaluation will treat as prohibited for the target, so weak wording produces weak evidence.

The reviewer should inspect each action and its context. Name the target behavior and the user role. Include the system state that makes the action prohibited. Remove vague entries that cannot produce a pass or fail judgment. Add missing actions that matter to the deployment.

Put the taxonomy on a plain monitor beside the application's actual policy, then read one row aloud. If the team cannot say what the expected response is, the row needs revision. This small physical check catches language that sounds formal but cannot drive an acceptance test.

I prefer a shorter reviewed taxonomy over a longer generated list. Evaluation volume is useless when the expected behavior remains ambiguous.

Predeployment and scheduled runs answer different questions

Microsoft supports predeployment work with larger combinations of attacks and risks. That mode helps a team examine a candidate release before promotion. Scheduled postdeployment runs can watch for behavior changes after model, prompt, agent, or policy updates.

Keep the two evidence streams separate. A predeployment result supports a release decision for the tested version. A scheduled result shows how the deployed target behaved under the later run conditions. Record model and target versions, configuration, taxonomy revision, evaluator selection, and execution time with each result.

The AI red-teaming workflow provides the wider test-fix-prove loop. The Azure agent can automate the test and evaluation lanes. The platform team still owns repair assignment, release decisions, and operation of the production control.

Scorecards need case-level evidence

Attack Success Rate compresses outcomes into a useful summary. The scorecard helps teams compare categories and runs. Neither artifact should replace the underlying attack-response record.

For every accepted finding, retain:

• the target and version;
• the attack strategy and generated input;
• the test identity and relevant state;
• the model or agent response;
• the evaluator and category;
• the expected behavior;
• the repair owner;
• the rerun result.

Review category movement over time, but close work at the case level. A lower aggregate rate can coexist with one severe reproducible failure. The owner needs the exact sequence to repair and replay that failure.

Remediation replay is the acceptance test

A finding becomes useful when the team can rerun it after repair. Preserve the attack input and target context before changing the system. Record the selected change, which may affect prompts, code, model configuration, permissions, or a route policy. Then repeat the case against the updated target.

Use the Azure agent's evaluation result as one part of closure. Retain evidence from the control that changed too. A prompt update needs the later model response. A tool-permission fix needs a denied action from the tool or API. A routed HTTP policy change needs the policy version and decision event from that request path.

The AI red-teaming tools comparison shows why one runner cannot prove every surrounding control. PyRIT-based traffic can expose a failure, while the repair evidence belongs to the system that owns it.

The HTTP boundary remains narrow

The Foundry agent can test model and agent behavior inside its supported workflow. A DeepInspect policy applies only when an authenticated model request is deliberately routed through DeepInspect over HTTP. The two paths can meet when the authorized target sends the reproduced case through that gateway.

Foundry-hosted state remains outside the proxy. The same applies to local execution and STDIO, plus native SaaS actions and downstream authorization that never crosses the route. An agent may initiate all of those activities, but initiation alone does not place them inside an HTTP model-policy boundary.

Document route alignment for each finding. If the unsafe behavior appears in the model request or response handled by the gateway, it can become a route policy test. If the failure happens in a tool, store, host, or identity system, assign it there.

DeepInspect

DeepInspect can serve as the production control for one defined lane: authenticated user or agent requests deliberately routed over HTTP to an LLM. A reproducible Azure AI Red Teaming Agent finding on that route can become an identity-aware policy test. The rerun can retain the route, active policy, decision, and timestamp outside the calling application's write path.

Foundry-hosted state, local commands, STDIO calls, retrieval permissions, and downstream service authorization sit outside the routed model transaction. Those surfaces require their own acceptance evidence.

Book a technical deep dive at deepinspect.ai.