Is AI penetration testing the same as red teaming?

They overlap and are scoped differently. A penetration test is usually a time-boxed, scope-bounded engagement that enumerates and validates specific vulnerability classes against a defined target, then reports findings mapped to a taxonomy like the OWASP LLM Top 10. AI red teaming is typically broader and more open-ended, closer to adversarial simulation against realistic objectives, and it is the term NIST AI 600-1 uses. For a deployed LLM system you generally want both: a structured pentest for coverage and comparability, and red-team exercises for creative attack chains. The AI agent red teaming breakdown covers the simulation side.

Which OWASP LLM Top 10 items should an AI pentest cover?

At minimum LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM05 Improper Output Handling, LLM06 Excessive Agency, and LLM07 System Prompt Leakage, because those are the risks a hostile user reaches through the request/response path. For RAG and agent systems, add the tool and retrieval-path tests. Supply chain (LLM03) and data poisoning (LLM04) usually belong in a separate review because they sit outside live HTTP traffic. Ask your tester to cite the 2025 edition IDs by number so findings stay comparable release to release.

Does an AI pentest replace network and host penetration testing?

No, they measure different attack surfaces. An AI-behavior pentest exercises the model and agent on the HTTP path: injection, exfiltration through outputs, tool abuse, authorization, and egress. Network and host penetration testing exercises segmentation, OS and service exploitation, sandbox and container escape, and credential theft against the infrastructure the model runs on. A defensible security program runs both as distinct engagements with distinct reports. Combining them under one fixed fee tends to underweight the AI testing, which is the newer and less automated of the two.

How does an AI gateway change what a pentest can prove?

It gives the engagement a control to defeat and an evidence trail to cite. When a tester lands an injection, the question that matters is whether enforcement blocked the resulting action, and an inline gateway is where that decision is made and recorded. The per-decision audit record then documents the identity, the classification, the policy version, and the permit, redact, or deny outcome for every attempt. That converts a pentest finding from an unverifiable claim into a signed record, and it is the same record a regulator asks for under frameworks like the EU AI Act. See signed audit logs for AI requests for the record structure.

AI Penetration Testing: Scoping a Real Engagement Against a Deployed LLM or Agent System

A penetration test against a deployed LLM system starts with a scoping call, and one question on that call decides whether the engagement is worth the invoice: are you testing the model's behavior on the HTTP path, or the infrastructure the model runs on? Those two are separate disciplines, and they use different tools, produce different findings, and often need different people. I want to walk through what AI penetration testing actually covers when the target is a live LLM or agent system in production, which named methodologies to hold your tester to, and the one section of the scope document that separates a real AI pentest from a network scan with a chatbot screenshot stapled to the cover.

Half the "AI pentest" reports I read are the second thing. They enumerate open ports on the inference host, flag a TLS cipher, and never send a single adversarial prompt through the application. That is a fine network pentest. It tells you close to nothing about whether a hostile user can make your agent wire money or read another tenant's records.

What the engagement covers on the AI path

An AI penetration test scoped to the model's behavior exercises five things. Each maps to a specific risk in the OWASP Top 10 for LLM Applications (2025), which is the baseline any tester should reference by ID in their report.

Prompt injection and jailbreak testing. Direct injection through the user field, and indirect injection through content the model retrieves (a RAG document, a web page, an email body, a tool result). This is OWASP LLM01:2025 Prompt Injection. The test is not "did the model say something rude." The test is whether an injected instruction changes what the system does downstream. Deep prompt-injection technique lives in a separate piece, prompt injection testing, and I keep the payload catalog and pass/fail scoring in the prompt injection benchmark.

Data exfiltration through outputs. Whether a crafted prompt causes the model to return secrets, another user's data, or the system prompt itself (OWASP LLM02:2025 Sensitive Information Disclosure and LLM07:2025 System Prompt Leakage). The mechanism is covered in data exfiltration via LLM.

Tool and function-call abuse. For agent systems, whether a compromised prompt can invoke a tool the user was never authorized to call, or pass poisoned arguments to a legitimate tool (OWASP LLM06:2025 Excessive Agency). A test that stops at chat responses misses the entire agent attack surface.

Authorization bypass on model access. Whether identity and role actually gate which models, routes, and tools a caller reaches, or whether a shared service credential flattens everyone into one privilege level. The per-request authorization model this validates is in per-request LLM authorization.

Egress-path testing. Whether a successful injection can make the system reach an unapproved destination, so the injected instruction becomes a live exfiltration channel to an attacker-controlled host. This is the class most testers skip, and it ties directly to AI egress control.

The methodologies to hold a tester to

Three named references keep an engagement honest, and a scope that cites none of them is a red flag.

OWASP Top 10 for LLM Applications, 2025 edition, gives you the risk taxonomy. Ask the tester to map every finding to an LLM ID so the report is comparable across vendors and across years. The related OWASP LLM Top 10 overview breaks each item down.

MITRE ATLAS is the ATT&CK-style matrix of adversary tactics and techniques against AI systems, with real-world case studies. It gives you attack-chain language (initial access, ML model access, exfiltration) that a board understands. ATLAS-aligned red-team planning is in AI agent red teaming, and the injection-specific ATLAS techniques are in MITRE ATLAS prompt injection.

NIST AI 600-1, the Generative AI Profile published July 2024 as a companion to the AI Risk Management Framework, names red-teaming and adversarial testing as measurement actions under the Measure function. It is the reference a US regulated buyer will expect to see cited.

A broader program view, past a single point-in-time test, sits in LLM security testing.

What sits outside an AI pentest

Classic infrastructure penetration testing is a separate discipline, and honest scoping says so out loud. Network segmentation testing, host and OS exploitation, container and sandbox escape, hypervisor attacks, and credential theft against the servers hosting the model all matter. They are not what an AI-behavior pentest measures, and a tester who conflates the two is padding the report.

The line is the transport. If the attack flows as HTTP requests and responses between an authenticated user or agent and an LLM, it belongs in the AI pentest. If the attack is a sandbox escape on the box running the model, a stolen API key used directly against the provider, or lateral movement across the VPC, it belongs in a traditional infrastructure engagement. Both should happen. Pricing one and calling it the other is how buyers end up with a false sense of coverage. My opinion, and a committee would soften it: a combined "AI + infra" fixed-fee quote almost always shortchanges the AI half, because the infra tooling is mature and the AI testing is the part nobody has automated yet.

The gateway is both the control under test and the evidence source

An identity-aware AI gateway plays two roles in a penetration test, and this is the part most pentest write-ups miss.

First, it is a control the findings validate. When the tester lands an indirect prompt injection that instructs the agent to POST customer records to attacker.example, the interesting result is not that the model complied. The interesting result is whether policy at the gateway blocked the egress and denied the unauthorized tool call. A finding of "injection succeeded at the model, enforcement blocked the action" is a passing control. A finding of "injection succeeded and the payload left the building" is a Sev-1. The pentest is how you learn which one you have.

Second, it is the evidence source that proves what a red-team payload actually reached. When the report claims a payload exfiltrated data, the per-decision audit record settles the question: what identity made the call, what the classifier saw in the prompt and the response, which policy version evaluated it, and whether the decision was permit, redact, or deny. Without that record, a pentest finding is one screenshot against the engineer's memory. The signed audit trail pattern is in signed audit logs for AI requests.

Here is the shape of an indirect-injection test case a tester would run against an agent, planted in a document the agent retrieves:

The expected audit-log entry when policy holds, and the artifact the tester should demand as proof, looks like this:

The deny with a named reason, tied to a verified identity and a policy version, is what turns a red-team claim into an auditable fact. Related tool-scoping enforcement is in AI tool use authorization.

DeepInspect

This is the layer a pentest is really measuring when the target is the AI path. DeepInspect is a stateless, identity-aware proxy on the HTTP path between authenticated users or agents and any LLM. It binds each request to the identity the application supplies, classifies prompt and response content, and applies a permit, redact, or deny decision before the request reaches the model or before a tool call executes.

For a penetration test, that gives you the two things a report needs. The gateway is the enforcement point your testers try to defeat, so a passing finding means policy actually blocked the injected egress or the out-of-scope tool call rather than the model merely declining. And every attempt, blocked or allowed, writes a signed per-decision record, so the engagement produces evidence a regulator or an auditor can read rather than a slide deck of screenshots. A policy gateway does not test your sandbox, your hosts, or your network, and it should not claim to. It governs the AI request path, and it proves what happened on it.

If you are scoping an AI penetration test and want the enforcement and audit layer in place before the testers arrive, book a technical deep dive at deepinspect.ai.