Do Bedrock Guardrails replace the need for an AI gateway?

They cover the Bedrock portion of the surface, which is the right answer for a single-provider deployment. The limitation is architectural rather than qualitative: Guardrails apply to calls made to Bedrock, so a developer calling an external provider directly, a SaaS tool with embedded AI features, or a self-hosted model behind an internal endpoint all sit outside that policy. An organization auditing its AI usage usually finds surfaces beyond the one it set out to govern. The practical test is whether your written AI usage policy applies to all of them or only to the one with a console that enforces it.

Does a VPC endpoint mean prompts are protected?

A VPC endpoint through PrivateLink keeps the traffic off the public internet and is worth configuring for the network-exposure reason. It changes nothing about what is in the prompt or who is allowed to send it. Private routing is a transport property; content classification is an inspection property. Teams routinely present a PrivateLink architecture diagram as evidence of data protection during a security review, and the diagram demonstrates that the packets took a private path to a third-party inference service carrying whatever the application put in them.

Is CloudTrail enough for an AI audit trail?

CloudTrail establishes that a call happened, by which principal, when, and against which model, which covers part of the question. It does not contain the prompt or response payloads, so it cannot answer what data was sent. Model invocation logging captures payloads and has to be enabled separately, and enabling it creates an S3 bucket full of raw prompt content that inherits every obligation attached to the data inside it. A record that stores classifications and decisions instead of raw payloads answers the audit question without building a second sensitive datastore.

How do we get per-user policy when everything runs under one execution role?

The application has to propagate the originating identity, typically as a signed header or an assertion carried alongside the request. No control downstream can recover an identity the application never sent. This is application architecture work rather than a gap in any security product, and it is worth doing before evaluating enforcement tooling, since per-user policy and per-user audit records both depend on it. Deployments that skip it get role-granularity policy and role-granularity evidence, which is the ceiling.

Does routing through a proxy add meaningful latency?

Inline policy evaluation runs in the low tens of milliseconds from internal DeepInspect testing, small against the model inference time the user is already waiting on. The more useful question during design is failure behaviour: an enforcement layer that fails closed refuses requests when it cannot evaluate them, which is the correct posture for regulated data and needs to be a deliberate decision with the application team rather than a discovery during an outage.

What about data AWS retains from Bedrock calls?

AWS publishes terms covering Bedrock data handling, including its position on customer content and model training, and those terms are the right primary source for a procurement review. The architectural point that survives whatever the terms say is that the content still left your environment, and the contractual position determines the exposure without changing the data movement. Vendor terms shift where liability sits. They do not stop an employee from putting something into a prompt that should never have been in one.

AWS Bedrock DLP: Classifying Prompt Content Before It Reaches the Inference Path

An IAM policy allowing bedrock:InvokeModel on a given model ARN answers one question: whether this principal may call this model. It has no opinion about the sixty lines of customer records inside the request body.

That is the whole of the AWS Bedrock DLP problem stated in two sentences. The authorization layer AWS gives you is excellent at the resource granularity it was designed for, and prompt content is a granularity below it.

I want to walk through the request path, the AWS-native controls that apply to it, the setting that determines whether you have any evidence at all, and where the enforcement point sits for organizations calling more than one provider.

The request path

A Bedrock call is an HTTPS request to a regional runtime endpoint, bedrock-runtime.us-east-1.amazonaws.com and its equivalents, signed with SigV4. The body carries the prompt.

The application issues a POST to the converse path for a model identifier such as anthropic.claude-3-5-sonnet-20241022-v2:0, and the body carries a messages array whose user content is free text: in this case an instruction to draft a reply, followed by the full ticket body.

Three properties of that request matter for data loss prevention. It is TLS-encrypted, so a network DLP appliance sitting at the perimeter sees a connection to an AWS endpoint and nothing about the ticket body inside it. It is signed with a principal identity, so AWS knows which role made the call. And the payload is unstructured text assembled by an application, so nothing upstream has classified it.

If the deployment uses a VPC endpoint through PrivateLink, the traffic never traverses the public internet, which is a real network-exposure improvement and changes nothing about content visibility. Private routing and content inspection are separate properties, and teams regularly credit the first for the second.

What the AWS-native controls cover

Bedrock Guardrails evaluate content against configured policies, including denied topics, content filters, word filters and sensitive-information filters that can block or mask categories such as personal identifiers.

That is a genuine control and it should be configured. Its architectural characteristic is where it runs: inside the AWS inference path, applied to calls made to Bedrock. For an organization whose entire AI surface is Bedrock, that placement is fine. For an organization whose developers also call OpenAI directly, whose data science team uses a self-hosted model, and whose SaaS vendors embed their own AI features, Guardrails cover the Bedrock slice and the other endpoints stay ungoverned. Policy written in one console applies to the traffic that console sees.

IAM handles the authorization half at resource granularity. A role may be permitted bedrock:InvokeModel on a specific model ARN and denied on others, which is how most organizations restrict which models are reachable. What IAM cannot express is a condition on prompt content, because the content is opaque to the policy language.

The default that decides whether you have evidence

CloudTrail records Bedrock control-plane activity, so you get a record that a call was made, by which principal, at what time, against which model.

Model invocation logging, which captures the request and response payloads themselves to S3 or CloudWatch Logs, is a separate feature that has to be turned on. That default is the most consequential one in the service, and it stays off in plenty of deployments eighteen months into production use. The team discovers it during the first incident, when the question is what was in the prompts and the honest answer is that nobody recorded them.

Turn it on before you need it. Then confront the second-order problem: those payloads are now sitting in an S3 bucket in full, unclassified and unredacted, which converts a visibility gap into a data-retention obligation. A log of raw prompts containing customer PHI is itself a system holding customer PHI, subject to the same access controls, retention limits and deletion obligations as any other. Classification at the boundary, storing decisions and classifications rather than raw payloads, avoids manufacturing that second problem while solving the first.

Where classification belongs

The classifier has to see the prompt before it is signed and sent, which puts the enforcement point between the application and the runtime endpoint.

Routing Bedrock traffic through a proxy is straightforward, because the AWS SDKs accept an endpoint override. Constructing the bedrock-runtime client with endpoint_url pointed at an internal gateway host, rather than at the regional AWS endpoint, is the entire application change. SigV4 signing, region selection and model identifiers stay as they were.

At that point the deployment has a single location where prompt content is classified, policy is evaluated against the identity that originated the request rather than the execution role, and the record is written. The same boundary handles calls to non-AWS providers, which is what makes the policy consistent across the estate rather than per-console.

The classes to classify are the organization's existing ones applied at prompt granularity: customer PII, PHI where healthcare data is in scope, NPI and pre-announcement material for financial services, CUI for federal contractors, source code with IP indicators. The practice is covered in prompt-level DLP.

Identity, and the execution-role problem

Bedrock sees the IAM role that signed the request. In most deployments that is an application execution role shared by every user of the application.

The consequence is that policy cannot be written per person or per team, and the audit record attributes every prompt to arn:aws:iam::123456789012:role/app-inference-role. When a regulator asks who submitted a specific prompt, the answer is a service account, which satisfies nobody.

Propagating the originating identity from the application into the request is upstream work that no downstream control can substitute for. The enforcement layer evaluates what the application gives it, and an application that gives it a shared role gets policy at the granularity of a shared role. This division is covered in the post-authentication gap for AI agents.

DeepInspect

This is the gap DeepInspect closes for multi-provider estates. DeepInspect sits at the AI request boundary as a stateless proxy between authenticated users or agents and the model endpoints they call, including Bedrock. Prompt content is classified before the request is forwarded, policy is evaluated against the identity the application propagates, and a per-decision record is written under your control rather than in a vendor console. Enforcement is inline and fails closed.

DeepInspect is model-agnostic, which is the point of putting it in front of Bedrock rather than relying only on controls inside it. The same policy covers the Bedrock calls, the direct OpenAI calls the platform team makes, the Azure OpenAI deployment in the European subsidiary and the self-hosted model behind the research cluster. It does not replace IAM, does not replace Guardrails, and does not manage your VPC.

If your Bedrock deployment is one of several AI surfaces and the policy only exists in one console, the other surfaces are already uncovered. Book a demo today.