Is AI cloud security different from cloud security posture management?

They cover adjacent layers of the same estate. CSPM and AI-SPM scan resource configuration and flag misconfigured model resources, exposed endpoints, or missing logging. They read state on a schedule. AI cloud security in the sense that matters for data governance concerns the outbound request from a workload to a managed model endpoint, where the prompt lives inside the HTTPS body for a few hundred milliseconds. Configuration scanning cannot see that body. You want CSPM for the resource layer and a request-path control for the inference layer, because they inspect different things.

Do IAM roles govern what my workloads send to Bedrock or Vertex AI?

IAM decides whether a role may call the model API at all, at the granularity of the API action and the resource ARN. An IAM policy can allow or deny bedrock:InvokeModel on a specific model. It evaluates request metadata, so it cannot express a rule about the content of the prompt, such as blocking source code or customer PII in the request body. Governing the body requires a control that reads the decrypted request, which sits above IAM on the same path.

How does an AI gateway relate to AWS PrivateLink or Azure Private Endpoints?

PrivateLink and Private Endpoints keep AI traffic on the provider's private network rather than the public internet, and they constrain which subnets can reach the endpoint. They govern the network path. A permitted private path still carries whatever the request body contains. An identity-aware gateway terminates the AI request on the decrypted path, classifies the content, applies policy, and forwards to the endpoint your PrivateLink route already permits. The two run together, one securing the network and the other governing the payload.

Does the gateway help with cross-region data residency?

Yes, at the request layer. When a workload in eu-west-1 calls a model endpoint in a US region, the prompt crosses jurisdiction. A gateway on the path can read the identity and the target region of the endpoint and block or redirect requests that would send classified data to a non-compliant region. It records the region of every call. Region-pinning at the network layer helps, and the per-request record proves which region each prompt actually reached.

What does the gateway not do?

It does not manage your cloud infrastructure. A policy gateway does not scan resource configuration, provision IAM roles, configure VPCs, or manage encryption keys, which stay owned by your platform team as cloud-native controls. The gateway governs the AI request and response on the HTTP path and writes the per-decision audit record. When a vendor claims a single AI gateway replaces your CSPM and your key management, treat that as a scoping error.

AI Cloud Security: Governing the Outbound Path from Your Workloads to Managed Model Endpoints

A workload in your AWS account calls bedrock-runtime.us-east-1.amazonaws.com over HTTPS, signs the request with SigV4 using the role attached to its task or node, and receives a model completion. Your CloudTrail record of that call shows the API action InvokeModel, the calling role ARN, the source IP, the model ID, and a timestamp. What that record omits is the prompt itself, the human or agent the workload was acting for, and the data classification that rode inside the request body. That gap is the whole of what makes AI cloud security a distinct problem, and everything else labelled "AI cloud security" is the cloud security you already run.

I want to walk through where the outbound path to managed model endpoints diverges from generic cloud posture, be precise about what the cloud-native controls cover, and show the one control layer that reads the request itself.

Where AI cloud security diverges from generic cloud security

Generic cloud security governs three things well: the configuration of your resources, the identities that can act on them, and the network paths between them. AWS Config, Azure Policy, and Google Security Command Center catch a public S3 bucket, an over-permissive IAM role, or a security group open to 0.0.0.0/0. Those tools inspect state and configuration.

The managed model endpoint breaks that model in one specific way. When a workload calls Bedrock, Azure OpenAI, or Vertex AI, the security-relevant content lives inside the HTTPS request body, not in any resource configuration. The prompt is a payload, not a state. A posture tool that reads your account configuration every fifteen minutes has nothing to inspect, because the sensitive material exists only for the few hundred milliseconds the request is in flight. Netwrix reported that 97% of organizations that suffered an AI-related breach lacked proper access controls for AI services (Netwrix, 2026). Those are access controls on the request path, which is exactly the layer configuration scanning cannot see.

The three managed endpoints and their address shapes

Each provider exposes its inference data plane at a predictable hostname, and knowing the exact shape matters because your egress and policy rules key off it.

The distinction that trips teams up: bedrock-runtime.{region}.amazonaws.com is the inference data plane, while bedrock.{region}.amazonaws.com is the control plane for managing models and guardrails. Allowlisting the second does not permit inference, and allowlisting the first without governing its body leaves every prompt ungoverned. Azure issues a distinct subdomain per resource, so contoso-eastus.openai.azure.com and contoso-westeu.openai.azure.com are separate hosts your policy has to enumerate. The AWS Bedrock endpoint reference, Azure OpenAI reference, and Vertex AI REST reference each publish the regional variants. If you run all three, the provider-specific mechanics differ enough that they warrant their own treatment in the Azure OpenAI gateway patterns, Vertex AI gateway patterns, and Bedrock gateway patterns writeups.

Provider-managed inference versus self-hosted

Managed inference on Bedrock, Azure OpenAI, and Vertex AI means the model runs in the provider's account and your workload reaches it as an authenticated API call. Self-hosted inference means the model runs on your GPUs, behind your VPC, reachable at an internal address such as a vLLM server on 10.0.4.12:8000. The security question stays constant across both. Which identity in your cloud is allowed to reach which model endpoint, and what may that identity send.

Provider-managed inference adds a wrinkle the self-hosted case avoids. The prompt leaves your trust boundary the instant the request crosses to the provider account. Once the body reaches bedrock-runtime.us-east-1.amazonaws.com, the data governance decision has already been made by whatever sat on the path before it left. If nothing inspected the body, the decision was made by default and the sensitive field is already gone. My opinion, and a committee would soften this: teams over-invest in provider-side model guardrails and under-invest in the outbound checkpoint, because the guardrail ships in the console and the checkpoint has to be built.

Cross-account and cross-region data flow

A single AI request can cross more account and region boundaries than the calling engineer expects. A workload in a spoke account assumes a role in a shared services account, which holds the Bedrock access, which routes to a model hosted in us-east-1 even though the workload runs in eu-west-1. That last hop matters for data residency. A prompt containing EU personal data that resolves to a US-region model endpoint has left the jurisdiction, and the Vertex AI data residency documentation and Azure's regional resource model exist precisely because customers need to pin inference to a region. Governing that requires a control that reads both the identity making the call and the region of the endpoint it targets, which the AI data residency controls breakdown covers in depth. Running the same model across several providers or regions is common enough that multi-cloud AI gateway patterns have their own design constraints.

What the cloud-native controls cover, and where they stop

The controls your cloud platform team already runs each do real work on this problem, and each stops before the request body.

Cloud Security Posture Management (CSPM) and AI security posture management scan configuration and flag a Bedrock resource with no logging enabled or a Vertex endpoint reachable from the public internet. They inspect state on a schedule. They never see a prompt in flight. The AI-SPM overview is clear about this scoping.

IAM roles and resource policies decide whether an identity may call InvokeModel at all. They operate at the granularity of the API action and the resource ARN. An IAM policy can permit or deny access to a model, and it cannot express "this role may call this model but may not send source code in the prompt," because IAM evaluates the request metadata, not the request body.

VPC and network configuration, including AWS PrivateLink for Bedrock, Azure Private Endpoints, and Google Private Service Connect, keep the traffic off the public internet and constrain which subnets may egress to the provider. Destination allowlisting through DNS or a forward proxy decides which provider hosts are reachable. These govern where the traffic goes, and a permitted destination still carries whatever the body contains, which the AI egress control piece traces in detail.

Key management (KMS, Azure Key Vault, Cloud KMS) protects the API credentials and encrypts the data at rest and in transit. It secures the keys and the TLS session. It has no view into the plaintext prompt inside that session.

Each of these is a cloud-native control worth running, and none of them reads the AI request, which is the seam that AI cloud security has to close.

Governing which identity reaches which model endpoint

Closing the seam requires a control on the decrypted HTTP path that evaluates three things per request: the identity the call is made for, the model endpoint it targets, and the classified content of the body. Expressed as policy, that looks like:

The rule binds a cloud identity to a specific set of model endpoints and a content policy, and it records every evaluation. An IAM policy handles the first clause at the API-action level. The content clause and the per-request audit record have to come from something sitting on the request itself.

DeepInspect

This is the gap DeepInspect closes, and it closes exactly two things, so I want to be precise about the boundary. DeepInspect is a stateless, identity-aware proxy on the HTTP path between your workloads and any managed or self-hosted model endpoint. It does two jobs, and only these two. It enforces identity-aware policy on each outbound AI request, binding the call to the identity your application supplies and deciding pass, redact, or block against the endpoint and the classified content before the request reaches bedrock-runtime.{region}.amazonaws.com, {resource}.openai.azure.com, or {region}-aiplatform.googleapis.com. And it writes a per-decision audit record containing the identity, the target endpoint, the region, the content classification, the policy version, and the outcome.

DeepInspect does not replace your CSPM, your IAM roles, your VPC and PrivateLink configuration, or your key management. Those remain the cloud-native controls that govern your infrastructure. A policy gateway does not scan resource configuration, issue roles, or manage encryption keys. It governs the AI request and response, and it produces the record that CloudTrail's InvokeModel entry lacks. This complements the identity-aware AI gateway model and the signed audit records that regulated teams need, and it sits alongside the provider mechanics in the AI gateway for regulated industries guide.

If your cloud platform runs AI workloads against Bedrock, Azure OpenAI, or Vertex AI and your only record of those calls is the provider's API log, book a demo today.