Azure OpenAI Security: The Controls Microsoft Ships and the Layer It Leaves to You
Azure OpenAI ships real security controls: Microsoft Entra ID authentication, managed identities, private endpoints, content filters, and diagnostic logging into Azure Monitor. Those controls secure the connection and the platform. They stop at the question of whether a specific authenticated caller is permitted to send a specific prompt to a specific deployment. This piece walks the controls Azure provides, shows where they stop, and gives the identity-aware authorization pattern that closes the post-authentication gap on Azure OpenAI traffic, with the request-level detail an engineer needs.
An Azure OpenAI request is an HTTPS POST to https://{resource}.openai.azure.com/openai/deployments/{deployment}/chat/completions, authenticated with either an api-key header or a Microsoft Entra ID bearer token. Everything Azure gives you to secure that call operates on the connection and the platform around it. What none of it evaluates is whether the authenticated caller behind the token is permitted to put this particular prompt in front of this particular deployment. I want to walk the controls Microsoft ships, mark where they stop, and show the authorization pattern that covers the part they leave to you.
The security controls Azure OpenAI ships
Azure gives an engineering team a genuine set of platform controls, and using them is the baseline.
Entra ID authentication and managed identities. You can disable API-key access and require Microsoft Entra ID tokens, then assign the Cognitive Services OpenAI User role through RBAC. A workload running in Azure can use a managed identity so no static key ever lives in code or config. This removes the shared-secret failure mode that plagues key-based access.
Private networking. A private endpoint places the Azure OpenAI resource on your virtual network, and disabling public network access forces all traffic through the private link. The model endpoint stops being reachable from the public internet.
Content filters. Azure's content filtering runs configurable severity thresholds on prompts and completions for categories like hate, violence, and self-harm, and supports a prompt-shield layer for jailbreak and indirect-injection patterns.
Diagnostic logging. Diagnostic settings stream request metadata to Azure Monitor, Log Analytics, or a storage account, giving you a platform-level record of who called the API and when.
Where the Azure controls stop
Each control has a boundary that matters for a real deployment. Entra ID authenticates the caller and RBAC grants coarse access to the resource, and neither evaluates the content of the request. Managed identity proves a workload's identity to Azure and says nothing about which end user delegated the task the workload is performing. Content filters score the prompt for safety categories and do not reason about your data classification, so a prompt full of customer PII scores clean because it is neither hateful nor violent. Diagnostic logs record that a call happened and are written by the platform for the platform, so they lack your policy state at the moment of decision.
The post-authentication gap on Azure OpenAI traffic
The gap sits after authentication. Entra ID confirms who is calling. It does not answer whether this caller, in this role, is permitted to send this prompt, carrying this data classification, to this model right now. A logged-in analyst with the Cognitive Services OpenAI User role can paste a spreadsheet of unreleased financials into a completion request, and every Azure control passes it: the token is valid, the network path is private, the content filter finds nothing violent, and the diagnostic log dutifully records the call. This is the post-authentication gap, and it is an authorization problem that lives at the request layer, above what the platform inspects.
What identity-aware authorization on the request looks like
Closing the gap means evaluating the request itself, on the path to the model, against the caller's identity and the data in the prompt. In practice that is a policy decision point in front of the Azure OpenAI endpoint. A raw call looks like this:
An identity-aware enforcement layer intercepts that request, reads the caller identity from the token, classifies the prompt content, evaluates policy, and either forwards the call or returns a decision like this before the model is ever reached:
The evaluation is deterministic and fail-closed, and it commits that record independently of the calling application before any response returns.
Compliance framing
For regulated workloads on Azure OpenAI, the diagnostic log is not the audit record a regulator wants. EU AI Act Article 12 requires automatic, traceable event logging for high-risk systems, including inputs and the natural persons involved, applying from August 2, 2026 per the European Commission. A platform log written by the platform, for its own view of the call, sits in a weaker position than an identity-bound record produced independently of the application under review.
DeepInspect
This is exactly what DeepInspect does. DeepInspect is a stateless proxy that sits inline in front of your Azure OpenAI deployments, and it is model-agnostic, so the same layer covers OpenAI, Anthropic, Bedrock, Vertex, and self-hosted endpoints in the same deployment. For every request it evaluates the caller's identity, role, the data classification in the prompt, and organizational policy, then returns a pass or block decision before the traffic reaches the model. Enforcement adds under 50 milliseconds in internal testing, invisible against the model's own response time, and the default on ambiguity is deny.
For every decision it commits an identity-bound audit record before the response returns, written independently of the calling application. That record carries the identity, policy, data classification, and outcome that Azure's diagnostic logs never held. DeepInspect does not replace Entra ID, private endpoints, or content filters. It adds the authorization-and-audit layer that sits above them, on the request itself. Book a demo today.
Frequently asked questions
- Does Microsoft Entra ID secure Azure OpenAI on its own?
Entra ID authenticates the caller and RBAC grants access to the resource, which is the right baseline. Neither evaluates the prompt content, so an authenticated caller can still send data they should not. Request-level authorization is a separate layer above authentication.
- Do Azure OpenAI content filters classify my sensitive data?
No. Content filters score prompts and completions for safety categories like hate and violence and for jailbreak patterns. They do not reason about your data classification, so a prompt containing PII or material nonpublic information passes if it is not otherwise harmful.
- Are Azure OpenAI diagnostic logs enough for an audit?
They record that a call happened, written by the platform for its own view. For a regulatory audit under EU AI Act Article 12, an identity-bound record produced independently of the calling application, carrying policy state and data classification, is the stronger evidence.
- How do I stop a legitimate user from sending prohibited data to a model?
Evaluate the request on the path to the model against the caller's identity and the prompt's data classification, and block it before it executes. That authorization decision sits above Entra ID and content filtering, at the AI request boundary.
- Does an inline enforcement layer add latency to Azure OpenAI calls?
The enforcement decision adds under 50 milliseconds in internal testing, against the 500 milliseconds to several seconds an Azure OpenAI completion takes. The overhead sits inside the model's own latency and is not perceptible to the caller.