Should the gateway use api-key or Entra ID for Azure OpenAI?

Prefer Entra ID. The static api-key authenticates the resource, so a gateway that forwards it learns nothing about the person behind the call, and a leaked key grants full access. With Entra ID, the client presents a bearer token the gateway validates to extract the natural-person identity, and the gateway authenticates to Azure with its own managed identity so the upstream credential never lives in the client. Keep the api-key path available only for legacy callers that cannot present a token, and scope it tightly.

Why does the api-version parameter matter to the gateway?

Azure OpenAI requires an api-version query parameter on every call, and the request fails without it. The gateway should carry the client's api-version through unchanged and reject a request that omits it, so a malformed call fails at the gateway rather than after a round trip to Azure. Pinning the versions your policies were written against also keeps a client from silently moving to an API shape the gateway has not been validated for.

Does a gateway replace Azure OpenAI content filtering?

No, and it is worth being precise. Azure's content filtering runs inside the model service and covers categories Microsoft defines, scoped to the Azure endpoint. A gateway adds the layer Azure's filter does not: per-request authorization bound to a natural-person identity, classification of regulated data types like PHI and MNPI, and a signed audit record independent of the resource. The two operate at different points, and the gateway is what produces the identity-bound evidence an auditor inspects.

How does per-deployment policy differ from resource-level access?

Azure role assignments govern who can administer or call a resource. Per-deployment policy at the gateway governs whether a specific person, in a specific role, may send a specific class of data to a specific deployment on that resource. Resource-level access is coarse and administrative. Deployment-level policy is per request and turns on the data in the prompt, so a PHI-approved deployment is reachable only by the role permitted to send PHI, and every other attempt fails closed and is recorded.

Azure OpenAI Gateway Setup: A Governed Proxy in Front of Your Deployments

Azure OpenAI does not look like the public OpenAI API, and the differences are exactly the parts a gateway has to handle. A call addresses a specific deployment on a resource-specific host, https://{resource}.openai.azure.com/openai/deployments/{deployment}/chat/completions?api-version={version}, and it authenticates with either an api-key header or a Microsoft Entra ID bearer token. Setting up a governed proxy in front of Azure OpenAI means terminating that shape, checking identity, and forwarding it unchanged.

This guide is the setup walk-through. For the deployment-topology decisions behind it, the token-governance and regional load-balancing choices and the model-layer content-filter gap, see Azure OpenAI gateway patterns. Here the goal is a working, identity-bound gateway in front of your deployments.

Step one: terminate the deployment-shaped request

The gateway presents an endpoint the client points at instead of the Azure host, and it preserves the Azure request shape so no client code changes beyond the base URL:

The deployment path segment and the api-version query parameter are load-bearing. The deployment names the model instance, and policy will turn on it. Carry both through unchanged, and reject a request that omits api-version, because Azure will reject it anyway and the gateway should fail fast.

Step two: validate the Entra ID token and bind the identity

Prefer Microsoft Entra ID over the static api-key. A key identifies the resource, not the person, and a gateway that only forwards a shared key has learned nothing about who is calling. Require an Entra ID bearer token from the client, validate it, and extract the natural-person identity:

The gateway then authenticates to Azure OpenAI itself, ideally with its own managed identity requesting a token for https://cognitiveservices.azure.com/.default, so the upstream credential never lives in the client. That closes the post-authentication gap: Azure sees the gateway's managed identity, and the gateway holds the record of the real person behind each call.

Step three: apply per-deployment policy

Because the deployment name is in the path, policy attaches to it. Different deployments serve different models and different approvals:

The gateway classifies the prompt content against regulated data types and evaluates the rule for the targeted deployment. A PHI-approved deployment is reachable only by the role that is permitted to send PHI, and every other path fails closed.

Step four: constrain egress to the Azure host

Identity and policy are only a guarantee if traffic cannot route around them. Restrict egress from the application tier so the only path to *.openai.azure.com is through the gateway:

Azure OpenAI resources are per-region and per-resource hosts, so the allow-list names the specific resource endpoints in use. A service that still holds an api-key and calls its resource directly bypasses the gateway, which is why egress control is part of setup, not an afterthought.

Step five: commit an audit record for every decision

Each pass-or-block decision writes a record naming the person, the deployment, the data class, the outcome, and the policy version, committed before the response returns and on a write path the application cannot alter:

Azure's own diagnostic logs record that the resource was called. They do not record which natural person the application was acting for, because the resource only saw the gateway's managed identity. The identity-to-decision binding lives in the gateway's record, which is the artifact an auditor asks for.

DeepInspect

This is the problem DeepInspect was built to solve. Azure OpenAI gives you deployment-addressed endpoints and a choice between a shared key and an Entra ID token. DeepInspect sits in front of those deployments as the governed proxy this guide describes.

DeepInspect terminates the deployment-shaped request, validates the Entra ID token, binds the call to the natural-person identity, classifies the prompt against regulated data types, and applies per-deployment policy before forwarding to {resource}.openai.azure.com with its own managed identity. The decision is deterministic and fail-closed, under 50 ms in internal testing, and every decision commits a signed, identity-bound audit record before the response returns, on a write path the application never controls. If you are standing up Azure OpenAI for a regulated workload and need identity-bound authorization and evidence on it, book a demo today.