AI Usage Monitoring: Requests, Identity, and Shadow AI
AI usage monitoring records who sends which prompt to which approved or shadow AI service, then preserves the policy decision for investigation.
AI usage monitoring records who sent which prompt to a model service such as OpenAI, then retains the policy decision on that request. At the LLM request boundary, the record links the authenticated requester and destination to a prompt classification. It also captures the decision and latency so an investigator can reconstruct the routed call.
Most enterprise monitoring stops at the TLS encryption boundary and treats traffic to api.openai.com, api.anthropic.com, or bedrock-runtime.amazonaws.com as a single destination with a stable IP range. The prompt content and the decision made about it remain invisible. This article maps the inspection point, the signals worth retaining, and the responsibilities that stay outside an HTTP AI gateway.
TL;DR
AI egress monitoring needs visibility into the request that actually reaches an LLM provider. A routed HTTP proxy can record the requester's identity, destination, classified content, policy outcome, and latency. Endpoint controls need their own evidence. Tool authorization and local execution belong with their own control owners. Downstream data access needs its own evidence as well.
AI usage monitoring needs separate evidence streams
AI usage monitoring describes two different jobs that share a label. An AI product team watches model quality and operational cost. A security team needs an inventory of approved services. It also needs evidence of shadow AI on managed devices and a record of sensitive data crossing a model boundary. The NIST AI Risk Management Framework separates related governance and measurement work, which is a useful way to keep those records distinct.
An HTTP AI gateway owns deliberately routed application or agent calls. It can attach the authenticated user or agent supplied by the calling system. The gateway inspects the prompt after TLS termination, applies policy, and writes the decision record. Browser telemetry and endpoint controls own discovery of consumer AI use that bypasses that route. DNS data or a secure web gateway can provide another discovery signal. I would resist calling any one of those feeds a complete AI usage inventory. A usable investigation joins the discovery signal with the request-path record only where the traffic was intentionally routed.
The distinction matters for the query "shadow AI monitoring." A new browser domain or endpoint event can establish that a managed device contacted an unapproved service. It cannot establish the prompt text or the identity behind a backend API call. The request-path audit record supplies those fields for managed HTTP model traffic, while the discovery controls keep their own evidence for every other route.
What AI egress monitoring has to observe
Egress monitoring at the AI boundary surfaces five observables.
Destination. Which LLM endpoint the request is going to. Major providers have public IP ranges and stable hostnames. A request going to api.openai.com is a sanctioned destination. A request going to a long-tail model provider the deployer has no contract with is shadow AI. The destination is the first filter.
Identity context: which user or agent is on whose behalf the request is being made. Without identity context, the traffic is anonymous from the deployer's perspective, even when the user is authenticated upstream. Static service credentials destroy identity context.
Prompt content: the actual text moving across the boundary carries the data classification. It can contain PII or PHI. Financial NPI and proprietary source code often appear with customer data, and all require the same request-path review. Network DLP runs underneath TLS and cannot inspect that prompt content.
Response content. The text the model returns can carry data the requester was not authorised to receive or evidence of a successful prompt injection. A gateway can inspect a routed HTTP model response. Tool authorization and downstream actions remain with the tool endpoint and the systems that execute them.
Latency: the end-to-end latency of the request surfaces inspection bottlenecks and upstream provider degradation. A widening tail is a security signal and a performance signal.
Why the existing stack does not see this
Enterprise egress monitoring stacks were built for a different traffic pattern.
Network DLP runs underneath TLS. Network DLP inspects unencrypted traffic at the network boundary, while AI traffic moves as HTTPS. The DLP sees encrypted bytes going to a known cloud endpoint. The prompt content is not visible without TLS termination at the egress layer, and TLS termination at the egress layer for AI provider domains is rarely configured.
CASB and SaaS controls miss API calls. CASB tools focus on sanctioned and unsanctioned SaaS applications. They observe browser-based usage of the SaaS UI. API calls to an LLM provider from a backend application do not show up in the CASB view because no browser session is involved.
Endpoint DLP inspects local actions, not API traffic. It examines file movement, clipboard activity, and process actions on a device. An employee pasting source code into a browser-based ChatGPT prompt is partially visible. The same employee submitting that source code through a Python script to api.openai.com is largely invisible.
Application logs are self-attested. Logs the application keeps about its own AI usage are produced by the application. The application that generated the AI decision is the system logging the AI decision. The self-attestation problem applies: selective logging, suppression, and loss on crash are all possible.
The architectural pattern that produces visibility
AI egress monitoring requires inspection at the layer where the prompt is decrypted. That layer is the AI request boundary, between the application and the model endpoint.
Inline proxy with TLS termination for AI provider domains. The proxy terminates TLS for traffic to LLM provider domains, inspects the prompt content, re-encrypts, and forwards the request to the provider. This gives the provider a request from the proxy's egress IP while the proxy and deployer can inspect the prompt.
Identity context attached at the request layer. The application's authentication context is attached to the proxy request as a header or token. The proxy reads the context, attaches it to the audit record, and uses it for policy evaluation. Pillar 1 of the NIST framework.
Per-decision audit record. Every inspected request produces a record with destination, identity, prompt classification, response classification, and policy outcome. It also captures latency. The record stream is the deployer's evidence layer. Tool invocation records belong with the tool endpoint.
Coverage across model providers. The proxy operates in front of any HTTP-based LLM endpoint. The deployer's compliance posture stays consistent across providers, which matters when the deployer uses more than one. The AI gateway architecture determines where this proxy belongs, while AI DLP and traditional DLP cover distinct inspection surfaces.
Operational signals worth tracking
Egress monitoring surfaces five signals the deployer's SOC and AI platform team read.
Destination distribution. The distribution of destinations over time. Shadow AI shows up as long-tail destinations the deployer has no contract with. A growing share of traffic to an unsanctioned provider signals the platform team needs to either onboard the provider or block it.
PII rate per route. The proportion of prompts on a route that contain PII. A rising rate on a route that should not see PII (an internal copilot processing public documents, for example) signals either a misconfiguration or a user behavior change worth investigating.
Cross-region traffic. The proportion of traffic crossing jurisdictional boundaries shows when EU residents' data reaches a US-hosted model endpoint, a GDPR Chapter V concern. The monitor surfaces that volume for review.
Failure rate by destination. The error rate the provider returned. A rising failure rate signals upstream provider degradation or a contract that has hit a limit. The deployer's AI platform team triages.
Latency tail. The p99 latency measures the slowest one percent of requests. A widening tail signals an inspection bottleneck or an upstream provider issue that the deployer's SRE team should triage.
Unauthorised destinations. Requests to destinations the deployer's policy does not authorise. The proxy blocks them at the egress and produces an audit record showing the attempted access.
DeepInspect
This is the request-path evidence DeepInspect produces. DeepInspect sits inline on the egress path between the application and the LLM provider. The proxy terminates TLS for traffic to LLM provider domains, inspects the prompt content against the deployer's policy, attaches identity context the application supplies, produces a signed audit record, and forwards the request to the provider.
For egress monitoring, the proxy is the visibility layer for destination, identity, prompt content, response content, policy decision, and latency. The audit record stream feeds the deployer's SIEM and AI observability stack. Tool invocation authorization, local execution, credential handling, and downstream effects require controls at their own enforcement points. The same request record can support compliance reporting under EU AI Act Article 12 and sector-specific regimes.
The proxy operates across model providers. A deployer using OpenAI, Anthropic, and Bedrock simultaneously sees a unified audit record stream covering all three. Shadow AI shows up as traffic to unsanctioned providers, with the same identity context attached, which lets the platform team decide whether to onboard or block.
If you are deploying AI in a regulated environment and your egress monitoring stops at the TLS boundary, the bulk of your AI risk is invisible. Book a demo today.
Frequently asked questions
- How is AI egress monitoring different from traditional DLP?
Traditional DLP inspects file movement, email, and endpoint actions. AI egress monitoring inspects HTTP API calls to LLM provider endpoints. The traffic patterns are different. Traditional DLP misses prompt content because the prompt travels as HTTPS to a cloud endpoint that DLP does not terminate. AI egress monitoring sees the prompt because it terminates the TLS at the AI request boundary and inspects the decrypted content. The two layers complement each other across the broader DLP surface.
- Can we run AI egress monitoring on top of an existing API gateway?
An AI egress monitor can run with an existing API gateway such as Kong or Apigee. The existing gateway handles transport-level concerns including TLS, rate limiting, and authentication. The AI monitor sits in front of the LLM provider call, reads the decrypted prompt, and applies AI-specific policy. The traditional gateway sees the traffic, while the AI monitor sees the content.
- Does the deployer's existing observability stack catch any of this?
The existing observability stack captures part of this picture. SIEM tools see network connection metadata: source IP, destination IP, byte counts, and timestamps. They do not see prompt content. The application's own logs may include sampled prompts, but the application produced those logs and the self-attestation problem applies. Endpoint DLP catches some browser-based AI usage but misses API-driven usage. The visibility gap is structural and the fix is at the AI request boundary.
- What does AI egress monitoring contribute to incident response?
The audit record stream is the SOC's primary evidence source for AI incidents. Detection of a prompt-leak, an unauthorised destination, or a tool-invocation anomaly comes from the egress monitor. The containment lever is a policy update at the egress proxy, which propagates within seconds. The postmortem evidence is the per-decision audit record.
- Do companies monitor AI usage?
Companies monitor AI usage through several evidence sources. Provider administration records can show activity in an approved tenant. Managed-device and browser telemetry can identify consumer AI services or browser extensions. DNS data and secure web gateways add further discovery. An HTTP AI gateway adds a per-request record for application and agent traffic that the organization deliberately routes through it. The combined inventory should name the source of every observation so the security team can distinguish a discovered service from a prompt-level audit record.
- How can I check for AI usage?
Start with the managed-device and network signals that show which AI services are contacted, then reconcile them with approved provider tenants and application ownership records. For the systems under your control, require model calls to use an authenticated HTTP route. Inspect the resulting audit record for requester identity and destination, then review its classification, decision, and time. That process exposes an unapproved destination without claiming that a domain lookup reveals the content of an encrypted prompt.
- How can AI be monitored?
Monitor AI at the control point that can observe the fact you need. Endpoint and browser controls observe managed-user activity. Provider records describe a sanctioned tenant. An AI gateway observes routed application or agent requests after TLS termination and can enforce policy before the model receives the prompt. Join the evidence during an investigation, but retain the control boundary on each record.
- How does AI egress monitoring intersect with the EU AI Act?
Article 12 of the EU AI Act requires automatic recording of events over the lifetime of a high-risk system. The audit record stream can contribute request-path evidence. Provider retention appears in Article 19, while the deployer six-month floor appears in Article 26(6). Article 15 expects cybersecurity controls appropriate to the system. A gateway produces evidence for routed HTTP model requests; the full system record also needs evidence from the application and other control owners.