Does the inspection point read every prompt the enterprise sends?

The inspection point reads every prompt that traverses it. The deployment pattern routes AI-bound traffic through the inspection point at the corporate egress (network policy directs known LLM provider hostnames through the proxy), the corporate identity provider (credentials issued by the corporate IdP authenticate against the proxy as the front door to the provider), and the application configuration (corporate-managed runtimes point at the proxy as the LLM endpoint). Traffic the deployment does not route falls into the shadow AI discovery layer, which surfaces the activity through other signals (browser extension, CASB integration, network telemetry).

How does the inspection point handle providers that change their API shape?

The inspection point normalizes the prompt content into an internal representation regardless of the provider's API shape. A new provider, a new model that uses a different request body shape, or a provider that updates the request format requires the inspection point to add a parser for the new shape. The classifier pipeline operates on the internal representation, so the classifier rules do not change when the provider format changes. The parser layer is the change surface.

Is the inspection point a privacy concern for employees?

The inspection point reads the prompt content under the corporate AI usage policy the employee agreed to as a condition of corporate AI access. The records are access-controlled at the audit store layer, which means the inspection point operator does not have ad-hoc access to past prompts. The audit reviewer access goes through the compliance review process the corporate environment runs for other sensitive logs (financial system access logs, HR system access logs, healthcare PHI access logs). The privacy posture is comparable to the existing endpoint DLP and email gateway investment, which inspect employee communications under the same corporate policy framework.

What happens to prompts the policy denies?

A denied prompt does not reach the model. The inspection point returns a structured denial to the application (a 403 with a JSON body identifying the policy rule that fired, the data classification that triggered the denial, and the policy version in effect). The application presents the denial to the user with a configurable message the deployment chose. The audit record captures the denied prompt, the policy decision, and the user-facing message. The compliance reviewer reads the record alongside the permits and the redacts.

Does the inspection point work for streaming responses and long context windows?

The inspection point handles streaming responses by buffering the stream up to a policy-defined chunk size, running the classification on each chunk, and forwarding the chunk to the caller after the policy decision. Long context windows (100,000 tokens, 1,000,000 tokens, or larger) are handled by running the classification pipeline against the full window; the classifier latency scales sublinearly with the window size in the configurations the deployment uses at enterprise scale.

AI Traffic Inspection: The Layer Where Prompt Content Becomes Visible to the Enterprise Stack

Cloud Radix found that 78% of employees use unauthorized AI tools at work and 86% of IT leaders are completely blind to those interactions. The blindness is not a tooling gap at the network layer. The enterprise stack already inspects traffic at the endpoint, the network egress, the cloud workload edge, and the email gateway. The blindness is at the prompt layer: the JSON request body that carries the actual content into the model. AI traffic inspection is the architectural layer that reads the prompt and the response as first-class data fields, where every other layer reads opaque bytes. I want to walk through what the inspection point reads, where each existing layer is blind, and what the 2026 compliance regimes expect from the inspection layer.

What each layer of the existing stack sees on AI traffic

Network telemetry sees the IP-layer fields, the TLS handshake metadata, the SNI hostname, and the byte count of the encrypted body. Network telemetry catalogs the existence of a session to api.openai.com and the bytes that moved. The prompt content is inside the TLS-encrypted body, and the byte count does not reveal what was in it.

The CASB sees the SaaS application category. The CASB catalogs that the user accessed chatgpt.com and that the session originated from a corporate endpoint. The CASB does not read the prompt body because the CASB operates at the application-traffic catalog layer.

The endpoint DLP sees file system events, USB writes, clipboard copies into authorized destinations, and uploads from the local file system. An employee pasting 800 lines of source code into a browser tab logs as a clipboard event without context. The endpoint cannot parse the JSON request body the browser composes when it sends the data to the LLM provider.

The email gateway sees outbound message bodies and attachments. AI traffic does not go through email. The email gateway is not in the path.

The result is that the existing stack catalogs the existence of AI traffic and the metadata around it. The content inside the traffic, which is the actual data that left the boundary, is invisible.

What the AI inspection point reads

The inspection point sits at the AI request boundary. The point terminates the outbound TLS session to the LLM provider, reads the JSON request body as plaintext, parses the prompt content out of the provider-specific shape, and applies classifiers that label segments by data classification.

The prompt shape varies by provider. For OpenAI-compatible endpoints the prompt sits in a messages array. For the Anthropic API the prompt sits in an input field. For Bedrock invoke calls the shape depends on the model the call targets. The inspection point handles each shape and normalizes the prompt content into a single internal representation the classifiers operate on.

The classifier pipeline runs pattern matching for known shapes (SSN, NPI, MRN, account number formats), named-entity recognition for person and address content, domain-specific classifiers (medical codes, financial instruments, ticker symbols, legal case citations), and policy-defined regular expressions for organization-specific data classes. The output is a labeled segmentation of the prompt: each span carries one or more data classifications and a confidence score.

The same operations run on the model response on the return path. The response carries the same classification pipeline, the same identity-aware policy, and the same audit record.

Identity binding at the inspection point

The point reads identity context from the token the caller supplies. The SSO assertion from the corporate IdP binds the request to a verified user. The OIDC bearer token binds an off-network user session. The workload identity certificate binds a corporate-managed service. The agent identity claim binds an autonomous agent runtime.

The verified identity attaches to the policy decision and the audit record. Two callers with different roles see different outcomes for the same prompt against the same model, because the policy authorizes them for different data classifications. A support-tier-1 caller cannot send PHI to the model; a clinical-records auditor can.

The identity binding is the property the existing stack lacks at the prompt layer. Network telemetry binds at the IP layer. CASB binds at the SaaS access layer. Endpoint DLP binds at the OS user layer. None of them carry the verified identity into the policy decision against the prompt content. The inspection point does.

What the inspection point reveals about the deployment

The first thing the inspection point reveals is the actual AI surface. Before the inspection point lands, the enterprise typically knows about the LLM use cases the IT and security teams sanctioned and a fraction of the shadow AI surface the discovery layer surfaced. After the inspection point lands, every AI request that the corporate environment routes is visible.

The IBM Cost of Data Breach Report 2026 found that one in five breached organizations experienced breaches linked to shadow AI. The inspection point reveals the shadow AI traffic structurally because it sees every request. The 247-day detection window IBM reported for shadow AI breaches reflects the inverse: the deployment without the inspection point sees the breach only when the data shows up outside the boundary, months later.

The second thing the inspection point reveals is the data classification distribution. The classifier output across a population of requests produces a histogram of classifications: what fraction of prompts contain PHI, PII, source code, MNPI, PCI. The histogram drives policy iteration. A policy that is too strict produces high false-deny rates the team reviews and tunes. A policy that is too permissive produces classifications the team needs to evaluate against the regulatory exposure.

The third thing the inspection point reveals is the identity-by-classification matrix. The matrix shows which identities are sending which classifications to which models. The matrix surfaces high-risk patterns the existing telemetry could not see: a single user sending PHI to public LLMs across many sessions, a service account routing source code through a model that the deployment policy did not authorize for code review, an agent runtime calling models outside the approved model class for its role.

How the inspection layer composes with the existing controls

The inspection layer does not replace the existing telemetry, CASB, endpoint DLP, or email gateway. The four existing layers continue to inspect the channels they were designed for. The inspection layer adds a fifth layer specialized for the AI request boundary.

The composition produces cross-layer evidence. A file movement event the endpoint DLP caught reconciles against a prompt event the AI inspection point caught for the same user. A session the CASB cataloged reconciles against the prompt activity the inspection point observed during the session. A network telemetry record of a session to api.openai.com reconciles against the per-decision audit records the inspection point wrote for that session.

The reconciliation produces the cross-layer evidence the compliance reviewer expects. The EU AI Act Article 12 obligation, the NIST AI RMF Manage function, and the ISO 42001 clause 8.3 expectation are satisfied by the inspection point at the AI request boundary alongside the existing layers at the other boundaries.

The audit record the inspection point writes

Every decision produces a per-decision audit record. The record carries the verified identity, the role and authorization context, the data classifications detected, the policy version in effect, the rules that matched, the decision outcome (permit, redact, deny), the timestamp with sufficient precision for cross-system correlation, and a tamper-evident signature.

The record commits to a write path the application has no access to. The application that made the call cannot suppress the record by crashing. The application cannot rewrite the record because it has no write access. The application cannot selectively log because the inspection point logs every decision.

The audit independence property is what distinguishes the inspection point record from an application-internal log. EU AI Act Article 12 expects the log to be admissible as evidence in regulatory review. An application-internal log is a self-attestation artifact. The inspection point record is the system of record at the AI request layer.

What the 2026 compliance set expects

EU AI Act Article 12 requires automatic logging over the system lifetime. Article 19 specifies the content (timestamps, input data, identification of natural persons) and retention floor (six months). The August 2, 2026 deadline applies. The inspection point produces the records the obligation expects.

NIST AI RMF Govern function expects policy. Map function expects context understanding. Measure function expects measurable evidence. Manage function expects incident response evidence. The inspection point feeds each function: the policy enforcement, the use case inventory, the per-decision metrics, and the incident response records all flow from the records the inspection point writes.

ISO 42001 clauses 8.2 and 8.3 expect operational controls and the evidence they produce. The inspection point is the operational control at the AI request boundary.

Fannie Mae LL-2026-04, effective August 6, 2026 per the Cooley legal analysis, expects lenders to produce evidence of how AI tools handled specific decisions. Texas TRAIGA, effective January 1, 2026, expects operators to maintain records of AI system operation. California AI Transparency Act, effective the same day, expects disclosure on AI use to specified user populations. Each is supported by the records the inspection point writes.

DeepInspect

This is exactly what DeepInspect provides at the AI request boundary. DeepInspect is the inspection point. The proxy terminates the outbound TLS session for traffic to known LLM provider endpoints, reads the JSON request body, runs the classification and identity-aware policy decision, writes the per-decision audit record, and forwards the call to the upstream provider.

The classifier covers PHI under HIPAA, PII under GDPR Article 4, MNPI under SEC and FINRA, PCI under PCI DSS, source code, and organization-defined data classes. The identity binding uses the SSO assertion, the OIDC bearer, the workload identity certificate, or the agent identity claim the caller supplies. The audit record commits to a write path the application has no access to and carries a tamper-evident signature.

Enforcement overhead runs under 50 milliseconds end-to-end in internal DeepInspect testing, against LLM inference latency of 500 milliseconds to 5 seconds. The inspection point operates alongside the existing endpoint DLP, network telemetry, CASB, and email gateway investment, and reconciles against the same enterprise identity directory and audit store.

If your existing stack catalogs AI traffic at the network and SaaS layer and has no inspection point at the prompt layer, book a demo today.