Does an internal audit by the security team count as the audit framework, or does it have to be external?

Both produce findings, but external audits carry the certifications customers ask for and the regulatory weight regulators expect. An internal audit using the same three-layer framework is the right preparation step. The framework is the same regardless of who is conducting the audit. External auditors typically test more deeply on layer 3 (the per-request evidence) because that is where most internal programs have not yet built the muscle.

Can a SIEM-based AI monitoring program substitute for an enforcement-layer audit record?

A SIEM that ingests AI request metadata can produce a partial record useful for forensic analysis. It typically does not satisfy properties 4 and 5 because the SIEM ingests after the fact and the application has had custody of the data along the way. The SIEM is a useful layer alongside the enforcement audit record. It is not a substitute for the per-decision record produced at the request boundary.

How does the framework apply to internal AI models the organization hosts itself?

The framework applies the same way. The architectural principle is that the audit record has to be independent of the application that uses the model. For internally hosted models, the enforcement layer sits between the application and the model API endpoint. The same six properties apply. The hosting choice does not change the audit requirements.

How long does the implementation typically take?

The two longest poles are the IdP integration (which is the property 1 prerequisite) and the data classification rule set (which is the property 2 prerequisite). Organizations that already have a working IdP can typically stand up the enforcement layer in weeks rather than months for an initial scope. Expanding coverage to all use cases (property 6) is an ongoing program, not a single project. The audit posture improves measurably as soon as the per-request evidence layer is in operation for the highest-risk use cases.

What does the audit framework not cover that we should still worry about?

The framework focuses on the control layer for AI usage. It does not cover model bias and fairness testing, model performance monitoring against drift, the procurement-side AI vendor risk assessment, or the broader AI ethics committee artifacts that some organizations produce. Those are adjacent programs that share inputs with the governance audit but produce their own evidence. A complete AI governance program addresses all of them. The framework here is the part that the per-request enforcement layer specifically operates.

AI Governance Audit Framework: What Auditors Actually Test

The AI governance audit is becoming a standard line item in SOC 2 Type II reviews, ISO 42001 certifications, and the readiness checks customers run against B2B SaaS vendors that process their data through AI. The framework auditors use to test the program covers three layers: the policy artifacts the organization has produced, the operation of the controls those policies describe, and the per-request evidence that the controls actually fired.

The programs that pass share six structural properties. The programs that fail typically fail at the evidence layer because the audit record does not exist or is under the same control as the application generating the request. That is the self-attestation problem applied to AI governance.

I want to walk through the framework, the six properties of programs that pass, and the architecture the framework depends on.

The three-layer audit

Layer 1: Policy artifacts

The auditor reads the policy documents. An AI usage policy, a model risk management policy if the organization runs internal models, a data classification policy that names the classes the AI program enforces against, a vendor management policy that covers AI vendors specifically, and an incident response policy that covers AI-related incidents.

The auditor checks for currency (was the policy reviewed in the audit period), specificity (does it name actual data classes and tools), and reach (does it cover the use cases the organization actually has). A policy that describes a generic acceptable-use position without naming the specific data classes the regime cares about fails the specificity test.

Layer 2: Control operation

The auditor sends a control test. The test asks: when an employee submits a request through Tool X containing data class Y, what happens? The expected answer is an evaluation, a decision (allow, redact, block), and an audit record. The auditor traces the test request through the enforcement layer and confirms each step.

Programs that have policies but not enforcement layers fail this test. The control on paper does not exist in operation. The test is one of the highest-yield audit findings auditors produce because it cuts past the documentation and looks at what actually happens.

Layer 3: Per-request evidence

The auditor samples production requests from the audit period. For each sampled request, the auditor expects to see an audit record containing the user identity, the timestamp, the tool, the data classification applied, the policy version in effect at that moment, and the decision outcome. The record has to be tamper-evident and under the deployer's control rather than the application's.

This is where most programs fail. The application that generated the AI request also wrote the application log. The application log is not an independent audit record. Auditors familiar with the financial services audit pattern recognize the issue immediately: the system under audit cannot be the system writing the audit record.

The six properties of programs that pass

Property 1: Identity binding at the request layer

The audit record names a verified user identity from the corporate IdP for every AI request. Static service credentials shared across applications fail this property. Personal accounts on otherwise-approved tools fail this property. The identity has to be present in the record, not inferred after the fact.

Property 2: Prompt-level data classification

The data classes the policy names are detected and classified in the prompt content. The classification is deterministic and runs in line with the request. The auditor can see, in the sampled records, the classification result that drove the decision.

Property 3: Policy version capture

The policy version in effect at the moment of the request is captured in the audit record. Programs evolve. A control test six months after a request was made needs to reference the policy that was in effect when the request happened, not the current policy.

Property 4: Tamper-evident records

The audit record is signed, hashed, or otherwise tamper-evident. The application that made the request cannot modify the record. The retention infrastructure produces evidence that the records have not been altered. Standard application logs do not satisfy this property because the application that owns the log can rotate, truncate, or modify it.

Property 5: Independent write path

The audit record is committed before the model response returns to the application. The application never has custody of the write path. This is the architectural property that solves the self-attestation problem at the AI request layer.

Property 6: Coverage of the actual surface

The per-request evidence covers all AI requests the organization is exposed to: sanctioned tools, vendor SaaS with embedded AI, internal models, and the shadow traffic the enforcement layer can intercept. Coverage gaps surface as the auditor walks through the use case inventory and finds requests the control did not see.

How the framework maps to regulatory regimes

The EU AI Act Article 12 obligation is the closest match to the audit framework. The requirement for "automatic recording of events (logs) over the lifetime of the system" maps to properties 3, 4, and 5. The Article 19 specification of what goes in the log maps to properties 1 and 2. The retention floor of six months sets the minimum on property 6 with respect to retention.

NIST AI RMF GOVERN and MEASURE functions describe the artifacts and the measurement controls. The MEASURE function specifically calls for monitoring of operation, which is the per-request evidence layer.

ISO 42001 requires an AI management system with documented controls. The certification audit tests the operation of the controls. The same three-layer pattern applies.

SOC 2 Type II testing of AI controls applies the standard SOC 2 framework to the AI usage policy, with the operation tested across the audit period. The auditor samples requests across the period rather than at a point in time.

Fannie Mae LL-2026-04, effective August 6, 2026 for mortgage lenders, requires disclosure on demand of AI tools, providers, and safeguards. The infrastructure that satisfies the framework satisfies the LL-2026-04 disclosure obligation.

Why programs fail

The dominant failure mode is the absence of property 5 (independent write path). The organization has a policy, has a tool the security team considers sanctioned, has application logs that name the tool and the timestamp. The auditor samples a request, asks for the record, and gets the application log. The application is the system that made the AI decision. The application is also the system that wrote the log. The auditor's finding is that the audit record is self-attested and does not provide independent evidence.

The fix is not procedural. The fix is architectural. The audit record has to live on a write path the application does not control.

The second failure mode is policy without coverage (property 6 gap). The organization has a policy that names sanctioned tools and data classes. The auditor's sample includes a request through a vendor SaaS tool that added an AI feature in a recent release. The enforcement layer did not see the request because the new feature was not on the inspected surface. The finding is that the program does not cover the actual surface.

The third failure mode is identity inference (property 1 gap). The audit record names a user, but on inspection the user was inferred from a session cookie or a stable hash rather than identified from the corporate IdP. The audit defense crumbles when the identification chain is examined.

DeepInspect

This is the architecture the audit framework depends on. DeepInspect sits at the AI request boundary as a stateless proxy between users and agents and any LLM. The corporate IdP identifies the user. The prompt is classified against the policy's data classes. The decision and the record are generated by the proxy, not the application. The record is signed and committed before the model response returns. The application does not have custody of the write path.

The six properties the audit framework tests against come out of the architecture by design. Identity binding from the IdP integration. Prompt-level classification in the proxy's detection layer. Policy version capture in the record. Tamper-evident records through the signing infrastructure. Independent write path through the proxy. Coverage through the proxy's interception of all configured endpoints.

For the SOC 2 Type II audit and the ISO 42001 certification, the proxy is the operating control the auditor samples against. For EU AI Act Article 12 readiness ahead of the August 2 deadline, the proxy is the automatic recording infrastructure the regulation requires. For Fannie Mae LL-2026-04 disclosure, the proxy is the disclosure-on-demand evidence.

If your AI governance program is audit-aware and the per-request evidence layer is not in place, the August deadline narrows the runway. If you are facing the August deadline, let's talk.