AI Compliance Audit Checklist: The Evidence a Reviewer Will Actually Request
An AI compliance audit fails on the evidence you cannot produce, not the policy documents you can. This checklist walks through what a reviewer requests for a high-risk AI system: the inventory, the identity mapping, the per-decision records, the retention proof, and the vendor-AI coverage, with the EU AI Act and Fannie Mae as the reference deadlines.
An AI compliance audit does not fail on the policy binder. It fails at the moment a reviewer asks you to produce the record behind a specific AI decision and the answer is a JSON log the application wrote about itself. The gap between having a governance policy and being able to evidence it under questioning is where most organizations are exposed. This AI compliance audit checklist is organized around the evidence a reviewer requests, not the documents you already have, because the requests are what a real review turns on.
I want to walk through each item, what the reviewer is testing, and the artifact that satisfies it.
Item 1: A current inventory of every AI system in scope
The first request is always the inventory. The reviewer wants a list of AI systems, their use cases, their risk classification, the models behind them, and the data each one touches. Under the EU AI Act, classification turns on the use case, and high-risk categories in Annex III include credit scoring, employment screening, and biometric identification.
The item that trips deployments is completeness. Shadow AI keeps the inventory perpetually out of date. Cloud Radix reported that 78% of employees use unauthorized AI tools at work and 86% of IT leaders are blind to those interactions. An inventory built by survey misses exactly the usage that creates the most exposure. The defensible version is derived from observed traffic, so the inventory reflects what is actually calling models rather than what teams remember to declare.
Item 2: Identity mapping from every AI request to a real person or agent
The reviewer next asks who initiated a given AI decision. Article 19 of the EU AI Act requires logs to identify the natural persons involved. This is the item most deployments cannot satisfy, because AI systems commonly call model APIs on static service credentials that identify the application rather than the human or agent behind the request.
The checklist test is concrete: pick a request from last month and name the person. If the answer resolves to a shared key, the identity mapping is missing. Satisfying it requires identity context attached at the request layer, following AI agent identity practice, so every AI call carries a verified principal. Without that mapping, the audit trail, the access-control evidence, and the disclosure records all inherit the same gap at once.
Item 3: Per-decision records with policy state, not application logs
Here the reviewer asks for the record behind a specific decision, and specifies what it must contain: the identity, the data classification, the policy in effect at the moment, and the outcome. Standard application logs fail this because they record the application's own view, not the policy state that governed the decision.
The distinction the checklist enforces is independence. A record the audited system can modify fails the traceability test, following the reasoning in audit-log immutability. The artifact that passes is a per-decision record, signed and tamper-evident, committed before the model response returns to the application. Test it by asking whether the application that made the decision could have altered or suppressed the record. If it could, the record is a convenience log wearing an audit label.
Item 4: Retention proof and data classification evidence
The reviewer asks how long records are kept and how prompt data is classified. Retention has a floor of at least six months under Article 19, extended by financial and healthcare record-keeping obligations to years in most regulated settings. The checklist wants proof of the retention mechanism, not a stated policy.
Classification is the paired item. The reviewer wants to see that the system evaluates what data entered a prompt, at the prompt level rather than the document level, since prompt-level classification is what determines whether a request exposed regulated content. Netwrix found only 37% of organizations have any AI-related governance policy in place, which means most cannot produce classification evidence at all. The artifact is a per-request classification result stored alongside the decision record, so retention and classification are answered from the same place.
Item 5: Coverage of vendor and embedded AI
The final request is the one that surprises teams: show the records for AI your vendors run on your behalf. The Fannie Mae Lender Letter LL-2026-04 holds the deploying organization liable for AI mistakes by subcontractors and vendors, and the disclosure obligation does not transfer to the vendor.
This is where due diligence and due care diverge, a distinction I drew in due diligence is not due care. A SOC 2 report on a vendor is a one-time procurement check. It does not evidence what that vendor's AI did with your data last Tuesday. The checklist item is satisfied when vendor-AI traffic that flows through your environment is subject to the same identity, classification, and recording as your own systems, and when contracts require vendor-side records for the traffic that does not.
DeepInspect
This is the infrastructure DeepInspect provides for the checklist. DeepInspect is a stateless proxy between your authenticated users and agents and any HTTP-based LLM endpoint. It derives an inventory from observed AI traffic, attaches identity to every request, classifies prompt data inline, and enforces per-route and per-role policy before the request reaches the model. Every decision produces a signed, per-decision audit record with identity, classification, policy version, and outcome, committed before the response returns.
Because the record is independent of the application and covers vendor traffic that transits your environment, the five checklist items are answered from one control point rather than five separate projects.
If you are preparing for an AI compliance audit against a 2026 deadline, let's talk today.
Frequently asked questions
- What is the difference between an AI compliance audit and a security audit?
A security audit tests whether controls prevent unauthorized access and detect attacks. An AI compliance audit tests whether you can evidence how AI systems made decisions, to the standard a specific regulation sets. The two overlap in the controls they examine, but the compliance audit turns on producible records rather than posture. A reviewer wants to see the per-decision trail, the identity mapping, the retention proof, and the vendor coverage for AI use, and to confirm those artifacts are independent of the systems that generated them. You can pass a security audit and fail a compliance audit if your controls work but your evidence is self-attested or incomplete, which is a common outcome for AI deployments built without an audit layer.
- What evidence do AI auditors ask for first?
The inventory, then identity. Auditors start by asking for a complete list of AI systems in scope, their risk classification, the models behind them, and the data they touch. Then they pick a specific decision and ask who initiated it, which tests whether your logs map AI requests to real people or agents rather than shared service credentials. These two requests expose the most common gaps quickly: an inventory built from surveys misses shadow AI, and identity mapping fails when systems call models on static keys. If those two items are weak, the rest of the audit inherits the same weakness, because the per-decision records, access-control evidence, and disclosure trails all depend on knowing what systems exist and who is behind each request.
- How do I prove AI audit log retention to a regulator?
Show the retention mechanism, not the policy statement. A regulator wants evidence that records are actually kept for the required period and cannot be silently truncated, which means demonstrating where the records live, how long they persist, and what prevents modification. The EU AI Act floor is at least six months, but financial and healthcare obligations commonly extend that to years, so the mechanism should support multi-year retention with integrity protection. The artifact that satisfies this is a store of signed, tamper-evident, per-decision records with a demonstrable retention configuration, ideally independent of the application that produced the decisions, so a reviewer can confirm the records exist and have not been altered rather than taking a policy document on faith.
- Does an AI compliance audit cover vendor AI tools?
Yes, and this is the item teams most often miss. Frameworks like Fannie Mae LL-2026-04 hold the deploying organization liable for AI mistakes made by subcontractors and vendors, and the disclosure obligation stays with the deployer. A SOC 2 report on the vendor is due diligence at procurement; it does not evidence what the vendor's AI did with your data on a given day. The audit expects coverage: vendor-AI traffic that flows through your environment should be subject to the same identity, classification, and recording as your own systems, and contracts should require vendor-side records for traffic that does not transit your environment. Without that, the vendor-AI portion of the audit is a gap you own.
- How often should an AI compliance audit run?
Governance frameworks generally expect at least an annual formal audit, but the useful cadence is continuous evidence with periodic formal review. Regulations change, AI inventories drift as teams adopt new tools, and shadow AI expands the scope between reviews, so an annual snapshot goes stale quickly. The stronger posture is an audit layer that produces per-decision evidence continuously, so a formal audit becomes a query against records that already exist rather than a scramble to reconstruct them. Regulatory references should be re-verified annually because effective dates and requirements shift, and the inventory should be refreshed from observed traffic on a rolling basis rather than rebuilt from memory at audit time.