AI Compliance Audit Checklist: The Evidence a Reviewer Will Actually Request
An AI compliance audit fails on the evidence you cannot produce, not the policy documents you can. This checklist walks through what a reviewer requests for a high-risk AI system: the inventory, the identity mapping, the per-decision records, the retention proof, and the vendor-AI coverage, with the EU AI Act and Fannie Mae as the reference deadlines.
An AI compliance audit fails when a reviewer asks for the record behind a specific AI decision and receives a JSON log the application wrote about itself. The gap sits between a governance policy and evidence that can withstand questioning. This AI compliance audit checklist is organized around the evidence a reviewer requests, including the deletion proof that shows regulated prompt data actually left the systems that held it.
I want to walk through each item, what the reviewer is testing, and the artifact that satisfies it.
TL;DR
An AI audit needs a current system inventory, verified requester identity, a per-decision evidence record, tested retention and deletion evidence, and coverage for vendor-operated AI. Start with one request and trace its data, policy decision, record location, retention setting, and deletion path. A policy binder cannot substitute for that trace.
Item 1: A current inventory of every AI system in scope
The first request is always the inventory. The reviewer wants a list of AI systems, their use cases, their risk classification, the models behind them, and the data each one touches. Under the EU AI Act, classification turns on the use case, and high-risk categories in Annex III include credit scoring, employment screening, and biometric identification.
The item that trips deployments is completeness. Shadow AI keeps the inventory out of date. An inventory built only by survey misses the usage that creates the most exposure. The defensible version combines owner attestation, procurement and OAuth records, cloud discovery, and observed routed model traffic, so it can be tested against what actually calls models.
Item 2: Identity mapping from every AI request to a real person or agent
The reviewer next asks who initiated a given AI decision. For high-risk systems, the EU AI Act's Article 12 record-keeping requirement makes automatic event logs part of the evidence needed to understand operation over time. This is difficult when AI systems call model APIs on static service credentials that identify an application rather than the human or agent behind the request.
The checklist test is concrete: pick a request from last month and name the person. If the answer resolves to a shared key, the identity mapping is missing. Satisfying it requires identity context attached at the request layer, following AI agent identity practice, so every AI call carries a verified principal. Without that mapping, the audit trail, the access-control evidence, and the disclosure records all inherit the same gap at once.
Item 3: Per-decision records with policy state, not application logs
Here the reviewer asks for the record behind a specific decision, and specifies what it must contain: the identity, the data classification, the policy in effect at the moment, and the outcome. Standard application logs fail this because they record the application's own view, not the policy state that governed the decision.
The distinction the checklist enforces is independence. A record the audited system can modify fails the traceability test, following the reasoning in audit-log immutability. The artifact that passes is a per-decision record, signed and tamper-evident, committed before the model response returns to the application. Test it by asking whether the application that made the decision could have altered or suppressed the record. If it could, the record is a convenience log wearing an audit label.
Item 4: Retention proof and data classification evidence
The reviewer asks how long records are kept and how prompt data is classified. For deployers of high-risk systems, EU AI Act Article 26(6) sets an at-least-six-month log-retention floor, subject to longer applicable law. Providers have their own automatic-log obligation in Article 19. The checklist wants proof of the retention mechanism, not a stated policy.
Classification is the paired item. The reviewer wants evidence that the system evaluates data submitted in each request, at request granularity rather than document granularity, since prompt-level classification determines whether regulated content was exposed. The artifact is a per-request classification result stored alongside the decision record, so retention and classification are answered from the same place.
Item 5: Deletion proof across the AI data path
The query that matters in a compliance review is precise: prove that AI data was deleted. A deletion request requires an evidence chain, not a support-ticket closure. Identify the prompt or record, the application store, the model provider or processor account, retrieval indexes, caches, exported audit packages, backup schedule, and the contract term that governs each location. GDPR Article 17 supplies the erasure right and its conditions; it does not turn a provider's generic retention page into proof for one data subject.
Run a deletion test against a non-production marker, then preserve the request ID, the identity that authorized it, processor confirmation, the object or record identifiers, timestamped deletion logs, and the date by which immutable backups age out. If cryptographic erasure is used, retain evidence that the relevant key was destroyed and that no alternate key can decrypt the data. The review package should name any legal hold or statutory-retention exception instead of calling the record deleted while a retained copy remains accessible.
Item 6: Coverage of vendor and embedded AI
The final request is the one that surprises teams: show the records for AI your vendors run on your behalf. The Fannie Mae Lender Letter LL-2026-04 holds the deploying organization liable for AI mistakes by subcontractors and vendors, and the disclosure obligation does not transfer to the vendor.
This is where due diligence and due care diverge, a distinction I drew in due diligence is not due care. A SOC 2 report on a vendor is a one-time procurement check. It does not evidence what that vendor's AI did with your data last Tuesday. The checklist item is satisfied when vendor-AI traffic that flows through your environment is subject to the same identity, classification, and recording as your own systems, and when contracts require vendor-side records for the traffic that does not.
DeepInspect
This is the infrastructure DeepInspect provides for the checklist. DeepInspect is a stateless proxy between your authenticated users and agents and any HTTP-based LLM endpoint. It derives an inventory from observed AI traffic, attaches identity to every request, classifies prompt data inline, and enforces per-route and per-role policy before the request reaches the model. Every decision produces a signed, per-decision audit record with identity, classification, policy version, and outcome, committed before the response returns.
Because the record is independent of the application and covers vendor traffic that transits your environment, the request-path portions of the checklist are answered from one control point. Provider deletion evidence, tool authorization, and local-system records retain their own control owners.
Let's talk today.
Frequently asked questions
- What is the difference between an AI compliance audit and a security audit?
A security audit tests whether controls prevent unauthorized access and detect attacks. An AI compliance audit tests whether you can evidence how AI systems made decisions, to the standard a specific regulation sets. The two overlap in the controls they examine, but the compliance audit turns on producible records rather than posture. A reviewer wants to see the per-decision trail, the identity mapping, the retention proof, and the vendor coverage for AI use, and to confirm those artifacts are independent of the systems that generated them. You can pass a security audit and fail a compliance audit if your controls work but your evidence is self-attested or incomplete, which is a common outcome for AI deployments built without an audit layer.
- What evidence do AI auditors ask for first?
The inventory, then identity. Auditors start by asking for a complete list of AI systems in scope, their risk classification, the models behind them, and the data they touch. Then they pick a specific decision and ask who initiated it, which tests whether your logs map AI requests to real people or agents rather than shared service credentials. These two requests expose the most common gaps quickly: an inventory built from surveys misses shadow AI, and identity mapping fails when systems call models on static keys. If those two items are weak, the rest of the audit inherits the same weakness, because the per-decision records, access-control evidence, and disclosure trails all depend on knowing what systems exist and who is behind each request.
- How do I prove AI audit log retention to a regulator?
Show the retention mechanism, not the policy statement. A regulator wants evidence that records are actually kept for the required period and cannot be silently truncated, which means demonstrating where the records live, how long they persist, and what prevents modification. EU AI Act Article 26(6) sets at least six months for deployers of high-risk systems, and financial or healthcare rules can extend that period. The artifact that satisfies this is a store of signed, tamper-evident, per-decision records with a demonstrable retention configuration, ideally independent of the application that produced the decisions, so a reviewer can confirm the records exist and have not been altered rather than taking a policy document on faith.
- Does an AI compliance audit cover vendor AI tools?
Yes, and this is the item teams most often miss. Frameworks like Fannie Mae LL-2026-04 hold the deploying organization liable for AI mistakes made by subcontractors and vendors, and the disclosure obligation stays with the deployer. A SOC 2 report on the vendor is due diligence at procurement; it does not evidence what the vendor's AI did with your data on a given day. The audit expects coverage: vendor-AI traffic that flows through your environment should be subject to the same identity, classification, and recording as your own systems, and contracts should require vendor-side records for traffic that does not transit your environment. Without that, the vendor-AI portion of the audit is a gap you own.
- How often should an AI compliance audit run?
Governance frameworks generally expect at least an annual formal audit, but the useful cadence is continuous evidence with periodic formal review. Regulations change, AI inventories drift as teams adopt new tools, and shadow AI expands the scope between reviews, so an annual snapshot goes stale quickly. The stronger posture is an audit layer that produces per-decision evidence continuously, so a formal audit becomes a query against records that already exist rather than a scramble to reconstruct them. Regulatory references should be re-verified annually because effective dates and requirements shift, and the inventory should be refreshed from observed traffic on a rolling basis rather than rebuilt from memory at audit time.