AI Governance Best Practices: Six Controls That Survive an Audit
Most AI governance best-practice lists stop at committees and policy documents. A reviewer tests the controls underneath them. This covers six AI governance practices that produce evidence rather than intent: an inventory derived from traffic, identity on every request, inline policy, independent audit records, vendor-AI coverage, and annual regulatory re-verification.
Most AI governance best-practice lists describe intent: form a committee, write an acceptable-use policy, run a risk assessment, train the staff. Those steps organize a program, and a reviewer does not test them directly. A reviewer tests the controls underneath the intent by asking for the record behind a specific AI decision. AI governance best practices that survive that moment are the ones that produce evidence as a byproduct of operating, rather than documents that describe what should happen. The gap between the two is where programs with thick policy binders still fail an audit.
I want to walk through six practices, each defined by the evidence it produces rather than the document it generates.
Practice 1: Derive the AI inventory from observed traffic
A governance program starts with an inventory of AI systems, and the standard practice is to build it by survey. That version is stale on arrival, because the highest-risk usage is the usage nobody declares. Cloud Radix found 78% of employees use unauthorized AI tools and 86% of IT leaders are blind to those interactions.
The practice that holds is to derive the inventory from AI traffic itself, so the list reflects what is actually calling models rather than what teams remember to report. Observed traffic surfaces the shadow AI a survey misses and keeps the inventory current as usage shifts. This is the foundation every other control rests on, because a governance program cannot enforce policy on or produce evidence for a system it does not know exists. An AI governance framework built on a declared inventory governs only the honest half of the deployment.
Practice 2: Attach identity to every AI request
The single most consequential technical practice is attaching a verified identity to every AI request. Article 19 of the EU AI Act requires logs to identify the natural persons involved, and most deployments cannot, because they call model APIs on static service credentials that name the application rather than the human or agent behind the request.
Without request-level identity, the entire governance program inherits one gap. The audit trail cannot name who initiated a decision. The access-control evidence cannot show least privilege. The disclosure record cannot answer who exposed which data. Fixing it means carrying identity context on every call, following AI agent identity practice, so the person or agent is resolvable at the moment of the request. This is Pillar 1 of the NIST agent identity framework, and it is the practice the other controls depend on most directly.
Practice 3: Enforce policy inline, not in application code
Governance policy that lives in application code drifts. Each team re-implements it against a different provider SDK, the versions diverge, and no one can state with confidence what policy is actually in force across the deployment. The practice that scales is to enforce policy at a single inline point on the AI request path, so one policy set applies uniformly to every model and route.
Inline placement also determines whether the control can prevent anything. A policy evaluated beside the request path, on a copy of the traffic, produces an alert after the request reached the model. A policy evaluated inline returns a permit, redact, or deny decision before the model sees the prompt, following inline enforcement practice. Mandiant measured the median attack handoff at 22 seconds, faster than any human response to an alert, which is why the enforcement decision has to be synchronous and in-path.
Practice 4: Generate audit records independent of the application
The audit record is the artifact a review turns on, and the practice that makes it defensible is independence. When the application that makes an AI decision also writes the record of that decision, the record is self-attested, and it carries three failure modes: selective logging, suppression, and loss when the application crashes before the log commits.
The practice is to generate the record at a decoupled point on the request path and commit it before the model response returns, following audit-log immutability practice. The record needs a verified identity, the data classification, the policy version, the outcome, a timestamp, and an integrity mechanism. Committed this way, it exists whether or not the application survives the transaction, and the application cannot alter it afterward. A record with those properties is a system of record. A record the audited system controls is a convenience artifact a reviewer discounts.
Practice 5: Cover vendor and embedded AI on the same terms
Governance programs routinely stop at their own systems and leave vendor AI ungoverned, which is where the largest liability sits. The Fannie Mae Lender Letter LL-2026-04 holds the deploying organization liable for AI mistakes by subcontractors and vendors, and the disclosure obligation does not transfer.
The practice is to treat vendor-AI traffic that flows through your environment with the same identity, classification, and recording as your own systems, and to require vendor-side records by contract for the traffic that does not transit your environment. This is the difference between due diligence and due care, which I drew out in due diligence is not due care. A SOC 2 report obtained at procurement is a one-time check. Ongoing supervision of what a vendor's AI does with your data is a continuous obligation the report does not satisfy.
Practice 6: Re-verify regulatory references annually
Regulations change, and stale references damage credibility faster than any other inaccuracy. The practice is to re-verify every regulatory fact your program relies on, effective dates, article numbers, penalty amounts, against the primary source at least annually, not against a secondary aggregator.
The 2026 calendar shows why. EU AI Act high-risk obligations take effect August 2, 2026, with penalties reaching €15 million or 3% of turnover under Article 99. Fannie Mae LL-2026-04 and Freddie Mac Section 1302.8 carry their own dates. State laws including the Texas Responsible AI Governance Act took effect January 1, 2026. A governance program that cites a superseded date in a board paper or an audit response undercuts the credibility of the controls it is describing. Annual re-verification, tied to a named owner, keeps the program's factual foundation current.
DeepInspect
Four of these six practices are technical controls on AI traffic, and DeepInspect implements them from one place. DeepInspect is a stateless proxy between your authenticated users and agents and any HTTP-based LLM endpoint. It derives an inventory from observed traffic, attaches identity to every request, enforces per-route and per-role policy inline, and generates a signed, per-decision audit record committed before the response returns and independent of the application. Vendor-AI traffic that transits the proxy is governed on the same terms as first-party traffic.
The committee, the policy documents, and the annual re-verification remain human practices. The controls a reviewer actually tests are the ones DeepInspect produces evidence for.
If you are turning AI governance intent into controls that survive an audit, let's talk today.
Frequently asked questions
- What are the most important AI governance best practices?
The practices that produce evidence rather than intent. A governance program benefits from committees, policies, and training, but a reviewer tests the controls underneath them by asking for the record behind a specific AI decision. The highest-value practices are deriving the AI inventory from observed traffic so it includes shadow AI, attaching a verified identity to every AI request, enforcing policy inline so it can prevent rather than only alert, generating audit records independent of the application, covering vendor and embedded AI on the same terms, and re-verifying regulatory references annually. These share a property: each produces a defensible artifact as a byproduct of operating. That is what distinguishes a governance program that survives an audit from one that documents what should happen.
- How is AI governance different from traditional IT governance?
Traditional IT governance manages systems, access, and change through relatively stable interfaces. AI governance has to account for a data channel that existing tools do not see and a decision layer that is probabilistic. Prompt content travels inside encrypted sessions to provider APIs, so network monitoring and legacy DLP miss it. Model behavior is trained rather than configured, so it cannot be treated as an enforceable control. And AI usage spreads through shadow adoption faster than software procurement did. These differences mean AI governance leans harder on request-level identity, prompt-level classification, and inline enforcement than IT governance historically needed, and it treats the audit record as a first-class artifact rather than a byproduct of application logging. The governance principles carry over; the control points move to the AI request boundary.
- What is the first step in building an AI governance program?
Build an accurate inventory, and build it from observed traffic rather than a survey. Every subsequent control depends on knowing which AI systems exist, what they do, and what data they touch, and a survey-based inventory is stale immediately because the highest-risk usage is undeclared. Deriving the inventory from AI traffic surfaces shadow AI and keeps the list current. With an accurate inventory, you can classify systems by risk, attach identity to their requests, apply policy, and generate records. Starting with policy documents before the inventory is accurate produces governance over the systems you already knew about and none over the ones creating the most exposure, which is the inverse of what a risk-based program should prioritize.
- How do AI governance best practices map to the NIST AI RMF?
The NIST AI Risk Management Framework organizes governance into GOVERN, MAP, MEASURE, and MANAGE functions, and the practices here slot into them. Deriving an inventory from traffic and classifying systems supports MAP. Attaching identity, enforcing policy inline, and generating independent records support MEASURE and MANAGE, because they produce the operational evidence and the runtime controls those functions call for. The GOVERN function covers the committee, the policies, and the accountability structure. NIST's separate agent identity and authorization framework maps even more directly: request-level identity is Pillar 1, per-request authorization is Pillar 2, and the independent audit record is Pillar 3, action lineage. Mapping your practices to these functions gives a board and an auditor a recognized structure to evaluate the program against.
- Do AI governance best practices apply to vendor AI tools?
Yes, and omitting vendor AI is a common and costly gap. Frameworks like Fannie Mae LL-2026-04 hold the deploying organization liable for AI mistakes made by subcontractors and vendors, and the disclosure obligation stays with the deployer. Good practice treats vendor-AI traffic that flows through your environment with the same identity, classification, and recording as first-party systems, and requires vendor-side records by contract for traffic that does not transit your environment. A SOC 2 report obtained at procurement is due diligence, a one-time check, while supervising what a vendor's AI does with your data over time is due care, a continuous obligation. Governance that stops at your own systems governs the smaller share of the risk and leaves the larger, vendor-driven share unaddressed.