AI Governance for the Public Sector Needs Program-Level Accountability

A benefits program adds an LLM to summarize case files for reviewers. The agency CIO can authorize the system, procurement can approve the contract, and the Chief AI Officer can place the use in an inventory. One decision remains at the program: who accepts the risk that a summary influences an eligibility action, and what evidence will show how the tool operated? AI governance for the public sector needs that chain to connect enterprise policy with program execution. I would reject any governance chart that ends at the agency board. Public consequences land in program offices.

TL;DR

• Use the Chief AI Officer and governance board to set agency policy, common review and inventory methods, plus escalation thresholds.
• Assign each use case to the program official accountable for its mission outcome, with independent review and signed risk acceptance for high-impact AI.
• Put testing access, data rights, monitoring and change-notice requirements, plus exit terms into AI acquisitions before award.
• Preserve request decisions as one evidence stream. Program outcomes, appeals, records, privacy and accessibility, along with security authorization, retain separate owners.

Enterprise governance sets the decision system

OMB Memorandum M-25-21, issued April 3, 2025, rescinded and replaced M-24-10. It directs each covered agency to retain or designate a Chief AI Officer. CFO Act agencies must convene an AI Governance Board chaired at the Deputy Secretary level or equivalent, with the CAIO as vice-chair and representation spanning IT, cybersecurity, data, budget, legal, privacy, civil rights, and civil liberties.

Those roles should establish one decision system. The board sets policy, shared intake fields, high-impact determination methods, criteria for independent review and controls for waivers, plus reporting. The CAIO maintains the agency inventory and centrally tracks high-impact determinations. Security and privacy officials apply their authorities. Acquisition staff own contract formation.

The existing public-sector compliance pillar covers the wider regulatory stack. This article focuses on operating architecture under the current OMB memoranda, with the program office holding the mission decision that enterprise bodies structure and oversee.

Program officials own use-case outcomes

M-25-21 tells agencies to enable trained and accountable officials at the lowest appropriate level to identify and assess AI risk, then mitigate and accept it. A footnote encourages assigning that work to officials accountable for the mission outcome. This creates a clean division of labor: the agency defines the process, while the benefits director, grants executive, inspection leader, or customer-service owner accepts use-case risk inside delegated authority.

A program record should name the mission outcome and affected service, plus the accountable official and technical owner. Add the system and model, intended users, data sources, people affected, output use, human decision point, appeal route and records schedule, plus change triggers. High-impact status and its rationale belong in the same file.

A central board approving every prompt would become a queue. Blanket delegation would hide ownership. The stronger design gives program officials bounded decision rights, sends defined cases to independent review, and lets the CAIO see inventory and exceptions across the agency.

High-impact AI needs a signed operational file

M-25-21 defines high-impact AI around outputs serving as a principal basis for decisions or actions with significant effects on rights, access to programs, health and safety, critical infrastructure, public safety, or strategic assets. The memorandum requires pre-deployment testing and an AI impact assessment for covered high-impact uses. It also requires ongoing monitoring, operator training, suitable human oversight, and a timely human review or appeal for affected individuals when appropriate.

The impact assessment includes intended purpose and expected benefit, data and model fitness, potential effects on privacy and civil rights, reassessment procedures, costs, independent-review results, and a risk-acceptance signature. That is an operating file rather than a score on a portfolio dashboard.

Picture a paper benefits notice clipped to a case folder. A reviewer should trace its program action to the approved use, assessment, model version, input source, production event and human disposition, plus the appeal. The AI governance framework supplies a lifecycle structure. The program file makes accountability concrete.

Acquisition determines the agency's future evidence

OMB Memorandum M-25-22, also issued April 3, 2025, rescinded and replaced M-24-18. It directs agencies to use cross-functional acquisition work, assess foreseeable use cases, and identify likely high-impact uses before award. For those uses, solicitations must communicate the documentation needed to support M-25-21 compliance.

Contract terms should give the agency access and time for independent evaluation. M-25-22 also addresses government data and IP rights, portability, ongoing testing and monitoring, vendor performance and feature notification, plus closeout access to data or derived assets. These terms decide what the program can inspect after deployment.

A demo against a clean sample file proves very little about production. Require tests that reflect agency networks and data conditions, then define the evidence the contractor must provide after a model or feature change. AI model inventory management can hold the route and version record joined to that contract.

State programs need the same chain under state authority

California's Executive Order N-12-23, signed September 6, 2023, directed state bodies to develop procurement and use guidelines for generative AI. It also directed them to develop training guidelines. Agencies under the governor's authority had to appoint senior management personnel and maintain an inventory of current high-risk generative AI uses. The order called for controlled pilot environments and review of effects on vulnerable communities.

That structure shows how a state can assign enterprise coordination while preserving department responsibility. A state technology office can define intake and approved environments. Procurement can set terms. The department running unemployment insurance, licensing, child welfare, or tax administration still owns the service outcome and applicable state authority.

This scope is broader than municipal AI governance, which uses a city service and record system as its operating unit. State programs need statewide policy plus department and program accountability, with federal grant or data conditions added where they apply.

Production evidence should connect controls to public actions

For customer-controlled HTTP model routes, the application can send employee or agent identity and program purpose to an external policy point. That point can classify prompt content, restrict the destination, apply a versioned rule, and record the outcome before forwarding an allowed request. The event supports monitoring and investigation for traffic on that path.

The wider program needs additional evidence. Impact assessments show design intent. Outcome monitoring tests performance and adverse effects. Case systems preserve official decisions and human review. Appeal systems hold remedies, while records officers assign retention and disclosure treatment. Privacy, civil rights, accessibility, and cybersecurity officials retain their authorities. Acquisition officials and Inspectors General retain theirs.

Continuous monitoring for federal AI addresses the security-control slice. The AI use-case file should link that evidence to program outcomes and change review. One dashboard cannot replace these records. A useful portfolio view points to named owners and source artifacts rather than compressing accountability into a green circle.

The HTTP boundary covers routed agency model calls

An HTTP policy gateway can inspect agency-controlled traffic deliberately routed between authenticated users or agents and LLM endpoints. It can enforce model-destination and content rules using the identity and context supplied upstream. It can also produce an independent record of its own decision.

Embedded AI in vendor-managed case systems may use inference paths the agency cannot redirect. Local models and offline analytics bypass the gateway. IAM establishes identity, while an authorization to operate covers the information system under its assigned process. Model evaluation and public consultation sit elsewhere. Appeals sit elsewhere, as do official program actions and contract oversight.

Show that division on the architecture diagram. Connect the agency application, policy point, approved endpoint, and evidence store with a solid routed line. Put vendor-managed inference and local execution in separate boxes. Link the request record to the use-case inventory and program file. The board can then see the exact control supplied by each component.

DeepInspect

DeepInspect supports the routed HTTP portion of public-sector AI governance. It sits as a stateless proxy between authenticated agency users or agents and HTTP-based LLM endpoints. The calling application supplies identity and program context. DeepInspect evaluates that context with prompt classification and model destination, plus the role and active policy version, before an allowed request reaches the model.

Each routed decision produces a signed, tamper-evident record outside the calling application's write path. DeepInspect leaves identity proofing, system authorization, model evaluation, procurement, records management, appeals, and official program actions with their assigned owners. Vendor-managed inference and local execution remain outside the proxy boundary. For traffic deliberately routed through it, agencies gain an enforceable request point and independent evidence tied to the use-case file. Book a technical deep dive at deepinspect.ai.