Public Sector AI Compliance: OMB M-24-10, NIST AI RMF, and the State AI Laws That Apply to Agencies

The Office of Management and Budget issued Memorandum M-24-10 on March 28, 2024, directing federal civilian agencies to advance AI governance, innovation, and risk management. The memorandum established the role of Chief AI Officer at each covered agency, the requirement for public AI use case inventories under Executive Order 13960 as amended, and the specific risk management practices required for rights-impacting and safety-impacting AI. The follow-on OMB Memorandum M-24-18, issued September 24, 2024, set federal AI acquisition expectations. The Department of Defense issued its Responsible AI Strategy and Implementation Pathway, with the DoD CIO and the Chief Digital and Artificial Intelligence Officer (CDAO) carrying the operational mandate. State legislatures have moved on parallel tracks: Colorado SB 24-205 takes effect February 1, 2026, Connecticut SB 2 was enacted, California AB 2930 advanced through committee. The combined regime sets specific architectural expectations on AI used by or for the public sector.

I want to walk through what OMB M-24-10 actually requires, how the NIST AI RMF connects, what state-level laws add, and the inspection architecture that closes the gap for agencies and contractors.

What OMB M-24-10 requires

M-24-10 applies to federal civilian agencies covered by the Chief Financial Officers Act of 1990 and a number of small agencies, with adaptations for the Intelligence Community and the Department of Defense. The memorandum sets three pillars:

Strengthening AI governance. Each covered agency designates a Chief AI Officer responsible for coordinating AI policy, advising the agency head on AI matters, and serving as the senior accountable official for AI risk management. Agencies designate AI Governance Boards. The Chief AI Officer reports to the Office of Management and Budget on the agency's AI inventory, risk management, and policy compliance.

Advancing responsible AI innovation. Agencies remove barriers to responsible AI use, identify priority use cases, and invest in AI workforce, infrastructure, and data foundations. The memorandum directs agencies to publish AI use case inventories annually and to make the inventories available to the public.

Managing risks from the use of AI. This is the pillar with the operational teeth. Agencies have to identify AI use cases that are rights-impacting or safety-impacting under the definitions in the memorandum. Rights-impacting AI includes use cases affecting access to government services, eligibility for benefits, employment decisions, criminal justice outcomes, civil rights protections, and similar consequential determinations. Safety-impacting AI includes use cases where errors could threaten human life, physical safety, or critical infrastructure.

For rights-impacting and safety-impacting AI, the agency has to complete impact assessment, real-world performance testing, ongoing monitoring, human oversight, public notice for the use case, and (for rights-impacting) consultation with affected communities. The agency cannot use rights-impacting or safety-impacting AI without these controls in place by the December 1, 2024 deadline, with waivers available only under specified conditions.

The Federal Acquisition Regulation amendments that flowed from M-24-18 set acquisition expectations for AI products and services the agency procures: vendor disclosure of AI use, AI bill of materials components, and contractual provisions for monitoring and incident notification.

How NIST AI RMF connects

The NIST AI Risk Management Framework 1.0, published January 26, 2023, provides the voluntary technical baseline for AI risk management. M-24-10 directs agencies to apply the framework's practices to their AI risk management programs.

The framework's four functions (Govern, Map, Measure, Manage) map to the M-24-10 operational expectations:

• Govern maps to the Chief AI Officer designation, the AI Governance Board, and the agency's AI policy infrastructure.
• Map maps to the AI use case inventory, the impact classification (rights-impacting / safety-impacting), and the identification of affected systems and stakeholders.
• Measure maps to the real-world performance testing, the bias and fairness testing, and the security testing requirements.
• Manage maps to the ongoing monitoring, the human oversight, the incident response, and the lifecycle management of the AI use case.

The NIST AI RMF Generative AI Profile, released July 26, 2024, addresses the specific risks of generative AI including hallucination, confabulation, prompt injection, and synthetic content generation. The profile is voluntary but increasingly cited by procurement reviewers as the baseline expectation for generative AI procurement.

What the state-level laws add

State legislatures have moved on AI use by state agencies and state contractors on parallel tracks to OMB M-24-10:

Colorado SB 24-205 ("Consumer Protections for Artificial Intelligence"), signed May 17, 2024, takes effect February 1, 2026. The law applies to developers and deployers of high-risk AI systems, with specific provisions for state-government use. Deployers (which include state agencies using AI) have to implement a risk management policy and program, complete impact assessments, notify affected consumers of AI use, and report algorithmic discrimination to the state Attorney General.

Connecticut SB 2, enacted in 2023 and amended, requires the Connecticut Department of Administrative Services to inventory and assess generative AI use by state agencies and to develop AI governance policies. It established AI literacy training requirements for state employees using generative AI.

California AB 2930, advanced through committee and at various points through 2024, addresses algorithmic discrimination in consequential decisions including state-government and state-contractor AI. While the bill's final form has continued to evolve, the architectural expectations track Colorado SB 24-205.

Texas HB 2060 (2023) directed state agencies to inventory AI systems and report on them, with subsequent legislation expanding governance obligations.

The state laws use vocabulary that differs from M-24-10 in places but converge on the same architectural expectations: agencies have to inventory AI use, classify by risk, complete impact assessments, document the risk management program, monitor real-world performance, and produce evidence on examination.

Where agency and contractor AI deployments are exposed

The exposure pattern across agencies running AI in benefit eligibility determination, fraud detection, hiring decisions, law enforcement support, and citizen-facing chatbots repeats:

The agency is using an AI system from a vendor (Salesforce Einstein, Microsoft Azure OpenAI, Google Gemini, AWS Bedrock, Palantir, Booz Allen) for a rights-impacting determination. The agency has the FedRAMP authorization for the infrastructure and the procurement record for the AI capability. The agency has impact assessment documentation completed at procurement time. The agency cannot produce, per request, the AI input, the AI output, the identity of the federal employee involved in the decision, and the policy that governed the request. The M-24-10 monitoring expectation, the NIST RMF Manage function, the Colorado SB 24-205 algorithmic discrimination notification expectation, and the agency's own Section 552 (FOIA) response obligation all sit on top of records that do not exist.

Shadow AI in agencies follows the enterprise pattern. The general workforce uses ChatGPT and Copilot for drafting, research, and analysis. Agency policy may permit certain enterprise-licensed uses and prohibit others. The agency cannot inventory the unsanctioned use without inspection at the network and identity layers. The data classes at risk include controlled unclassified information (CUI), Personally Identifiable Information collected under the Privacy Act, and information protected by agency-specific authorities (HIPAA for HHS, FERPA for ED, IRS 6103 for Treasury, ITAR/EAR for export-controlled data at State and Commerce).

The contractor exposure mirrors the agency exposure. A contractor performing work on a covered agency contract is bound by the same risk management expectations through the contract. The FAR amendments and the agency-specific contract clauses (DFARS for DoD, HHSAR for HHS, DHS-specific clauses for DHS contracts) push the requirements onto the contractor's operating environment.

The inspection architecture that closes the gap

The architecture has the same shape as the architecture for private-sector high-risk AI compliance. Public-sector calibration adds CUI and Privacy Act data class detection, federal identity (PIV / CAC) attribution, and FedRAMP boundary considerations.

Inline inspection sits at the HTTP AI request boundary between authenticated agency users and any LLM endpoint, including the FedRAMP-authorized enterprise services and the public consumer AI services agency policy may permit or restrict. The inspection includes detection for the data classes relevant to public-sector workloads: CUI markers, PII under the Privacy Act, PHI for HHS workloads, FERPA-protected information for education workloads, IRS 6103 information for Treasury workloads, and export-controlled data markers for State and Commerce workloads.

Identity attribution names the federal employee or contractor employee behind each request, linked to the PIV or CAC identity used to authenticate the session. For agent-based workflows, the agent identity and delegated authority appear in the record alongside the human identity.

Per-decision audit records satisfy the M-24-10 ongoing monitoring expectation, the NIST AI RMF Manage function, the Colorado SB 24-205 algorithmic discrimination reporting, the FOIA response obligation for AI use records, and the agency's incident response framework. Each record contains timestamp, identity, data class, policy version, decision outcome, and a tamper-evident signature. Retention follows the agency's records schedule under the Federal Records Act and the National Archives and Records Administration GRS schedules.

FedRAMP boundary considerations apply where the inspection layer itself processes data inside the agency's authorization boundary. The inspection layer deployment pattern of choice is one where the layer runs inside the agency's FedRAMP-authorized environment, processes the AI request data inside the boundary, and produces the audit record inside the boundary, without sending the prompt content to an external SaaS for inspection.

Where this connects to the broader public-sector regulatory stack

OMB M-24-10 and the state AI laws do not stand alone for public-sector AI. Other authorities apply in parallel:

• FedRAMP (Federal Risk and Authorization Management Program) sets the security baseline for cloud services agencies use. AI services accessed by an agency have to operate under a FedRAMP authorization at the appropriate impact level.
• The Privacy Act of 1974 applies to system-of-records information processed through AI. The agency's System of Records Notice (SORN) covers AI-assisted processing of covered information.
• Section 508 of the Rehabilitation Act applies to accessibility of AI-assisted citizen interfaces.
• The Paperwork Reduction Act applies to information collection through AI-assisted citizen interaction.
• Agency-specific authorities apply: HIPAA at HHS, FERPA at ED, IRS 6103 at Treasury, ITAR / EAR at State and Commerce, BSA / AML / SAR at FinCEN.
• For DoD specifically, the DoD Responsible AI Strategy and the CDAO mandate add operational expectations including the DoD AI Ethical Principles.

The architecture that satisfies M-24-10 produces records that contribute to Privacy Act SORN evidence, FOIA response, FISMA continuous monitoring, agency Inspector General review, GAO audit, and the Office of Special Counsel's authorities. The infrastructure is shared. The vocabulary differs by authority.

DeepInspect

This is the architecture DeepInspect was built to provide for public-sector AI compliance. DeepInspect sits inline between authenticated agency and contractor users and any HTTP-based LLM endpoint. The inspection includes detection for CUI markers, Privacy Act PII, PHI for HHS workloads, FERPA-protected information for education workloads, IRS 6103 information for Treasury workloads, and export-controlled data markers for State and Commerce workloads.

Every request produces a per-decision audit record containing federal-employee or contractor-employee identity, timestamp, data class, policy version, decision outcome, and a tamper-evident signature. The records support the M-24-10 ongoing monitoring expectations, the NIST AI RMF Manage function, the Colorado SB 24-205 reporting obligations, FOIA response on AI use records, and the agency's records schedule under the Federal Records Act. For Chief AI Officers, agency CISOs, contractor program managers, and contracting officers facing the M-24-10 operational deadlines and the cascading state and contract obligations, the inspection layer is the architectural component that produces the evidence each authority expects from the same infrastructure. Book a demo today.