CJIS AI Audit Evidence: Build a Review Package Investigators Can Replay

An auditor selects an AI request made by an authenticated criminal justice user and asks who initiated it and which model received it. The sample must also show what information crossed the boundary and which policy ran. Finally, it must show what happened next. CJIS AI audit evidence has to answer those questions with records that preserve the event's content and order. A screenshot of a dashboard leaves the reviewer dependent on whoever chose the screen.

I would build the package so another person can repeat the selection and retrieval without calling the application developer. That is the practical standard for this evidence.

TL;DR

• Define the complete population of in-scope AI requests before anyone selects samples, including permitted and redacted events alongside denied and failed events.
• Preserve identity and destination, policy version and classification, decision and time, plus response disposition in a protected record tied to the original sequence.
• Give reviewers saved queries and sample-selection logic. Include integrity checks and weekly review evidence, with at least one year of searchable retention.
• Keep DeepInspect claims within authenticated HTTP traffic routed through its enforcement point. IAM and endpoint teams retain separate responsibilities, as do records policy and investigations.

Fix the evidence population before sampling

The FBI CJIS Security Policy version 5.9.5 organizes audit accountability through AU controls covering event selection and record content, review and reporting, protection and retention, plus generation. For an AI service, the evidence population should begin with all in-scope HTTP requests during a named UTC window. It should include successful model calls alongside redactions and denials. Policy errors and missing-identity failures belong in the population too, as do response blocks.

Export a population manifest containing a stable event identifier and timestamp, originating principal and calling application, model route and policy version, plus action and record location. Freeze the manifest before sample selection. Let the reviewer choose identifiers or document a repeatable selection method. My opinion is simple: owner-picked screenshots are demonstrations. A frozen population with reviewer-selected rows is audit evidence.

A white spreadsheet with filtered-out denials may look tidy on screen. Its missing rows become obvious when the reviewer compares event counters with the manifest.

Bind the sample to the original decision

AU-3 requires records that establish what occurred and when and where it occurred. They must also establish the source and outcome, along with the identities involved. NIST SP 800-53 Revision 5 provides the control structure used by the CJIS policy. An AI event needs additional decision context: prompt classification and requested model, policy rule and version, enforcement result and response disposition.

The originating identity deserves special attention. Many applications authenticate a justice user, then call an LLM through one shared service account. The provider record identifies the relay while the human originator disappears. Evidence should join the upstream authentication event to the outbound request with a stable correlation identifier. IA-2 supports the upstream authentication objective. AC-3 and AC-4 frame access and information-flow decisions on the routed transaction.

For each selected sample, include the identity assertion and validation result, policy input and decision output. Include the request record and response record, along with any review disposition. Protected prompt or response content may sit behind separate access controls. If the design stores a fingerprint, the package must also document controlled retrieval of the source record.

Preserve integrity and original order

CJIS AU-9 requires protection of audit information and audit tools against unauthorized access, modification, and deletion. The evidence package should identify the record writer and destination store. It should identify access roles and deletion permissions, then document the integrity mechanism and verification procedure. It should also show that the calling application lacks a path to silently rewrite its own decision history.

Run a staged tamper test against a copied event. Alter one decision field and execute the documented integrity check. Record the failure output and reviewer. Then test an application administrator's attempted modification or deletion against the protected store. These tests prove custody more clearly than a policy paragraph.

Search and reporting under AU-7 should preserve original content and time ordering. Produce one report for a principal and another for a model route. Produce a separate report for a policy version. Each result should retain source identifiers and UTC sequence rather than flattening events into an untraceable summary. The audit-log chain of custody guide covers the same custody problem at a deeper implementation level.

Prove weekly review and historical retrieval

CJIS AU-6 calls for audit review and analysis at least weekly, with reporting of findings to designated personnel. The useful artifact is a completed review record with its query scope and execution time. It should also name the reviewer, findings, escalation, and disposition. A dashboard proves that records are available for inspection. Signed review output proves the operating process ran.

Use queries tied to AI failure modes inside the assessed route. Retrieve requests with missing identity and traffic to unapproved model destinations. Review repeated denials and regulated-data classifications. Track changes in policy version. The review owner should connect findings to a ticket or incident record, approved exception, or closed false positive.

AU-11 establishes at least one year of audit-record retention. Test historical retrieval against the oldest available period, then compare the returned record with its integrity value. Confirm its original ordering separately. The exercise should record query time and any archive restoration step. The LLM audit log retention guide explains how retention design affects evidence availability, while the CJIS package remains governed by the agency's approved records and security process.

Assemble a replayable package

Keep the index small enough to use during a live review:

• Boundary: AI data-flow diagram and routed HTTP path, external model connections, plus adjacent systems.
• Population: assessment window and manifest, event totals and exclusions. Document the selection method used for the sample.
• Samples: selected event records with identity and policy, content handling, plus response links.
• Integrity: write-path design and permissions, tamper test, plus verification output.
• Review: weekly queries and sign-off, findings and tickets. Include the recorded disposition for each finding.
• Retention: configuration and oldest-period retrieval, restored records, plus ordering check.
• Crosswalk: CJIS and NIST control identifiers pointing to exact files and tests.

Name exclusions in the boundary folder. Local model execution and browser sessions that bypass the route need other controls and evidence. So do stolen credentials and endpoint compromise. Opaque inference embedded inside a vendor service sits outside the route as well. The package gains credibility when it marks those edges plainly.

DeepInspect

DeepInspect sits between authenticated users or agents and HTTP-based LLM endpoints. For traffic the application routes through it, DeepInspect evaluates supplied identity and policy context, then classifies the request. It applies a permit or deny decision, with redaction where policy calls for it. It inspects the response and records the result. Signed, tamper-evident per-decision records can support population exports and selected samples. They can also support integrity checks and saved queries, along with historical retrieval.

The surrounding CJIS evidence still depends on upstream IAM and enforced routing. Endpoint controls and approved retention remain separate, as do weekly human review and the organization's incident process. The tamper-evident AI audit logs guide describes the request-layer record design. Book a technical deep dive at deepinspect.ai.