ISO/IEC 5338 AI Compliance Checklist for Lifecycle Controls
ISO/IEC 5338:2023 extends system and software life cycle processes for machine-learning and heuristic AI systems. This checklist turns its agreement, organizational, technical-management, and technical process groups into inspectable work: scope the system, control suppliers and changes, engineer data, validate continuously, preserve runtime evidence, maintain the system, and prove disposal.
ISO/IEC 5338:2023 defines AI system life cycle processes across acquisition, organizational support, technical management, engineering, operation, maintenance, and disposal. It adapts established system and software engineering processes to AI characteristics such as probabilistic behavior, reliance on data, measurable decay, iterative requirements, and potential autonomy.
I use this checklist as an evidence test rather than a policy inventory. Each checked item needs a named system, owner, baseline, operating record, and last test date. A checkbox beside “continuous validation” tells an assessor nothing. A change ticket linked to the model version, threshold result, production decision record, and closure approval tells the story.
1. Fix the scope and conformity basis
Record the AI system boundary, business purpose, lifecycle stage, internal and external components, owners, users, model endpoints, data sources, and environments. Identify which lifecycle processes apply and document any tailoring decision.
Anchor the work to ISO/IEC 5338:2023, including the edition and approval date. The standard defines lifecycle processes and a conformance concept. It is separate from ISO/IEC 42001:2023, which specifies requirements for an AI management system and supports management-system certification.
Evidence: scope statement, system context diagram, process applicability matrix, tailoring rationale, standard edition record.
2. Give every AI system a stable identifier
Assign one ID that survives model changes, supplier changes, and application releases. Use it across requirements, risks, architecture, test results, runtime records, incidents, maintenance, and disposal.
The inventory should distinguish the application, AI system, model or service, API route, data stores, and evidence store. One row called “customer-service AI” hides too much. Name the summarizer, retrieval component, approved model endpoint, production route, and owner.
Evidence: AI inventory, component register, route inventory, ownership record, data-flow diagram.
3. Control acquisition and supply
For acquired models and AI services, document the approved purpose, supplier, service version, data handling, security conditions, acceptance criteria, change-notification path, incident terms, and exit obligations. Connect procurement evidence to the same system ID used in engineering and operations.
The acceptance test should exercise the real integration route. A supplier questionnaire cannot prove that the production application sends only approved data classes to the contracted endpoint.
Evidence: supplier assessment, contract version, acceptance plan, test result, endpoint allowlist, change-notification record.
4. Define requirements that can be tested
Translate stakeholder and system requirements into observable criteria. “Protect sensitive data” needs a classification, an approved destination, an enforcement action, and a pass condition. “Maintain accountability” needs identity fields, policy versioning, integrity protection, retrieval rules, and a sample query.
I would reject any requirement containing the phrase “appropriate controls” until the author names the decision point and evidence. Vague requirements breed vague validation.
Evidence: requirement IDs, acceptance criteria, traceability matrix, architecture decisions, policy specifications.
5. Govern knowledge and AI data engineering
Document the origin, permission, quality criteria, transformations, versions, splits, and retention for training, test, validation, retrieval, and production-input data. Heuristic systems also need knowledge-acquisition records showing the source, reviewer, refinement, and approved representation.
For runtime prompts, define data classes and inspection behavior before the HTTP request reaches a model. Include attachments, structured fields, copied tables, and agent-generated follow-up calls in the test set.
Evidence: data register, lineage, quality results, knowledge-source approvals, dataset versions, prompt-classification test results.
6. Baseline architecture and configuration
Record model versions, prompt templates, retrieval configuration, tools, policy versions, provider endpoints, classifiers, thresholds, and deployment artifacts as configuration items. Tie each production request to the relevant runtime versions.
A screenshot of a console at 16:40 captures appearance, not a reproducible baseline. Store machine-readable configuration, approval metadata, effective timestamps, and integrity hashes.
Evidence: configuration baseline, version repository, deployment manifest, approval record, signed runtime sample.
7. Verify, validate, and continuously validate
Verification checks implementation against specified requirements. Validation checks the AI system against its intended use and stakeholder needs. ISO 5338 also names continuous validation as a lifecycle process, reflecting the fact that production data and desired behavior can change.
Define measures, thresholds, sample populations, review cadence, and actions when a threshold fails. Include model quality and drift where applicable, plus request-boundary measures such as sensitive-data decisions, unauthorized destinations, identity gaps, and policy errors.
Evidence: verification plan, validation report, continuous-validation schedule, threshold history, failed-test corrective action.
8. Authorize transition into operation
Require a release decision that references the approved requirements, architecture, risk treatment, configuration baseline, test results, operating procedures, rollback condition, and support owner. Record the exact time the production version became effective.
Then retrieve the first production decision record after release. It should show the expected model and policy version. Print that JSON record beside the release ticket during the review. Mismatched versions expose a weak transition process immediately.
Evidence: release approval, readiness review, rollback plan, effective timestamp, first-production sample.
9. Bind production requests to identity and policy
For HTTP-based LLM traffic, attach the authenticated caller, delegated user where relevant, role, system ID, route, business purpose, destination model, and data classification to the policy decision. Static service credentials often identify only the application, leaving the person or agent behind the request unresolved.
Define permit, redact, and deny outcomes. Fail closed when identity, route authorization, or required classification is missing. Test human calls, agent calls, scheduled workloads, and retry paths.
Evidence: identity mapping, authorization policy, delegated-authority record, enforcement tests, decision samples.
10. Keep independent operational evidence
A request record should contain the request ID, timestamp, identity, system and route IDs, model destination, classification, policy version, decision, and enforcement action. Add tamper-evident integrity and separate access to the evidence store from application administration.
Write-path independence addresses selective logging, suppression, and crash loss. The policy decision should commit before a permitted model response returns to the application.
Evidence: signed decision records, evidence-store access review, integrity verification, retention schedule, reproducible sample query.
11. Control maintenance and exceptions
Treat model swaps, endpoint changes, prompt updates, classifier changes, threshold changes, and policy edits as controlled maintenance. Record the reason, affected configuration items, risk review, approval, tests, release time, rollback condition, and observed result.
Every exception needs an owner, scope, reason, compensating control, expiry date, and closure test. “Temporary” is not a date.
Evidence: maintenance ticket, change diff, approval, test result, exception register, expiry alert, closure record.
12. Prove disposal
Plan retirement while the AI system is still operating. Identify route removal, credential revocation, supplier termination, data deletion or retention, evidence preservation, dependency updates, and final ownership.
After cutoff, send a controlled request to the retired route and preserve the denial. Check that applications, agents, scheduled jobs, and fallback configurations have stopped using the endpoint.
Evidence: disposal approval, revoked credentials, removed route, supplier notice, deletion result, denied post-cutoff test, final inventory update.
Grade the checklist with evidence
Use four states for each item: absent, designed, operating, and evidenced. “Operating” means the control runs in the declared scope. “Evidenced” means a second reviewer can retrieve the same record using a documented query and verify its integrity.
Assign one owner, one evidence location, one test cadence, and one remediation date to every row. Sample permit, redact, deny, exception, model-change, supplier-change, maintenance, and disposal events. Keep gaps visible. A red cell with an owner and date is more useful than a green cell supported by a policy PDF.
This scoring preserves the distinction between adopting 5338 lifecycle processes and obtaining a 42001 management-system certificate. The checklist can support both projects, but it cannot turn a lifecycle process mapping into a certificate.
DeepInspect
DeepInspect supplies the operating and evidence layer for HTTP traffic between authenticated users or agents and LLMs. It evaluates identity, role, data classification, route, model authorization, and policy before forwarding a permitted request. Each permit, redact, and deny action produces a tamper-evident decision record tied to the active policy version.
Those records support the checklist items for configuration, transition, operation, continuous validation, maintenance, exceptions, and disposal testing. DeepInspect leaves acquisition contracts, model development, dataset quality, workforce competence, and management-system certification to their proper owners.
Book a technical deep dive at deepinspect.ai.