BSI AIC4 Criteria: What Runtime AI Traffic Can Prove
The BSI AIC4 criteria catalogue is a full-lifecycle assurance framework for cloud AI services and an extension to BSI C5. Runtime model traffic can support request logging, request monitoring, and incident reconstruction. It cannot prove provider training, validation, data quality, explainability, bias controls, or operating effectiveness across the service.

The BSI AIC4 criteria catalogue evaluates a cloud AI service across development, testing, validation, deployment and monitoring. It is an assurance catalogue built as an extension to BSI C5, not a checklist for an LLM gateway. Runtime traffic records can contribute evidence for request logging under RE-02 and request monitoring under RE-03. The other criteria require provider processes and model tests, plus data records, documentation and independent audit work that sit beyond the request path.
My opinion is simple: mapping every AIC4 row to a gateway would make the control matrix look complete and the assurance case weaker.
TL;DR
- AIC4 covers the AI service lifecycle and depends on C5 for general cloud controls.
- Routed traffic can support RE-02 logging fields and RE-03 monitoring, along with selected incident reconstruction.
- A gateway cannot prove training controls, model validation, data quality, explainability or bias assessment. It also cannot prove provider operating effectiveness.
- Treat runtime evidence as one input to an AIC4 assurance file, then preserve the independent auditor's scope and findings.
AIC4 is a provider assurance catalogue
The BSI AIC4 criteria catalogue defines an AI service as a cloud service using AI methods in public or private cloud infrastructure. Its eight areas cover preliminary criteria, security against attacks, performance and functionality, reliability, data quality, data management, explainability and bias. The catalogue says the criteria set the minimum audit scope, while supplementary text gives implementation guidance.
AIC4 also requires general cloud compliance through BSI C5 under criterion PC-01. The BSI AIC4 overview describes the catalogue as an AI-specific extension to that cloud assurance base. This matters during evidence design. Identity management and change control retain their C5 owners, alongside infrastructure operations and conventional cloud logging. A routed model request can supply an event record without replacing those controls.
The document was published in 2021 and targets standard machine-learning services that improve through training data. It predates the current generative LLM wave. Its lifecycle and assurance structure still applies to relevant cloud AI services, but every criterion should be interpreted against the service actually under audit. For a managed LLM offering that combines a provider model, retrieval layer, customer data store, safety filters and several subcontracted infrastructure services, that interpretation begins by drawing one service boundary the system description, evidence population and auditor all use.
The audit covers more than control design
AIC4 describes independent assurance using established audit methods. It calls for reasonable assurance. A type 1 report addresses the system description and the design and implementation of controls at a specified date. A type 2 report also addresses operating effectiveness across a specified period. The catalogue recommends recurring audits include operating effectiveness, with design-only work reserved for an initial engagement.
That distinction stops a common evidence error. A configuration export can show that a logging policy exists. It says nothing about missing events during the review period, failed collectors, undocumented route changes, or an incident in which a control was bypassed. Operating effectiveness needs the event population and review period. It also needs exceptions and testing evidence.
AIC4 also keeps customer judgment in view. Its introduction says an independent report does not establish that the AI method suits a particular application or customer. The cloud customer must use the report and perform its own risk evaluation. The BSI does not select the auditors or perform the audits, and it does not check the resulting reports. An AIC4 badge without the report scope and exceptions is thin procurement evidence. Scope decides the claim.
RE-02 defines a narrow runtime evidence set
Criterion RE-02 requires request logging that supports investigation of failures and incidents and allows events to be traced to specific AI models. The catalogue names several minimum fields: request type, processing time including timestamps, and metadata about the requesting user. It calls for an application-appropriate retention interval of at least three months. It also requires documented policies and safeguards around logging, plus disclosure of log content and storage periods in the system description.
An authenticated HTTP gateway can supply part of that evidence. Record the subject and role supplied by the calling application. Add the source application and request type, then the destination endpoint and model identifier. Preserve start and completion times, processing duration and policy version, plus the disposition. A request identifier should join the event to provider and application logs without forcing full prompt content into every telemetry store.
The AI decision records pattern adds a useful policy layer. For AIC4, the assurance file also needs ownership and retention, then access and integrity safeguards. Coverage evidence must prove that the selected records represent the audit period.
RE-03 needs monitoring evidence and follow-up
Criterion RE-03 calls for continuous checks, at least monthly, for irregularities in user requests that may indicate malicious activity against in-scope models. Its supplementary text names unusually large request volume and similar request content as examples. It also points to model theft and data poisoning scenarios defined elsewhere in the catalogue.
Runtime monitoring can produce useful artifacts for this criterion. Preserve the monitored route population and the detection rule version. Record the time window and threshold, followed by the signal and affected request identifiers. Keep the triage owner and case reference, then the resulting policy or incident action. An alert count alone leaves the auditor guessing about coverage and disposition.
Imagine an auditor tracing one red row in a monthly review sheet. The row should open the request sequence and caller metadata, alongside the active rule, route, and case record. It should also reveal any collection gap. That is far stronger than a dashboard tile saying "27 anomalies." The AI incident response runbook owns the wider containment and recovery process. Gateway evidence supplies only the model-traffic segment.
Incident reconstruction crosses several systems
AIC4 criterion RE-05 requires the provider to handle AI-specific security incidents and feed identified incidents back into threat scenarios and countermeasure review. Type 2 reporting also addresses significant events and conditions during the review period. Runtime route records can help reconstruct request sequences and policy outcomes, especially when they sit outside the calling application's write path.
The reconstruction file should connect the gateway request identifier to the provider event and the application transaction. Add the incident case, affected model, rule or policy version, and collection health for the relevant period. A blocked request has no provider response, so the record should make that absence explicit. A permitted request followed by a harmful business action requires another join to the application or target system.
Direct provider calls and local inference create separate evidence populations. So do batch training jobs and traffic outside the managed route. List them in the coverage statement. The wider AI audit trail requirements remain an evidence architecture problem, and AIC4 adds a provider assurance scope that one traffic layer cannot own.
Most AIC4 criteria need provider-side evidence
Security criteria require threat assessment and attack testing, together with countermeasure evaluation and risk acceptance. Performance criteria require model selection and training and validation records, followed by business testing, drift review and retraining evidence. Data quality and management criteria require source and lineage, annotation and access, plus quality checks and dataset separation. Explainability and bias sections require assessments and methods, backed by measurements, tests and documented limitations.
A gateway sees selected inference traffic. That view can reveal input patterns, route decisions, response characteristics, and operational incidents. It cannot inspect an independent test set that never crossed the route. It cannot prove that training data were credible, annotations passed review, a bias metric stayed within tolerance, or a provider tested adaptive attacks. Provider documentation and technical evidence own those claims. The independent auditor evaluates their design and operating effectiveness.
The same limit applies to C5. A request record cannot prove physical security or personnel controls. It also cannot prove backup recovery, subcontractor oversight, or every infrastructure change. A control matrix should mark these items as external evidence dependencies instead of stretching the gateway artifact until it loses meaning.
A release test should prove coverage and limits
Select one in-scope model route in a nonproduction environment. Send a permitted request with a marker such as AIC4-RE02-731. Confirm that the event preserves request type and authenticated subject metadata, followed by the model, timestamps, processing duration, policy version and disposition. Verify the retention class and log integrity path. Then locate the same request in the provider or application record through the shared identifier.
Next, send a burst designed to trigger a nonproduction RE-03 monitoring rule. The expected artifact includes the monitored population and rule version, together with the threshold, detection time, request identifiers and triage case. Test a blocked route as part of the same run; its gateway record should exist while the provider response remains absent.
Finish the exercise with a bypass attempt. A direct call outside the authenticated HTTP route should create no gateway record. That result belongs in the coverage statement and a separate control test. The release package should also list AIC4 claims this test cannot support, including model training and validation, data quality and explainability, plus bias and provider operating effectiveness. Honest exclusions make an assurance map usable.
DeepInspect
DeepInspect is a stateless proxy for authenticated HTTP traffic deliberately routed between users or agents and LLM endpoints. Before forwarding, it can evaluate application-supplied identity and route, plus the model, data classification and policy. Each permit, redaction, reroute, or block can produce a signed, tamper-evident record outside the calling application's write path.
For AIC4, those records can contribute to RE-02 request logging and RE-03 request monitoring, along with selected incident reconstruction. They can preserve request type and user metadata, followed by model destination, timestamps, processing duration, policy version and outcome. DeepInspect does not provide an AIC4 attestation and cannot prove provider-side development, training, validation, data quality, model resilience, explainability, bias controls, C5 compliance, or operating effectiveness across the cloud service. Local execution, direct provider calls and training pipelines remain outside its coverage. If traffic does not travel through the authenticated HTTP proxy, DeepInspect does not see or control it.
Book a technical deep dive at deepinspect.ai.
Frequently asked questions
- Is AIC4 an LLM security standard?
AIC4 is an assurance catalogue for cloud AI services. The 2021 document covers standard machine-learning methods and the full service lifecycle. Its language includes natural language processing, but it was not written as a generative LLM gateway standard. Apply the criteria to the actual service scope and preserve the catalogue's stated exclusions and assumptions.
- Does an AIC4 report prove the service fits our use case?
The catalogue says the report supports customer risk analysis and does not establish suitability for a particular application. Customers should review the scope, applicable criteria, exceptions, subservice treatment, complementary customer controls, and audit period. They then need their own use-case risk decision.
- What is the difference between type 1 and type 2 evidence?
Type 1 addresses a specified date and covers the system description plus control design and implementation. Type 2 covers a specified period and also addresses operating effectiveness. Runtime events can contribute to the period evidence, but the auditor still needs a complete population, exception handling, collection health, and tests tied to the provider's control description.
- Can gateway logs satisfy RE-02 by themselves?
They can supply several required fields when all in-scope requests use the managed route. The assurance case also needs logging policies, safeguards, retention, access, integrity, system-description disclosures, coverage evidence, and treatment of gaps. Provider and application logs may hold additional facts. The auditor decides whether the combined evidence supports the criterion.