EU AI Act for Healthcare: Why AI in Diagnostics, Triage, and Clinical Decision Support Lands in the High-Risk Category

The EU AI Act treats healthcare AI as high-risk through two independent paths, and most providers find themselves caught by both. Annex III point 5 lists AI systems used in access to essential private and public services as high-risk, which captures clinical triage tools and AI used in benefits determinations. The Medical Device Regulation (MDR), which the EU AI Act explicitly references in Article 6(1), pulls in any AI that meets the definition of a medical device, which covers most diagnostic imaging, clinical decision support, and triage software. The combination means a hospital deploying an AI radiology tool owes both the MDR conformity assessment and the EU AI Act high-risk obligations. The August 2, 2026 deadline for high-risk system requirements is firm, and the record-keeping infrastructure most hospitals run today fails the Article 12 test.

I want to walk through which healthcare AI use cases land as high-risk, how the AI Act and MDR overlap in practice, where the record-keeping architecture must change, and what an audit visit will actually look like.

Mandate

The high-risk classification for healthcare AI flows from three legal sources operating together.

Annex III essential-services classification

Annex III point 5(a) covers AI systems used in the evaluation of eligibility for essential public services and benefits, which captures AI used in insurance claims, public health benefit determinations, and access to publicly funded services. Annex III point 5(b) covers AI used in evaluating the creditworthiness of natural persons, which can apply where an AI tool scores patient financial risk for treatment planning. Annex III point 5(c) covers AI used for risk assessment and pricing in life and health insurance, which captures most insurer AI deployments.

MDR overlap under Article 6(1)

Article 6(1) classifies as high-risk any AI system that is a safety component of a product covered by Union harmonization legislation listed in Annex I, including the MDR. AI that meets the definition of a medical device under MDR Article 2(1) and is itself a Class IIa, IIb, or III device falls in the high-risk category. Most diagnostic imaging AI, clinical decision support systems, and triage tools land here. The MDR conformity assessment route under Annex VII of the AI Act applies in parallel with the MDR conformity assessment.

Article 26 deployer obligations

The hospital, clinic, or insurer that uses the AI system is the deployer. Article 26 requires the deployer to operate the system in accordance with its instructions for use, to assign human oversight to natural persons with the necessary competence, to ensure input data is relevant to the intended purpose, to monitor system operation, to keep the automatically generated logs under Article 19, and to inform the provider and the market surveillance authority of serious incidents. The deployer obligation does not transfer to the AI vendor through a contract clause.

Compliance gap

Most healthcare AI deployments today have no architecture that produces the records the Article 26 deployer obligation presumes exist.

The application-controlled audit log fails the Article 12 test

A clinical decision support tool that records its own decisions in its own database fails the write-path independence test. The hospital's regulator asks: produce the audit record for AI request 47832 against the radiology AI tool used to flag the patient case, including the clinician identity, the data classification of the imaging study, the policy version that governed the flagging threshold, and the decision outcome. The vendor's internal log contains some of these fields but the hospital cannot independently verify the record. The deployer obligation under Article 26 makes the hospital the regulated party.

Identity context is missing at the request layer

The AI tool runs against a service credential issued by the hospital. The credential identifies the EHR integration, not the radiologist or the ED clinician reviewing the patient case. The Article 19 record fails the identity check. The deployer cannot demonstrate which clinician initiated which request without reconstructing the context from session traces, which a regulator does not accept as a contemporaneous record.

PHI handling is not evaluated at decision time

Healthcare AI prompts routinely contain PHI: patient identifiers, diagnostic codes, imaging metadata, clinical narratives. The HIPAA covered entity in a US deployment, or the GDPR data controller in an EU deployment, owes a contemporaneous record of which categories of personal data flowed through which model at which time. Most AI tools do not evaluate prompt-level classification at request time. The classification is reconstructed weeks later, if at all.

Vendor-embedded AI is invisible to the deployer

A material share of healthcare AI usage flows through SaaS vendors that embed model calls under the hood. The transcription vendor uses an LLM to summarize clinical notes. The coding vendor uses ML to assign CPT codes. The patient-engagement vendor uses an AI agent to triage messages. The deployer's environment never sees the prompt or the response. The Article 26 obligation does not transfer to the vendor. The hospital owns the disclosure on demand.

Mandate vs Compliance

The text of the EU AI Act reads at one level of abstraction. The infrastructure that survives a regulatory review in a hospital setting operates several levels lower. The gap between the two is where most healthcare deployers are exposed.

Disclosure test

A national competent authority opens an investigation following a serious incident. The authority asks for the contemporaneous audit record for the AI tool involved, with identity context, classification, policy state, and decision outcome. A compliant deployer produces the record within minutes. A non-compliant deployer produces vendor logs that lack the hospital's identity context and reconstructed traces that lack the classification evaluated at request time. The disclosure failure itself can produce a finding under Article 99 Tier 3, even if the underlying clinical decision was correct.

Vendor liability

The AI vendor providing the radiology tool is the provider under Article 3. The hospital is the deployer under Article 26. Both bear obligations, and the obligations do not collapse into the vendor's. A serious-incident report under Article 73 originates from the deployer. The compliance file under Article 12 is the deployer's responsibility for the records of use.

Compliance gap

The compliance gap is architectural. The records the deployer obligation presumes exist are not produced by EHR application logs, by network appliances, or by the AI vendor's internal database. They are produced by an inspection layer on the AI request path that writes the per-decision record independent of the application that originated the request and independent of the AI vendor that responds.

DeepInspect

This is the architecture the EU AI Act presumes for healthcare deployments. DeepInspect sits at the AI request boundary as an external enforcement layer that operates as a stateless proxy between authenticated clinicians or agents and the AI vendor endpoint. Every HTTP request is evaluated against per-route, per-role policies using identity context the EHR integration supplies. The per-decision record is committed by the proxy, independent of the AI vendor's database, before the model response returns to the EHR.

The record contains a verified identity for the clinician behind the request, the role and authorization context, the data classification applied to the prompt (including PHI detection), the policy version that governed the decision, the decision outcome, and a cryptographic signature that prevents post-hoc modification. The proxy enforces per-route policies on which AI vendors a given role can call, which categories of PHI can flow to which endpoint, and which policy bypasses require independent approval. Production overhead stays under 50 ms in internal testing, which keeps the enforcement layer outside the clinical workflow latency budget.

If you are facing the August deadline, let's talk.

Beyond the EU AI Act

The same architecture satisfies adjacent regimes that apply to healthcare AI. HIPAA in the US deployments requires the covered entity to maintain an accounting of disclosures of PHI for six years; an inspection layer that writes per-decision records produces the accounting source-of-truth. The MDR conformity assessment expects post-market surveillance records, which the inspection layer feeds directly. The California AI Transparency Act requires disclosure for AI systems with 1M+ monthly users. Each regime uses different vocabulary for the same infrastructure requirement: an inspection layer on the AI request path that writes a tamper-evident, identity-bound, classification-aware record at decision time.