Shadow AI in Medical Devices: Design Records, Complaints, and Model Routes

A quality engineer pastes a complaint narrative into an AI assistant to turn it into a concise investigation summary. The browser tab sits beside an open device history record. The pasted text carries a serial number and failure mode. It also includes the hospital name and technician note. That HTTP request can move regulated quality evidence and patient-linked context to a model service that supplier quality never approved.

Shadow AI in medical devices starts at this request path. It can appear in design engineering and regulatory writing. Complaint handling and cybersecurity work create more routes. So does field service. The model may sit outside the manufacturer's approved supplier controls even when the employee has legitimate access to the source record.

TL;DR

• Medical-device shadow AI can expose design files and complaint records. Test results, cybersecurity findings, and patient-linked service data can leave through the same unauthorized HTTP request.
• FDA's Quality Management System Regulation places records and complaint handling inside the manufacturer's quality system. An unapproved model route breaks the documented handling path.
• FDA's February 2026 cybersecurity guidance covers device design and labeling. It also covers premarket documentation for devices with cybersecurity risk. Shadow AI can expose that same security material.
• DeepInspect governs authenticated HTTP AI traffic routed through it. On-device inference, local models, direct browser bypass, and non-HTTP traffic require separate controls.

Quality records lose their wrapper inside a prompt

A controlled record has a document number and revision. It names an approver. Access and retention rules govern it. Copying one paragraph into a prompt strips away much of that wrapper while preserving the sensitive substance. A complaint excerpt may still identify the product and event. A test failure can reveal an unreleased design change. A vulnerability triage note may expose the affected component and exploit condition, along with the planned remediation.

The shadow AI pillar explains the general category. Device manufacturers carry an additional chain between the design history and quality record, followed by the supplier decision and regulatory file. An unauthorized model request creates a branch that the approved record system may never capture.

I would treat a personal AI account used for complaint summarization as an uncontrolled processor of quality data, even when the resulting prose is later pasted into the official system. The clean paragraph in the complaint file says nothing about the route that received the source material.

QMSR makes the record path operational

FDA's Quality Management System Regulation final rule became effective on February 2, 2026. The rule amended 21 CFR Part 820. It incorporated ISO 13485:2016 by reference while retaining additional FDA requirements. Its discussion specifically addresses risk-management records and complaint-handling records.

That framework matters because shadow AI changes where quality information travels. A manufacturer may have approved systems for complaints and nonconformities. Separate controls may cover CAPA and design review. Supplier records have their own path. An employee can still copy content out of those systems and send it to an unassessed model endpoint. The official record then shows only the edited output.

Request-level control should identify the user and source workflow before transmission. It should classify complaint terms and product identifiers. Adverse-event details require another check. So do design language and investigation findings. Policy can then bind those detections to an approved endpoint and purpose, or deny the request before the model receives it.

Cybersecurity files contain concentrated device intelligence

FDA's February 2026 medical-device cybersecurity guidance provides recommendations on device design and labeling, with separate recommendations for premarket documentation for devices with cybersecurity risk. It also addresses section 524B of the Federal Food, Drug, and Cosmetic Act for cyber devices.

Security teams hold threat models and architecture diagrams. Their files also contain software bills of materials and vulnerability assessments. Penetration-test findings and remediation plans complete the picture. These artifacts give a model provider more than generic product information. A short prompt asking for a clearer vulnerability description can disclose the affected interface and attack condition. A regulatory writer may expose the mitigation and release schedule in the same request.

The approved use may permit public vulnerability research while blocking product-specific findings. Provider approval alone cannot express that distinction. The enforcement decision needs the authenticated role and product or project context. It also needs the content class and destination. The registered purpose must apply to that exact HTTP request.

Complaint handling creates a patient-linked exposure

Complaint and service workflows often combine device facts with information about a patient, clinician, or facility. A narrative may include age and procedure details. It can show the outcome and location. The device identifier and service history add more context. Removing a patient name can leave enough context to identify the event. The model route therefore needs classification beyond simple name and account-number patterns.

A manufacturer should define policy for the complete workflow. Complaint intake may use an approved application and endpoint. Investigation may permit a narrower data set. Regulatory reporting requires qualified review tied to the controlled source record. The AI response can assist drafting only inside the approved process.

The AI data lineage guide describes how to connect source data and downstream artifacts. For shadow AI, lineage starts before transmission. Record the source application and data category. Add the model endpoint and policy version. Then record the decision and controlled reference to the originating record. Avoid copying full complaint text into a second audit repository unless the retention design expressly requires it.

Supplier approval and request approval answer different questions

Supplier quality can evaluate a model provider's contract and subprocessors. The review also covers its security program and retention terms, along with change notices. That review establishes an approved relationship. It cannot prove that a specific engineer was permitted to send a specific design record on Wednesday afternoon.

Per-request policy closes that operating gap for routed traffic. A service account assigned to regulatory writing may receive permission for public guidance and approved templates. The same route can deny a prompt containing an unreleased submission section or complaint identifier. An engineering role may use a model for public code examples. Product source code and vulnerability findings trigger another rule.

Embedded AI creates a separate boundary. A quality-management or product-lifecycle vendor may make model calls inside its own environment. The manufacturer's gateway sees no request when the traffic stays inside the vendor service. Supplier assessment and contract rights must cover that path. Configuration evidence and vendor exports must cover it too.

Discovery and enforcement need a precise boundary

DeepInspect inspects authenticated HTTP AI traffic deliberately routed through its proxy. A managed engineering assistant or quality application can use that route. So can an internal agent that sends a complaint excerpt to an approved LLM API. The policy decision occurs before an allowed request reaches the model.

A personal chatbot opened through an unmanaged browser can bypass the route. Endpoint telemetry and enterprise-browser controls address part of that surface. Secure web gateways and DNS monitoring add visibility. Egress controls address the remaining route. The shadow AI detection guide explains the role of those discovery layers.

On-device inference and offline models also sit outside an HTTP gateway. Non-HTTP transports do too. Device validation and clinical performance remain with the manufacturer's qualified functions. The same applies to CAPA and complaint assessment. Regulatory submissions remain there as well. This article owns the unauthorized request-path angle; it does not recast lifecycle governance or FDA's September 2026 GenAI discussion paper as a proxy problem.

DeepInspect

DeepInspect sits inline between authenticated medical-device applications or agents and HTTP-based LLM endpoints. The application supplies identity and approved workflow context. DeepInspect classifies the routed prompt and response. It evaluates role and destination against versioned policy, then either permits the traffic or blocks it through denial or redaction.

Each decision creates an identity-bound audit record for the managed request path. On-device inference and local models remain outside this boundary. So do browser bypass and embedded vendor inference. Non-HTTP traffic is also outside it. Device validation and clinical evidence remain with the manufacturer's qualified owners. Complaint conclusions and CAPA stay there too, as do regulatory decisions.

Book a technical deep dive at deepinspect.ai.