HIPAA Business Associate Agreements for AI Vendors: The Clauses That Actually Matter When PHI Reaches an LLM

The HIPAA Privacy Rule requires covered entities to have a Business Associate Agreement with any vendor that creates, receives, maintains, or transmits protected health information on their behalf. Every major LLM provider (OpenAI, Anthropic, AWS Bedrock, Google Cloud, Azure OpenAI) now offers a HIPAA-eligible tier with a BAA. A signed BAA does not, on its own, make an AI deployment HIPAA-compliant. The BAA covers the vendor relationship. The deployer still owns the administrative, physical, and technical safeguards under 45 CFR 164.308, 164.312, and 164.316. I want to walk through the BAA clauses that matter when PHI actually flows into an LLM, and the deployer-side safeguards the BAA does not discharge.

The BAA is the paperwork. The safeguards are the architecture.

What a BAA discharges and what it does not

A BAA discharges the covered entity's obligation to obtain satisfactory assurances that the business associate will safeguard PHI. It flows the HIPAA Security Rule requirements onto the vendor, allocates breach notification duties, and permits the covered entity to share PHI with the vendor without a separate patient authorization. That is the whole scope.

What a BAA does not discharge is the covered entity's own safeguards. The Security Rule requires administrative, physical, and technical safeguards on the deployer's side regardless of what the vendor implements. Section 164.312 lists the technical safeguards: access control (unique user identification, emergency access, automatic logoff, encryption), audit controls, integrity, person or entity authentication, and transmission security. Each safeguard applies to the deployer's environment where PHI is created and where the call to the vendor originates.

The training-on-PHI clause

Every AI-vendor BAA has to answer one question that a conventional cloud-vendor BAA never had to: does the vendor use the deployer's PHI to train models. The default in every major provider's HIPAA tier is no. The clause that codifies the default has to be explicit.

OpenAI's Enterprise agreement excludes API traffic from training. Anthropic's commercial terms exclude API traffic from training. AWS Bedrock, Google Vertex AI, and Azure OpenAI each maintain HIPAA-eligible tiers with training exclusion. The BAA should reference the specific service tier and the training exclusion clause by section number. A BAA that references only "the AI service" without specifying the tier can permit the vendor to move workloads between training and non-training tiers without breach.

The subprocessor question follows. The vendor's LLM inference may run on a third-party GPU cloud. The BAA should require flow-down of HIPAA obligations to subprocessors and give the deployer visibility into subprocessor changes. The source library has the current text from major providers' HIPAA supplements.

The audit trail the BAA cannot supply

The Security Rule's audit control requirement (164.312(b)) requires the covered entity to implement hardware, software, and procedural mechanisms that record and examine activity in information systems that contain or use electronic PHI. The BAA does not discharge this. The deployer owns the audit trail of PHI activity on the deployer's side.

Two failure modes reoccur when LLMs enter production. First, the deployer relies on the vendor's audit log as the primary record. The vendor's log shows the calling service account, not the natural person behind the calling application. The clinician who prompted the model, the assistant who executed the workflow, the developer who tested the endpoint all appear as the same service account. The ai audit logs format spec covers the fields the deployer's own record must include.

Second, the deployer's application logs record that a call happened but not the classification of data in the prompt. The record shows "clinical note summarization request" but not that the note contained the patient's diagnosis, medications, and social history. Without prompt-level data classification captured at the moment of the request, the deployer's audit trail fails the reconstructability test the OCR investigation guide applies.

Accounting of disclosures at the AI boundary

Section 164.528 gives patients the right to an accounting of disclosures of their PHI over the preceding six years. Disclosures for treatment, payment, and healthcare operations are exempt from the accounting. Disclosures to a business associate to perform a function on behalf of the covered entity fall under the operations exemption.

The exemption reduces the accounting burden but does not eliminate the underlying evidence requirement. When a patient files an OCR complaint alleging improper disclosure, the covered entity has to produce the record of what data was disclosed to which vendor for which purpose. If the AI vendor received a prompt containing the patient's PHI as part of a clinical decision support workflow, the covered entity has to produce that record. The HIPAA PHI redaction piece covers the redaction pattern that reduces the disclosure surface.

Colorado SB 26-189 and the operations exemption erosion

On May 14, 2026, Colorado Governor Polis signed SB 26-189, which scaled back the Colorado AI Act's HIPAA covered-entity carve-out. Effective January 1, 2027, the "consequential decision" test applies to clinical AI even when HIPAA authorizes the underlying disclosure. The BAA does not shield the deployer from Colorado AG enforcement under SB 26-189, which means the deployer's audit trail has to satisfy both HIPAA and the Colorado statute. The Tennessee AI chatbot law piece covers the parallel state trend in mental-health AI.

DeepInspect

This is exactly what DeepInspect does. DeepInspect sits inline between the clinical applications, assistants, and agents in the deployer's environment and the LLM APIs they call. Every request binds to a verified identity claim. Every prompt is inspected for PHI classification at the boundary. Every response passes back through the same layer. The audit record includes the clinician or agent identity, the data classification, the policy decision, the model that responded, and the cryptographic evidence of integrity.

The proxy architecture is stateless. Vendor credentials stay in the credential store. The deployer's audit log lands in an append-only store the vendor never touches, which is the record OCR asks for during an investigation.

If you are running clinical AI in production, let's talk today.