Table of contents
Getting a vendor to sign a Business Associate Agreement has become easier than most compliance officers expected. Major cloud providers sign them. A growing number of AI vendors will sign them too.
The problem is that a signed BAA is not the same as a compliant deployment.
A BAA is a contract. It defines scope - which systems, which data flows, which activities fall under the covered arrangement. Outside that scope, the BAA is silent. And most AI BAAs are drafted with a scope that is narrower than the way teams actually use the tool.
Why does the BAA’s scope matter more than the signature?
A Business Associate Agreement covers the services described in it - not the vendor’s entire product.[1] If your AI vendor’s BAA covers “cloud infrastructure services” but your PHI flows through a language model API that sits outside that scope, you have a signed BAA and an uncovered data flow.
This happens more often than it should, for a simple reason: AI vendors are under pressure to offer compliance quickly. The path of least resistance is to extend an existing cloud BAA to include the new AI product - even when the AI product introduces materially different PHI handling risks that the original BAA language never contemplated.
When you look at a BAA, the relevant questions are not “did they sign it?” but:
- Which services are explicitly named?
- Is the AI inference path in scope?
- Is PHI processing in scope, or just storage?
- Who is the actual sub-processor handling the LLM calls?
That last question matters because the AI vendor you contracted with may not be the entity handling your data at inference time.
What happens to your data between your app and the model?
The typical AI API call moves through more infrastructure than most buyers realize. Understanding exactly where your data flows matters more than the signature at the top of the agreement.
Your application sends a request to the AI vendor’s API. The vendor routes it to one or more inference providers. The inference provider processes the prompt and returns a response. Along the way, the data may pass through edge infrastructure, CDN nodes, and logging systems.
Each one of these hops is a potential sub-processor. Your BAA needs to cover them, or the vendor needs to hold them under a separate BAA that your arrangement relies on.
The HIPAA Security Rule’s requirements for transmission security apply to every segment of the data path - not just the segment between your application and the first API endpoint.[2] If a sub-processor in the chain isn’t covered, the chain is broken.
The impact of data breaches at the vendor level has reached record highs. In 2024, business associate breaches accounted for 85% of all individuals affected by healthcare data incidents — exposing more than 200 million records in a single year.[3]
This doesn’t mean you can’t use AI. It means the BAA needs to accurately describe the data path, not just the vendor you’re contracting with directly.
What does training data policy have to do with HIPAA?
Some AI vendors use API traffic to improve their models. Most enterprise-tier agreements opt out of this. But “opting out” is not always the default - you have to read the usage policy and make sure the data processing agreement matches the privacy policy.
HIPAA’s minimum necessary standard requires that PHI be used only for the minimum amount necessary to accomplish the intended purpose.[1] If your AI vendor uses PHI-containing API calls to improve its model - even in a de-identified form - that use may fall outside the scope of treatment, payment, or healthcare operations, and therefore outside the coverage of a standard BAA.
The question to ask is not “do you train on my data?” but “what happens to my data after inference?” The answer should be documented in the data processing agreement, not buried in a support FAQ.
What about de-identification - does it get you out of HIPAA?
De-identification is a legitimate HIPAA compliance path. If PHI is properly de-identified under the Safe Harbor or Expert Determination method, HIPAA’s rules no longer apply to that data.[4]
The practical issue is that de-identification is harder than it looks. The Safe Harbor method requires removing 18 specific identifiers, including dates of service, geographic data below the state level, and any other “unique identifying number, characteristic, or code.” Clinical notes often contain combinations of quasi-identifiers that don’t appear on the list but could still re-identify a patient in context.
Expert Determination requires a qualified statistician to certify that the risk of re-identification is very small. That’s a meaningful bar. “We removed the name” is not Expert Determination.
If your AI deployment relies on de-identification to avoid HIPAA coverage, the de-identification should be done before data reaches the model - not at the storage layer afterward. And the method should be documented, auditable, and reviewed by someone with relevant expertise.
What should an AI audit trail look like under HIPAA?
HIPAA’s Audit Controls standard (45 CFR 164.312(b)) requires covered entities and their business associates to implement hardware, software, and procedural mechanisms to record and examine activity in information systems containing PHI.[2]
For an AI system, that means at minimum:
- A record of who submitted what prompt
- A record of what the model returned
- A timestamp tied to a reliable time source
- A mechanism to verify the record hasn’t been altered
Most AI vendors log API traffic for operational purposes. Very few produce audit logs that meet the integrity requirements a healthcare compliance officer would expect. Logs that can be modified, deleted, or accessed without controls do not satisfy the audit controls requirement - even if they capture the right events.
The distinction worth drawing is between operational logging (useful for debugging) and compliance-grade audit records (admissible as evidence, tamper-evident, retained under the covered entity’s record retention policy).
The practical checklist
Before you call your AI deployment HIPAA-compliant, verify four things:
- Scope: The BAA explicitly names the AI inference service, not just the cloud infrastructure it runs on.
- Sub-processors: The vendor has BAAs with every entity in the data path that handles PHI: the inference provider, any intermediate routing layer, any logging provider.
- Data use: The data processing agreement prohibits use of PHI for model training or improvement, and this prohibition applies to de-identified derivatives as well as raw data.
- Audit trail: The system produces tamper-evident logs with timestamps that can be independently verified, retained under your record retention policy.
These four checks are not a one-time exercise. A BAA’s scope is only accurate on the day it is signed. Vendors add inference providers, change how prompts are routed, and ship new product tiers — and any of those can move PHI onto a path the original agreement never named. Re-verify scope whenever a vendor announces a material change to how its AI service works, and treat a newly added sub-processor the same way you would treat a brand-new vendor: as something that needs coverage before any PHI reaches it.
A BAA signature is necessary. It is not sufficient. The gap between the two is where most AI compliance failures happen. BAA-included AI breaks down what a BAA has to cover before a deployment is compliant, and the HASP trust page documents every sub-processor and what each one handles.
Sources
U.S. Department of Health & Human Services. “Minimum Necessary Requirement.” HHS.gov.
hhs.gov
U.S. Department of Health & Human Services. “Security Rule Guidance Material.” HHS.gov.
hhs.gov
HIPAA Journal. “2024 Healthcare Data Breach Report” (2025). Analysis of HHS OCR breach portal data.
hipaajournal.com
U.S. Department of Health & Human Services. “Guidance on De-identification of Protected Health Information.” HHS.gov.
hhs.gov