HIPAA and AI: What Healthcare Organizations Need to Govern Beyond Data Privacy

Key Takeaway

HIPAA creates three governance obligations that AI deployments trigger and that most compliance programs have not yet addressed: risk analyses must now explicitly include AI systems as a distinct category, Business Associate Agreements with AI vendors must address model training data use, output retention, and AI-generated PHI, and the minimum necessary standard applies to what AI systems access, not just what they produce. These are gaps within HIPAA compliance. But HIPAA-compliant AI is not well-governed AI. HIPAA addresses data handling. It does not address oversight design, decision rights, escalation authority, or what happens when an AI system causes harm without involving a privacy breach. That governance infrastructure must come from somewhere else.

The Assumption That Creates Exposure

The assumption is widespread and understandable: your organization has a HIPAA compliance program, your compliance team has reviewed your AI tools, your vendors have signed Business Associate Agreements. AI is governed. The assumption is wrong, not because HIPAA compliance is unimportant, but because it addresses a specific set of obligations that do not cover most of what makes AI risky in a clinical environment.

HIPAA governs the privacy and security of protected health information. It was enacted in 1996 and last significantly updated in 2013. Its risk analysis requirements, its BAA provisions, and its minimum necessary standard were designed for a world where the primary data risk was unauthorized human access to patient records. They were not designed for AI systems that generate PHI as outputs, that process entire EHR datasets to improve model performance, or that make clinical recommendations with no human meaningfully in the review loop.

The result is that most healthcare organizations have two distinct problems: HIPAA obligations that AI deployments have created but that their compliance programs have not yet addressed, and governance obligations that AI creates entirely outside HIPAA's scope. Both carry risk. This post addresses the first. The second determines whether the first is sufficient.

Three HIPAA Obligations That AI Deployments Trigger, and That Most Programs Have Not Addressed

HHS's January 2025 proposed update to the HIPAA Security Rule made explicit what OCR had been signaling for some time: AI systems that create, receive, maintain, or transmit electronic PHI must be included in organizations' technology asset inventories, and risk analyses must address those systems as a distinct category. This is not a new obligation in principle; it is an existing obligation that most organizations have not applied to their AI deployments.

The three compliance gaps below are the ones most commonly uncovered when we work with health systems on AI governance readiness. Each represents a HIPAA compliance failure, not just a governance gap.

Risk Analysis: Why Your Existing Assessment Likely Has an AI Gap

HIPAA Security Rule — 45 CFR 164.308(a)(1)

Risk Analysis: The AI Gap

The HIPAA Security Rule requires covered entities and business associates to conduct an accurate and thorough assessment of the potential risks and vulnerabilities to the confidentiality, integrity, and availability of ePHI. OCR's enforcement position, reinforced in the 2025 proposed rule, is that AI systems handling ePHI must be assessed as part of this requirement.

The practical problem is timing. Most healthcare organizations updated their risk analyses in the 2015-to-2020 period following the 2013 Omnibus Rule. AI deployments in those organizations began accelerating in 2022 and 2023. The risk analyses predate the AI systems they are supposed to cover by several years.

An organization whose risk analysis was last updated before its AI deployments began has a documented HIPAA compliance gap. The AI systems processing ePHI are not assessed. The vulnerabilities specific to those systems — including model inference attacks, prompt injection risks, PHI leakage to hosted model providers, and AI-generated PHI retention — are not addressed. OCR's guidance is clear that this is not acceptable.

In practice: A risk analysis update that treats AI systems as a distinct category, identifies ePHI touchpoints specific to each AI deployment, and documents the controls in place for AI-specific risk vectors. This is not a checkbox addition to an existing risk analysis; it requires the participation of technical teams who understand how the AI systems actually handle data.

Business Associate Agreements and AI Vendors: What Standard BAAs Don't Cover

HIPAA Privacy Rule — 45 CFR 164.308(b)

BAAs and AI Vendors: The Provisions That Are Missing

A Business Associate Agreement is required whenever a covered entity discloses PHI to a vendor who performs services on its behalf. Most health systems have BAA programs. The gap is not in whether BAAs exist; it is in whether the BAAs they have signed address the specific ways AI vendors handle PHI.

Standard BAA templates were written for vendors who store, process, or transmit existing PHI. They were not written for vendors whose AI systems may use that PHI for model training, generate new PHI as outputs, retain conversational context containing PHI between sessions, or transmit PHI to third-party AI providers as part of their service architecture.

Three provisions are absent from most AI vendor BAAs that OCR's guidance on AI and emerging technologies now signals as necessary: an explicit prohibition on using customer PHI to train or fine-tune AI models without separate authorization; clear retention and deletion terms for AI-generated outputs that contain PHI; and disclosure requirements when the vendor's AI system itself relies on a third-party model provider, creating a downstream BAA obligation.

For organizations deploying agentic AI systems, the BAA gap is larger still. An agentic AI that retains memory across sessions, calls external tools, and generates outputs that influence clinical decisions is not adequately covered by a standard BAA that describes PHI handling in static storage and transmission terms.

In practice: A BAA audit against current AI vendor relationships, a revised BAA template that addresses model training data use, AI-generated PHI, output retention, and downstream AI provider disclosure, and a vendor onboarding process that treats BAA adequacy as a procurement gate, not a post-signature review.

Has your AI governance posture been assessed against current HIPAA expectations?

Health-Vision.AI's AI Readiness & Maturity Assessment evaluates your governance infrastructure across eight dimensions, including data integrity, access controls, and vendor governance.

Start the Assessment

The Minimum Necessary Standard Applied to AI Access

HIPAA Privacy Rule — 45 CFR 164.502(b)

Minimum Necessary: AI Access Scope as a Compliance Question

The minimum necessary standard requires covered entities to make reasonable efforts to limit PHI use and disclosure to the minimum necessary to accomplish the intended purpose. For human data access, this principle is well understood and commonly implemented through role-based access controls.

For AI systems, the application of the minimum necessary standard is rarely evaluated at deployment. The commercial pressure runs in the opposite direction: broader EHR access produces better model performance, and AI vendors frequently request broad dataset access to improve the quality of their recommendations. Organizations grant that access because better model performance serves legitimate clinical purposes.

The HIPAA problem is that clinical utility does not override the minimum necessary standard. An AI system that accesses a patient's full medication history, social determinants of health data, and historical encounter records to generate a care management recommendation may be accessing substantially more PHI than the minimum necessary for that specific purpose. If the broader access is not justified by the specific intended use, it is a compliance exposure regardless of whether the AI's output is appropriate.

In practice: A data access review for each AI deployment that documents the PHI categories the system accesses, the clinical purpose those categories serve, and a documented minimum necessary determination. For AI systems that require broad access to function, the determination must be documented, not assumed.

"AI systems that access broad EHR data to produce better clinical recommendations may violate the minimum necessary standard, even if their outputs are appropriate. Access scope is a HIPAA compliance question that AI deployment decisions rarely evaluate."

What HIPAA Doesn't Govern, and What Has to Come After

Addressing the three gaps above brings an organization's AI deployment into HIPAA compliance. It does not make that AI deployment well-governed. The distinction matters because the failure modes that most frequently cause patient harm, institutional liability, and regulatory exposure in clinical AI settings are not HIPAA violations.

HIPAA does not require organizations to define who owns accountability when an AI system produces a harmful output. It does not require documentation of escalation paths or stop conditions. It does not address oversight design: whether the human reviewer of an AI recommendation has the criteria, time, and authority to meaningfully evaluate it. It does not require continuous performance monitoring of AI systems in production. And it does not address decision rights: who has the authority to pause or retire an AI deployment when something goes wrong.

These are the governance infrastructure gaps that produce the four failure patterns we document in our work: invisible decisions, nominal oversight, diffused accountability, and retrofitted controls. None of them are addressed by HIPAA. All of them are addressed by AI governance infrastructure.

The regulatory frameworks that do address these dimensions — the NIST AI Risk Management Framework and the EU AI Act's Article 14 human oversight requirements among them — were designed specifically because HIPAA and similar data privacy regulations were recognized as insufficient for governing AI systems in high-stakes decision environments. HIPAA compliance is the required starting point. It is not the destination.

Governance Principle

HIPAA compliance is necessary but not sufficient. An organization that has addressed the three HIPAA gaps above and has no AI governance infrastructure beyond those obligations has satisfied its data privacy obligations. It has not addressed the governance conditions under which its AI systems operate, which is where the patient safety risk and institutional liability actually live.

Key Takeaways

AI deployments trigger three specific HIPAA obligations that most compliance programs have not addressed: risk analysis gaps, BAA inadequacy, and minimum necessary standard violations from broad AI data access.
Risk analyses must now explicitly address AI systems as a distinct category. An organization whose risk analysis predates its AI deployments has a documented compliance gap under current OCR guidance.
Standard BAA templates were not designed for AI vendors. They do not address model training data use, AI-generated PHI retention, or downstream AI provider disclosure requirements. BAAs signed before generative AI deployments are likely inadequate.
The minimum necessary standard applies to what AI systems access, not just what they produce. Broad EHR access that improves model performance but exceeds what is necessary for the specific clinical purpose is a compliance exposure regardless of output quality.
HIPAA-compliant AI is not well-governed AI. HIPAA addresses data handling. It does not address oversight design, decision rights, escalation authority, or performance monitoring. Those requirements come from AI governance infrastructure, not from HIPAA.
HIPAA compliance is the floor. The governance infrastructure that limits patient safety risk and institutional liability sits above it, in territory that requires deliberate design.