These questions come from compliance teams I talk to every month. They're the ones handed an AI pilot project last quarter and are now trying to figure out what HIPAA actually requires when your vendor's large language model is ingesting patient notes. They're not asking theoretical questions. They're asking: "Do I need to file a breach report?" and "What do I tell the auditor?"
Here's what they want to know.
Q1: If we're using AI to process patient data, do we need to map every single data flow?
Yes, and you probably should've done this already.
A data flow inventory isn't AI-specific; it's a foundational HIPAA Security Rule practice. AI implementations expose how many organizations have been running on incomplete data maps for years.
Your inventory needs to document:
- How you're ingesting data (APIs, flat files, direct EHR integration)
- Where processing happens (on-premises, vendor cloud, hybrid)
- Storage locations and retention periods
- Which vendors touch the data and what they do with it
- How data exits your systems (reports, dashboards, third-party analytics)
If you don't know where protected health information lives and who's accessing it, you can't secure it. That's not an AI problem; it's a fundamental safeguards problem. But AI projects force you to confront gaps you've been deferring.
Start with your AI pilot. Map that data flow completely. Then use that exercise as a template to backfill your existing systems.
Q2: Our vendor says their AI model "learns" from our patient data. Is that a permissible use under HIPAA?
It depends on what "learns" means, and this is where you need specifics in your Business Associate Agreement (BAA).
If the vendor is using your protected health information to improve their general model, training it on your patients' data to make the product better for all their customers, that's likely outside HIPAA's permissible uses unless you have explicit patient authorization or a waiver from an Institutional Review Board for research.
HIPAA permits use of protected health information for treatment, payment, and healthcare operations. Model training for the vendor's commercial benefit doesn't fit neatly into those categories.
Your BAA should specify:
- Whether the vendor can use your data for model training
- If so, what de-identification standard applies (Safe Harbor, Expert Determination)
- Whether you retain any rights to audit training datasets
- How the vendor handles model outputs that might contain patient information
If your vendor can't answer these questions clearly, you're looking at a potential breach notification scenario if protected health information ends up in their training corpus without proper authorization.
Q3: We're three months into an AI project and just realized we never documented the data flows. What do we do?
Document them now, then assess whether you have a reportable breach.
Here's your immediate action plan:
First, create the data flow diagram you should've had at kickoff. Work with your vendor and your technical team to map exactly what data has moved where.
Second, compare what actually happened against what your BAA permits. If the vendor accessed or used protected health information outside the scope of your agreement, that's an impermissible disclosure.
Third, apply the HIPAA Breach Notification Rule's risk assessment framework. You need to evaluate:
- The nature and extent of the protected health information involved
- Who accessed or received it
- Whether it was actually acquired or viewed
- The extent to which risk has been mitigated
If your risk assessment concludes there's a low probability that the protected health information has been compromised, you don't have a reportable breach. But document that assessment thoroughly, your OCR auditor will want to see it.
If you can't demonstrate low probability of compromise, you're looking at breach notification within 60 days of discovery.
Q4: Our AI vendor wants to store patient data in their cloud. What should we be asking them?
Start with the same questions you'd ask any Business Associate, then add AI-specific layers.
Standard BAA requirements:
- Where is data stored geographically?
- What encryption standards apply (at rest and in transit)?
- Who has access to the data and under what access controls?
- What's their incident response process?
- How do they handle data deletion requests?
AI-specific additions:
- Is patient data used for model training, or only for inference?
- If used for training, what de-identification methods apply?
- How do they prevent model outputs from revealing protected health information?
- What monitoring exists to detect data leakage through model behavior?
- How do they handle model updates, does retraining create new data processing risks?
You also need technical safeguards documentation. Under the HIPAA Security Rule, you're required to implement technical safeguards that are reasonable and appropriate. That means you need to understand your vendor's architecture well enough to determine whether their controls meet that standard.
If your vendor can't explain their data handling in these terms, they're not ready to be your Business Associate.
Q5: How do we handle patient consent for AI processing?
HIPAA doesn't require patient consent for treatment, payment, or healthcare operations, but your state law might, and your risk tolerance should factor in.
From a HIPAA compliance perspective, if you're using AI within the scope of treatment or healthcare operations (clinical decision support, administrative efficiency), you don't need separate consent beyond your standard Notice of Privacy Practices.
But consider:
Many states are passing AI-specific healthcare laws. Some require disclosure when AI is used in clinical decisions. Some require opt-out mechanisms. Track your state's legislative activity.
Your Notice of Privacy Practices should describe how you use technology, including AI, to deliver care. If you're implementing AI after your last notice, you may need to update it.
Patient trust matters. Even when consent isn't legally required, transparency about AI use can prevent complaints and potential OCR investigations. Consider adding a plain-language AI disclosure to your intake process.
If your AI application involves Automated Individual Decision-Making and Profiling under the General Data Protection Regulation (for any EU patient data), you have additional disclosure and opt-out obligations.
Q6: What's the biggest mistake you see organizations make with AI and HIPAA?
Treating AI as a technology problem instead of a data governance problem.
Your technical team will focus on model accuracy, latency, and integration. That's important. But from a compliance perspective, AI is fundamentally about data, what data you're using, how you're using it, who has access to it, and whether that use is permissible under your existing legal framework.
The organizations that handle AI implementations well are the ones who involve their privacy and compliance teams from day one of the pilot. They map data flows before the first API call. They negotiate BAA terms before they share test datasets. They document their permissible use analysis before they flip the switch.
The organizations that end up filing breach reports are the ones who let the AI project run for six months before asking their privacy officer to take a look.
AI doesn't change your HIPAA obligations. It just makes the consequences of ignoring them much more expensive.
Where to go for more
Start with HHS Office for Civil Rights guidance on AI and HIPAA, they've published several fact sheets addressing common scenarios.
Review your existing data flow documentation (or create it if you don't have it). The NIST Cybersecurity Framework (CSF) 2.0 Govern function provides a solid structure for data inventory practices.
If you're implementing agentic AI specifically, pay attention to how autonomous decision-making affects your permissible use analysis. These systems often make data access decisions dynamically, which means your data flow isn't static, it's a set of rules that need governance.
And talk to your legal counsel before you sign your next AI vendor contract. The BAA terms you negotiate now will determine whether your next OCR audit is routine or catastrophic.
