A health system had acquired four community hospitals over six years. Each acquisition made strategic sense at the time. Together, they created a problem nobody had fully mapped until a clinical AI deployment made it impossible to ignore.
The product was a risk stratification model, designed to identify high-risk patients for proactive outreach before they deteriorated. The vendor had a strong track record. The model had been validated. The pilot at the flagship hospital had produced results the CMO had presented at a board meeting.
Then the team tried to roll it out across the full system.
What they found was that the four acquired hospitals had never been fully migrated onto the flagship’s EHR instance. Two were on a different version of the same platform. One was on a different platform entirely. A fourth had a hybrid setup that nobody could fully explain. Diagnosis codes were mapped differently across instances. Medication lists used different terminologies. Nursing documentation followed different templates in each facility. Social determinants data existed in one hospital and was almost entirely absent in the others.
The model had been trained on clean, well-governed data from the flagship. It was now being asked to run on four datasets that looked superficially similar but were structurally inconsistent in ways that took three months to fully catalogue. The rollout stalled. The vendor relationship became strained. The internal team that had championed the project spent the next two quarters doing data remediation work that nobody had scoped, budgeted, or anticipated.
The AI hadn’t failed. The data strategy that should have preceded it had never existed.
The Evolution Nobody Planned For
To understand how health systems end up in this position, it helps to trace how their data environments actually evolved, because almost none of them were designed. They accumulated.
Through the 1990s and early 2000s, most health systems digitized opportunistically. Different departments adopted different systems. Radiology had its own platform. The lab had its own. The pharmacy had its own. The EHR, when it arrived, was layered on top of this existing infrastructure rather than replacing it. Data lived in silos that were connected, when they were connected at all, through point-to-point interfaces that were configured, maintained, and occasionally forgotten by IT teams that turned over regularly.
Then came the acquisition wave. Health system consolidation accelerated through the 2010s, driven by reimbursement pressures, population health mandates, and the scale economics of value-based care. Each acquisition brought a new set of legacy systems, a new set of data models, and a new set of governance gaps. Full EHR migrations are expensive, disruptive, and politically complicated. Most acquired facilities operated on their original systems for years, sometimes indefinitely, with integration handled through whatever interface could be stood up quickly enough to satisfy the most urgent operational needs.
The result, in most large health systems today, is a data environment that reflects twenty years of opportunistic decisions rather than a coherent architecture. Multiple EHR instances. Inconsistent data models. Terminology mapping that was never standardized. Governance policies that vary by facility, by department, and sometimes by who was running IT at the time a particular system was configured.
This is the environment clinical AI products are being deployed into. And it is the environment most AI vendors, and many internal teams, have significantly underestimated.
The Technical Debt Nobody Budgeted For
Technical debt in software development refers to the accumulated cost of shortcuts taken earlier that have to be paid back later, usually at a higher price than if they had been done correctly the first time. Healthcare data has a version of this that is broader, messier, and more expensive than most enterprise leaders fully appreciate.
It is not just code debt. It is data debt: years of inconsistent documentation practices, incomplete data entry, terminology mismatches, and governance gaps that have compounded across systems, facilities, and acquisitions. And unlike software technical debt, which can sometimes be refactored in isolation, data debt touches every downstream use case simultaneously. You cannot clean up your medication data in one part of the system without affecting how medication-related analytics work across the entire organization.
For clinical AI specifically, this debt surfaces in three ways.
Training data that doesn’t represent production data. A model trained on data from one well-governed facility will behave unpredictably when deployed across facilities with different documentation practices, different coding conventions, and different rates of data completeness. The model isn’t wrong. It is operating outside the distribution it was built for.
Garbage in, confident predictions out. AI models do not flag uncertainty the way a clinician does. A clinician with incomplete information will say “I’m not sure, I need more data.” A model with incomplete or inconsistent input will often produce a confident-looking output anyway. In a risk stratification context, that means patients getting incorrectly stratified, care resources being allocated to the wrong populations, and clinical staff losing trust in the tool after the first few unexplained predictions.
Normalization as a perpetual prerequisite. Before a clinical AI product can work reliably across a multi-facility health system, the underlying data needs to be normalized: consistent terminology, consistent coding, consistent documentation structures, consistent completeness standards. That normalization work is unglamorous, expensive, and time-consuming. It is also almost never scoped into an AI deployment project, because the vendor assumes the data is cleaner than it is, and the health system assumes the vendor will handle it.
Red flag: If an AI vendor’s implementation plan doesn’t include a data quality assessment as a first step, they either haven’t deployed in a complex multi-facility environment before, or they are planning to discover the data problems after the contract is signed.
The Governance Paradox
Here is the tension that enterprise leaders rarely name directly: the organizations that most need AI to work, the large, complex, multi-facility health systems with the highest patient volumes and the most to gain from risk stratification and clinical decision support, are precisely the organizations whose data environments are least ready to support it reliably.
Smaller, more focused organizations with a single EHR instance, consistent governance, and clean data pipelines can deploy clinical AI and see results relatively quickly. The large integrated delivery networks, the ones with the budget and the strategic appetite for AI at scale, are carrying the most data debt and the most normalization complexity.
This creates a governance paradox. The health system needs AI to manage complexity. But the complexity of its data environment prevents AI from working reliably until governance is in place. And establishing governance across a fragmented, multi-instance, multi-facility data environment requires organizational will, budget, and time that competes directly with the pressure to show AI results quickly.
The teams that navigate this well are the ones that resist the pressure to skip the governance step. They treat data strategy as a prerequisite for AI strategy, not a parallel workstream. They invest in data normalization and quality infrastructure before they evaluate AI vendors. And they scope their AI deployments to start where the data is cleanest, build evidence, and expand as governance matures.
If you only remember one thing: an AI strategy built on top of a weak data strategy is not a strategy. It is a timeline to a failed deployment. Fix the foundation first.
The Structured Versus Unstructured Reality
There is a specific version of data debt that clinical AI teams consistently underestimate, and it connects directly to the narrative data argument from earlier in this series.
Structured data in a clinical record, diagnosis codes, lab values, vitals, medication lists, is relatively tractable. It can be normalized, mapped to standard terminologies, and fed into a model pipeline with manageable effort. It is incomplete and imperfect, but it is at least consistently shaped.
Unstructured data, clinical notes, care summaries, assessment and plan sections, nursing observations, is where the actual clinical reasoning lives. It is also where documentation practices vary most dramatically across facilities, departments, and individual clinicians. One physician writes three-sentence progress notes. Another writes detailed narrative assessments. A third copies forward from the previous encounter and edits minimally. Across a multi-facility health system, the variation in documentation culture can be as significant as the variation in data models.
For AI products that depend on natural language processing or that use clinical notes as a feature in their models, this variation is not a minor inconvenience. It is a core reliability problem. A model that performs well on the documentation style of one facility may degrade significantly at another where the same clinical concepts are expressed differently, documented less completely, or structured according to different templates.
This is not a problem that better algorithms solve. It is a problem that requires documentation standardization, governance, and in many cases, a realistic conversation about which facilities are actually ready to be included in an AI deployment and which ones need remediation work first.
What Good Looks Like in Five Years
The cautionary argument above is not an argument against clinical AI. It is an argument for sequencing it correctly. And the health systems that are doing the sequencing work now are building something that will be genuinely differentiated in five years.
What good looks like is not a single platform or a single vendor. It is a set of foundational capabilities that make AI deployable reliably and repeatedly across the organization.
A unified data layer. Not necessarily a single EHR, which may never be realistic for large integrated systems, but a governed data platform that normalizes data from multiple source systems into a consistent model. FHIR-based data lakes are becoming the most common architecture for this, with varying degrees of maturity. The health systems that have invested in this infrastructure are already seeing faster AI deployment cycles and more reliable model performance.
Terminology standardization. SNOMED, LOINC, RxNorm, and ICD-10 mappings that are consistently applied across facilities, with a governance process for maintaining them as source systems change. This sounds unglamorous. It is also the difference between a model that works across the system and one that works in one building.
Documentation governance. Structured note templates, documentation standards by specialty, and clinical informatics oversight that ensures the narrative layer of the record is consistent enough to be used reliably as model input. This requires physician engagement and change management, which is why most organizations haven’t done it. The ones that have are sitting on a significantly more valuable data asset.
A data quality program with teeth. Ongoing monitoring of data completeness, consistency, and accuracy, with feedback loops to the source systems and accountability for remediation. Not a one-time cleanup project but a continuous operational function.
AI governance infrastructure. Model validation processes, bias monitoring, clinical oversight frameworks, and clear policies on how models are updated and how performance is tracked post-deployment. The regulatory environment around clinical AI is evolving, and the organizations building governance infrastructure now will be ahead of requirements rather than scrambling to meet them.
The health systems that build these foundations over the next three to five years will not just deploy AI more reliably. They will be significantly more attractive partners for digital health vendors, more credible in value-based care arrangements, and more capable of generating the kind of longitudinal insights that drive meaningful clinical and operational improvement.
The ones that keep buying AI products without fixing the data environment underneath will keep having the same conversation: promising pilot, stalled deployment, expensive remediation, reset.
Before You Buy the Next AI Product
Whether you are an enterprise leader evaluating a clinical AI vendor or a founder trying to understand the environment you are selling into, these questions are worth having answered before a contract gets signed.
- Has a data quality assessment been completed for the specific facilities and data sources the AI product will depend on?
- Are the EHR instances and data models consistent enough across the deployment scope for the model to perform reliably, or does normalization work need to happen first?
- Is structured data alone sufficient for this use case, or does the product depend on narrative or unstructured data? If the latter, how consistent is documentation across the target facilities?
- Has the vendor deployed in a comparable multi-facility environment before, and can they share what the data remediation scope looked like?
- Is there a data governance owner on the health system side who has authority and budget to address data quality issues that surface during deployment?
- Is the AI deployment scoped to start where the data is cleanest, or is it attempting full system rollout from day one?
- Is there a model monitoring plan post-deployment, including performance tracking across facilities and a process for flagging degradation?
Closing
The promise of clinical AI is real. Risk stratification that catches deteriorating patients earlier. Decision support that reduces diagnostic error. Operational models that allocate resources more efficiently. None of that is hype. It is happening, in organizations that built the data foundation to support it.
What is also real is the graveyard of AI deployments that looked promising in a pilot and collapsed in production because the data environment underneath them was never ready. Not because the AI was bad. Because strategy did not precede tooling.
The argument I find myself making consistently, across conversations with founders, CTOs, and enterprise leaders, is the same one: the AI is not the hard part. The data is the hard part. And the governance of the data is harder still. The organizations that internalize this early and invest in the foundation before they buy the product are the ones that will have something real to show in five years.
Everyone else will still be doing data remediation and wondering why the pilot didn’t scale.
