AI Intelligence Ecosystem: From Data Procurement to Data Curation

The most consequential misconception about AI in medicine is that there is a single thing called AI. By 2050 the physician's competence will rest on understanding that there is not, and on knowing where each kind of intelligence ends.

Exhibit 4.1 divides the ecosystem into four interoperating sectors, spanned by a band that names the current frontier forms: generative AI, agentic AI, physical AI, affective systems AI, and superintelligence.

Prediction and classification systems, built on machine learning and deep learning, excel where labeled data is dense and ground truth is verifiable. Language systems, built on large language models, handle summarization, drafting, and triage with a fluency that is not the same as comprehension. Robotic systems couple perception to physical action. Workflow and EHR systems operate ambiently and increasingly agentically, acting across multistage tasks inside the record itself.

Three modes of intelligence cut across those four sectors, and the physician's exposure differs in each. Augmented intelligence keeps a person in the loop by design, and is the mode most of medicine currently occupies. Artificial intelligence proper performs the task, with the human at the boundary rather than inside it. Emergent intelligence warrants the most attention, because capability arises from the interaction of systems at scale rather than from anything a designer specified.

Its near relative is annotation-free intelligence, in which systems learn without the curated human labelling that anchored earlier models to judgments a person had checked. Both point the same way. As capability detaches from human specification, what a system is optimizing for stops being answerable by reading its training data, and oversight must shift from auditing inputs to interrogating behavior. The boundary this report draws holds even at the limit. A system may exceed any physician at inference and still hold no view about what is worth wanting.

If there is no single AI, then the organizing concept becomes interoperability, the connection of many systems into a working whole across four planes: AI to AI, AI to robotics, AI to humans, and AI to workflows and health records. Clinical value depends less on any single model's accuracy than on whether models, data standards, and human workflows can exchange information without loss. A fragmented estate of brilliant but isolated tools will underperform a well-integrated whole.

Interoperability of that kind tends to separate the systems that recommend from the systems that act, since the two are governed differently, fail differently, and must be certified differently. Healthcare accordingly becomes modular, assembled from components that decide and components that execute, and the physician's work moves toward orchestration across that architecture: composing the modules for a given patient, setting the thresholds at which a case escalates to a person, and carrying the accountability that no module holds.

The constraint that governs the entire enterprise is data quality.

AI performance depends entirely on large, high-quality, validated data, yet much of medicine's data is poorly coordinated, shifts as procedures and equipment change, and is thinnest precisely where the most valuable cross-domain relationships lie. We may know that genetic factors influence a cancer's prognosis yet lack datasets pairing imaging with full genetic profiles, which are the uncommon relationships at which AI excels, and for which coordinated data does not yet exist.

The strategic error has been to treat this as a procurement problem: acquire more data, sign more agreements, buy more repositories. It is a curation problem. Volume without provenance, harmonization, and maintenance produces confident models with no defensible relationship to reality.

Curation is easier to demand than to locate, and locating it requires seeing that data is layered. Beneath any clinical recommendation sit raw data, processed data, modelled outputs, derivative datasets, and the conclusions built on top of those layers. Statistical validation enters at each transition, not only at the last.

Validation must therefore occur at multiple points: on preprocessing steps; on the statistical methods applied; on derivative datasets; and on downstream conclusions, and not solely on the final model output that happens to be visible. A system validated only where it is observable can be sound at the surface and unsound three layers down. AI need not be only a consumer of data in this arrangement. It can be turned on the statistical approaches themselves, checking its own methods as well as its outputs.

The failure modes at those lower layers are specific. Data drift erodes models trained on distributions that have since moved. Representativeness is frequently weak in ways the dataset does not disclose. And a feedback loop opens once AI-generated output is used to train subsequent systems, producing loss of variance and amplification of error that terminates, at the limit, in model collapse, a system trained increasingly on its own reflection.

The concrete cases make the point better than the abstractions. Contaminated genomic sequences carrying Epstein-Barr virus fragments propagated through the literature and remained embedded in the scientific record, so that flawed source data poisoned every derivative result built upon it. Errors of that kind do not announce themselves, and they are not detectable from the output.

This points toward an institutional remedy rather than a technical one, and of everything the working group considered on data, this proposal carries the most weight. What is wanted is a monitoring schema, certification apparatus, or trusted organizational function that validates or sanitizes biomedical data and derivative analytics before they are used broadly. In effect it would be a seal of good housekeeping for datasets, statistical methods, and downstream analytic products, or, in the plainest formulation, an FDA-like function for health data and data analytics.

The objection is obvious and should be conceded: such an apparatus would be a major undertaking and would slow things down considerably. The answer is that the present arrangement, in which no body certifies the substrate on which clinical AI rests, cannot support trustworthy biomedical AI at scale, and the choice is therefore between a deliberate friction and an accumulating one.

The proposal needs a sponsor, a standard, and a reason for the profession to hold the pen. No single body has the standing to convene it, and the plausible route is a consortium of specialty societies, academic health systems, and the certifying boards, formed with regulators present though not in charge, on the model Chapter XII describes for negotiating as a bloc.

What would be certified is provenance and maintenance, not accuracy: where the data came from; which population it represents; what transformations have been applied at each layer; when it was last revalidated; and what is known to have drifted since. And the reason for the profession to hold the pen is the one this report keeps returning to. Industry cannot certify itself, the state moves on a legislative clock, and the physician is the party who must act on the output and answer for the result.

Radiology supplies the sharpest version of the caution, being where medical AI is furthest advanced. Daiju Ueda and colleagues at Osaka Metropolitan University have cataloged what goes wrong, and their findings generalize well beyond imaging. Three constraints recur.

The base-rate problem: a model performing impressively on a balanced test set will generate a crushing burden of false positives when deployed against a low-prevalence population, because predictive value follows prevalence rather than accuracy. Limited generalizability and bias: models validated internally degrade on external data, with specificity falling furthest, and they carry forward whatever inequities are latent in the populations and institutions that produced their training data. And practical barriers of workflow, regulation, and trust, which decide whether a technically sound tool is ever used well.

Ueda's group has pressed the fairness question hardest. Bias in clinical AI, they argue, is a predictable consequence of how medical data are generated, not an incidental defect to be patched, and mitigating it requires cooperation among clinicians, developers, and regulators in place of a technical fix applied at the end. Their more recent work extends the argument to alignment: as systems grow more capable, the danger shifts from errors of accuracy to optimization against the wrong objective, with a model tuned to a proxy that diverges from what the patient actually needs.

The remedies they propose are the ones this report keeps arriving at from other directions: external validation on diverse cohorts before deployment; uncertainty-aware designs that refer the ambiguous case to a human instead of guessing; federated approaches that let models learn across institutions without extracting their data; and prospective trials that measure outcomes rather than concordance. The ecosystem is only as good as its data, and only as safe as the judgment surrounding it.

Who does the evaluating, and when, is a question the profession has largely left to procurement. Evaluation should run the length of a system's working life instead of clustering at the moment of purchase. It should seat practicing clinicians, nurses and pharmacists, patients, data scientists, ethicists, and the institution's own safety function at the same table from selection through decommissioning.

Four things belong on the standing schedule: performance against prospectively defined clinical endpoints; bias analysis repeated as the served population shifts; measured effect on the efficiency of physician workflow and on the quality of care delivered; and measured effect on the working lives of the clinicians using it. The last is routinely asserted and almost never quantified, which is how a system that saves ten minutes of documentation and costs twenty minutes of vigilance comes to be counted as progress.

One further constraint lies outside the clinic and cannot be delegated away from it. The same generative capabilities that design a protein for therapeutic purpose can design one for harm, and the National Academies' 2025 assessment of AI-enabled biological tools concludes both that the risk is real and that the same tools materially strengthen biodefense, by interpreting environmental pathogen data and accelerating countermeasure development, so that monitoring and mitigation are required on both counts.³ Physicians are not the primary governors of that boundary. They are, however, among the few practitioners who will meet both sides of it. Dual-use literacy therefore belongs in clinical formation, not in national policy alone.

Both constraints imply a single competence, narrower than technical fluency and more demanding than familiarity. The physician must be able to interrogate a system without inspecting its internals. That means asking where its data came from and whom they represent; what objective it was actually optimized against, as distinct from the one it is described as serving; and where its failure envelope lies, meaning the conditions under which its performance is known to degrade and the cases it should refuse. Those three questions can be put to a vendor, to a colleague, or to a model card, and none requires reading code. Data-aware medicine, as this report uses the term, is the disposition and the standing to ask them, which is what Chapter XI must find a way to teach.

A closing observation about the ecosystem concerns what it rests on, which is rarely counted in clinical discussions of AI. These systems require data pipelines, validation processes, oversight, and, more prosaically, computers, electricity, and money, and the question of who can build, maintain, govern, and evaluate such capability over time is not settled by any clinical argument.

The consequence for the physician is a form of dependency with no real precedent in the profession's history. Clinical judgment becomes contingent on system design, institutional access, and implementation decisions taken elsewhere, and if healthcare organizes around a small number of large-scale platforms, the implications for access, dependency, and control will be felt first by the clinicians whose practice those platforms mediate. A physician who cannot see the infrastructure is not thereby independent of it.