This is the third post in our series on AI in healthcare.
Read our previous posts, “Medicine 3.0: The Next Era of Healthcare Has Already Begun” and “Healthcare Has More Data Than Any Human Can Process. Here’s Where AI Comes In.”
Artificial intelligence has the potential to become one of the most powerful medical technologies ever developed. An AI system can process information at extraordinary speed, but speed is not the same as wisdom – and consensus is not always the same as truth. That makes an important question increasingly urgent – who decides what the AI believes? As physicians and patients increasingly depend upon artificial intelligence to navigate an expansive medical information environment, we must think seriously about what information is being used to train medical AI models and the quality, independence, privacy, and integrity of medical AI systems.
AI Can Inherit the Biases of Its Information Environment
When people hear the phrase “AI bias,” they often think about demographic bias involving race, sex, age, or socioeconomic characteristics. Those are legitimate concerns, but there is another form of bias that deserves equal attention: information bias.
Artificial intelligence learns from information created by human beings and human institutions. That information environment can contain publication bias, financial incentives, institutional preferences, incomplete evidence, political pressures, selective reporting, and prevailing scientific assumptions. If an AI system simply reproduces whatever viewpoint appears most frequently in its source material, it may become extremely effective at repeating conventional wisdom, without necessarily evaluating whether conventional wisdom is correct.
Medicine cannot afford that.
Science progresses because accepted ideas remain open to challenge. Many concepts physicians accept as obvious today were controversial when first proposed. Hence, a trustworthy medical AI should not simply tell physicians what most people believe, it should help them understand why most people believe it. What is the evidence? How strong is that evidence? What are the limitations? Are there credible competing interpretations? Which conclusions are well established? Which remain uncertain? These distinctions are essential.
Medicine Needs AI That Can Say “We Don’t Know”
One of the most dangerous tendencies in both human decision-making and artificial intelligence is false certainty. Medical science is frequently complicated – studies conflict, patient populations differ, risks and benefits change depending upon the individual, and evidence evolves. A responsible medical AI should be comfortable communicating uncertainty. Sometimes the most scientifically accurate answer is: The evidence is not yet strong enough to know. That is not weakness. It is intellectual integrity.
COVID-19 illustrated how difficult it can be when scientific questions collide with public policy, politics, media, and rapidly changing evidence. Questions involving treatments, masking, social distancing, vaccines, and the origin of SARS-CoV-2 became highly polarized. Competing interpretations were often discussed publicly with levels of certainty that exceeded what the available evidence could support at various points during the pandemic.
But the lesson is broader than any single COVID debate. Medical AI should be capable of showing the prevailing recommendation while also identifying the supporting evidence, meaningful contrary evidence, methodological limitations, and unresolved questions. Otherwise, it risks turning temporary consensus into permanent computational certainty.
Your Medical Data Deserve Special Protection
There is another issue that may be even more important to patients: privacy. Medical information is unlike almost any other type of data. A health record may contain information about genetics, medications, family history, mental health, reproductive health, behavioral health, previous illnesses, and predictions about future disease. Patients can now upload medical records, laboratory results, or deeply personal health histories into consumer AI platforms within seconds. Before doing so, they should understand an important distinction: a consumer technology company is not necessarily operating within the same privacy and legal framework as a physician, hospital, health plan, or HIPAA-regulated organization.
Before using AI to interpret medical data, patients should ask: Where is my information stored? Who can access it? How long will it be retained? Can it be used to improve an AI model? Can it be shared with another company? Can I delete it? What happens if the company is acquired? What exactly did I consent to when I uploaded my information? These questions will become increasingly important as healthcare and consumer technology converge.
Privacy is Also About Human Freedom
Medical privacy is frequently discussed as a cybersecurity issue. It is certainly that, but it is also something deeper – it is a question of human autonomy.
As data systems become capable of predicting increasingly personal aspects of human behavior and health, those systems must not be allowed to diminish individual freedom. Medical information should never become a mechanism for coercion, manipulation, discriminatory profiling, or opaque decision-making that affects a person’s opportunities without meaningful accountability. Technology should help people make better decisions, not quietly make important decisions for them.
The Human Person Must Remain at the Center
Artificial intelligence can be extraordinarily powerful, but its purpose in healthcare must remain clear: the patient does not exist to provide data for the algorithm, the algorithm exists to help the patient. That principle should shape how medical AI is designed, governed, and implemented.
At America’s HealthShare, we believe innovation and ethics should advance together. We should aggressively pursue technologies that can detect disease earlier, personalize treatment, reduce administrative burdens, and improve the patient experience. But innovation should come with strong expectations for privacy, transparency, clinical accountability, informed choice, and responsible stewardship of data. The central principle is simple:
Technology should serve the human person. The human person should never become a servant of the technology.
Artificial intelligence will undoubtedly become more capable. The challenge before us is ensuring that our wisdom grows alongside it.
In Health,
Dr. John Oertle
Chief Medical Officer