The history of medical diagnosis is a march through meticulous observation. Ancient Egyptian physicians first diagnosed urinary tract infections by observing patterns in patients' urine. To diagnose diseases of the heart and lungs, medieval physicians added key elements of the physical examination: pulse, palpation and percussion. The 20th century saw the addition of laboratory studies, and the 21st century of advanced imaging and genetics.
Despite the advances, however, diagnosis has remained largely a human endeavor, with doctors relying on so-called disease scripts: clusters of signs, symptoms, and diagnostic findings that characterize an illness. Medical students spend years memorizing such scripts, training themselves to identify, for example, the submillimeter variations in electrocardiogram waveforms that can alert them to a heart attack.
But of course people do digress. Sometimes a misdiagnosis occurs because a doctor misses something — when the disease patterns fit the script but the script is misread. This happens in an estimated 15% to 20% medical encounters. Other times, misdiagnosis occurs because the disease has features that don’t fit known patterns — they don’t fit the script, as when a heart attack occurs without obvious symptoms or ECG findings.
Artificial intelligence can help solve these two fundamental problems, if it has sufficient financial support and is deployed in the right way.
First, AI is less susceptible to common factors that lead doctors to make diagnostic errors: fatigue, lack of time and cognitive bandwidth in the treatment of many patientsholes of knowledge and trust in mental shortcutsEven when diseases conform to scripts, computers are sometimes better than humans at identifying details hidden in massive health care data.
Using AI to improve the accuracy and timeliness with which doctors can identify diseases could mean the difference between life and death. For example, ischemic stroke is a life-threatening emergency in which a blocked artery blocks blood flow to the brain. Brain scans confirm the diagnosis, but those scans must be performed and interpreted quickly and accurately by a radiologist. Studies show that AI, through superhuman pattern recognition skills, can identify strokes seconds after the scans are performed — tens of minutes rather than by often busy radiologists. Similar capabilities have been demonstrated in diagnosing blood poisoning, pneumoniablood clot in the lungs (pulmonary embolism), acute kidney injury and other circumstances.
Second, computers can be useful for diseases that we haven’t developed the right scripts for. AI can actually diagnose diseases using new patterns that are too subtle for humans to identify. Consider hypertrophic cardiomyopathy, a rare genetic condition in which the heart muscle has grown more than it should, eventually leading to heart failure and sometimes death. Experts estimate that only 20% of those affected are diagnoseda process that requires consultation with a cardiologist, an echocardiogram, and often genetic testing. What about the remaining 80%?
Researchers across the country, including at the Mayo Clinic And University of San Franciscohave shown that AI can detect complex, previously unrecognized patterns to identify patients likely to have hypertrophic cardiomyopathy. This means that AI-driven algorithms will be able to screen for the condition in routine ECGs.
AI was able to recognize these patterns after examining the ECGs of many people with and without the disease. The rapid growth of healthcare data — including detailed electronic health records, imaging, genomic data, biometrics, and behavioral data — combined with advances in artificial intelligence technology has created a major opportunity. Because of its unique ability to identify patterns from the data, AI has helped radiologists to find hidden cancerspathologists to characterize liver fibrosis and ophthalmologists to detect retinal disease.
One challenge is that AI is expensive and requires large-scale data to train computer algorithms and the technology to do so. As these tools become more ubiquitous, so may the associated intellectual property difficult to protect, discouraging private investment in these products. More generally, diagnostics have long been considered unattractive investments. Unlike their therapeutic counterparts, which see some $300 billion in research and development investment diagnostics receive a modest annual 10 billion dollars in private financing.
Then there is the question of who pays for the use of AI-based tools in medicine. Some applications, such as stroke detection, save insurers money (by preventing costly ICU stays and subsequent rehabilitation). These technologies tend to be reimbursed more quickly. But other AI solutions, such as detecting hypertrophic cardiomyopathy, can lead to higher spending on expensive follow-up therapies to treat newly identified chronic diseases. While the use of AI can improve the quality of care and long-term outcomes in such cases, reimbursement and therefore adoption may be slow without financial incentives for insurers.
Life sciences companies have on rare occasions agreed to to subsidize development or reimbursement of AI-based diagnostics. This will help close the gap, but the federal government may need to play a larger role. Federal support for COVID diagnostics During the pandemic, rapid development of crucial tests and the cancer moonshot project were stimulated has helped stimulate R&D in screening and new treatments.
It's usually hard to mobilize funding on the scale needed for new medical frontiers. But the National Academies of Medicine have estimated that tens of billions of dollars and countless lives could be saved by improving diagnostics in medicine.
Artificial intelligence offers a path to that end. It should complement, rather than replace, the human expertise that is already saving so many lives. The future of medical diagnosis doesn’t mean handing the keys to AI, but rather leveraging what it can do that we can’t. This could be a special moment for diagnosis, if we invest enough and get it right.
Gaurav Singal is a computer scientist and physician at Harvard Medical School And was previously chief data officer of Foundation Medicine, a cancer diagnostics company. Anupam B. Jena is an economist, physician, and professor at Harvard Medical School and co-author of “Random Acts in Medicine: The Hidden Forces That Influence Doctors, Impact Patients, and Shape Our Health” and the Random actions of medicine substack.
