AI Models in Healthcare Are Not Generalizable When Predicting Disease

Artificial intelligence (AI) in healthcare has made claims that large datasets can be mined to predict and identify diseases and the best course of care for future patients. Unfortunately, we do not know how these models would perform in terms of predicting disease and treatment on new patients because they are rarely tested prospectively on truly independent patient samples. AI models of schizophrenia have proven themselves no better than chance prediction.

Chekroud et al. (2024) have found that machine learning models routinely achieve perfect performance in one dataset even when that dataset is a large international multisite clinical trial. However, when that exact model was tested in truly independent clinical trials, performance fell to chance levels. Even when building what should be a more robust model by aggregating across a group of similar multisite trials, subsequent predictive performance remained poor. 

Reports that statistical models can improve decision-making related to medical treatments pervade the media. However, partly because of the cost and scarcity of medical outcomes data, these models are typically based on investigators observing a model’s success in one or two datasets or clinical contexts. When Checkroud et al (2024) scrutinized how well a machine learning model performed across several independent clinical trials of antipsychotic medication for schizophrenia, the models predicted patient outcomes with high accuracy within the trial in which the model was developed but performed no better than chance when applied out-of-sample. In an attempt to improve performance of the model, data were pooled across trials to predict outcomes but did not improve predictions. Their results suggest that models predicting treatment outcomes in schizophrenia are highly context-dependent and may have limited generalizability.

In other words, AI models can be good at “predicting” what’s already known (and that is progress), but if you want to predict something new in healthcare in terms of diagnosing and treating a condition, go out and buy yourself a coin to flip, it’s much cheaper than buying an AI program.