Every time I think I know what's going on, suddenly there's another layer of complications.
2024年1月17日星期三
2024年1月11日星期四
An interesting paper from Science on predictive models
"A central promise of artificial intelligence (AI) in healthcare is that large datasets can be mined to predict and identify the best course of care for future patients. Unfortunately, we do not know how these models would perform on new patients because they are rarely tested prospectively on truly independent patient samples. Chekroud et al. showed that machine learning models routinely achieve perfect performance in one dataset even when that dataset is a large international multisite clinical trial (see the Perspective by Petzschner). However, when that exact model was tested in truly independent clinical trials, performance fell to chance levels. Even when building what should be a more robust model by aggregating across a group of similar multisite trials, subsequent predictive performance remained poor. —Peter Stern"
Adam M. Chekroud et al. Illusory generalizability of clinical prediction models.Science383,164-167(2024).DOI:10.1126/science.adg8538