This AI study is a fun experiment claiming to replicate the clinical gestalt generated by a physician’s initial synthesis of visual information. The ability to rapidly assess the stability and acuity of a patient is part of every experienced clinician’s refined skills – and used as a pre-test anchor for application of further diagnostic and management reasoning.
So, can AI do the same thing?
Well, “yes” and “of course not”.
In this demonstration project, these authors set up a mobile phone video camera at the foot of patients’ beds in the emergency department. Patients were instructed to perform a series of simple tasks (touch your nose, answer questions, etc.) while being recorded. Then, AI models were trained off images from these videos to predict the likelihood of admission.
The authors performed four comparisons: AI video alone, AI video + triage information (vital signs, chief complaint, age), triage information alone, and emergency severity index (ESI). In this fun demonstration, all four models were basically terrible at predicting admission (AUROCs ~0.6-0.7). But, the models incorporating video basically held their own, clearly outperforming ESI, and video + triage information was incrementally better than triage information alone.
There is very clearly nothing here suggesting this model is remotely clinical useful, or that it somehow parallels the cognitive processes of an experienced clinician. It is solely an academic exercise, though describing it as such ought not minimize the novelty of incorporating image analysis with other clinical information. As has been previously seen with other image analysis, AI models frequently trigger off image features unrelated to the clinical aspects of a case. The k-fold cross-validation used on their limited sample of 723 patients likely overfits their predictive model to their training data, leading to artificial inflation of performance. Then, “admission to hospital”, while operationally interesting, is a poor surrogate for immediate clinical needs and overall acuity. Finally, the authors also note several ethical and privacy challenges around video capture in clinical settings.
Regardless, a clever contribution to the AI clinical prediction literature.
“Hospitalization prediction from the emergency department using computer vision AI with short patient video clips”
https://www.nature.com/articles/s41746-024-01375-3