Month: January 2025

Variation Exists! Outcomes Exist!

This little article has made the rounds, primarily by those who critique it for its many flaws. However, the underlying themes can still be valid, even if an article has limitations.

This is a “there is variation in emergency physician admitting practices” article. Literally every practicing physician working in a hospital environment knows there is a broad spectrum of skill, approach to acute illness, and level of risk-tolerance. These attributes manifest in different ways, and, in emergency physicians, one is the differing likelihood two clinicians might have to admit same patient to the hospital.

In this fashion, this descriptive study is basically fine. Over time, with a few exceptions, clinicians all basically see similar distributions of patients. Thus, it is very reasonable for this study to estimate there is a 90th percentile admission rate for “chest pain” of around 56%, and a 10th percentile admission rate around 32%. The underlying principle has face validity, even if the precise numbers do not.

The second part of the analysis involves the downstream outcomes after these patients are seen and/or admitted following their emergency department visit. The first point involves whether the subsequent inpatient stay was less than 24 hours, and the second point involves downstream short- and long-term mortality. The authors also tried to evaluate the frequency and outcomes of laboratory and radiology tests ordered by emergency physicians.

Without getting too granular into the data presented, the gross pattern is that clinicians with higher admission rates were also associated with higher likelihoods of <24 hour inpatient stays. This association was most prominent, unsurprisingly, in the cohort of patients with “chest pain”. Patterns were slightly less prominent, but still present, between higher rates of radiology and laboratory testing and subsequent admission.

The kicker from this study, and the mildly controversial portion, is where these authors tie this all back to the mortality data: no association between admission rate and mortality. The general implication vilifying those clinicians with higher rates of admission, as these behaviors are generating only short (read: unnecessary) admissions of no value (no mortality difference).

Everything here is almost assuredly imprecise and unable to be generalized outside the VA system involved. There are going to be issues with confounding, mis-coded data, and variation across sites. That said, the underlying principle here is probably true – some clinicians over-test, over-consult, and over-admit to no patient-oriented benefit.

However, what is to be done? Changing clinician behavior is fraught, and it is unclear whether reduced admission rates from the highest-admitting cohort would safely target only those whose admissions were unnecessary. Worse still, attempting to change behaviors in the U.S. involves more than patient-level considerations, but issues of health system and tort culture. The best path forward probably has little to do with specifically targeting individual clinicians, or even broad complaints like “chest pain”, but identifying the specific uncertainties upon which decisions are made. Then, evidence or tools may be generated to address the specific clinical questions giving rise to the variation.

“Variation in Emergency Department Physician Admitting Practices and Subsequent Mortality”
https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/2828189

Mobile Stroke Unit Propaganda Writ Large

This is yet another one of those “Get With The Guidelines” stroke analyses, a retrospective dredge with massive imbalances between groups – followed by statistical adjustments capable of turning out whichever result suits an author list with a full, dense printed page of pharma and stroke technology conflicts of interest.

In that respect, the study is unremarkable. Patients with potential stroke who were transported by Mobile Stroke Units were more likely to be functionally independent at baseline and more likely to be transported to a comprehensive stroke center. Thus, patients transported by Mobile Stroke Unit were more likely to be ambulatory and functionally independent at hospital discharge. Everything between the intake and output is just diversions.

Where it becomes further disagreeable is the accompanying editorial, written by two individuals who run Mobile Stroke Unit programs, arguing federal reimbursement ought to cover their pet projects. After a brief brush with the limitations of these data, they assert:

“it convincingly demonstrates through a large, representative, multicenter study that in real-world clinical practice, MSUs are associated with improved short-term patient outcomes”
… quite the over-glamourization of a secondary analysis of quality improvement registry data.

“the magnitude of benefit conferred by MSUs is comparable to that of other widely accepted acute stroke interventions, such as IVT in a 3-hour to 4.5-hour window and specialized stroke units”
… after multiple statistical adjustments of a grossly imbalanced cohort.

“this study demonstrates that MSUs not only benefit patients with AIS eligible for IVT, but also patients with AIS who are ineligible for IVT and patients with other forms of stroke”
… so, even if the MSU – whose mission in life is to provide tip-of-the-spear IVT – doesn’t provide acute treatment, it still confers benefit due to its soothing glow?

“This may be explained by faster imaging and blood pressure control in patients with intracerebral hemorrhage.”
… admission blood pressure for patients with SAH in this cohort was identical between MSU and EMS.

“this study rebuts concerns that by reaching and treating patients with suspected stroke earlier in their clinical course, MSUs could lead to unnecessary IVT treatments and higher rates of hemorrhagic complications. In fact, this study demonstrated the opposite: MSU care was associated with lower rates of stroke mimics”
… yes, as is the typical approach to coding these data, early administration of IVT virtually dictates a patient be coded. Once a patient has received IVT, only strong evidence to the contrary permits consideration of alternative causes of transient neurologic dysfunction – a happy accident also precluding any sICH occurring in “stroke mimics”, because there are none. To wit: only 24 of 4,218 (0.56%) of all MSU responses were “stroke mimics”, whereas 2,114 of 104,466 (2.0%) of all EMS responses were stroke mimics. When all you have is a hammer, everything you see looks like a stroke.

“Furthermore, for the broader population presenting with suspected stroke regardless of final diagnosis, the data suggest the potential for a lower risk of death.”
Again, this is magical thinking. As above, observing benefits outside the scope of the capabilities of an MSU ought prompt reconsideration statistical adjustments rather than plaudits.

These data are simply unsuited to support this sort of unabashed enthusiasm for MSUs. Rather than this editorial supporting their argument to consider funding and reimbursement structures for these tools, their biases shine through to diminish it. Regrettably, as per usual, guidelines and policy will be made by those sponsored to make the most persuasive contortion of data, rather than the most accurate.

“Mobile Stroke Unit Management in Patients With Acute Ischemic Stroke Eligible for Intravenous Thrombolysis”
https://jamanetwork.com/journals/jamaneurology/fullarticle/2824954

“Mobile Stroke Units—Time for Legislation and Remuneration”
https://jamanetwork.com/journals/jamaneurology/fullarticle/2824955

The AI Will Literally See You Now

This AI study is a fun experiment claiming to replicate the clinical gestalt generated by a physician’s initial synthesis of visual information. The ability to rapidly assess the stability and acuity of a patient is part of every experienced clinician’s refined skills – and used as a pre-test anchor for application of further diagnostic and management reasoning.

So, can AI do the same thing?

Well, “yes” and “of course not”.

In this demonstration project, these authors set up a mobile phone video camera at the foot of patients’ beds in the emergency department. Patients were instructed to perform a series of simple tasks (touch your nose, answer questions, etc.) while being recorded. Then, AI models were trained off images from these videos to predict the likelihood of admission.

The authors performed four comparisons: AI video alone, AI video + triage information (vital signs, chief complaint, age), triage information alone, and emergency severity index (ESI). In this fun demonstration, all four models were basically terrible at predicting admission (AUROCs ~0.6-0.7). But, the models incorporating video basically held their own, clearly outperforming ESI, and video + triage information was incrementally better than triage information alone.

There is very clearly nothing here suggesting this model is remotely clinical useful, or that it somehow parallels the cognitive processes of an experienced clinician. It is solely an academic exercise, though describing it as such ought not minimize the novelty of incorporating image analysis with other clinical information. As has been previously seen with other image analysis, AI models frequently trigger off image features unrelated to the clinical aspects of a case. The k-fold cross-validation used on their limited sample of 723 patients likely overfits their predictive model to their training data, leading to artificial inflation of performance. Then, “admission to hospital”, while operationally interesting, is a poor surrogate for immediate clinical needs and overall acuity. Finally, the authors also note several ethical and privacy challenges around video capture in clinical settings.

Regardless, a clever contribution to the AI clinical prediction literature.

“Hospitalization prediction from the emergency department using computer vision AI with short patient video clips”
https://www.nature.com/articles/s41746-024-01375-3