Category: Clinical Decision Rules

Quick Hit: Elders Risk Assessment

A few words regarding an article highlighted in one of my daily e-mails – a report regarding the Elders Risk Assessment tool (ERA) from the Mayo Clinic.

The key to the highlight is the assertion this score can be easily calculated and presented in-context to clinicians during primary care visits, allowing patients with higher scores to be easily identified for preventive interventions. With an AUC of 0.84, the authors are rather chuffed about the overall performance. In fact, they close their discussion with this rosy outlook:

The adoption of a proactive approach in primary care, along with the implementation of a predictive clinical score, could play a pivotal role in preventing critical ill- nesses, benefiting patients and optimizing healthcare resource allocation.

Completely missed by their limitations is that prognostic scores are not prescriptive. The ERA is based on age, recent hospitalizations, and chronic illness. The extent to which the management of any of these issues can be addressed “proactively” in the current primary care environment, and demonstrate a positive impact on patient-oriented outcomes, remains to be demonstrated.

To claim a scoring system is going to better the world, it is necessary to compare decisions made with formal prompting by the score to decisions made without – several steps removed from performing a retrospective evaluation to generate an AUC. It ought also be appreciated some decisions based on high ERA scores will increase resource utilization without a corresponding beneficial effect on health, while lower scores may likewise inappropriately bias clinical judgement.

This article has only passing applicability to emergency medicine, but the same issues regarding the disutility of “prognosis” apply widely.

“Individualized prediction of critical illness in older adults: Validation of an elders risk assessment model”
https://agsjournals.onlinelibrary.wiley.com/doi/abs/10.1111/jgs.18861

Potpourri

Just a quick-hit collection of articles I’ve wanted to highlight/catalogue for future reference, but couldn’t find the time for deep dives into each:

Shared Decision Making in Patients With Suspected Uncomplicated Ureterolithiasis: A Decision Aid Development Study.
For this common clinical scenario in the Emergency Department, the authors have developed a patient-facing packet to facilitate shared decision-making. However, more important than the product, is the process these authors have described for its creation. A similar roadmap could be followed to address similar opportunities in your department.

Reduction of Inappropriate Antibiotic Use and Improved Outcomes by Implementation of an Algorithm-Based Clinical Guideline for Nonpurulent Skin and Soft Tissue Infections.
Amazing – using the correct antibiotics reduces treatment failures and, likewise, treatment failures necessitating admission to the hospital. This is an effort-intensive intervention featuring provider education and individual prescribing feedback, but, given the limitations, can be considered a change management success. Whether this can be replicated at your institution will depend on many cultural factors.

Utility of INR For Prediction of Delayed Intracranial Hemorrhage Among Warfarin Users with Head Injury.
Here’s a topic with a ton of practice variation – do you admit patients with closed head injury on anticoagulation for observation? This retrospective review of those patients just on warfarin tries to make the case patients with INR <2 are safe for discharge, whereas those with higher scores are not. Again, however, the yield of observation is somewhere south of 1% in their entire therapeutic cohort, making it truly challenging to find the inflection point of value. Another opportunity for shared decision-making?

Performance of Novel High-Sensitivity Cardiac Troponin I Assays for 0/1-Hour and 0/2- to 3-Hour Evaluations for Acute Myocardial Infarction: Results From the HIGH-US Study.
A detailed look at high-sensitivity Troponin I rule-in/rule-out algorithms suggests a 0/1-hour strategy is similar to a 0/3-hour strategy. Overall, while the disposition of patients is likely to be more rapid from the 0/1 hour strategy, a greater proportion of patients ultimately fall into the “intermediate” zone requiring further observation and diagnostics. Certainly, combinations of hsTnI and other risk-stratification instruments ought to mean the majority of patients with straightforward chest pain presentations may be discharged from the Emergency Department.

Randomized Clinical Trial of IV Acetaminophen as an Adjunct to IV Hydromorphone for Acute Severe Pain in Emergency Department Patients.
In this trial, patients receiving hydromorphone were randomized to receive adjunctive treatment with IV acetaminophen or placebo. With 159 patients, they found advantages to the multi-modal approach favoring the addition of acetaminophen – but the confidence interval for their primary outcome crossed unity by 0.01. The authors conclude this is a negative trial, but it rather seems to me there’s certainly no harm in adding acetaminophen (it need not be IV) – adding it likely has a favorable effect, even if the effect size may not be large.

Effect of No Prehydration vs Sodium Bicarbonate Prehydration Prior to Contrast-Enhanced Computed Tomography in the Prevention of Postcontrast Acute Kidney Injury in Adults With Chronic Kidney Disease: The Kompas Randomized Clinical Trial.
In news surprising no one, another trial fails to show benefit of prehydration in staving off post-contrast exposure acute kidney injury. As seen on Twitter, rather than “contrast-induced nephropathy”, the clinical paradigm is effectively “contrast-adjacent nephropathy.” The impairment in renal function is associated with the underlying medical illness and not the exposure to IV contrast. Thus, no intervention – such as prehydration – can prevent such.

Coronary CT Angiography in Patients With Non-ST-Segment Elevation Acute Coronary Syndrome.
This interesting observational study evaluated patients with a diagnosis of non-ST elevation acute coronary syndrome using coronary CT angiography prior to invasive coronary angiography. The good news: CT angiography was probably useful at excluding obstructive coronary disease. The bad news: nearly 70% of patients had a coronary stenosis identified on invasive angiography, so patient selection prior to CT angiography will be important to improve the value of using it as a screen to prevent invasive angiography.

Industry Payment to Vascular Neurologists: A 6-Year Analysis of the Open Payments Program From 2013 Through 2018.
As we watch our healthcare delivery system struggle and groan under the various strains and burdens, one of the culprits has always been the influence of pharmaceutical/device manufacturers targeting investments to improve uptake of their products. In this observational analysis of the OpenPayments database, these authors identified the recipients of financial support from the manufacturers of endovascular devices. About 16% of vascular neurologists received funding from industry, but over 75% could be identified as “influencers” – chiefs of staff, department chairs, or similar. Pharma et al should always be remembered they are serving the interests of owners and shareholders, and not patients and our healthcare system.

The Clinical Impact of SAH Decision Rules

The Ottawa Subarachnoid Hemorrhage Rule has been around a long time now, dating back to 2013. The “six hour CT” rule has been around even longer, dating back even to 2011. They’ve become entwined in at least the discussion around the evaluation of SAH, if not clinical practice.

… but are they actually useful?

This is the “before and after” study from Perry and Stiell (not to be confused with Penn and Teller), in which the practice of Canadian physicians was examined around the time these rules were under development and in publication. These authors gathered data on patients presenting with atraumatic headache spanning the time period between 2011 and 2016, looking at resource utilization and missed SAH before and after adoption of both the Ottawa SAH Rule and the 6-hour CT Rule. Specifically, practicing clinicians were instructed not to use decision rules for the basis of patient care until June 2013, at which point clinicians were actively encouraged to do so.

The basic findings:

  • The Ottawa SAH Rule doesn’t change much.
  • The 6-hour CT Rule probably reduces downstream lumbar puncture/CTA.

Again, with concern for generalizability, a full 5.1% of their qualifying atraumatic headaches were diagnosed with SAH across the study period. The rate of investigation of these patients remained high, about 88%, regardless of study period – and regardless whether the Ottawa Rule criteria were met. However, for patients presenting within 6 hours of headache onset, the rate of subsequent LP dropped from 31.3% to 15.1%. The Ottawa SAH Rule showed its expected specificity of about 12%, and, therefore, was 100% sensitive. The 6-hour CT Rule “missed” 5 of 111 patients, however, for various reasons – one radiology misread, a false-positive owing to profound anemia, a non-aneurysmal SAH from dural vein fistula, and two cases of false-positive LPs meeting their study criteria for false-negative CT.

A long story made short, 1) keep using the 6-hour CT rule with the caveat of known potential confounders to visible blood (anemia); 2) the Ottawa Rule is only clinical useful as a one-way decision instrument owing to its poor positive likelihood ratio.

“Prospective Implementation of the Ottawa Subarachnoid Hemorrhage Rule and 6-Hour Computed Tomography Rule”

https://www.ncbi.nlm.nih.gov/pubmed/31805846

Yet Another Febrile Infant Rule

The Holy Grail in the evaluation of infants of less than 60 days remains safe discharge without a lumbar puncture. Boston, Philadelphia, Rochester, Step-by-Step and others have tried to achieve this noble goal over the years. And now, the Febrile Young Infant Research Collaborative has tossed their hat into the ring.

In this retrospective query of their Pediatric Health Information System and other electronic medical records, these authors identified 181 non-ill appearing patients across 11 Emergency Departments with invasive bacterial infection, defined as bacteremia in either blood or cerebrospinal fluid. Using 362 matched controls as a comparison cohort, these authors used the typical logistic regression route to tease out the strongest predictors of IBI – age in days, observed temperature, absolute neutrophil count, and urinalysis result. Subsequently, they condensed the continuous variables into cut-offs maximizing area under the curve. These cut-offs were then incorporated into a scoring system based on the strength of their adjusted odds ratio, and then the final output was validated on the derivation set using k-fold cross-validation with 10 sets.

The final result using their best cumulative score cut-off: sensitivity of 98.8% (95% CI 95.7-99.9) with 31.3% specificity. The two cases missed were that of a 3-day old and a 40-day old otherwise afebrile in the ED with normal UA and an ANC <5185. The authors ultimately conclude their score, if validated, may have best value as a one-way prediction tool primarily to reduce current routine invasive testing, owing to its poor specificity. Certainly, I agree it does not have much value in those who might otherwise not undergo testing; a more specific risk score may be better, if not clinician gestalt.

The other tidbit I might mention is whether there could be value in incorporating time-of-onset of fever into their evaluation. We’ve seen in other studies a few of the fallouts with regard to sensitivity of IBI stem from recency of illness onset, and it may be falsely reassuring to find a normal ANC early in an illness course. Furthermore, these authors do not specifically mention whether the lack of fever in the ED could have been associated with prehospital antipyretic use. Finally, their data collection does not appear to incorporate respiratory swab results; readily available respiratory viral panel results may also prove useful in ruling out IBI.

While these data are certainly alluring, considering the desire to avoid invasive procedures in young infants, substantial prospective work is still likely required.

As a sad aside, the authors state:

However, these criteria were developed >25 years ago, and the epidemiology of serious bacterial infections has changed considerably since that time.

Unfortunately, as vaccination frequency continues to decline, even since patients were enrolled for this study, our “modern” cohort may better begin to resemble that of 25 years ago.

“A Prediction Model to Identify Febrile Infants ≤60 Days at Low Risk of
Invasive Bacterial Infection”

https://www.ncbi.nlm.nih.gov/pubmed/31167938

PE in Pregnancy & YEARS Protocol

So, I generally like the YEARS protocol. It’s an incremental step forwards towards incorporating a pretest likelihood of disease into the interpretation of the D-dimer result. At the least, considering our reckless overdiagnosis of pulmonary embolism, it doesn’t make the situation worse.

Now, pregnancy. There is a mild increase in risk for PE while pregnant, and subsequent puerperal risk is even higher. Unfortunately this leads to a paradox: the index of suspicion for PE during pregnancy is so high, the yield of testing is frighteningly low – on the order of 5% or less for those undergoing evaluation for PE. Little prospective data regarding safe exclusion of PE during pregnancy are available.

These investigators – the Artemis Study – applied the YEARS algorithm, to the diagnosis of PE during pregnancy, attempting generally to demonstrate its safety while describing its yield and test characteristics. The primary modification to the baseline YEARS algorithm was patients having clinical signs of deep-vein thrombosis underwent compression ultrasonography prior to being evaluated with D-dimer and potential CTPA.

They enrolled 498 patients in whom PE was considered a relevant clinical diagnosis, about half of whom met no “high-risk” YEARS criteria. About a fifth of the patients were first trimester, and the remainder were split between second and third. The news is good and bad, unfortunately. Overall, the incidence of PE in their cohort was only 4.0% – typical of our deranged gestalt for PE in pregnancy. In the first trimester, their algorithm excluded PE without CTPA in 65% of those enrolled – meaning only 26 CTPA were indicated to diagnose the 5 PEs in this cohort. This is a reasonable yield.

However, second and third trimester excluded only 46% and 32% of patients from CTPA – meaning 261 CTPAs would be allowed to diagnosis 15 PEs – a yield of only 5.7%. This is better than performing a CT on everyone, but it’s still abysmal. This results, effectively, from the gradual physiologic increase in D-dimer throughout pregnancy – from a median of 505 ng/mL in their first trimester cohort, to 730 and 1,120 in subsequent trimesters.

So, while their algorithm is clearly safe – only one DVT occurred within their 3-month follow-up period, basically the expected rate of occurrence in their enrolled cohort – it’s not the final answer with regard to pregnancy. The next likely step required is to use our observational data to test specific trimester-related normals and pretest-related multipliers to find the optimal cut-offs, such that second and third trimester performance may approach that of the first.

“Pregnancy-Adapted YEARS Algorithm for Diagnosis of Suspected Pulmonary Embolism”
https://www.nejm.org/doi/full/10.1056/NEJMoa1813865

To COPD, or Not to COPD

This is yet another typically Canadian study, which is to say it’s Ian G. Stiell, et al, producing yet another high-quality risk-stratification profile for Emergency Department patients.

In this week’s episode, we find our heroes prospectively validating the Ottawa COPD Risk Scale (OCRS), a set of 10 criteria identifying COPD patients in the ED at highest-risk for short-term serious outcomes. Short-term serious outcomes, by their definition, was the occurrence of death, admission to a monitored unit, intubation, non-invasive ventilation, myocardial infarction, or hospital re-admission. Their target population was effectively patients with COPD who weren’t otherwise obviously ill, with a set of exclusion criteria for those who clearly necessitated admission on the index visit.

Within their 10 items, there are 16 points available to assign, and the expected risk of short-term outcomes ranged from 2.2% at a score of 0, up to 91.4% at a score of 10. Their validation, however, demonstrated potentially important differing results – with nearly two-fold greater risks for short-term poor outcomes in the lowest (and most common) strata. Then, at the high-end, there simply weren’t enough patients with scores >4 to come to any reliable consensus regarding the accuracy of the risk stratification. In fact, of the 65 patients with scores >4, the overall incidence of serious outcome was only 23% – while the expected risk from their derivation scale would probably be upwards of 40 or 50%. Their explanation: these higher-risk patients were all hospitalized at the index visit, thereby decreasing their expected rate of short-term serious outcome. This may, in fact, be true, but it is rather a hypothesis rather than a well-supported conclusion.

The other question, even assuming this tool is valid, is “what now?” The authors emphasize these data and risk levels are not necessarily prescriptive, but rather give the clinician an objective tool to supplement their decision-making. Will incorporating this tool this improve care delivery and outcomes? Unfortunately, it is impossible to tell. These authors have done the “easy” research, a data collection and number-crunching exercise. Determining its effect on clinical care is another, much harder step – and fraught with limitations due to generalizability from one practice setting to another. I would be wary of incorporating this into your clinical care until better data is available regarding its effect on patient-oriented and resource utilization outcomes – unless you’re ready to prospectively study it!

“Clinical validation of a risk scale for serious outcomes among patients with chronic obstructive pulmonary disease managed in the emergency department”

http://www.cmaj.ca/content/190/48/E1406

Rochester v. Philadelphia, Pediatric Edition

It’s a little tough for Rochester to go head-to-head against Philadelphia – with apologies to the Americans, Red Wings, Rhinos, Knighthawks, and Razorsharks. The playing field of … the playing field … is just on another level in Philadelphia. The playing field of febrile infants, however, is another matter.

This small study re-analyzed prospective data from 135 febrile children ≤60 days of age with documented invasive bacterial illness, and applied the Rochester and modified Philadelphia criteria for risk-stratification. IBI was defined as having a positive blood or CSF culture, if obtained. In this small sample, both Rochester and Philadelphia were 100% sensitive for all cases of meningitis in infants greater than 28 days of age, but each missed similar numbers of those with bacteremia. A comparison for those below 28 days is frankly irrelevant, as the modified Philadelphia criteria specifically applies only to those >28 days of life – so, yes, it is comically 100% sensitive and 0% specific in neonates. The Rochester criteria, which does not mandate CSF, if applied to those ≤28 days, would have missed two cases of meningitis, and is therefore not suitable for use.

The takeaway here is not so much which criteria is superior to the other – the elements of each are virtually identical. Moreso, it is the recognition that each is about 83% sensitive, and all children in this age range evaluated in the ED and discharged will require close follow-up for re-evaluation of clinical status.

“Risk Stratification of Febrile Infants ≤60 Days Old Without Routine Lumbar Puncture”
https://www.ncbi.nlm.nih.gov/pubmed/30425130

Clinical Policy: Sanity Returns to ACS

This may be the most important recent sentence in modern emergency medicine:

“… based on limitations in diagnostic technology and the need to avoid the harms associated with false-positive test results, the committee based its recommendations on the assumption that the majority of patients and providers would agree that a missed diagnosis rate of 1% to 2% for 30-day MACE in NSTE ACS is acceptable.”

It’s no longer the domain of rogue podcasters and throwaway magazine editorialists to declare our zero-miss culture destructive and self-defeating – it’s finally spelled out in black & white by our speciality society. This is not a license to kill, of course, but it is now utterly reasonable to feel as though the wind is at your back when sending an appropriately-evaluated patient home.

This clinical policy statement does not address terribly many questions, but it does jam a lot of evidence into one document in their review. Specifically, these authors ask:

1. In adult patients without evidence of ST-elevation ACS, can initial risk stratification be used to predict a low rate of 30-day MACE?

In short, yes. These authors recommend HEART as their decision instrument du jour, but also acknowledge other scores that simply do not yet have enough diverse evidence to support their use. Interestingly, they also note clinical gestalt may be just as good as any decision instrument, at least when the ECG and troponin are negative for new ischemia. Again, more prospective evidence would be required to formally enshrine such a recommendation into a clinical policy statement.

2. In adult patients with suspected acute NSTE ACS, can troponin testing within 3 hours of ED presentation be used to predict a low rate of 30-day MACE?

Here the authors have only Level C recommendations, which means their recommendations are based on low levels of evidence. Overall, they are weakly in favor of using of high-sensitivity troponins alone, or repeat conventional troponin testing as part of a risk-stratification or accelerated diagnostic pathway.

3. In adult patients with suspected NSTE ACS in whom acute MI has been excluded, does further diagnostic testing (eg, provocative, stress test, computed tomography [CT] angiography) for ACS prior to discharge reduce 30-day MACE?

Please no: “Do not routinely use further diagnostic testing (coronary CT angiography, stress testing, myocardial perfusion imaging) prior to discharge in low-risk patients in whom acute MI has been ruled out to reduce 30-day MACE.”  Take that, CCTA proponents.  They give an expert consensus recommendation of 1 to 2 week primary care follow-up when feasible, or consideration of observation when no follow-up is possible.

The fourth question posed deals with use of P2Y12 and
glycoprotein IIb/IIIa inhibitors in the ED, and is met basically with a shrug.

So!  Go forth and provide good medical care – specifically, high-value medical care, further freed from the mental oubliette of zero-miss.

“Clinical Policy: Critical Issues in the Evaluation and Management of Emergency Department Patients With Suspected Non–ST-Elevation Acute Coronary Syndromes”
https://www.ncbi.nlm.nih.gov/pubmed/30342745

Adult Head CT Decision Instrument Showdown

Every country seems to have their own pediatric imaging rule for minor head trauma, featuring PECARN, CHALICE, and CATCH. Recently, a head-to-head-to-head comparison (no pun intended) found the clear winner was: clinical judgement in Australia and New Zealand. Adoption of any of the rules would not have reliably increased sensitivity, but all would dramatically increase imaging.

Now, what about adult head trauma? The same story of every-country-has-a-flavor seems to be the case, with the CT in Head Injury Patients rule, the New Orleans criteria, the Canadian CT Head rule, and the National Institute for Health and Care Excellence guideline. This time, we have the Dutch performing the comparison.

In this multicenter, observational study conducted in 2015 and 2016, the authors enrolled neurologically-intact patients aged greater than 16 years and presenting with blunt head trauma within 24 hours of injury. Clinical data with the elements necessary for each decision instrument were completed by treating clinicians and collected by study staff. Decisions to perform imaging were based on individual clinician discretion, but primarily based on the CHIP rule. Outcomes were ascertained by electronic record review.

There were 5,839 patients entered in their study database, 5,517 meeting eligibility criteria. At three centers, only patients undergoing CT were entered in the database, while the remaining six centers included a handful of patients who did not undergo CT. Obviously, this grossly limits the descriptive capacity of the study, as clearly a massive number of patients with minor head injury who did not undergo CT were not followed for outcomes.

Overall, 384 of the 3,742 patients undergoing CT had positive traumatic findings. Most were small skull fractures, but about half had intracranial bleeding of some variety or another, with a further 74 being judged potential neurosurgical lesions. The most sensitive of the decision instruments in this study was the New Orleans criteria, while NICE guidelines were the least. Of course, the New Orleans criteria also would have recommended CT in all but 189 patients, for a specificity of 4.2%.

Ultimately, there’s no clear “winner” in this study, and, unfortunately, there’s also no obvious superior “clinician judgement” comparison lurking. The underlying rate of imaging was effectively the same as CHIP, as this was the national guideline in the Netherlands at the time of the study. Whether this is the “best” depends on tolerance for risk and the reliability of their estimate of “potential neurosurgical lesion”. Then, regardless of the decision instrument chosen, each still recommends imaging in thousands of patients in order to pick up the few with positive findings. Considering data from children, it seems we ought to be able to do much better – but current practice does not appear to be moving in that direction.

“External validation of computed tomography decision rules for minor head injury: prospective, multicentre cohort study in the Netherlands”
https://www.bmj.com/content/362/bmj.k3527

Don’t Rely on the EHR to Think For You

“The Wells and revised Geneva scores can be approximated with high accuracy through the automated extraction of structured EHR data elements in patients who underwent CTPA in the emergency department.”

Can it be done? Can the computer automatically discern your intent and extract pulmonary embolism risk-stratification from the structured data? And, with “high accuracy” as these authors tout in their conclusion?

IFF:  “High accuracy” means ~90%. That means one out of every ten in their sample was misclassified as low- or high-risk for PE. This is clinically useless.

The Wells classification, of course, depends highly upon the 3 points assigned for “PE is most likely diagnosis.” So, these authors assigned 3 points positive for every case.  This sort of probably works in a population that was selected explicitly because they underwent CTPA in the ED, but is obviously a foundationally broken kludge.  Revised Geneva does not have a “gestalt” element, but there are still subjective examination features that may not make it into structured data – and, obviously, it performed just as well (poorly) as the Wells tool.

To put it mildly, these authors are overselling their work a little bit. The electronic health record will always depend on the data entered – and it’s setting itself up for failure if it depends on specific elements entered by the clinician contemporaneously during the evaluation. Tools such as these have promise – but perhaps not this specific application.

“Automated Pulmonary Embolism Risk Classification and Guideline Adherence for Computed Tomography Pulmonary Angiography Ordering”
https://onlinelibrary.wiley.com/doi/abs/10.1111/acem.13442