A Window Into Your EHR Sepsis Alert

Hospitals are generally interested in detecting and treating sepsis.  As a result of multiple quality measures, however, now they are deeply in love with detecting and treating sepsis.  And this means: yet another alert in your electronic health record.

One of these alerts, created by the Cerner Corporation, is described in a recent publication in the American Journal of Medical Quality.  Their cloud-based system analyzes patient data in real-time as it enters the EHR and matches the data against the SIRS criteria.  Based on 6200 hospitalizations retrospectively reviewed, the alert fired for 817 (13%) of patients.  Of these, 622 (76%) were either superfluous or erroneous, with the alert occurring either after the clinician had ordered antibiotics or in patients for whom no infection was suspected or treated.  Of the remaining alerts occurring prior to action to treat or diagnose infection, most (89%) occurred in the Emergency Department, and a substantial number (34%) were erroneous.

Therefore, based on the authors’ presented data, 126 of 817 (15%) of SIRS alerts provided accurate, potentially valuable information.  Unfortunately, another 80 patients in the hospitalized cohort received discharge diagnoses of sepsis despite never triggering the tool – meaning false negatives approach nearly 2/3rds the number of potentially useful true positives.  And, finally, these data only describe patients requiring hospitalization – i.e., not including those discharged from the Emergency Department.  We can only speculate regarding the number of alerts triggered on the diverse ED population not requiring hospitalization – every asthmatic, minor trauma, pancreatitis, etc.

The lead author proudly concludes their tool is “an effective approach toward early recognition of sepsis in a hospital setting.”  Of course, the author, employed by Cerner, also declares he has no potential conflicts of interest regarding the publication in question.

So, if the definition of “effective” is lower than probably 10% utility, that is the performance you’re looking it with these SIRS-based tools.  Considering, on one hand, the alert fatigue, and on the other hand, the number of additional interventions and unnecessary tests these sorts of alerts bludgeon physicians into – such unsophisticated SIRS alerts are almost certainly more harm than good.

“Clinical Decision Support for Early Recognition of Sepsis”
http://www.ncbi.nlm.nih.gov/pubmed/25385815

A Laughable tPA “Systematic Review”

Over 200,000 physicians belong to the American Medical Association.  The Journal, therefore, of this Association has a significant audience and a long tradition.  Continuing Medical Education inserts in JAMA may represent the basic education of many new developments for general practitioners.

Unfortunately, the authors of this most recent CME portion seem to require their own education on the conduct of a “systematic review”.

A properly performed systematic review utilizes a precise, replicable, well-described search strategy with which to canvass the evidence for synthesis.  The assembled evidence is then evaluated based on pre-specified criteria for inclusion or exclusion.  The end-result, hopefully, is a knowledge translation document based on the entire scope of published literature, accounting for controversy and irregularity in the context of a larger summary.

These authors perform a systematic review on “acute stroke intervention”.  They identify and review 145 abstracts utilizing multiple combinations of MeSH terms and synonyms for “brain ischemia/drug therapy, stroke drug/therapy, tissue plasminogen activator, fibrinolytic agents, endovascular procedures, thrombectomy, time factors, emergency service, treatment outcome, multicenter study, and randomized controlled trial”.  A massive undertaking, to be sure – considering these authors are also including intra-arterial and mechanical therapy in their review.

Yet, as indicated in their evidence review chart in the supplement, this strategy managed to identify only 17 RCTs – in the whole of systemic and endovascular therapy.  As an example for comparison, the latest Cochrane Review of thrombolytics for acute ischemic stroke included 27 trials of fibrinolytic agents alone.  And, as covered in their text and cited in their References, the RCT evidence regarding systemic therapy for acute stroke consists of: NINDS and ECASS III.

That’s it.

No MAST-E, MAST-I, or ASK.  No mention of the smaller imaging-guided trials, EPITHET, DEDAS, DIAS, or DIAS II.  Or, even excluding non-tPA trials, no ECASS, ECASS II, ATLANTIS, or, even the largest of flawed acute stroke trials, IST-3.  And, even with such limited coverage, certainly no mention of any of the controversy over imbalances in NINDS, nor flaws in ECASS III pertaining to tPA’s persistent non-approval by the FDA for the 3-4.5 hour time window.

If this were simply a commentary for the lay press regarding the bare minimum highlights of the last 20 years of stroke treatment, perhaps this would suffice.  And, frankly, these authors do much better regarding their reporting on the recent endovascular trials.  But, a CME publication in a prominent medical journal failing to address 90% of the evidence on a particular topic – yet calling itself a “systematic review” – is retraction-worthy.

“Acute Stroke Intervention – A Systematic Review”
http://jama.jamanetwork.com/article.aspx?articleid=2247149

Too Many Tests! Or, So We Believe ….

Yes, Virginia, we order too many tests.  And, we know it – as evidenced by such conferences on overdiagnosis and costs of care.  And, even more relevant than such academic exercises, as this study indicates, even the general clinician seems to have a fair bit of self-awareness.

In this survey consisting of 435 respondents, 85% of emergency physicians believed excessive testing occurred in their Emergency Department.  Most frequently, such testing was motived by fear of missing even rare diagnoses, but defensive medicine and malpractice came a close second.  Patient expectations, local practice patterns, and time saving were also substantially cited as motivators for ordering.  Thankfully, administrative and personal motivations to increase reimbursement were rarely reported as reasons.

Despite the protestations of some policy-makers, the clinicians surveyed believed the most helpful change to the system would be malpractice reform.  Interestingly, the next ranked helpful interventions included educating patients and increasing shared decision-making.  While the first item may be logistically (or politically) unachievable, there are no obstacles to integrating improved communication behaviors into routine practice.  It does, however, show a need for increased availability of tools for clinicians to use at the point of care.

There are flaws in these sorts of perception-based surveys with regard to the accuracy of such anecdotal self-assessment.  Physician assessment of their own practice and that of others can certainly be questioned.  It must be admitted, however, a more intensive just-in-time surveying method would likely impact the variables measured.

There are also some highly entertaining outliers in Figure 2, of course, the perception of self vs. colleague ordering.  There is a handful of physicians who believe they, themselves, order over 80% of their CTs and MRIs unnecessarily – but that no one else in their group does.  Likewise, there is a handful with just the opposite perception – that their colleagues over-order, while they, themselves rarely do.  I wonder if they work in the same department?

Regardless, first step is admitting you have a problem.  We have many steps yet to go.

“Emergency Physician Perceptions of Medically Unnecessary Advanced Diagnostic Imaging”
http://onlinelibrary.wiley.com/doi/10.1111/acem.12625/abstract

Sometimes, The Stick Doesn’t Work

Pressure ulcers, catheter-associated UTIs, central-line infections, and injuries from falls are all iatrogenic injuries associated with healthcare and hospitalization.  Fewer of all these events would be ideal.

Of course, since asking nicely isn’t much of a motivation for healthcare delivery systems to improve practice, Medicare had a different solution – non-payment.  In 2008, Medicare ceased allowing hospitals to claim higher severity diagnosis related group codes to account for costs incurred by eight “never event” complications.  Money, on the other hand, is a strong motivator for change.  This study tries to evaluate just how successful such a heavy stick is at influencing care delivery.

These authors looked at the National Database of Nursing Quality Indicators, counting reported ulcers, falls, CLABSI, and CAUTI occurring between 2006 and 2010.  The trends reported for each differ starkly.  For CLABSI and CAUTI, in the quarters leading up to CMS policy change, the prevalence of each was gradually increasing.  After 2008, however, both trends show abrupt and consistent reversal and downward movement.  For pressure ulcers and injurious falls, however, the prevalence was gradually decreasing at the time of CMS policy implementation, and the slope of the line after 2008 is consistent with that same gradual decline.

The authors go into the limitations of each data source, but, the general takeaway is likely still valid – some “never events” just aren’t consistently, systematically preventable.  There are concerted, teachable best-practices involved with decreasing CLASBI and CAUTI.  Fall prevention and pressure ulcer prevention, on the other hand, are less amenable to care bundles, and seem to depend on gradual cultural changes and vigilance.  Thus, while outcomes-focused quality improvement using a financial motivator, while a reasonable method to try, will probably have the greatest impact and yield where a validated, evidence-based strategy can be implemented.

“Effect of Medicare’s Nonpayment for Hospital-Acquired Conditions Lessons for Future Policy”
http://archinte.jamanetwork.com/article.aspx?articleid=2087876

SIRS – Insensitive, Non-Specific

In what is almost certainly news only to quality improvement administrators, this newly published work out of Australia and New Zealand confirms what most already knew: the Systemic Inflammatory Response Syndrome criteria are only modestly associated with severe sepsis.

This is a retrospective evaluation of 13 years of data from the Australia and New Zealand Intensive Care Society Adult Patient Database, comprising routinely collected quality-assurance data.  Of 1,171,797 patients admitted to adult ICUs, 109,663 were identified as having both an infection and organ failure – the general, clinical definition of severe sepsis.  First, the good news:  over the 13 year study period, mortality dropped substantially – from over 30% down to close to 15%.  Then, the bad news:  12.1% of patients in the severe sepsis cohort manifested 0 or 1 SIRS criteria.  Mortality was lower in SIRS-negative severe sepsis, but hardly trivial at 16.1% during the study period, compared with 24.5% in the SIRS-positive patients.

So, the traditional SIRS-criteria definition of severe sepsis, previously thought to have at least sensitivity at expense of specificity will miss 1 in 8 patients with organ failure and an underlying infection.  Considering only approximately 1/3rd of patients with two or more SIRS criteria in the Emergency Department have an underlying infection, the utility of these criteria is substantially less reliable than previously thought.  Sadly, I’m certain many of you are suffering under SIRS criteria-based alerts in your Electronic Health Record – and, if such alerts are introducing cognitive biases by decreased vigilance and alert fatigue, it ought to be obvious we’re simply harming ourselves and patients.

“Systemic Inflammatory Response Syndrome Criteria in Defining Severe Sepsis”
http://www.nejm.org/doi/full/10.1056/NEJMoa1415236

Early Goal-Directed Waste For Sepsis

First there was ProCESS.  Then there was ARISE.  Now there is ProMISe.

If the prior two trials hadn’t already been celebrated and dissected, there would be much more to write regarding this one.  This, like the others, randomized patients to Early Goal-Directed Therapy for severe sepsis versus “usual care”.  This, like the others, found the basic components of resuscitation – intravenous fluids and early antibiotics – are far more important than the specific targets and protocols enshrined by Rivers et al.

These authors screened 6,192 patients to randomize 1,260.  Half had refractory hypotension, and the mean lactate levels were 7.0 and 6.8 in the EGDT and usual care arms.  Patients were enrolled within 6 hours of presentation and randomized within 2 hours of meeting inclusion criteria, with the EGDT arm receiving catheter insertion capable of SCVO2 monitoring within ~1 hour.   EGDT protocol was adhered to for 6 hours following enrollment.

As expected, randomization produced some divergence in treatment due to the EGDT protocol.  The EGDT cohort received more frequent red cell transfusions during both the protocolized period and subsequent care.  Likewise, dobutamine use in the EGDT arm exceeded usual care.  However, some differences occurred outside of the protocol.  EGDT arm patients were more likely to be admitted to an ICU setting, more likely to receive any sort of central line, more likely to receive invasive blood pressure monitoring, and more likely to be placed on vasopressors.  The remaining treatment – crystalloid resuscitation, colloid resuscitation, and other transfusions were similar.

And, finally, 90-day mortality was similar: 29.5% EGDT vs. 29.2% usual care.

A financial analysis found EGDT was more costly, but the result did not reach statistical significance.  However, the cost analysis was performed using different financial models that may not be generalizable to the billing structure in the United States.  The difference in ICU admission and length-of-stay alone certainly has important ramification both from a cost and a resource utilization standpoint.

So, finally, we have the publication of the last of the triumvirate of EGDT trials.  If there were any lingering doubts (hopes?) regarding the necessity of the most resource-intensive interventions, they ought to be laid to rest.  However, as with each of these negative trials, it is important to acknowledge the role of Rivers’ work in aggressively seeking, recognizing, and treating severe sepsis.  Even as we discard the components of his protocol, the main thrust of his work has saved many, many lives.

“Trial of Early, Goal-Directed Resuscitation for Septic Shock”
http://www.nejm.org/doi/full/10.1056/NEJMoa1500896

Flights of the Minimally Injured

Helicopter transport of trauma patients is a controversial topic.  Most agree there is a cohort of severely and specifically injured patients who receive important benefits from HEMS versus ground transportation.  However, it is reasonably suggested from registry studies those patients are rather few.  And, if only a subset of seriously injured patients benefit from HEMS, then, certainly the minimally injured patient does not.

But, unfortunately, flights of the minimally injured are hardly infrequent.

This is a retrospective review of all trauma transports at a single academic center in Arizona.  “Minimally injured” was defined as an ISS of 5 or lower, and who did not require intensive care or operative intervention.  Over the six years of the study period, this center received 3,992 ground transports, 39% of which were minimally injured.  They also received 981 HEMS arrivals – 27% of which were minimally injured.

Or, approximately $4.8 million burned for no benefit on just pre-hospital transportation by helicopter.

The authors’ title says it all:

“Overuse of helicopter transport in the minimally injured: A health care system problem that should be corrected”
http://www.ncbi.nlm.nih.gov/pubmed/25710420

Rampant Underreported Research Misconduct

It is not surprising to hear clinical trials sometimes struggle with data integrity and quality issues.  Such undertakings can be logistically challenging, and certainly any substantial scope of effort leads to the occasional cutting of corners.

However, there are also millions (or billions) of dollars in revenue, along with multiple professional reputations, at stake.  This creates fertile territory for the more nefarious sort of data corruption.  In some instances, the Food and Drug Administration performs site monitoring as evaluation for misconduct.  And, as this study indicates, the FDA sometimes discovers serious issues – issues almost always swept under the rug.

Using a variety of methods, including Freedom of Information Act requests, FDA.gov site exploration, and other FDA published warnings, these authors compiled a list of 421 serious irregularities identified by FDA audit.  However, heavily redacted language in many of the documents discovered precluded linkage to clinical trials – resulting in only 57 published trials that could be linked to serious violations.  These 57 trials resulted in 78 identifiable publications – only 3 of which mentioned or addressed the issues raised by the FDA.  Those three specifically noted data excluded due to protocol errors, data falsification, or inappropriate monitoring.  The remaining 75 publications did not.

A couple examples:

  • 8 of 16 FDA inspections of sites for RECORD 4, a rivaroxaban trial for DVT prophylaxis, identified unblinding, falsification of records, and randomization improprieties.  The associated study publications do not mention such issues.
  • A clinical site in China falsified data regarding apixiban in ARISTOTLE.  Excluding such data from the final study report would eliminate any apparent mortality benefit, but publications continue reporting mortality benefit analyses based on the entire data set.

The lack of transparency and apparent action regarding what is certainly just the tip of the iceberg is staggering.  How is it our own drug safety organization fails to protect patients on such a scale?  Is it any wonder so few clinical trial results hold up on re-examination?

“Research Misconduct Identified by the US Food and Drug Administration”
http://www.ncbi.nlm.nih.gov/pubmed/25664866

More Tests, Longer Turnaround, Longer LOS

In this Friday’s edition of “It’s Science!”, we cover this recent publication demonstrating, essentially, what we already knew:  throughput suffers when test turnaround times are longer!

To do so, however, requires (apparently) cross-classified random-effect modeling, linking Emergency Department information systems to laboratory test data.  These authors evaluated a retrospective, multi-site cohort consisting of 27,656 linked ED and laboratory encounters and modeled the attributable effect of test turnaround time on ED length-of-stay.  They discovered, rather obviously, patients receiving more tests had longer ED LOS.  Working backwards, furthermore, they found for every 30 minute increase in laboratory test turnaround time, there was an approximately 17 minute increase in median ED length-of-stay.  It’s not a 1:1 relationship – as you can imagine situations where the ED LOS is dictated rather by radiography, procedures, or consultations, rather than laboratory testing – but increases in a relatively linear fashion.

So, depending on the structure and flow of your Emergency Department, there may be substantial benefits to focusing on improved laboratory turnaround times.  And, likewise, you can probably improve all your times by simply ordering fewer tests!

“The Effect of Laboratory Testing on Emergency Department Length of Stay: A Multihospital Longitudinal Study Applying a Cross-classified Random-effect Modeling Approach”
http://www.ncbi.nlm.nih.gov/pubmed/25565488

Merry Christmas!

If you truly must read literature on Christmas, then I direct you to thebmj, and a selection of articles from its Christmas issue:

“Televised medical talk shows—what they recommend and the evidence to support their recommendations: a prospective observational study”
http://www.bmj.com/content/349/bmj.g7346

“CARTOONS KILL: casualties in animated recreational theater in an objective observational new study of kids’ introduction to loss of life”
http://www.bmj.com/content/349/bmj.g7184

“When somebody loses weight, where does the fat go?”
http://www.bmj.com/content/349/bmj.g7257

“Are some diets “mass murder”?”
http://www.bmj.com/content/349/bmj.g7654