Too Many Tests! Or, So We Believe ….

Yes, Virginia, we order too many tests.  And, we know it – as evidenced by such conferences on overdiagnosis and costs of care.  And, even more relevant than such academic exercises, as this study indicates, even the general clinician seems to have a fair bit of self-awareness.

In this survey consisting of 435 respondents, 85% of emergency physicians believed excessive testing occurred in their Emergency Department.  Most frequently, such testing was motived by fear of missing even rare diagnoses, but defensive medicine and malpractice came a close second.  Patient expectations, local practice patterns, and time saving were also substantially cited as motivators for ordering.  Thankfully, administrative and personal motivations to increase reimbursement were rarely reported as reasons.

Despite the protestations of some policy-makers, the clinicians surveyed believed the most helpful change to the system would be malpractice reform.  Interestingly, the next ranked helpful interventions included educating patients and increasing shared decision-making.  While the first item may be logistically (or politically) unachievable, there are no obstacles to integrating improved communication behaviors into routine practice.  It does, however, show a need for increased availability of tools for clinicians to use at the point of care.

There are flaws in these sorts of perception-based surveys with regard to the accuracy of such anecdotal self-assessment.  Physician assessment of their own practice and that of others can certainly be questioned.  It must be admitted, however, a more intensive just-in-time surveying method would likely impact the variables measured.

There are also some highly entertaining outliers in Figure 2, of course, the perception of self vs. colleague ordering.  There is a handful of physicians who believe they, themselves, order over 80% of their CTs and MRIs unnecessarily – but that no one else in their group does.  Likewise, there is a handful with just the opposite perception – that their colleagues over-order, while they, themselves rarely do.  I wonder if they work in the same department?

Regardless, first step is admitting you have a problem.  We have many steps yet to go.

“Emergency Physician Perceptions of Medically Unnecessary Advanced Diagnostic Imaging”
http://onlinelibrary.wiley.com/doi/10.1111/acem.12625/abstract

Sometimes, The Stick Doesn’t Work

Pressure ulcers, catheter-associated UTIs, central-line infections, and injuries from falls are all iatrogenic injuries associated with healthcare and hospitalization.  Fewer of all these events would be ideal.

Of course, since asking nicely isn’t much of a motivation for healthcare delivery systems to improve practice, Medicare had a different solution – non-payment.  In 2008, Medicare ceased allowing hospitals to claim higher severity diagnosis related group codes to account for costs incurred by eight “never event” complications.  Money, on the other hand, is a strong motivator for change.  This study tries to evaluate just how successful such a heavy stick is at influencing care delivery.

These authors looked at the National Database of Nursing Quality Indicators, counting reported ulcers, falls, CLABSI, and CAUTI occurring between 2006 and 2010.  The trends reported for each differ starkly.  For CLABSI and CAUTI, in the quarters leading up to CMS policy change, the prevalence of each was gradually increasing.  After 2008, however, both trends show abrupt and consistent reversal and downward movement.  For pressure ulcers and injurious falls, however, the prevalence was gradually decreasing at the time of CMS policy implementation, and the slope of the line after 2008 is consistent with that same gradual decline.

The authors go into the limitations of each data source, but, the general takeaway is likely still valid – some “never events” just aren’t consistently, systematically preventable.  There are concerted, teachable best-practices involved with decreasing CLASBI and CAUTI.  Fall prevention and pressure ulcer prevention, on the other hand, are less amenable to care bundles, and seem to depend on gradual cultural changes and vigilance.  Thus, while outcomes-focused quality improvement using a financial motivator, while a reasonable method to try, will probably have the greatest impact and yield where a validated, evidence-based strategy can be implemented.

“Effect of Medicare’s Nonpayment for Hospital-Acquired Conditions Lessons for Future Policy”
http://archinte.jamanetwork.com/article.aspx?articleid=2087876

SIRS – Insensitive, Non-Specific

In what is almost certainly news only to quality improvement administrators, this newly published work out of Australia and New Zealand confirms what most already knew: the Systemic Inflammatory Response Syndrome criteria are only modestly associated with severe sepsis.

This is a retrospective evaluation of 13 years of data from the Australia and New Zealand Intensive Care Society Adult Patient Database, comprising routinely collected quality-assurance data.  Of 1,171,797 patients admitted to adult ICUs, 109,663 were identified as having both an infection and organ failure – the general, clinical definition of severe sepsis.  First, the good news:  over the 13 year study period, mortality dropped substantially – from over 30% down to close to 15%.  Then, the bad news:  12.1% of patients in the severe sepsis cohort manifested 0 or 1 SIRS criteria.  Mortality was lower in SIRS-negative severe sepsis, but hardly trivial at 16.1% during the study period, compared with 24.5% in the SIRS-positive patients.

So, the traditional SIRS-criteria definition of severe sepsis, previously thought to have at least sensitivity at expense of specificity will miss 1 in 8 patients with organ failure and an underlying infection.  Considering only approximately 1/3rd of patients with two or more SIRS criteria in the Emergency Department have an underlying infection, the utility of these criteria is substantially less reliable than previously thought.  Sadly, I’m certain many of you are suffering under SIRS criteria-based alerts in your Electronic Health Record – and, if such alerts are introducing cognitive biases by decreased vigilance and alert fatigue, it ought to be obvious we’re simply harming ourselves and patients.

“Systemic Inflammatory Response Syndrome Criteria in Defining Severe Sepsis”
http://www.nejm.org/doi/full/10.1056/NEJMoa1415236

Early Goal-Directed Waste For Sepsis

First there was ProCESS.  Then there was ARISE.  Now there is ProMISe.

If the prior two trials hadn’t already been celebrated and dissected, there would be much more to write regarding this one.  This, like the others, randomized patients to Early Goal-Directed Therapy for severe sepsis versus “usual care”.  This, like the others, found the basic components of resuscitation – intravenous fluids and early antibiotics – are far more important than the specific targets and protocols enshrined by Rivers et al.

These authors screened 6,192 patients to randomize 1,260.  Half had refractory hypotension, and the mean lactate levels were 7.0 and 6.8 in the EGDT and usual care arms.  Patients were enrolled within 6 hours of presentation and randomized within 2 hours of meeting inclusion criteria, with the EGDT arm receiving catheter insertion capable of SCVO2 monitoring within ~1 hour.   EGDT protocol was adhered to for 6 hours following enrollment.

As expected, randomization produced some divergence in treatment due to the EGDT protocol.  The EGDT cohort received more frequent red cell transfusions during both the protocolized period and subsequent care.  Likewise, dobutamine use in the EGDT arm exceeded usual care.  However, some differences occurred outside of the protocol.  EGDT arm patients were more likely to be admitted to an ICU setting, more likely to receive any sort of central line, more likely to receive invasive blood pressure monitoring, and more likely to be placed on vasopressors.  The remaining treatment – crystalloid resuscitation, colloid resuscitation, and other transfusions were similar.

And, finally, 90-day mortality was similar: 29.5% EGDT vs. 29.2% usual care.

A financial analysis found EGDT was more costly, but the result did not reach statistical significance.  However, the cost analysis was performed using different financial models that may not be generalizable to the billing structure in the United States.  The difference in ICU admission and length-of-stay alone certainly has important ramification both from a cost and a resource utilization standpoint.

So, finally, we have the publication of the last of the triumvirate of EGDT trials.  If there were any lingering doubts (hopes?) regarding the necessity of the most resource-intensive interventions, they ought to be laid to rest.  However, as with each of these negative trials, it is important to acknowledge the role of Rivers’ work in aggressively seeking, recognizing, and treating severe sepsis.  Even as we discard the components of his protocol, the main thrust of his work has saved many, many lives.

“Trial of Early, Goal-Directed Resuscitation for Septic Shock”
http://www.nejm.org/doi/full/10.1056/NEJMoa1500896

Flights of the Minimally Injured

Helicopter transport of trauma patients is a controversial topic.  Most agree there is a cohort of severely and specifically injured patients who receive important benefits from HEMS versus ground transportation.  However, it is reasonably suggested from registry studies those patients are rather few.  And, if only a subset of seriously injured patients benefit from HEMS, then, certainly the minimally injured patient does not.

But, unfortunately, flights of the minimally injured are hardly infrequent.

This is a retrospective review of all trauma transports at a single academic center in Arizona.  “Minimally injured” was defined as an ISS of 5 or lower, and who did not require intensive care or operative intervention.  Over the six years of the study period, this center received 3,992 ground transports, 39% of which were minimally injured.  They also received 981 HEMS arrivals – 27% of which were minimally injured.

Or, approximately $4.8 million burned for no benefit on just pre-hospital transportation by helicopter.

The authors’ title says it all:

“Overuse of helicopter transport in the minimally injured: A health care system problem that should be corrected”
http://www.ncbi.nlm.nih.gov/pubmed/25710420

Rampant Underreported Research Misconduct

It is not surprising to hear clinical trials sometimes struggle with data integrity and quality issues.  Such undertakings can be logistically challenging, and certainly any substantial scope of effort leads to the occasional cutting of corners.

However, there are also millions (or billions) of dollars in revenue, along with multiple professional reputations, at stake.  This creates fertile territory for the more nefarious sort of data corruption.  In some instances, the Food and Drug Administration performs site monitoring as evaluation for misconduct.  And, as this study indicates, the FDA sometimes discovers serious issues – issues almost always swept under the rug.

Using a variety of methods, including Freedom of Information Act requests, FDA.gov site exploration, and other FDA published warnings, these authors compiled a list of 421 serious irregularities identified by FDA audit.  However, heavily redacted language in many of the documents discovered precluded linkage to clinical trials – resulting in only 57 published trials that could be linked to serious violations.  These 57 trials resulted in 78 identifiable publications – only 3 of which mentioned or addressed the issues raised by the FDA.  Those three specifically noted data excluded due to protocol errors, data falsification, or inappropriate monitoring.  The remaining 75 publications did not.

A couple examples:

  • 8 of 16 FDA inspections of sites for RECORD 4, a rivaroxaban trial for DVT prophylaxis, identified unblinding, falsification of records, and randomization improprieties.  The associated study publications do not mention such issues.
  • A clinical site in China falsified data regarding apixiban in ARISTOTLE.  Excluding such data from the final study report would eliminate any apparent mortality benefit, but publications continue reporting mortality benefit analyses based on the entire data set.

The lack of transparency and apparent action regarding what is certainly just the tip of the iceberg is staggering.  How is it our own drug safety organization fails to protect patients on such a scale?  Is it any wonder so few clinical trial results hold up on re-examination?

“Research Misconduct Identified by the US Food and Drug Administration”
http://www.ncbi.nlm.nih.gov/pubmed/25664866

More Tests, Longer Turnaround, Longer LOS

In this Friday’s edition of “It’s Science!”, we cover this recent publication demonstrating, essentially, what we already knew:  throughput suffers when test turnaround times are longer!

To do so, however, requires (apparently) cross-classified random-effect modeling, linking Emergency Department information systems to laboratory test data.  These authors evaluated a retrospective, multi-site cohort consisting of 27,656 linked ED and laboratory encounters and modeled the attributable effect of test turnaround time on ED length-of-stay.  They discovered, rather obviously, patients receiving more tests had longer ED LOS.  Working backwards, furthermore, they found for every 30 minute increase in laboratory test turnaround time, there was an approximately 17 minute increase in median ED length-of-stay.  It’s not a 1:1 relationship – as you can imagine situations where the ED LOS is dictated rather by radiography, procedures, or consultations, rather than laboratory testing – but increases in a relatively linear fashion.

So, depending on the structure and flow of your Emergency Department, there may be substantial benefits to focusing on improved laboratory turnaround times.  And, likewise, you can probably improve all your times by simply ordering fewer tests!

“The Effect of Laboratory Testing on Emergency Department Length of Stay: A Multihospital Longitudinal Study Applying a Cross-classified Random-effect Modeling Approach”
http://www.ncbi.nlm.nih.gov/pubmed/25565488

Merry Christmas!

If you truly must read literature on Christmas, then I direct you to thebmj, and a selection of articles from its Christmas issue:

“Televised medical talk shows—what they recommend and the evidence to support their recommendations: a prospective observational study”
http://www.bmj.com/content/349/bmj.g7346

“CARTOONS KILL: casualties in animated recreational theater in an objective observational new study of kids’ introduction to loss of life”
http://www.bmj.com/content/349/bmj.g7184

“When somebody loses weight, where does the fat go?”
http://www.bmj.com/content/349/bmj.g7257

“Are some diets “mass murder”?”
http://www.bmj.com/content/349/bmj.g7654

Bayesian Statistics: We’re Dumb as Rocks

A guest post by Justin Mazzillo, a community doc in New Hampshire.

Physicians are often required to interpret medical literature to make critical decisions on patient care. Given that it is often in a hectic and hurried environment a strong foundation of evidence-based medicine is paramount. Unfortunately, this study from JAMA showed that physicians at all levels of training have anything but that.

This group surveyed a mix of medical students, interns, residents, fellows, attending physicians and one retired physician. They were asked to answer the following question:

“If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease, assuming you know nothing about the person’s symptoms or signs?”

Unfortunately three-quarters of the subjects got the answer wrong. The results were consistent across all levels of training. The most commonly given answer was almost as far from correct as possible.

I’ve withheld the answer for those who want to try out the questions themselves, and I know all the dedicated EMLoN readers will fare much better.

“Medicine’s Uncomfortable Relationship With Math: Calculating Positive Predictive Value”
http://archinte.jamanetwork.com/article.aspx?articleid=1861033

[I gave this same test to our residency program – and the results were almost identical.  A few sample “answers” below. -Ryan]


It’s a Patient Hand-Off Miracle

Transitions of care – more frequent now in medicine than ever before – are fertile opportunities for error and miscommunication.  Most institutions have developed, at least, informal protocols to exchange patient information during hand-off.  But, certainly, everyone has some anecdotal tale of missed information leading to a near-miss or actual patient harms.

This study tells the story of I-PASS, a handoff bundle implemented and measured as an error prevention strategy by a pre- and post-intervention study design.  Across 9 pediatric residency training programs, residents were observed for six months for time spent in hand-offs, time spent in patient care, and a variety of classifications of preventable and non-preventable errors.  Then, the I-PASS bundle was introduced – a structured sign-out mnemonic, a 2-hour workshop on communication skills, a 1-hour role-playing and simulation intervention, a faculty development program, direct-observation tools, and a culture-change campaign with a logo, posters, and other promotional activities.

Following the intervention, residents were, again, observed for six-months.  And, in general, preventable medical errors decreased a small absolute amount, along with a larger absolute decrease in near misses.  2 of 9 hospitals had increases in medical errors after the interventions, and the bulk of the effect size was a result of improvements at two hospitals whose baseline error rate was double that of the other 7 facilities.

The authors, then, are very excited about their I-PASS bundle.  But, as they note at the end of their discussion: “Although bundling appears to have been effective in this instance, it prevents us from determining which elements of the intervention were most essential.”  And, on face validity, this is obvious – the structured sign-out sheet was only one of many quality improvement interventions occurring simultaneously.  A decisive change in culture will trump the minor components of implementation anytime.

The final takeaway: if your institutional audit reveals handoff-related errors are pervasive and troublesome, and if reductions in such errors are prioritized and supported with the correct resources, you will probably see a reduction.  The I-PASS tool itself is not important, but the principles demonstrated here probably are.

“Changes in Medical Errors after Implementation of a Handoff Program”
http://www.ncbi.nlm.nih.gov/pubmed/25372088