A Muddled Look at ED CPOE

Computerized Provider Order Entry – the defining transition in medicine over the last couple decades.  Love it or hate it, as UCSF’s CEO says, the best way to characterize the industry leader is that it succeeds “not because it’s so good, but because others are so bad.”  A fantastic sentiment for a trillion-dollar industry that has somehow become an unavoidable reality of medical practice.

But, it’s not all doom and gloom.  This systematic review of CPOE in use in the Emergency Department identified 22 articles evaluating different aspects of EDIS – and some were even helpful!  The main area of benefit – which has been demonstrated repeatedly in the informatics literature – was a reduction in medication prescribing errors, overdoses, and potential adverse drug events.  There was no consensus regarding changes in patient flow, length of stay, or time spent in direct patient care.  Then, on the flip side, some CPOE interventions were harmful – the effect of order set use as decision-support was implementation dependent, with some institutions seeing increased testing while others saw decreases.

A muddled look at a muddled landscape with, almost certainly, a muddled immediate future.  There are a lot of decisions being made in boardrooms and committees regarding the use of these systems, and not nearly enough evaluation of the unintended consequences.

“May you live in interesting times,” indeed.

“The Effect of Computerized Provider Order Entry Systems on Clinical Care and Work Processes in Emergency Departments: A Systematic Review of the Quantitative Literature”
www.ncbi.nlm.nih.gov/pubmed/23548404

Negative Tests Fail to Reassure Patients

This article touches in a topic that we encounter all the time in Emergency Medicine – testing with the intent of “reassurance”.  The assumption is, wouldn’t a patient with symptom concerns be less anxious regarding their illness if they received a favorable negative test result?

That assumption, according to this meta-analysis and systematic review, is wrong.  These authors gathered together 14 trials evaluating the effect of non-diagnostic testing on downstream patient outcomes.  These tests included endoscopy for mild dyspepsia, radiography for low back pain, and cardiac event recording for palpitations.  This is a difficult article to interpret, particularly because there’s so much heterogeneity between the included studies, but the general conclusion is that tests performed in the setting of low pretest probability do not decrease subsequent primary care utilization, symptom recurrence, or anxiety regarding illness.

It’s rarely easy to tell a patient no testing is indicated – but this is yet another example illustrating the minimal benefits to over-testing.

“Reassurance After Diagnostic Testing With a Low Pretest Probability of Serious Disease”
http://www.ncbi.nlm.nih.gov/pubmed/23440131

The Case of the Missing Appendix

The correct initial diagnostic imaging test to evaluate pediatric abdominal pain for appendicitis is an ultrasound.  It carries none of the risks associated with CT imaging – except for the increased risk of a non-diagnostic evaluation.  It is also highly operator dependent and suffers in centers without sufficient volume of abdominal ultrasonography.

This study evaluates the subset of ultrasonography reports with the dreaded result “Appendix not visualized.”  Overall, 37.7% of 662 consecutive ultrasonographic studies at the authors’ institution failed to visualize the appendix.  Of interest to these authors were the “secondary signs” of appendicitis – free fluid, pericecal inflammatory changes, prominent lymph nodes, and phlegmon.

Their results are quite complicated – and, woefully, not terribly helpful.  Free fluid in females – useless.  Free fluid in males – more helpful if there’s a lot, but still only 2 cases of appendicitis out of the 5 males with a moderate/large amount of free fluid.  Lymph nodes – useless.  Pericecal fat changes – 1 out of 4.  Phlegmon – 2 out of 2.

So, there’s some information here.  Secondary signs with “Appendix not visualized” are typically not diagnostic alone – but, depending on the summation of other clinical findings, may yet be enough to obviate supplemental CT.

“Appendix Not Seen: The Predictive Value of Secondary Inflammatory Sonographic Signs” 
www.ncbi.nlm.nih.gov/pubmed/23528502

Chloride-Restriction & More JAMA Inadequacy

“The implementation of a chloride-restrictive strategy in a tertiary ICU was associated with a significant decrease in the incidence of AKI and use of RRT.”

Pretty clear, eh?  This article is one of several in a line of folks working to divorce us from normal saline.  The argument is that this hypernatremic, hyperchloremic solution, when given for large-volume resuscitation in the critically ill, leads to metabolic acidosis and decreased urine output.  This study, sponsored by Baxter, the makers of Plasma-Lyte, is an open-label, before-and-after design.  One year, they gave whatever fluid they wanted – mostly saline.  The next year, saline-containing fluids were restricted, and they used 20% albumin, lactated ringers (Hartmann’s solution), or Plasma-Lyte.

Firstly, the primary outcome doesn’t match their clinicaltrials.gov registration.  They’ve changed it from mean base excess during hospital stay to two primary outcomes that weren’t even both previously defined as secondary outcomes – increase in creatinine from baseline and incidence of acute kidney injury according to the RIFLE classification.

Then, they offer two positive results from their study – a decrease in the incidence of AKI and the use of renal replacement therapy.  The authors use RIFLE as their indicator of AKI – but they don’t pre-define which categories of RIFLE they use, and lump “Injury” and “Failure” together to a composite endpoint to gain statistical significance.  Otherwise, it’s a 7.4% control and 5.4% intervention difference in “Failure” that doesn’t reach statistical significance – and considering the mean baseline creatinine was lower in the intervention period, it ought to be expected to reach the failure definition less frequently.

The difference in rise of creatinine reaches statistical significance – but they’ve hidden the details in their online supplement  The mean serum creatinine in the baseline period rises from 10.4 mmol/L to 11.0 mmol/L, and in the intervention period from 10.3 mmol/L to 10.7 mmol/L.  This might be statistically significant, but hardly clinically significant.  Luckily, the authors use a skewed y-axis to distort and magnify the difference in their graph of these results.

Lastly, the RRT difference reported in their six-month study period is befuddling.  The overall rate of RRT in the entire year of their baseline period is 7.9%, while the rate of RRT in the entire year of their intervention period is 7.4%.  Yet, in the six months reported for this study, they report RRT use of 10% in the baseline period and 6.3% for the intervention period.  This implies the authors retrospectively selected their study period in order to magnify the effect of the RRT difference.  This difference in RRT also doesn’t match the 2% absolute difference in RIFLE classification for “failure” during the study period.  This implies the open-label nature of the study influenced the frequency of RRT use, as the authors may have exerted control over an outcome measure.

As far as patient-oriented outcomes go, after all this splitting of hairs, ICU length of stay was no different, the incidence of long-term dialysis was no different, and mortality was no different.  This is also a “bundle-of-care” study, with multiple different chloride-poor and chloride-rich fluids in use, which confounds the generalizability of the results.

Maybe chloride-sparing therapy is important.  But these authors are guilty of distorting and misleading with their presentation of results – and the JAMA editors, again, have failed us.

“Association Between a Chloride-Liberal vs Chloride-Restrictive Intravenous Fluid Administration Strategy and Kidney Injury in Critically Ill Adults”
www.ncbi.nlm.nih.gov/pubmed/23073953

Credit for much of the insight into this article goes to Greg Press, who prepared this article for last month’s Journal Club at UT-Houston – but he is in no way responsible for this unhinged rant.

Tongue Blade For Mandible Fractures

The “tongue blade test” is one of the fun, functional tests in Emergency Medicine.  If you’ve got facial trauma and you’re concerned about a mandible fracture, simply align a wooden tongue blade over the molars in a patient’s mouth and have them bite down firmly.  Then, twist the blade medially.  If the patient is not limited by pain, they’ll be able to hold the blade until it breaks.  If they’re limited by pain, such as in the presence of a mandible fracture, the patient won’t be able to hold the blade until it breaks.

This is an observational study enrolling 190 eligible patients for the tongue blade test in the presence of suspected mandible fracture.  66 patients had negative (normal) tongue blade tests, while 124 had positive (abnormal) tests.  All patients received a CT for definitive diagnosis.  There were 5 false negatives and 29 false positives.  Therefore, the sensitivity of the test is 95% and specificity 68%.  These results are consistent with some prior reviews of this test’s characteristics.

Not a “zero miss” test, but, depending on the pre-test likelihood based on other clinical factors, a very useful screening test.

“Re-evaluating the diagnostic accuracy of the tongue blade test: still useful as a screening tool for mandibular fractures?”
www.ncbi.nlm.nih.gov/pubmed/23490109

Don’t Get Sick on the Weekend

Quite bluntly, you’re more likely to die.

These authors analyzed the 2008 Nationwide Emergency Department Sample, using 4,225,973 patient encounters as the basis of their observational analysis.  The absolute mortality differences between weekday emergency department presentations and weekend emergency department presentations is tiny – about 0.2% difference.  However, this difference is very consistent across type of insurance, teaching hospital status, and hospital funding source.

The NEDS sample did not offer these authors any specific explanation of the “weekend effect”, but they expect it is due to decreased resource availability on weekends.  The authors note specific systems in place (e.g., trauma centers, PICU, stroke centers) where weekend staffing is unchanged have demonstrated the ability to eliminate such weekend phenomena.  However, it’s probably never going to be the case that weekend shifts are less desirable – so we’re probably stuck with this slight mortality bump on weekends.

“Don’t get sick on the weekend: an evaluation of the weekend effect on mortality for

patients visiting US EDs”

www.ncbi.nlm.nih.gov/pubmed/23465873

Critical Deficiencies in Pediatric EM Training

This article is an overview of the critical procedures performed over a one-year period at Cincinnati Children’s, a large, well-respected, level 1 trauma center with a pediatric emergency medicine fellowship program.  In theory, this facility ought to provide trainees with top-flight training, including adequate exposure to critical life-saving procedures.

Not exactly.

In that one year period, the PEM fellows performed 32 intubations, 7 intraosseus line placements, 3 tube thoracostomies, and zero central line placements.  This accounted for approximately 25% of all available procedures – attending physicians and residents poached the remainder of procedures during the year.  Therefore, based on this observational data, these authors conclude the training in PEM might not be sufficient to provide adequate procedural expertise.  Then, the authors note pediatric emergency departments have such routinely low acuity – 2.5 out of every 1,000 patients requiring critical resuscitation – that it is inevitable these skills will deteriorate.

Essentially, this means the general level of emergency physician preparedness for a critically ill child is very low.  PEM folks might have more pediatric-specific experience – but very limited procedural exposure – while general emergency physicians perform procedures far more frequently – but on adults.  The authors even specifically note 63% of PEM faculty did not perform a single successful intubation throughout the entire year.

Their solution – which I tend to agree with – is the development of high-quality simulation tools to be used for training and maintenance of skills.  Otherwise, we won’t be providing optimal care to the few critically ill children who do arrive.

“The Spectrum and Frequency of Critical Procedures Performed in a Pediatric Emergency Department: Implications of a Provider-Level View”
www.ncbi.nlm.nih.gov/pubmed/22841174

The Boondoggle of Step 2 CS

Recent medical school graduates are familiar with the Step 2 Clinical Skills examination, a day-long charade of simulated clinical encounters intended to screen out medical students who are incapable of functioning in a clinical setting.  This test was adapted from the ECFMG Clinical Skills Assessment, intended essentially to screen out foreign medical graduates with inadequate communication skills to safely practice medicine in the United States.

However, U.S. and Canadian medical school graduates pass this test 98% of the time on the first attempt, and 91% of the time on a re-attempt.  This means each year $20.4 million are expended in test fees – and probably half again that amount in travel expenses – to identify 30-odd medical school graduates who are truly non-functional.  The authors of this brief letter in the NEJM suggest, with interest compounding secondary to medical school debt repayments, it costs over a million dollars per failed student.

Clearly, some medical students are not capable of functioning as physicians.  However, clinical skills teaching, evaluation, and remediation ought to be part of the purview of the medical school training program that has multi-year longitudinal experience with the student, not a one-day simulation.  I’m sure some of the few who fail Step 2 CS twice are capable of safely practicing medicine, and certainly many who pass Step 2 CS still require additional teaching.  I agree with these authors that this test is an expensive and ineffective farce.

Then again, as this NYTimes vignette points out, medical schools are having a tough time failing folks for poor clinical skills.  However, the solution is not to pass the buck along to the NBME.

“The Step 2 Clinical Skills Exam — A Poor Value Proposition”
www.nejm.org/doi/full/10.1056/NEJMp1213760

EM Lit of Note on KevinMD.com

Featured today as a guest blog, revisiting the JAMA Clinical Evidence synopsis critiqued last month on this blog, here and here.

It’s rather an experiment in discovering just how influential social media has become – open access, crowdsourced “peer review” – and whether this mechanism for addressing conflict-of-interest in the prominent medical journals is more effective than simply attempting a letter to the editor.

KevinMD.com – “The filtering of medical evidence has clearly failed“