Category: Pediatrics

What Does a Sepsis Alert Gain You?

The Electronic Health Record is no longer simply that – a recording of events and clinical documentation.  Decision-support has, for good or ill, morphed it into a digital nanny vehicle for all manner of burdensome nagging.  Many systems have implemented a “sepsis alert”, typically based off vital signs collected at initial assessment. The very reasonable goal is early detection of sepsis, and early initiation of appropriately directed therapy. The downside, unfortunately, is such alerts are rarely true positives for severe sepsis in broadest sense – alerts far outnumber the instances in a change of clinical practice results in a change in outcome.

So, what to make of this:

This study describes a before-and-after performance of a quality improvement intervention to reduce missed diagnoses of sepsis, part of which was introduction of a triage-based EHR alert. These alerts fired during initial assessment based on abnormal vital signs and the presence of high-risk features. The article describes baseline characteristics for a pre-intervention phase of 86,037 Emergency Department visits, and then a post-intervention phase of 96,472 visits. During the post-intervention phase, there were 1,112 electronic sepsis alerts, 265 of which resulted in initiation of sepsis protocol after attending physician consultation.  The authors, generally, report fewer missed or delayed diagnoses during the post-intervention period.

But, the evidence underpinning conclusions from these data – as relating to improvements in clinical care or outcomes, or even the magnitude of process improvement highlighted in the tweet above – is fraught. The alert here is reported as having a sensitivity of 86.2%, and routine clinical practice picked up nearly all of the remaining cases that were alert negative.  The combined sensitivity is reported to be 99.4%.  Then, the specificity appears to be excellent, at 99.1% – but, for such an infrequent diagnosis, even using their most generous classification for true positives, the false alerts outnumbered the true alerts nearly 3 to 1.

And, that classification scheme is the crux of determining the value of this approach. The primary outcome was defined as either treatment on the ED sepsis protocol or pediatric ICU care for sepsis. Clearly, part of the primary outcome is directly contaminated by the intervention – an alert encouraging use of a protocol will increase initiation, regardless of appropriateness. This will not impact sensitivity, but will effectively increase specificity and directly inflate PPV.

This led, importantly, for the authors to include a sensitivity analysis looking at their primary outcome. This analysis looks at the differences in overall performance if stricter rules for a primary outcome might be entertained. These analyses evaluate the predictive value of the protocol if true positives are restricted to those eventually requiring vasoactive agents or pediatric ICU care – and, unsurprisingly, even this small decline in specificity results in dramatic drops in PPV – down to 2.4% for the alert alone.

This number better matches the face validity we’re most familiar with for these simplistic alerts – the vast majority triggered have no chance of impacting clinical care and improving outcomes. It should further be recognized the effect size of early recognition and intervention for sepsis is real, but quite small – and becomes even smaller when the definition broadens to cases of lower severity. With nearly 100,000 ED visits in both the pre-intervention and post-intervention periods, there is no detectable effect on ICU admission or mortality. Finally, the authors focus on their “hit rate” of 1:4 in their discussion – but, I think it is more likely the number of alerts fired for each each case of reduced morbidity or mortality is on the order of hundreds, or possibly thousands.

Ultimately, the reported and publicized magnitude of the improvement in clinical practice likely represents more smoke and mirrors than objective improvements in patient outcomes, and in the zero-sum game of ED time and resources, these sorts of alerts and protocols may represent important subtractions from the care of other patients.

“Improving Recognition of Pediatric Severe Sepsis in the Emergency Department: Contributions of a Vital Sign–Based Electronic Alert and Bedside Clinician Identification”

http://www.annemergmed.com/article/S0196-0644(17)30315-3/abstract

PECARN, CATCH, CHALICE … or None of the Above?

The decision instrument used to determine the need for neuroimaging in minor head trauma essentially a question of location. If you’re in the U.S., the guidelines feature PECARN. In Canada, CATCH. In the U.K., CHALICE. But, there’s a whole big world out there – what ought they use?

This is a prospective observational study from two countries out in that big remainder of the world – Australia and New Zealand. Over approximately 3.5 years, these authors enrolled patients with non-trivial mild head injuries (GCS 13-15) and tabulated various rule criteria and outcomes. Each rule has slightly different entry criteria and purpose, but over the course of the study, 20,317 patients were gathered for their comparative analysis.

And, the winner … is Australian and New Zealand general practice. Of these 20,000 patients included, only 2,106 (10%) underwent CT. It is hard to read between the lines and determine how many of the injuries included in this analysis were missed on the initial presentation, but if rate of neuroimaging is the simplest criteria for winning, there’s no competition. Applying CHALICE to their analysis cohort would have increased their CT rate to approximately 22%, and CATCH would raise the rate to 30.2%. Application of PECARN would place 46% of the cohort into CT vs. observation – an uncertain range, but certainly higher than 10%.

Regardless, in their stated comparison, the true winner depends on the value-weighting of sensitivity and resource utilization. PECARN approached 100% or 99% sensitivity, missing only 1 patient with clinically important traumatic brain injury out of ~10,000. Contrawise, CATCH and CHALICE missed 13 and 12 out of ~13,000 and ~14,000, respectively. Most of these did not undergo neurosurgical intervention, but a couple missed by CHALICE and CATCH would. However, as noted above, PECARN is probably substantially less specific than both CATCH and CHALICE, which has relatively profound effect on utilization for a low-frequency outcome.

Ultimately, however, any of these decision instruments is usable – as a supplement to your clinical reasoning. Each of these rules simplifies a complex decision into one less so, with all its inherent weaknesses. Fewer than 1% of children with mild head injury need neurosurgical intervention and these are certainly rarely missed by any typical practice. In settings with high CT utilization rates, any one of these instruments will likely prove beneficial. In Australia and New Zealand – as well as many other places around the world – potentially not so much.  This is probably a fine example of the need to compare decision instruments to clinician gestalt.

“Accuracy of PECARN, CATCH, and CHALICE head injury decision rules in children: a prospective cohort study”

http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(17)30555-X/abstract

Making Urine Cultures Great Again

As this blog covered earlier this month, the diagnosis of urinary tract infection – as common and pervasive as it might be – is still fraught with diagnostic uncertainty and inconclusive likelihood ratios. In practice, clinicians combine pretest likelihood, subjective symptoms, and the urinalysis to make a decision regarding treatment – and invariably err on the side of over-treatment.

This is an interesting study taking place in the Nationwide Children’s Hospital network regarding their use of urine cultures. In retrospect, these authors noted only half of patients initially diagnosed with UTI had the diagnosis ultimately confirmed by contemporaneous urine culture. Their intervention, then, in order to reduce harm from adverse effects of antibiotics, was to contact patients following a negative urine culture result and request antibiotics be stopped.

This tied into an entire quality-improvement procedure simply to use the electronic health record to accurately follow-up the urine cultures, but over the course of the intervention, 910 patients met inclusion criteria. These patients were prescribed a total of 8,648 days of antibiotics, and the intervention obviated 3,429 (40%) of those days. Owing to increasing uptake of the study intervention by clinicians, the rate of antibiotic obviation had reached 61% by the end of the study period.

There are some obvious flaws in this sort of retrospective reporting on a QI intervention, as there was no reliable follow-up of patients included. The authors report no patients were subsequently diagnosed with a UTI within 14 days of being contacted, but this is based on only 46 patients who subsequently sought care within their healthcare system within 14 days, and not any comprehensive follow-up contact. There is no verification or antibiotics actually being discontinued following contact. Then, finally, antibiotic-free days are only a surrogate for a reduction the suspected adverse events associated with their administration.

All that said, this probably represents reasonable practice. Considering the immense frequency with which urine cultures are sent and antibiotics prescribed for dysuria, the magnitude of effect witnessed here suggests a potentially huge decrease in exposure to unnecessary antibiotics.

“Urine Culture Follow-up and Antimicrobial Stewardship in a Pediatric Urgent Care Network”
http://pediatrics.aappublications.org/content/early/2017/03/14/peds.2016-2103

The Solution to Dilution is ….

Do we order a lot of urinalyses? Does the sun rise in the east? Does a bear ….

For a test we order with great frequency, there is actually quite a bit of complexity in its interpretation. The combination of symptoms, clinical context, the balance between sample contamination and presence of white blood cells, of nitrites and/or leukocyte esterase, and so on, can make it a relatively tricky test to interpret. The gold standard remains a urine culture.

Now – if you haven’t been already – you probably ought to be taking into account the urine specific gravity, as well.

This retrospective analysis of 14,971 children for whom paired urinalyses and urine cultures were available describes the test characteristics of WBCs/hpf, LE, and nitrites as stratified by urine specific gravity. There are a lot of numbers in this article – a “zillion” to be precise – across eighteen dense tables of +LR/-LR, sensitivity/specificity, and PPV/NPV, but the basic gist of the matter is: variations in urine concentration diminish the value of the test in different ways. As urine specific gravity increases, it becomes more likely a patient will not have a positive urine culture despite having typically diagnostic amounts of WBCs/hpf, +LE, and/or +nitrites. Likewise, with dilute urine, a lower threshold for WBCs/hpf may be needed to have adequate sensitivity.

Just one more layer to consider in this frequently used test of under-appreciated complexity.

“The Importance of Urine Concentration on the Diagnostic Performance of the Urinalysis for Pediatric Urinary Tract Infection”
https://www.ncbi.nlm.nih.gov/pubmed/28169050

Let’s Hyperpronate!

There are two ways to treat a Nursemaid’s elbow (radial head subluxation) – supination/flexion or hyperpronation. I’ve done both. I’m a fan of hyperpronation, so, therefore, I’m highlighting a study that agrees with my practice pattern.

This systematic review covers 7 studies enrolling 701 patients, comparing the success rate and perceived pain of each technique. Trials were, generally and for some obvious reasons, limited in terms of blinding and outcome assessment. Pooled failure rate with hyperpronation 9.1% was while failure with supination /flexion was 27.3%. In studies reporting pain scores, subjective external rating of child pain during procedure also favored the hyperpronation group.

Use it!

This video demonstrates both techniques, although I’ve seen variations on hyperpronation both using extension and flexion.

“Effectiveness of reduction maneuvers in the treatment of nursemaid’s elbow: A systematic review and meta-analysis”
https://www.ncbi.nlm.nih.gov/pubmed/27836316

Imprecise Dosing of Liquid Medications

“Many parents are overdosing their kids, study says”. Is this true? Are parents poisoning their own children, as the headline implies?

Of course not; this is not in fact a study regarding overdose incidence at all. It is, quite simply, a measurement precision study.

This study involves 2,110 parents randomly assigned to measure doses of liquid medication in various quantities using either a cup, a 0.2mL syringe, or a 0.5mL syringe. Approximately a quarter of parents were >20% off with their measurement, and 2.9% doubled the instructed dose. Taking these results as a surrogate for overdose depends on the therapeutic range for a medication – so, while the headline is not technically incorrect, the implication is an exaggeration.

With regard to measurement and dosing errors, there were a few important trends to note. Health literacy had a large influence on dosing errors – regardless of whether teaspoons or mL were used in the instructions. Then, the cup: avoid the cup when possible. Almost three-quarters of parents committed measurement or dosing errors when asked to provide a 2.5mL dose in the cup. Stick to the syringe and target round numbers (5mL) to minimize errors.

With regard to the premise of overdose – for medications with a wide therapeutic range, these data are not quite as clinically relevant. However, for high-risk medications, more time and effort should be taken to demonstrate proper dosing with parents.

“Liquid Medication Errors and Dosing Tools: A Randomized Controlled Experiment”
http://pediatrics.aappublications.org/content/early/2016/09/08/peds.2016-0357

The Febrile Infant Step-by-Step

You’ve heard of the Philadelphia Criteria. You’ve heard of the Rochester Criteria. But – Step-by-Step?

This is an algorithm developed by European emergency physicians to identify low-risk infants who could be safely managed without lumbar puncture nor empiric antibiotic treatment. After retrospectively validating their algorithm on 1,123 patients, this is their prospective validation in 2,185 – looking for IBI or “Invasive Bacterial Infection” as their primary outcome.

The easiest way to summarize their algorithm and results is by this figure:

Step by Step

Sensitivity and specificity, respectively, were as follows:

  • Rochester – 81.6% and 44.5%
  • Lab-score – 59.8% and 84.0%
  • Step-by-Step – 92.0% and 46.9%

The authors attribute 6 of the 7 missed by Step-by-Step to evaluation early in the disease process – presentation within 2 hours of onset of fever.

Their algorithm is reasonable at face validity, and could be incorporated into a protocol with close follow-up to re-evaluate those early in their disease process. We still have, however, a long way to go regarding specificity.

“Validation of the “Step-by-Step” Approach in the Management of Young Febrile Infants”
http://www.ncbi.nlm.nih.gov/pubmed/27382134

Putting Children to the Flame

Many readers here are students, trainees, or otherwise academic-affiliated, and have limited exposure to the world of community practice.  In these settings, frequently, our pediatric exposure is supervised by clinician-educator sub-specialists in Pediatric Emergency Medicine.  We see the very best evidence translated into acute care of children in the Emergency Department.

The real world is a little different.

These two articles describe the shortcomings of advanced imaging practice in community pediatric settings – in the diagnosis of appendicitis, and in the evaluation of closed head injury.

In the appendicitis article, the authors compare two settings both staffed by PEM physicians – an academic medical center with in-house pediatric surgical coverage, and a community center with consultation available only by phone.  Each site had similar rates of appendicitis diagnoses – 4.7% vs. 4.0% at the academic and community site, respectively.  The academic site, however, evaluated fewer patients with abdominal pain with blood work, and then fewer still of those went on to advanced imaging.  Then, of those receiving advanced imaging, the rates were 10.8% CT at the academic center vs. 28.1% CT at the community center.  Ultrasound however, was employed in 16.6% of cases at the academic center versus 6.5% at the community practice.  Nearly all this difference, however, seemed to be made up of patients admitted to the hospital without any operative intervention.  The obvious reality, then:  radiation in lieu of observation.

The second article here describes the neuroimaging (CT or MRI) of patients evaluated following trauma, along with their ultimate disposition.  Of 2,679 patients reviewed, there were 94 patients with important non-surgical, trauma-related diagnoses, and an additional 16 patients who required neurosurgical intervention.  These authors, however, based on GCS estimates recorded and the distribution of outcomes in the PECARN study, estimate the prevalence of entry criteria into appropriate scanning would have obviated >2000 of these scans.  While I believe they are probably mis-applying the evidence and overstating the inappropriateness of CT, the rarity of serious diagnoses suggests at least a majority of these CTs probably could have been avoided.

In short, we’re still doing too many CTs on children.  Some of the contributing issues are systems based, and some are related to practice re-education.  More ultrasound and more observation, please – and less nuking of children.

“Imaging for Suspected Appendicitis: Variation Between Academic and Private Practice Models”
https://www.ncbi.nlm.nih.gov/pubmed/27050738

“Neuroimaging Rates for Closed Head Trauma in a Community Hospital”

Where Acute Otitis Media is Born

Is it 3 AM in your Emergency Department?  Is there a febrile infant with their still-awake parents straggling in the door?  Do you hear the first few bars of the “it’s just a virus” song start playing over Spotify?

This little study prospectively enrolled healthy infants at birth and followed them to their first episode of acute otitis media or 12 months of age.  They were followed specifically to determine predictive clinical and epidemiological factors influencing the first diagnosis of AOM.  Additionally, as they aged and during illness, nasopharyngeal swabs were taken to evaluate viral and bacterial flora.

Based on a sample of 367 infants followed for a total of 286 child-years, there were 887 presentations for viral URIs and 180 presentations for AOM –and all but two of AOMs were preceded by a URI.  The median time from URI presentation to AOM diagnosis was 3 days.  These authors also present a fair bit of microbiologic data regarding specific risks for URI and AOM, although these are not specifically modifiable and of lesser clinical relevance.  From a modifiable environmental outlook, however, there are a few interesting tidbits tying into what we already suspected to be true:  breastfeeding is good, the new PC13 vaccine is good, and daycare is a cesspool.

Overall, this would tend to support our typical advice to parents to have their children present for a recheck 48-72 hours following Emergency Department visit, particularly if there has not been clinical improvement or in the context of apparent clinical re-worsening.

“Acute Otitis Media and Other Complications of Viral Respiratory Infection”
https://www.ncbi.nlm.nih.gov/pubmed/27020793

You Can Make Unnecessary Care Go Away

Low-value care is such a pervasive problem, ABIM developed the Choosing Wisely initiative.  However, Choosing Wisely is, unfortunately, a disengaged and toothless activity.  And it hasn’t worked.

But, as this study shows, you can eliminate unnecessary care with a more proactive and involved approach.

This is a quality improvement collaborative across 21 hospitals aimed at reducing the use of unnecessary or ineffective care relating to bronchiolitis.  As we’ve seen time and time again, if anything works at all for bronchiolitis, the ambiguity over its effectiveness probably means the effect size is clinically meaningless.  To this end, these hospitals banded together to deploy a QI program targeting reductions in bronchodilator use, steroid use, chest radiography, and other process measures.  Across all measures, the pre- and post-intervention measures demonstrated pooled meaningful and statistically significant improvement.  Bronchodilator use dropped from 46.2% to 32.7%, steroids from 10.9% to 2.2%, and CXRs from 12% to 6.7%.  A secondary effect of these interventions was a reduction in length of stay by 5 hours, from 49.6 to 44.6.

Success!

Of course, the QI intervention did not have the same effect at all participating hospitals.  Some, clearly, were on the ball, and almost entirely eliminated some unnecessary care (steroids).  Others, however, had no change from baseline, or, even, an increase – like two hospitals demonstrating 150%+ increase in bronchodilator use, and three hospitals with 100%+ increases in CXR use.  It would be interesting to see some qualitative analysis regarding the lack of improvement at certain hospitals.

But, in general, widespread improvement in unnecessary care can be realized.  In contrast to Choosing Wisely, it requires motivated agents of change and constant feedback.  The Choosing Wisely lists and their elements, unfortunately, seem adrift.

“A Multicenter Collaborative to Reduce Unnecessary Care in Inpatient Bronchiolitis”