The checklist has reached ascendant status in medicine. As introduced into the mainstream by Atul Gawande, they have begun to permeate every nook of healthcare delivery. However, evidence of benefit when applied to one particular problem in one particular setting is no guarantee of universal utility.
These authors performed a study in Brazilian intensive care units, using a cluster-randomized pre/post design to evaluate the effect of a quality improvement effort built around a checklist. Each element on the checklist represented a consensus or evidence-based practice associated with improvement in surrogate markers for patient outcomes. The combined intervention was hoped to improve overall in-hospital mortality for ICU patients at the intervention hospitals.
It didn’t – mortality increased similarly for both intervention and control ICUs.
In fact, for all secondary clinical outcomes – catheter-related infections, ventilator-associated pneumonia, urinary tract infections, ICU days, etc. – there were no significant improvements over the baseline period, and no difference compared with controls. There were small improvements in processes of care, such as VTE prophylaxis, catheter use, and appropriate tidal volumes during ventilation – but without corresponding clinical outcome improvement.
Interestingly, clinicians working in the intervention ICUs typically felt as though their ICUs were safer. They were more likely, sometimes significantly so, to provide answers reflecting positive associations regarding their working conditions and safety climate. Indeed, the intervention was perceived as so likely to be beneficial even prior to the start of the study that a short duration was mandated for the trial so all ICUs could eventually start using the checklist.
These authors have several justifications for why their checklist did not function appropriately, focusing on various details regarding the trial. I think the simplest expression regarding the effectiveness of a checklist relates to the magnitude of effect and the baseline frequency of adherence. Unless a significant magnitude of effect is seen by improving compliance with an intervention, and the intervention itself is infrequently performed, returns will diminish dramatically. A checklist such as this, with multiple low-yield elements, is unlikely to return substantial patient-oriented outcome improvements. Indeed, the resources devoted to checklist rounding and adherence may even dilute the focus on important clinical considerations.
“Effect of a Quality Improvement Intervention With Daily Round Checklists, Goal Setting, and Clinician Prompting on Mortality of Critically Ill Patients”
https://www.ncbi.nlm.nih.gov/pubmed/25928627
I think the simplest expression regarding the effectiveness of a checklist relates to the magnitude of effect and the baseline frequency of adherence.