OP-15 for imaging effectiveness in atraumatic headache is coming, vigorously opposed by many. To date, most of the opposition has been in principle, or with specific clinical concerns.
This is a different approach to the problem – looking at whether the patients that CMS identified as “inappropriate” were actually appropriate exceptions. This was a retrospective chart review of 748 charts that were referred back to 21 hospitals as “inappropriate” following a “dry run” of OP-15. Based on individual chart review, the authors found documentation of one of the exclusion criteria in 489 of them – 479 based on the clinical criteria, and 35 based on administrative criteria (some met both). They then look at those 259 patients for whom there is no CMS exception for their CT, and they claim that 136 of those were clinically warranted. They therefore conclude that only 125 of the original 748 were accurately identified by this quality measure as inappropriate use of CT in atraumatic headache, and that this measure is garbage.
And, a quick Google News search finds an extensive parade of indignant headlines pulled from ACEP’s press release, condemning the measure.
But, this study misses the point. It’s not CMS’ responsibility to comb through individual charts to find these exclusion criteria. The onus is on clinicians and hospitals to ensure their documentation clearly expresses the indications for CT in those cases that meet the exclusion criteria, and the purpose of this dry run is to help hospitals identify where the information they are supplying to CMS is deficient.
Then, I expect CMS to take a low opinion of the additional patients in whom these authors felt the imaging was clinically warranted. Of the 78 patients for whom the authors felt ACEP guidelines for imaging were met, 73 of them met only the Level C recommendation: >50 years of age with a new type of headache and a normal neurologic examination. Then, there is another set of patients with headaches on warfarin, who had recent neurosurgery, or had known hydrocephalus that they claim are misclassified by CMS – but I can’t see how the misclassification isn’t on the documentation end, as headaches in all those patients should meet ICD-9 339.44 “Other complicated headache syndrome,” which is an exclusion to the rule as well.
So, even just on first pass, I’m not sure this is an effective tool with which to influence revision of OP-15. I expect this measure to go into effect as planned – and it will be up to us to document appropriately and thoroughly, and then to monitor and demonstrate that compliance results in measurable patient harms.
“Assessment of Medicare’s Imaging Efficiency Measure for Emergency Department Patients With Atraumatic Headache”
http://www.annemergmed.com/webfiles/images/journals/ymem/FA-JDSchuur.pdf
Ryan,
First of all, thanks for writing your blog. I read your updates every time you post them and often pass them along – they’re a great way to initiate discussion about new articles pertaining to EM.
Secondly, thanks for writing about our article. The more light we can shine on the OP-15 issue, the better. While reducing inappropriate utilization of imaging in the ED is definitely a laudable goal, this ill-conceived measure was absolutely the wrong way to do it. I did, however, want to address a couple of your concerns.
While it’s not CMS’ responsibility to pore through charts, it is their responsibility to develop a measure able to capture the information the purport to capture. Their measure uses only billing data to attempt to determine appropriateness – however, there are no ICD-9 or CPT codes for “patient refused the LP after I explained to her that I was worried about a subarachnoid hemorrhage” or “this 84 year old woman on warfarin and clopidogrel fell and hit her head and there aren’t any decisions rules that even remotely address this situation so I decided to get a head CT”. While you might imagine that these patients would have been excluded by CMS based on “complicated headache” or “trauma”, they weren’t (please see the examples of actual cases listed in Figure 4 in our paper) – which is exactly the problem with the measure.
You mentioned that the onus should be on “clinicians and hospitals to ensure their documentation clearly expresses the indications for CT in those cases that meet the exclusion criteria”. What we found on our chart review was that this was almost always already the case, which is how we were able to complete the study. Emergency physicians were documenting everything in these patients’ charts, but since there are usually only three diagnosis codes selected for billing and procedures not performed (like the LP above) are not billed for, there is no way to capture that information in billing data.
Unfortunately, billing for a code like ICD-9 339.44 "Other complicated headache syndrome" may not work either. The definitions of a complicated headache vary in the literature but one common one includes headaches with focal neurologic signs associated with them. None of the patients you suggest (“patients with headaches on warfarin, who had recent neurosurgery, or had known hydrocephalus”) would meet the definition without focal neurologic signs but I think they probably still needed their head CTs. Requiring emergency physicians to add on specific ICD-9 friendly diagnoses seems inappropriate for another reason: if this is truly about better care then teaching us how to play a game by calling a headache “complicated” while still doing the exact same thing clinically is an unnecessary hassle without any gain for the patient whatsoever.
Again, thanks for discussing our article – it’s definitely a controversial topic and, regardless of whether or not you believe our piece is “an effective tool with which to influence revision of OP-15”, I’m glad you’re making your readers aware of it. Keep up the good work!
Sincerely,
Ali
Ali S. Raja MD, MBA, MPH, FACEP
Department of Emergency Medicine
Center for Evidence-Based Imaging
Brigham & Women's Hospital
Assistant Professor
Harvard Medical School
Ryan,
First of all, thanks for writing your blog. I read your updates every time you post them and often pass them along – they’re a great way to initiate discussion about new articles pertaining to EM.
Secondly, thanks for writing about our article. The more light we can shine on the OP-15 issue, the better. While reducing inappropriate utilization of imaging in the ED is definitely a laudable goal, this ill-conceived measure was absolutely the wrong way to do it. I did, however, want to address a couple of your concerns.
While it’s not CMS’ responsibility to pore through charts, it is their responsibility to develop a measure able to capture the information the purport to capture. Their measure uses only billing data to attempt to determine appropriateness – however, there are no ICD-9 or CPT codes for “patient refused the LP after I explained to her that I was worried about a subarachnoid hemorrhage” or “this 84 year old woman on warfarin and clopidogrel fell and hit her head and there aren’t any decisions rules that even remotely address this situation so I decided to get a head CT”. While you might imagine that these patients would have been excluded by CMS based on “complicated headache” or “trauma”, they weren’t (please see the examples of actual cases listed in Figure 4 in our paper) – which is exactly the problem with the measure.
You mentioned that the onus should be on “clinicians and hospitals to ensure their documentation clearly expresses the indications for CT in those cases that meet the exclusion criteria”. What we found on our chart review was that this was almost always already the case, which is how we were able to complete the study. Emergency physicians were documenting everything in these patients’ charts, but since there are usually only three diagnosis codes selected for billing and procedures not performed (like the LP above) are not billed for, there is no way to capture that information in billing data.
Unfortunately, billing for a code like ICD-9 339.44 "Other complicated headache syndrome" may not work either. The definitions of a complicated headache vary in the literature but one common one includes headaches with focal neurologic signs associated with them. None of the patients you suggest (“patients with headaches on warfarin, who had recent neurosurgery, or had known hydrocephalus”) would meet the definition without focal neurologic signs but I think they probably still needed their head CTs. Requiring emergency physicians to add on specific ICD-9 friendly diagnoses seems inappropriate for another reason: if this is truly about better care then teaching us how to play a game by calling a headache “complicated” while still doing the exact same thing clinically is an unnecessary hassle without any gain for the patient whatsoever.
Again, thanks for discussing our article – it’s definitely a controversial topic and, regardless of whether or not you believe our piece is “an effective tool with which to influence revision of OP-15”, I’m glad you’re making your readers aware of it. Keep up the good work!
Sincerely,
Ali
Ali S. Raja MD, MBA, MPH, FACEP
Department of Emergency Medicine
Center for Evidence-Based Imaging
Brigham & Women's Hospital
Assistant Professor
Harvard Medical School
Thanks for the feedback! There are always complexities and issues between the lines that don't always come across in the text of the article. It's great that modern discourse allows this sort of discussion, which will only help to better educate consumers of medical literature.
Where I feel the strength of this paper lies, from the points you make above and from the article itself, is that CMS simply cannot implement the rule they have created. Patients with obvious exclusion criteria are either not documented clearly enough, or CMS is failing to pick up on this clear documentation, and it is an invalid public measure of "quality". And, to say that CMS fails to pick up half of what they claim to measure, that's a strong condemnation of the rule as implemented.
Where I felt your argument was a little weaker was the incorporation of the validity of the rule, in which the remaining cases were reviewed to see if those CTs were clinically appropriate. Mixing together the numbers from the reliability and validity points – while both important critiques of OP-15 – I felt inflated the numbers in a flawed fashion, leading to the exaggerated hyperbole in the lay media.
It is a critically important issue to protect our patients in the face of poorly conceived measures such as OP-15, and this is an important paper to help describe these issues. However, I feel there would have been more integrity to the evidence presented if the reliability and validity arguments hadn't been combined.
Thanks for the feedback! There are always complexities and issues between the lines that don't always come across in the text of the article. It's great that modern discourse allows this sort of discussion, which will only help to better educate consumers of medical literature.
Where I feel the strength of this paper lies, from the points you make above and from the article itself, is that CMS simply cannot implement the rule they have created. Patients with obvious exclusion criteria are either not documented clearly enough, or CMS is failing to pick up on this clear documentation, and it is an invalid public measure of "quality". And, to say that CMS fails to pick up half of what they claim to measure, that's a strong condemnation of the rule as implemented.
Where I felt your argument was a little weaker was the incorporation of the validity of the rule, in which the remaining cases were reviewed to see if those CTs were clinically appropriate. Mixing together the numbers from the reliability and validity points – while both important critiques of OP-15 – I felt inflated the numbers in a flawed fashion, leading to the exaggerated hyperbole in the lay media.
It is a critically important issue to protect our patients in the face of poorly conceived measures such as OP-15, and this is an important paper to help describe these issues. However, I feel there would have been more integrity to the evidence presented if the reliability and validity arguments hadn't been combined.