Clinical Informatics Exam Post-Mortem

I rarely break from literature review in my blog posts (although, I used to make the occasional post about Scotch).  However, there are probably enough folks out there in academia planning on taking this examination, or considering an Emergency Medicine Clinical Informatics fellowship – like the ones at Mt. Sinai, BIDMC, and Arizona – to make this diversion of passing interest to a few.

Today is the final day of the 2015 testing window, so everyone taking the test this year has already sat for it or is suffering through it at this moment.  Of course, I’m not going to reveal any specific questions, or talk about a special topic to review (hint, hint), but more my general impressions of the test – as someone who has taken a lot of tests.

The day started out well, as the Pearson Vue registration clerk made a nice comment that I’d gone bald since my picture at my last encounter with them, presumably for USMLE Step 3.  After divesting myself of Twitter-enabled devices, the standard computer-based multiple-choice testing commenced.

First of all, for those who aren’t aware, this is only the second time the American Board of Preventive Medicine has administered the Clinical Informatics board examination.  Furthermore, there are few – probably zero – clinicians currently taking this examination who have completed an ACGME Clinical Informatics fellowship.  They simply don’t exist.  Thus, there is a bit of a perfect storm in which none of us have undergone a specific training curriculum preparing us for this test, plus minimal hearsay/experience from folks who have taken the test, plus a test which is essentially still experimental.

Also, the majority (>90%) of folks taking the test use one of AMIA’s review courses – either the in-person session or the online course and assessment.  These courses step through the core content areas describe for the subspecialty of Clinical Informatics, and, in theory, review the necessary material to obtain a passing score.  After all, presumably, the folks at AMIA designed the subspecialty and wrote most of the questions – they ought to know how to prep for it, right?

Except, as you progress through the computer-based examination, you find the board review course has given you an apparently uselessly superficial overview of many topics.  Most of us taking the examination today, I assume, are current or former clinicians, with some sort of computer science background, and are part-time researchers in a subset of clinical informatics.  This sort of experience gets you about half the questions on the exam in the bag.  Then, about a quarter of the course – if you know every detail of what’s presented in the review course regarding certification organizations, standards terminologies, process change, and leadership – that’s another 50 out of 200 questions you can safely answer.  But, you will need to really have pointlessly memorized a pile of acronyms and their various relationships to get there.  Indeed, the use of acronyms is pervasive enough it’s almost as though their intention is more to weed out those who don’t know some secret handshake of Clinical Informatics, rather than truly assess your domain expertise.

The last quarter of the exam?  The ABPM study guide for the examination states 40% of the exam covers “Health Information Systems” and 20% covers “Leading and Managing Change”.  And, nearly every question I was trying to make useful guesses towards came from those two areas – and covered details either absent from or addressed in some passing vagueness in the AMIA study course.  And, probably as some consequence of this being one of the first administrations of this test, I wasn’t particularly impressed the questions – which were heavy on specific recall, and not hardly on application of knowledge or problem solving.  I’m not sure exactly what resources I’d use to study prior to retaking if I failed, but most of the difference would come down to just rote memorization.

However, because the pass rate was 92% last year, and nearly everyone taking the test used the AMIA course, an average examinee with the average preparation ought yet to be in good shape.  So, presumably, despite my distasteful experience overall – one likely shared by many – we’ll all receive passing scores.

Check back mid-December for the exciting conclusion to this tale.

Update (as noted in comments below):  Passed!

Hopefully future editions of prep courses will gradually attune themselves to the board content, once a few iterations have progressed.  Individuals taking this exam, in the meantime, will need to rely heavily on their medical or prior technical experience, particularly as the curricula for fellowships are fleshed out.  Additionally, the CI exam content is so broad, fellowship trainees will need to specifically target their coursework to areas they lack – for, example, “Leading and Managing Change” as a major content area of the examination will definitely force many informaticians into a knowledge gap.

Interesting times!