Tuesday, April 1, 2008

Fleecing Medical Students: A Cost-Benefit Analysis of Step 2 CS

By: Abhas Gupta
    Update (04/02/08): This analysis refers exclusively to US-based M.D. Students, not D.O. or Foreign Medical Graduates.

    As 4th-year rolls around, the inevitable costs of applying to residencies begin to weigh one down. The list seems endless: application fees, flights, lodging, new suit, new tie, PDA, and of course, licensing exams—Step 2 CS, alone, now costs $1050. Further investigation into the costs and benefits of Step 2 CS suggest that not all licensing requirements are constructive and perhaps, this time, medical students are being taken for a ride.

    Why is Step 2 CS administered in the first place? The USMLE website states:
    Step 2 CS uses standardized patients to test medical students and graduates on their ability to gather information from patients, perform physical examinations, and communicate their findings to patients and colleagues.
    The NBME® reports that in the 2005 academic year, 16,936 medical students took the Step 2 CS and 98% passed2. Of the test takers, 325 had failed previously but overwhelmingly passed this time around (97%) . Put another way: In the 2005 academic year, 4th-year medical students were made to spend $17.8 million (16,936 students x $1050/student) on Step 2 CS alone so that 363 students with unsatisfactory clinical skills could be identified and retested. After retesting, only about 11 students would still fall short of satisfactory clinical proficiency.

    The cost of Step 2 CS to students is substantial; the $17.8 million calculated above does not even include the ancillary costs of traveling and housing—an additional $2 - $3 million—required to attend one of the only five nationwide testing centers. But what about the test's benefits, both to the student and to society? Presumably, identifying poor-performers is better for students and better for patients; students that need remediation are made aware of their weaknesses and patients are safer at the hands of well-vetted interns.

    The relationship between Step 2 CS and these benefits is not as direct as suggested above. There are numerous mechanisms already in place to identify poor-performers both at the medical school and residency levels. A 1999 study published in Academic Medicine2 found that 48% of medical schools had incorporated standardized patients to evaluate the clinical skills of their students. One can reasonably expect that this fraction has increased since the study's publication. Also, all residency programs have numerous internal teaching tools and safety nets as well as external licensing requirements to cultivate and monitor their interns. The value of Step 2 CS can thus be considered minimal in the short term and all-but-useless by residency.

    The justifications for an expensive Step 2 CS are therefore few and trivial. The following are three recommendations that could potentially mitigate the sizeable cost to students and still preserve the test's professional purpose:
    1. Implement other metrics to identify students at high-risk for failing the Step 2 CS - factors such as scores on Step 1 or Step 2 CK, serious clinical infractions, or consistently poor-performance in clerkships could help identify students with clinical skills weaknesses. The Step 2 CS could then be made mandatory for these students and these students only

    2. Minimize redundancy in clinical skills testing – many institutions already test clinical skills, albeit with different formats and styles. The NBME could establish a set of national guidelines dictating how the evaluation should be conducted. Students at institutions that comply with these guidelines would then be exempt from taking the Step 2 CS

    3. Eliminate Step 2 CS from residency programs' entrance criteria – leadership from these programs is necessary for students and medical school administrators to consider any changes to the current system. These programs should be made aware of the negligible benefit of Step 2 CS and come to the aid of their future interns
    These three actions would make the Step 2 CS much more palatable to medical students without detracting from the licensing exam's purpose. I am eager to hear more from other medical students—post your comments anonymously, if you prefer. If there's enough interest, perhaps we can consider broaching this issue with the NBME.

    With the excitement of the democratic primaries, I have been negligent in posting to this blog. If you have specific topics you would like to see discussed, please forward them here. I am currently considering writing on the health care payment system and pay-for-performance incentives.

    References:
    1. 2006 USMLE Performance Data.
    2. Kassebaum, DG, Eaglen, RH. Shortcomings in the Evaluation of Students’ Clinical Skills and Behaviors in Medical School

    3 comments:

    Unknown said...

    Abhas, I'm in complete agreement with your analysis and have often expressed the same sentiments to my classmates. If a medical school is accredited to give medical degrees to its students then it stands to reason that its graduates should be competent in their physical exam skills. After all, this is a major component of the third and fourth year rotations and the sole focus of the Compass evaluations! In addition, I have heard from a few current fourth-years that our school's Compass exam is more rigorous and difficult than the Step II CS.

    Thus the only worthwhile purpose of Step II CS is to provide proof of achieving adequate competency, and it stands to reason that the only students for which it is necessary are those who have shown themselves to be less than competent or borderline. If a med school cannot identify these students on its own, then it does not deserve accreditation. Either way, the issue ostensibly solved by Step II CS actually needs to be addressed at the level of med school administrations rather than on the level of each individual student.

    The only scenario in which it could be argued that the Step II CS is useful in its current format is as another means to stratify students and filter them for residency programs. In its current form, Step II CS does not do this. Another meaningless score to represent who we are is not something we need; however, I suspect that is where we are headed. So let's get rid of the burden of Step II CS before things get worse for us all!

    Unknown said...

    AMEN!

    Anonymous said...

    There are consistent (albeit unsubstantiated) rumors that CS became a requirement for us US-based medical students (vs. the previous FMG-only requirement), after it was found that the testing centers had cost the NBME millions of dollars to build.

    These centers then needed to become income generators. So the policy was then to require all medical students to take CS, and pay the fee (fine..?.), as noted, currently $1050.

    Rumor also has it that CS will be mandatory at least until these testing centers have been well paid off.

    But if business as usual presides (and trust that the NBME is a business), then there will need to be tremendous pressure to eliminate the CS requirement, and ergo disrupt a large volume of cash (over 17 million by your calculations) from entering the coffers of the NBME annually.

    It sounds like a national boycott is in order.