PERSPE C T I V E

1. Medicaid and CHIP Payment and Access

Commission. Report to Congress on Medicaid
and CHIP. March 2011. (http://healthreform
.kff.org/~/media/Files/KHS/docfinder/
MACPAC_March2011_web.pdf.)
2. 42 U.S.C. 1396a(a)(30)(A).
3. Rosenbaum S. Medicaid payment rate

Medicaid and Access to Health Care

lawsuits: evolving court views mean uncertain future for Medi-Cal. Oakland: California
HealthCare Foundation, October 2009.
(http://www.chcf.org/~/media/Files/PDF/M/
PDF%20MediCalProviderRateLitigation
.pdf.)
4. Toby Douglas v. Independent Living Cen-

ter of Southern California, Inc., Nos. 09-958,

09-1158 and 10-283 (May, 2011).
5. Centers for Medicare & Medicaid Services. Medicaid program: methods for assuring access to covered Medicaid services. Fed
Regist 2011;76(88):26342-62.
Copyright 2011 Massachusetts Medical Society.

Money for Nothing? The Problem of the Board-Exam

Coaching Industry
Joshua Tompkins, B.A.

o paraphrase Mark Twain,

reports of the demise of
for-profit medical education are
greatly exaggerated. Although
proprietary medical schools vanished from the United States
nearly a century ago in the aftermath of the Flexner Report, the
for-profit sector has seen a recent
revival: standardized tests such as
the Medical College Admission
Test (MCAT) and the U.S. Medical
Licensing Exam (USMLE) have
spawned a multi-million-dollar industry in commercial exam-preparation assistance. Seizing upon
and perhaps fueling student
anxiety over high-stakes board exams, test-prep firms market aggressively on campuses, publish
encyclopedic study guides, and offer expensive classroom courses.
Unburdened by accreditation requirements, medical exam-preparation companies have become
de facto adjuncts to the formal
curriculum by doing what colleges
and medical schools arent supposed to do: teaching to the tests.
The source of demand for
exam coaching, as the industry
is known in academic circles, is
easy to trace. First, the increasing competition for spots in medical school is causing applicants
anxiety. The annual number of
candidates has increased by 24%
since 2002. George Washington
University Medical School, for example, recently received 14,008
104

applications for a class of 177.

Nationwide, those who get accepted not only have higher grade
point averages than the applicant
pool as a whole (3.67 vs. 3.53 out
of a maximum of 4.0), but their
MCAT scores are also much better
(31.1 vs. 28.3 out of a maximum of
45). Studies have shown both metrics to be strong predictors of medical school performance, and admissions committees weight them
accordingly. This means that a single exam, the MCAT, can dash an
applicants dream of a medical career, and the resulting evaluation
apprehension the sociological phenomenon of feeling anxious about being judged sends
students running to the coaching
firms.
In medical school, the pressure
grows even more intense. Students
must take the USMLE Step 1 before starting their hospital clerkships, and residency programs in
the more competitive specialties
are known to use stringent cut
points for Step 1 scores to winnow
their applicant pools. This is a
questionable practice (Step 1 scores
have little correlation with residency performance1) that is not
endorsed by the tests creators,
who intended the exam merely
as the first checkpoint in the
journey to licensure. But because
medical students are aware of
residency programs tactics, the
threat of mathematical elimina-

tion has bred a widespread sense

of desperation regarding Step 1.
Given the high stakes, many
preclinical students are tempted
to seek outside help. And that
help is plentiful: USMLE World,
USMLERx, USMLE Consult,
USMLEasy the gamut of supposed performance enhancers for
Step 1 eclipses even the array of
MCAT prep programs. Shelves at
the campus bookstore are lined
with workbooks, flashcard sets,
and software. (Far less help is
available for the USMLE Steps 2
and 3, because students consider
those exams easier and less important.) One thick guidebook
encapsulating the essential material, First Aid for the USMLE Step 1,
has become so cherished that
some second-year students read
it from cover to cover repeatedly.
This monomania subverts the
process of high-quality learning.
Preparation for Step 1, which is a
test of basic science, entails cramming for instant recall instead
of long-term retention. Memorizing the exams well-known tropes
and idiosyncrasies, students learn
to interpret information as test
takers, not clinicians, by employing pattern recognition instead
of differential diagnosis. Because
Step 1 is also believed to focus
on obscure minutiae and rare
conditions, students spend less
time studying the common diseases they will face during clerk-

n engl j med 365;2 nejm.org july 14, 2011

The New England Journal of Medicine

Downloaded from nejm.org at UNIV OF LOUISVILLE on July 14, 2011. For personal use only. No other uses without permission.
Copyright 2011 Massachusetts Medical Society. All rights reserved.

PERSPECTIVE

ships, residency, and practice.

Many neglect their coursework
as their test date approaches.
For students who want more
structured assistance in preparing, several companies still offer
private tutoring and live classroom courses for prospective takers of Step 1, the MCAT, and
other exams. The biggest name
in the business is Kaplan, a company that was founded by Stanley
Kaplan in 1938 and is now a subsidiary of the Washington Post
Company. Kaplans Test Preparation division, which reported
revenue of $416 million in 2010,
offers Step 1 assistance ranging
from its popular Qbank, which
features thousands of online practice questions for $109 a month,
to several packages that combine
video-based review with live lectures for $4,399 to $9,499.2
How many medical students
would pay that kind of money?
Statistics are scarce, but a 3-year
study of students at Loma Linda
University School of Medicine
found that 21% had signed up
for some form of commercial
Step 1 coaching.3 Such reliance
on coaching may continue after
graduation: a nationwide survey
of chief residents in radiology
found that 83% had attended a
commercial coaching course to
prepare for the American Board
of Radiology exam, and 84% had
been granted time off from their
residency duties to do so.
So do expensive prep courses
result in higher scores? No study
has shown that Step 1 courses
do so, and several have demonstrated that they dont.4 In the
Loma Linda study, investigators
using regression analysis concluded that students hadnt even
benefited by as much as 1 additional correct answer on Step
1 after taking a commercial review course. For the MCAT, pro-

The Problem of the Board-Exam Coaching Industry

fessional coaching yields trivial

improvement at most a halfpoint according to several
large-scale surveys of test takers.
Kaplans Web site doesnt guarantee superior results, nor do the
sites of other test-prep companies.
Because the average medical
student graduates with $158,000
in student-loan debt, creating an
environment in which students
feel compelled to spend thousands more for ineffective exam
coaching seems unconscionable.
For Step 1, the best alternative may
be for medical schools to take an
active, yet extracurricular, role
in helping students prepare. The
Ohio State University College of
Medicine recently experimented
with a 1-year peer-assisted Step
1 review course, in which upperclassmen taught some of the second-year students. The average
Step 1 score of those students
was eight points higher than the
average among classmates who
hadnt taken the course.5
Meanwhile, I believe, medical
educators can alleviate some of
the pressure surrounding Step 1
by developing other assessment
tools. First, if deans offices adhered closely to the guidelines
for Medical School Performance
Evaluations (a.k.a. deans letters)
set forth by the Association of
American Medical Colleges, it
would facilitate the comparison
of applicants from different
schools. Second, residency program directors might rely less
heavily on Step 1 scores and
adopt a more holistic approach
to appraising candidates. To assist in that effort, I believe that
USMLE officials should start
disclosing scores on the Step 2
Clinical Skills exam (a series of
simulated patient encounters that
was added to the USMLE in
2004) so that residency applicants
can include them in their files.

These changes would help

bandage the system until the
next complete overhaul takes
place. A radical shake-up of the
USMLE series has been in development since 2006, and the revision, whose exact timetable
has not been announced, will replace the current three-step sequence with one broad array of
exams (possibly stressing skills
such as communication and professionalism) near the end of medical school and another at the conclusion of residency training.
Yet for-profit medical education will persist. The new USMLE
format, while addressing many of
the deficiencies of the current one,
could scare more students and residents into seeking professional
help in preparing for terra incogni
ta. And with at least 14 new medical schools now in various stages
of planning and accreditation
making competition for residency
positions even more bruising
medical students will feel especially pressured to cinch a victory.
But the only guaranteed winners
will be the test-prep firms.
Disclosure forms provided by the author
are available with the full text of this article at NEJM.org.
From the Keck School of Medicine, University of Southern California, Los Angeles.
1. McGaghie WC, Cohen ER, Wayne DB. Are
United States Medical Licensing Exam Step
1 and 2 scores valid measures for postgraduate medical residency selection decisions?
Acad Med 2011;86:48-52.
2. Kaplan Medical. USMLE prep. (http://
www.kaptest.com/Medical-Licensing/
USMLE-Prep/step-1.html.)
3. Werner LS, Bull BS. The effect of three commercial coaching courses on Step One USMLE
performance. Med Educ 2003;37:527-31.
4. McGaghie WC, Downing SM, Kubilius R.
What is the impact of commercial test preparation courses on medical examination performance? Teach Learn Med 2004;16:202-11.
5. Alcamo AM, Davids AR, Way DP, Lynn DJ,
Vandre DD. The impact of a peer-designed
and -led USMLE Step 1 review course: improvement in preparation and scores. Acad
Med 2010;85:Suppl:S45-S48.
Copyright 2011 Massachusetts Medical Society.

n engl j med 365;2 nejm.org july 14, 2011

The New England Journal of Medicine

Downloaded from nejm.org at UNIV OF LOUISVILLE on July 14, 2011. For personal use only. No other uses without permission.
Copyright 2011 Massachusetts Medical Society. All rights reserved.

105