USMLE Step 1 Exam Facts

USMLE Step 1 Exam Facts

When should I take Step 1?

Most medical students take the USMLE Step 1 test after finishing their basic science classes and before starting the clinical clerkships, usually in the summer between the second and third years at most programs. Many institutions offer a 1-2 month break during this time for students to study and take the test, with some students deciding to take it earlier and have a longer break whereas others use as much time as possible for studying. This decision may have a large impact, given the heavy emphasis placed on the test results by program directors.

A 2002 study looked at student scores based on the timing of when they took the USMLE Step 1. Students who took the exam earlier tended to have a better academic record based on preclinical grades and MCAT scores. Many students who took the exam later did so because of personal issues or a need for remediation after second year classes. As might be expected, the pass rate among students who took the exam earlier was higher than it was among those who took it later, but this is confounded by the difference in academic record between these two groups. When performance was adjusted based on these variables there was no significant difference in scores related to when the exam was taken.

One can approach this data in several ways. Decreased retention of material with longer delays after finishing the basic science classes is unlikely to be much of a factor as there is a relatively short interval between when most students finish basic science classes and take the exam anyway. This is in contrast to Step 2 where there is often a longer interval between when students finish clinical clerkships and take the exam, a factor which has been correlated with performance on the test . In addition, students who fear they will do poorly on the test and take extra time to study may not do any better on the test than if they had taken it earlier.

Reference

Pohl CA, Robeson MR, Hojat M, Veloski JJ. Sooner or later? USMLE Step 1 Performance and Test Administration Date at the End of Second Year. Academic Medicine 77(10), October Supplement 2002, S17-S19.

How much basic science material from USMLE Step 1 will I end up remembering in the long term?

The retention and relevance of basic science material learned during medical school has long been a concern. This is part of the reason there has been a shift in medical education towards including more clinically relevant material during the first two years of medical school. There has also been a shift towards more patient cases being included in USMLE Step 1; previously only 30% of cases used a patient-based format, whereas now roughly 70% of cases do.

A 2008 study examined the results of including unscored Step 1 questions on the Step 2 CK exam. Students answering the questions as a component of the Step 2 CK exam, i.e. later in their medical training, performed approximately 6-7% poorer than students answering the questions as a component of the Step 1 exam. This decline was greater for questions that were presented in a style other than a patient vignette. The question category with the largest drop in performance was biochemistry, followed by microbiology and pharmacology. The drops were less dramatic for physiology, anatomy, and pathology. Performance actually increased for human behavior related questions. Changes in performance likely reflect emphasis placed during clinical clerkships, where biochemical principles are rarely discussed but aspects of human behavior such as communication and mental disorders are frequently discussed.

Similar studies have been conducted over the past three decades. Interestingly, the results of the most recent study were similar to those of earlier studies despite the changes made to medical education. In fact, the magnitude of decline was actually worse on the more recent study than those from years prior to changes in medical school curricula. Some of this may be secondary to fourth year medical students taking Step 2 later in the year, therefore increasing the interval since Step 1 and decreasing retention of the material.

Reference

Ling Y, Swanson DB, Holtzman K, Bucak SD. Retention of Basic Science Information by Senior Medical Students. Academic Medicine, Vol. 83, No. 10 / October 2008 Supplement, 582-585.

How much is my performance on the USMLE exams affected by the medical school I attend?

A variety of factors go into a student's choice of medical school. Schools vary in geographic location, curriculum, teaching quality, and reputation. It is difficult to compare medical schools in overall quality, except in the form of published rankings, which are of questionable validity. It is also unclear if differences between medical schools significantly impact a student's eventual performance in clinical situations or on standardized tests.

A 2008 study compared USMLE scores across multiple medical schools. Overall USMLE scores did vary among institutions, as would be expected. However, the majority of these differences could be accounted for by differences in incoming students, particularly MCAT scores. In other words, schools admitting students who are better test-takers tended to have higher average USMLE scores. Approximately 85% of the variation between schools can be traced to this variation among incoming students.

Only about 15% of the variation in USMLE score across different medical schools could be traced to factors related to the school itself. Little of this difference could be traced to differences in curriculum or school level-educational policies. Private school students tended to perform better on Step 1 but worse on Step 3. The geographic location of the school also impacted scores. This study did not directly address the quality of teaching that students receive, in part because this is difficult to assess and quantify. Overall these findings suggest that the majority of variation in USMLE performance among different schools is likely secondary to traits that the students bring into the medical school environment, rather than differences between medical schools themselves.

Reference

Hecker K, Violato C. How Much Do Differences in Medical Schools Affect Student Performance? A Longitudinal Study Employing Hierarchical Linear Modeling. Teaching and Learning in Medicine 20(2), 2008, 104-113.

Are my MCAT scores predictive of my USMLE and medical school performance?

Much as the USMLE Step 1 score is an important variable in the residency application process, the MCAT score is given important weight in the medical school application process. Medical school admissions officers are faced with the daunting task of determining which students are the most qualified, comparing applicants from different institutions of different quality and grading schemes. The MCAT is meant as an objective and consistent means of comparison, though other factors are certainly considered in the medical school application process.

A 2007 meta-analysis evaluated the relationship between the MCAT and medical school performance. There was a small to moderate correlation of MCAT performance with performance during the preclinical years, with the r value equal to roughly 0.39. There was also a correlation of certain subtests with preclinical performance, particularly the biological sciences subtest. The correlation was less strong between MCAT performance and performance during the clinical years. The writing subtest of the MCAT was shown to have no correlation with either performance during the preclinical years or the clinical clerkships.

A small to moderate correlation was also found between MCAT and USMLE scores, with r values ranging from 0.38 to 0.60. This correlation was highest for USMLE Step 1. Among the different MCAT subsets, the highest correlation was found for the biological sciences and verbal sections. There was near zero correlation between the writing subtest and USMLE scores.

Reference

Donnon T, Paolucci EO, Violato C. The Predictive Validity of the MCAT for Medical School Performance and Medical Board Licensing Examinations: A Meta-Analysis of the Published Research. Academic Medicine 82(1), January 2007, 100-106.

Are my USMLE scores affected by whether my medical school has a problem-based or traditional curriculum during the basic science years?

Over the past several decades, many medical schools have changed their curriculum from a more traditional format to problem-based learning (PBL). Studies have shown that graduates of PBL curriculums tend to have better clinical performance during residency, though not to a statistically significant degree. Students also generally prefer the PBL curriculum when given the choice. However, some worry the PBL approach may compromise basic science education, potentially hindering performance on USMLE exams, particularly Step 1. Multiple studies have examined this concern and the results have been mixed. The majority of these studies have demonstrated no significant difference in test performance between students on the two tracks. Exceptions include one study of students at the University of New Mexico which demonstrated a statistically significant increase in Step 1 scores among students taking a traditional curriculum and a trend towards improved Step 2 performance in students taught with a PBL curriculum. A study of University of Missouri students demonstrated improved performance on Steps 1 and II with a PBL curriculum. Students at Wake Forest University and Michigan State University showed similar test performance regardless of teaching style. While these studies do not draw any hard and fast conclusions, they suggest that the two tracks are roughly equivalent in terms of how they prepare students for the USMLE. A potential confounder in these studies is that students with less basic science background entering medical school may favor programs offering a PBL-style curriculum, as decreased basic science background is an independent predictor of poorer USMLE performance.

Reference

Enarson C, Cariaga-Lo L. Influence of Curriculum Type on Student Performance in the United States Medical Licensing Examination Step 1and Step 2 exams: Problem-based Learning vs. Lecture-based Curriculum. Medical Education 35, 2001,1050-1055.