Reliability Evidence for Examination Cut Scores within a Medical School

Abstract

Establishing credible cut scores for performance-type examinations in health professions education can be
challenging. The authors aimed to compare the pass-fail cut-score reliability with the maximum reliability
cut-score from multiple-choice tests (MCTs) designed on different undergraduate disciplines. Using the
cross-sectional evaluation of 1370 tests from six disciplines from Porto medical school, Portugal, in 2010, the
pass-fail cut-score reliability was obtained from the one-parameter logistic model of item response theory model.
The test information curve achieved maximum reliability for ability levels ranging from -1.40 to -0.01 standard
deviations below the average. The pass-fail cut score for estimated ability ranged from -1.36 to 0.25. These
results showed that all MCTs had a pass and fail threshold of competence, and that was appropriate for the
maximum information obtainable from the examination to occur at the pass and fail level; nevertheless, the
maximum information was not achieved in the pass and fail level.