Constructing a validity argument for the mini-Clinical Evaluation Exercise: a review of the research. 2010

Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
American Board of Medical Specialties, Chicago, Illinois 60601, USA. rhawkins@abms.org

OBJECTIVE The mini-Clinical Evaluation Exercise (mCEX) is increasingly being used to assess the clinical skills of medical trainees. Existing mCEX research has typically focused on isolated aspects of the instrument's reliability and validity. A more thorough validity analysis is necessary to inform use of the mCEX, particularly in light of increased interest in high-stakes applications of the methodology. METHODS Kane's (2006) validity framework, in which a structured argument is developed to support the intended interpretation(s) of assessment results, was used to evaluate mCEX research published from 1995 to 2009. In this framework, evidence to support the argument is divided into four components (scoring, generalization, extrapolation, and interpretation/decision), each of which relates to different features of the assessment or resulting scores. The strength and limitations of the reviewed research were identified in relation to these components, and the findings were synthesized to highlight overall strengths and weaknesses of existing mCEX research. RESULTS The scoring component yielded the most concerns relating to the validity of mCEX score interpretations. More research is needed to determine whether scoring-related issues, such as leniency error and high interitem correlations, limit the utility of the mCEX for providing feedback to trainees. Evidence within the generalization and extrapolation components is generally supportive of the validity of mCEX score interpretations. CONCLUSIONS Careful evaluation of the circumstances of mCEX assessment will help to improve the quality of the resulting information. Future research should address issues of rater selection, training, and monitoring which can impact rating accuracy.

UI MeSH Term Description Entries
D007388 Internal Medicine A medical specialty concerned with the diagnosis and treatment of diseases of the internal organ systems of adults. General Internal Medicine,Medicine, Internal,Internal Medicine, General,Medicine, General Internal
D007396 Internship and Residency Programs of EDUCATION, MEDICAL, GRADUATE training to meet the requirements established by accrediting authorities. House Staff,Internship, Dental,Residency, Dental,Residency, Medical,Dental Internship,Dental Internships,Dental Residencies,Dental Residency,Internship,Internship, Medical,Internships, Dental,Medical Residencies,Medical Residency,Residencies, Dental,Residencies, Medical,Residency,Residency and Internship,Internships, Medical,Medical Internship,Medical Internships,Residencies,Staff, House
D008487 Medical History Taking Acquiring information from a patient on past medical conditions and treatments. Medical History, Previous,Past Medical History, Family,Previous Medical History,Family Health History,Family History, Health,Family History, Medical,Family Medical History,History Taking, Medical,Family Health Histories,Family Medical Histories,Health Family Histories,Health Family History,Health History, Family,History, Previous Medical,Medical Family Histories,Medical Family History,Medical Histories, Previous,Medical History, Family,Previous Medical Histories
D010808 Physical Examination Systematic and thorough inspection of the patient for physical signs of disease or abnormality. Physical Exam,Examination, Physical,Physical Examinations and Diagnoses,Exam, Physical,Examinations, Physical,Exams, Physical,Physical Examinations,Physical Exams
D011594 Psychometrics Assessment of psychological variables by the application of mathematical procedures. Psychometric
D002983 Clinical Competence The capability to perform acceptably those duties directly related to patient care. Clinical Skills,Competence, Clinical,Clinical Competency,Clinical Skill,Competency, Clinical,Skill, Clinical,Skills, Clinical,Clinical Competencies,Competencies, Clinical
D004503 Education, Medical, Graduate Educational programs for medical graduates entering a specialty. They include formal specialty training as well as academic work in the clinical and basic medical sciences, and may lead to board certification or an advanced medical degree. Medical Education, Graduate,Education, Graduate Medical,Graduate Medical Education
D004521 Educational Measurement The assessing of academic or educational achievement. It includes all aspects of testing and test construction. Assessment, Educational,Graduate Records Examination,Educational Assessment,Examination, Graduate Records,Measurement, Educational,Assessments, Educational,Educational Assessments,Educational Measurements,Examinations, Graduate Records,Graduate Records Examinations,Measurements, Educational
D006801 Humans Members of the species Homo sapiens. Homo sapiens,Man (Taxonomy),Human,Man, Modern,Modern Man
D015203 Reproducibility of Results The statistical reproducibility of measurements (often in a clinical context), including the testing of instrumentation or techniques to obtain reproducible results. The concept includes reproducibility of physiological measurements, which may be used to develop rules to assess probability or prognosis, or response to a stimulus; reproducibility of occurrence of a condition; and reproducibility of experimental results. Reliability and Validity,Reliability of Result,Reproducibility Of Result,Reproducibility of Finding,Validity of Result,Validity of Results,Face Validity,Reliability (Epidemiology),Reliability of Results,Reproducibility of Findings,Test-Retest Reliability,Validity (Epidemiology),Finding Reproducibilities,Finding Reproducibility,Of Result, Reproducibility,Of Results, Reproducibility,Reliabilities, Test-Retest,Reliability, Test-Retest,Result Reliabilities,Result Reliability,Result Validities,Result Validity,Result, Reproducibility Of,Results, Reproducibility Of,Test Retest Reliability,Validity and Reliability,Validity, Face

Related Publications

Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
December 2015, Advances in health sciences education : theory and practice,
Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
January 2018, Teaching and learning in medicine,
Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
December 2015, Advances in health sciences education : theory and practice,
Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
October 2007, Medical teacher,
Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
January 2018, Advances in medical education and practice,
Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
December 2015, Advances in health sciences education : theory and practice,
Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
January 2007, The Journal of medicine and philosophy,
Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
September 2002, Academic medicine : journal of the Association of American Medical Colleges,
Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
October 2003, Academic medicine : journal of the Association of American Medical Colleges,
Richard E Hawkins, and Melissa J Margolis, and Steven J Durning, and John J Norcini
August 2008, The Medical journal of Australia,
Copied contents to your clipboard!