OBJECTIVE To develop an assessment of clinical competence of graduates of foreign medical schools and to determine the reliability and validity of the assessment and the feasibility of large-scale administration. METHODS The Educational Commission for Foreign Medical Graduates (ECFMG) clinical competence study included (1) clinical encounters with standardized patients to assess history taking, physical examination, and communication skills; (2) laser videodisk pictorials to assess identification and interpretation of diagnostic procedures; (3) written clinical vignettes to assess diagnosis and management skills; and (4) assessment of spoken English. A uniform method of operating the test centers and of training the standardized patients was developed. METHODS Medical schools and their primary teaching hospitals and affiliated hospitals. METHODS Six hundred twenty-four first-year residents, of whom 525 are graduates of foreign medical schools. METHODS Scores, reliability coefficients, validity measures, feasibility of multisite administration, trends of scores over time, and acceptability by examinees. RESULTS The ECFMG clinical competence assessment was conducted at four geographically separate test centers. Reliability coefficients were high (.85) for the integrated clinical encounter and were in a reasonable range (.71 to .82) for all test components. The assessment adds to the predictability of the residents' performance in the hospital over that of current ECFMG certification examinations. Test security was addressed by demonstrating no consistent pattern of change in scores over testing dates. Virtually all examinees thought the assessment was appropriate. Standardized patients were able to assess spoken English accurately. CONCLUSIONS The feasibility of conducting a reliable and valid test of clinical competence for graduates of foreign medical schools was demonstrated for this test population.