Global ratings of videotaped performance versus global ratings of actions recorded on checklists: A criterion for performance assessment with standardized patients

Purpose. To test whether global ratings of checklists are a viable alternative to global ratings of actual clinical performance for use as a criterion for standardized-patient (SP) assessment. Method. Five faculty physicians independently observed and rated videotaped performances of 44 medical students on the seven SP cases that comprise the fourth-year assessment administered at The Morchand Center of Mount Sinai School of Medicine to students in the eight member schools in the New York City Consortium. A year later, the same panel of raters reviewed and rated checklists for the same 44 students on five of the same SP cases. Results. The mean global ratings of clinical competence were higher with videotapes than checklists, whereas the mean global ratings of interpersonal and communication skills were lower with videotapes. The correlations for global ratings of clinical competence showed only moderate agreement between the videotape and checklist ratings; and for interpersonal and communication skills, the correlations were somewhat weaker. Conclusion. The results raise serious questions about the viability of global ratings of checklists as an alternative to ratings of observed clinical performance as a criterion for SP assessment.