Simulation-based emergency medicine resident self-assessment

Background: Videotaped recordings of simulation-based performance may allow learners the opportunity to review, evaluate, and reflect upon their own performance.

Objectives: To determine the accuracy of resident performance self-assessment after a simulation-based encounter; compare low- and high-scoring residents' abilities to evaluate their performance; and determine if video-assisted performance review improves self-assessment accuracy.

Methods: Emergency Medicine residents participated in a videotaped simulation-based assessment. Residents evaluated their performance immediately after completing simulated cases, and after reviewing the session's video. Self-ratings were compared to the faculty observers, and scores were divided based on the median.

Results: Seventeen residents participated, providing 270 self-ratings before, and 269 after, video review. Before video review, residents accurately graded their performance in 73.7% of the items. High- and low-scoring residents accurately self-assessed 83.9% and 62.2% of items, respectively. The odds of a high scorer accurately rating their own performance were 3.2 times that of a low scorer (95% confidence interval [CI] 1.9-5.2, p<0.001). After video review, resident self-assessments were accurate for 73.6% of the items. High scorers were accurate in their post-video self-assessment in 83.3% of the items, vs. 62.4% for low scorers. After video review, the odds of a high scorer accurately self-rating their performance were 3.0 times that of a low scorer (95% CI 2.1-4.1, p<0.001).

Conclusions: Residents' abilities to self-assess vary, and performance quality may influence self-assessment. Video review did not significantly increase self-assessment accuracy. Improving self-assessment skills may assist residents in identifying practice gaps, thereby allowing them to focus their energy toward filling that gap.