Association for surgical educationAssessment of critical appraisal skills
Section snippets
Examination
Three articles were chosen from leading medical journals by a clinical epidemiologist. They were chosen to be relatively good methodologically, and to represent different methodological topics (effectiveness of treatment, the natural history or prognosis of disease and risk factors or etiology of disease). A clinician with no formal epidemiology training also assessed the articles to ensure agreement that the articles were interesting and clinically relevant [4], [5], [6].
The examination
Results
Forty-four residents participated in the examination. All but one resident completed the examination in three hours or less. The mean score on the total examination was 52.4% (SD 8.6%).
The internal consistency of the 55-item examination was 0.77. The correlation between the clinical epidemiologist and the nonepidemiologist surgeon was 0.91, and between the clinical epidemiologist and the nurse was 0.78. Thus interrater reliability was excellent when a clinician performed the ratings, and was
Comments
Evidence-based medicine has been heralded as a new paradigm for medical education [14]. Teaching residents and practicing clinicians to be effective at critical appraisal is one of the major steps for enabling the use of EBM in clinical practice. Norman and Shannon [2] reviewed the literature on the effectiveness of instruction in critical appraisal. These authors found that studies that evaluated the teaching of critical appraisal to residents showed disappointing results, with minimal gains
References (16)
- Muller S. Physicians for the twenty-first century: report of the project panel on the general professional education of...
- et al.
Effectiveness of instruction in critical appraisal (evidence-based medicine) skillsa critical appraisal
Can Med Assoc J
(1998) Graduate medical education training in clinical epidemiology, critical appraisal, and evidence-based medicinea critical review of curricula
Acad Med
(1999)- et al.
Epidural patient-controlled analgesia in the elderly after major abdominal surgery
Anesthesiology
(2000) - et al.
Prognosis of abdominal aortic aneurysms—a population-based study
N Engl J Med
(1989) - et al.
Long-term risk of breast cancer in women with fibroadenoma
N Engl J Med
(1994) - et al.
Evidence-based medicinehow to practice and teach EBM
(2000) - et al.
Teaching residents to read the literature
J Med Educ
(1980)
Cited by (19)
Acquisition of evidence-based surgery skills in plastic surgery residency training
2011, Journal of Surgical EducationCitation Excerpt :Similar findings were noted in a meta-analysis on effectiveness of instruction in critical appraisal, in which residents showed a small change in knowledge (mean gain: 1.3%; standard deviation [SD]: 1.7%), even less than medical students (mean gain: 17.0%; SD: 4.0%).23 It is possible that substantial gains in EBS knowledge might not be detected by our posttest examinations because of a lack of valid assessment of critical appraisal skills.24 Multiple obstacles are likely to inhibit the effective teaching of EBS.
A moderated journal club is more effective than an internet journal club in teaching critical appraisal skills: Results of a multicenter randomized controlled trial
2010, Journal of the American College of SurgeonsStructured assessment format for evaluating operative reports in general surgery
2008, American Journal of SurgerySurgical Journal Clubs: A Bridge Connecting Experiential Learning Theory to Clinical Practice
2007, Journal of the American College of SurgeonsCitation Excerpt :Unfortunately, previous assessment tools have several methodologic problems. Very few studies have used a pretest–posttest design to confirm that the journal club increased critical appraisal ability.21 The validity and reliability of the journal club evaluative instrument was determined only in a single study.25