There are now a number of reports indicating that people trained in Advance Cardiac Life Support (ACLS) are not effective in resuscitating patients from ventricular fibrillation in under 3 minutes.^1,2 Human patient simulation has been used for many years for teaching purposes^3–8 and was recently reported to be effective in training team response to trauma resuscitation.⁹ Although simulation training is in its infancy in the medical field and is most commonly focused on individual skills such as endotracheal intubation, central venous catheter placement, and endoscopy training, its use is likely to increase. Settings that are especially prone to errors are those that are high acuity (risk) or low frequency, and those that require teamwork.

There are few reports of training multidisciplinary teams of individuals to manage ACLS and other medical crisis situations, and those that do use performance assessment tools that are relatively non-specific. We have created a crisis team training course that integrates web based and computerized human simulator technology and emphasizes a team approach to medical emergency team (MET) resource management. We believed that teaching organization and communication skills would improve completion of key tasks and simulated outcome in a medical crisis. Completing key tasks during a crisis situation is likely to improve efficiency of treatment delivery, patient outcome and, ultimately, patient safety.

DESIGN AND SETTING

We embarked on this project as part of our hospital’s quality improvement program to improve crisis response. Previous reports have focused on using a team leader to assign tasks as the crisis develops. It is possible that this methodology for organizing a team may detract from the team leader’s focus on treating the patient. We hypothesized that a focus on organizational strategies using predetermined (but not pre-assigned) roles would help organize diverse members into a more coordinated MET team. Organization did not depend on the “team leader” assigning tasks. We presumed that, if all members of the team (including the team leader) assume predetermined roles, they could immediately focus on treatment tasks. We also hypothesized that increased performance of organizational tasks would correlate with simulated patient outcome.

Our objective was to utilize a novel curriculum that used specified roles and goals, to rehearse during various simulated patient care scenarios, and improve performance. In this report we describe our preliminary experience in training multidisciplinary teams to respond in a coordinated fashion using pre-designed roles and goals to in-hospital crisis events.

Setting

The University of Pittsburgh Medical Center (UPMC) Winter Institute for Simulation Education and Research (WISER) is a medical education center equipped with 12 full body Laerdal SimMan simulators as well as many partial task trainers (described below).

Simulators

The full body simulator is a computer based mannequin with human physiology emulation capability. It is life size and trainees may interact with it in a fashion that is very similar to human-to-human interactions. For example, one uses a sphygmomanometer and a stethoscope to determine blood pressure; a stethoscope is used on the mannequin’s chest to ausculate heart and breath sounds. The mannequin can be programmed to create a variety of lung and heart sounds such as rales, ronchi, wheezes, and murmurs. A partial task trainer is a device that simulates a part of the human body—for example, a forearm for inserting intravenous catheters. They were not used for this exercise. All simulated patient encounters and the SimMan patient monitor are video recorded on digital video recorders (Model VT400; EZCam Inc, Trenton, MI, USA). This system allows instant playback and review.

Interdisciplinary trainees

Trainees for the crisis team training course consisted of critical care nurses, respiratory therapists, and physicians. All were ACLS certified within 2 years of their simulation training. Physicians were predominantly trainees including fellows in critical care medicine and pulmonary/critical care medicine, junior and senior residents in internal medicine, anesthesiology, and emergency medicine, and several attending physicians who volunteered to take the course. All but the attending physicians were required to take the course as part of the hospital’s MET training initiative. The hospital is in the process of training all the critical care nursing staff, respiratory care staff, and trainees in the above noted specialties. Simulation training is being used for crisis teams to improve patient safety in high risk situations.

Curriculum

The crisis team training course is composed of four components: (1) a web based power point presentation that trainees view before coming to WISER; (2) a brief didactic reviewing key concepts of team performance; (3) skills performance: three simulation scenarios using the Laerdal SimMan computer programmable human simulator with web based video recording; and (4) facilitator moderated debriefings aided by a customized Excel spreadsheet for performance evaluation. The second, third and fourth components take place in the simulator center during a single 3 hour session.

The web based curriculum presentation was developed by two of the authors (MD, JS); it describes the need for crisis teams, the criteria for initiating a crisis team response, barriers to error free team response, and team member roles and goals (table 1). All participants in the crisis team training program were required to view the presentation and complete a pretest.

Scenarios

Five pre-programmed, semi-automated simulator scenarios of similar complexity were developed for the training sessions (table 2). During each course, trainees responded to three different simulated crisis scenarios. The scenarios (and their order of sequence) for each course were predetermined, and no scenario was repeated during a course. Each scenario simulation was considered a “session”. Each scenario began with a “floor nurse” who was read a scenario setting, evaluated the patient, and then initiated a crisis team response (either condition C for “crisis” or condition A for “cardiopulmonary arrest”, in our hospital system terminology).^10,11 The team then responded, assessed, and treated the “patient”. A session was stopped after 5 minutes or when the patient was appropriately treated and a triage decision made.

Trainees were instructed not to play the same role more than once during the entire course. The purpose of this requirement was to improve cross training and to avoid improvement due to rehearsing the same role over and over. Our goal was to foster the ability of individuals to perform well in a clinical setting regardless of what professions arrive at an emergency. We do this because, at any crisis, the patient has treatment needs even if the “best” professional has not yet arrived. We wanted to foster the ability of team members to rapidly organize and fulfil patient needs.

KEY MEASURES FOR IMPROVEMENT

The primary outcome in this study was successful crisis management resulting in mannequin “survival”; secondary outcomes were completion of organizational and patient care tasks.

Crisis management goals

For the purposes of these exercises, successful crisis management consisted of achieving three goals: (1) the crisis team was required to manage all airway, breathing, circulation and neurological problems (ABCD) effectively; (2) each scenario contained a “definitive therapy” (DT) that was considered a key element of successful crisis response (for example, defibrillation of a patient in ventricular fibrillation or delivering naloxone to a patient with opioid induced hypopnea); (3) there were time constraints: identification and management of ABCD had to be completed within 60 seconds and delivery of DT was required to be delivered within 3 minutes of the start of the scenario. The scenarios were constrained so that simulated patient “survival” required the crisis team to manage ABCD + DT within the stipulated time limit.

Organization goals

The team members were required to assume roles during each scenario and to verbalize to other members what role they each undertook. Organizational tasks were graded.

Scenario outcomes

There were three possible “clinical” outcomes of each scenario: (1) if the crisis team successfully managed ABCD and delivered the DT within the time limit, the patient “survived”; (2) if the crisis team successfully managed ABCD but did not deliver DT (such as adequately bag mask ventilating the mannequin but failing to give naloxone for opioid overdose), a “critical incident” designation was assigned: critical incidents were still considered “survival”; (3) if ABCD was not delivered, the simulated patient “died”.

Process measures of performance

The secondary outcome was the crisis task completion rate (TCR). By consensus of the course authors, a set of 29 tasks was defined for each scenario in each of three domains: (1) patient assessment and treatment (for example, assessing cardiac rhythm, delivering defibrillation); (2) organizing the response (for example, delivering the equipment, positioning personnel in appropriate locations, allocating work); (3) communication (for example, using closed loop communication, data transfer). Task completion was assessed by consensus of the trainees and the facilitator after reviewing the video of the simulation. TCR was defined as the percentage of required tasks completed during the simulated crisis. Only tasks that were applicable to the scenario were counted towards the TCR. TCR was defined for each individual role and for the whole team.

Debriefing sessions

After the team responded to a simulated crisis, the team reviewed the web video of the scenario under the supervision of the facilitator. The team and facilitator rated the crisis team’s performance. Session scoring was recorded at the time of debriefing on a preformatted Excel spreadsheet and projected on a large screen to facilitate collective review by the crisis team trainees. Required tasks were scored 1 (completed) and 0 (not completed). When a task was not applicable to the scenario—for example, assisted ventilation for patients with adequate breathing—no score was given and the “denominator” number of tasks to be completed was decreased by 1.

Team debriefing focused on three types of performance: (1) assuming a specific role; (2) completing the tasks associated with those roles; and (3) cooperation. An example of cooperation is the deployment of equipment from the crash cart. The defibrillator pads need to be handed by the crash cart manager to the bedside nurse and then affixed to the patient’s thorax with the assistance of the chest compression person. Team performance and its relationship to individual performance were promoted. Focus on multistep processes needed to accomplish “simple” tasks was emphasized, often using a brief root cause analysis technique to underscore all the errors leading to a clinical failure.

We emphasized organization, teamwork, and crisis resource management during debriefing. This was done by focusing the trainees on their performance in assuming the roles designed, their ability to complete the tasks assigned to those roles, and their ability to communicate and cooperate with each other. At the end of each debriefing the team reviewed strategies to improve performance.

Statistical methods

All statistical analyses were performed using Microsoft Excel XP and SPSS version 11 (SPSS Inc, Chicago, IL, USA). The results were analyzed using descriptive and non-parametric statistics. Because of the repeated measures nature of the data, improvement of simulator “survival” across the three simulation sessions for each crisis team was determined using Cochran’s Q, which assesses changes in binary outcomes across repeated measurements. Likewise, improvement in the TCR across the three scenarios was determined using Kendall’s W. Although TCR was defined on a continuous scale, we used this non-parametric equivalent of repeated measures ANOVA because of the limited number of participants. Post hoc analyses were performed to determine if survival or TCR improvement occurred primarily between the first and second or the second and third sessions. A Bonferroni corrected critical p value of 0.017 was used to determine statistical significance for all primary and post hoc analyses.

OUTCOMES

Between March 2002 and May 2003 we ran 10 courses and trained 138 individuals including respiratory therapists, nurses, and physicians to function in teams each composed of eight people. The disciplines of the individuals trained are shown in table 3. One specialist in hospital medicine and three intensive care specialists participated. All courses had at least 10 participants and none had more than 20.

Overall simulator “survival” improved from 0% to 90% across the three sessions in a day’s course. This difference was statistically significant (Cochran’s Q = 12.6, p = 0.002). Post hoc analysis showed that most of the improvement in survival was observed between the first and second sessions (p = 0.014) rather than between the second and third sessions (p = 0.180).

The mean TCR improved overall from 31% to 89%, and each simulator role improved from 10–45% during the first session to 80–95% during the third session (fig 1). The improvement in overall TCR was statistically significant (Kendall’s W = 0.91, p<0.001). Post hoc analyses revealed improvement of overall TCR between both the first and second sessions (p = 0.002) and between the second and third sessions (p = 0.011). In addition, with training the TCR appeared to improve for each role (fig 2).

LESSONS LEARNT AND NEXT STEPS

We embarked on this training program with the hope that—by designing a crisis response that had highly specific roles and goals for each role, using a human simulation environment equipped with video recording capability, and debriefing focused on the relationship between organizational tasks and team performance—we could improve team performance and simulated patient outcome. Our preliminary data support the conclusion that this form of team training does improve both team performance and outcome.

The TCR improved with each successive debriefing and simulation session, even though team members played no role twice. Our data seem to support the notion that stressing team coordination skills such as role delineation and communication resulted in improved integration and execution of these tasks. If true, this is an important finding. It suggests that, in addition to knowledge and procedural skill training, team training should be part of ACLS instruction. In addition, other medical situations where team members must work in a coordinated fashion (such as conscious sedation, endoscopy) might also benefit from team training using human simulation.

Our data show preliminary evidence that a structured, interactive, human simulator based team training curriculum can facilitate an improved team response to simulated medical crises. Although we used a relatively modest number of participants, we were able to detect notable improvements in both the primary (simulator “survival”) and secondary (task completion) outcomes. These improvements were observed for every participating MET.

Every MET performed poorly at the outset of the educational intervention, even though all trainees were already ACLS certified and all had both reviewed precourse materials describing expected behaviors and passed a pretest of the materials. This is an important finding that corroborates the reports of others. We believe one problem is that, although ACLS training effectively indicates what needs to be done (crisis management), it does not focus on skills directed at how to get it done (crisis resource management). Accomplishing apparently simple tasks can be especially difficult when a large number of professionals from disparate disciplines respond and must cooperate. This failure to perform well may occur because prior training methods focus on individuals learning facts (knowledge) in isolation, rather than rehearsing performance (skills) in a team setting to coordinate task completion (organization) and group problem solving towards a common goal (so-called collective thinking). This finding implies that MET members should receive training in addition to ACLS.

A crisis situation requires a number of simultaneous, sequential, or coordinated interventions performed by a variable number of responders who are arriving in an uncoordinated order. For example, in our institution the team members vary by time of day, day of week, and the location of the event. In addition, team members arrive at a crisis situation in a haphazard fashion, making it difficult to make sure that all tasks are accomplished rapidly. Nevertheless, the patient needs to be assessed and treated. We have structured our training program to attend to this need. Responders assume predetermined roles and then complete the tasks associated with that role. If another responder arrives who possesses better skills for a certain role, then a “switch” of that person into the role would be appropriate. For example, a patient with respiratory insufficiency needs better oxygenation and ventilation, usually initially provided with a bag mask device. If an airway expert does not arrive immediately the respiratory insufficiency still needs to be supported. Our program teaches responders to assume any of the roles (in this example, perhaps a nurse or a respiratory therapist might take on the airway manager role and begin bag ventilation) and to provide the support the patient needs. Later, should a better airway manager arrive, that person could take over the airway manager role. This sort of training, if successful, may improve the quality of crisis response, especially in the early stages.

Sica et al,¹² Reznek et al,¹³ and Gaba et al¹⁴ have studied crisis resource management in the setting of a radiology suite, the emergency department and the operating room, respectively. Lighthall et al¹⁵ have studied interdisciplinary training for crisis response, but their methodology focused more on crisis management and their results related more to attitudes and perceptions of the learning environment than on completion of key tasks. To our knowledge, there are no previous reports of how to organize a crisis response for inpatients with this degree of detail, using a fully interdisciplinary approach, and correlating task completion with simulated outcome.

The Institute of Medicine has recommended the establishment of team training programs using simulation,¹⁶ as exemplified in civil aviation crew resource management training.^17,18 Our preliminary experience confirms that team training is not only feasible for ad hoc crisis response teams in hospitals, but may also result in both superior resuscitation process and outcome. Our training program focuses on improving the organization, efficiency, and reliability of a crisis team response, not on the improvement of individual ACLS skills. Our data suggest that communication and integration of individual activities into a team response may be as important as individual clinical or procedural skills.

There are data which indicate that caregivers recognize their own poor performance. Pittman et al¹⁹ surveyed cardiac arrest team members. Respondents perceived that communication was poor during and after a crisis response and resulted in poor performance.

In our crisis team training course we emphasized four overarching elements for organizing crisis team response:

• The responding MET members must be self-identified.

• The tasks to be accomplished need to be characterized and prioritized.

• The required individual steps in achieving specific goals must be sequenced.

• The roles and specific task responsibilities need to be delegated and rehearsed.

We believe that this approach not only forced the crisis team trainees to focus on organizing the team according to necessary roles and responsibilities, but also encouraged the teams to work together to achieve role-specific as well as team goals. We believe that use of the human simulator was a key element in the observed improvement in performance because it enabled trainees to observe the consequences of their actions on others, enabled rehearsal to improve skills, and provided feedback indicating that this new organizational methodology improved performance.

Knowledge of what to do in a crisis is not enough to ensure successful team performance. Every MET member we trained was already ACLS certified, was an experienced clinician, and all had passed the pretest covering the web based curriculum. We therefore believe that it is highly unlikely that there existed a knowledge deficit that was corrected during our simulator training session. Nevertheless, the teams uniformly performed poorly at the outset. Review, debriefing, and rehearsal seemed to improve performance even though each person was asked to assume a different role for each session. Figure 2 shows that performance improved in the second and third scenarios in spite of the crisis being different and the changing roles of the trainees.

Our report has several limitations. This was not a controlled trial but a preliminary report of a novel educational effort to improve quality of MET performance. We did not evaluate inter-rater reliability of performance ratings. Gaba et al,¹⁴ Sica et al,¹² and Reznek et al¹³ all assessed performance using either Likert scales (of overall performance) or attitude surveys. Instead, we focused on an objective measure: specific task completion. We believe objective measures are less susceptible to inter-rater differences. Like ACLS, completion of the training does not necessarily correlate with improved performance in the hospital. “Survival” in our training program is a measure of performance of the ABC of resuscitation and delivery of critical treatments in a timely fashion. The clinical impact of this type of training has not been tested and is worthy of future investigation.

We believe that these preliminary data show that simulation training improves MET organization and performance. Furthermore, our performance assessment tool provides objective task performance measurement. This assessment may be very important to behavior change.

Finally, the scenarios themselves might be subtly different from course to course—for example, the simulator response to the same intervention may not have been identical. Future investigators might program the simulator to exhibit standard physiological responses to treatments.

CONCLUSION

We have demonstrated the feasibility of using a computerized human simulator for teaching MET organization and team skills in response to medical crisis situations. Our preliminary results suggest that multidisciplinary team training using a human simulator and emphasizing organization results in improvement of both resuscitation process elements and simulated outcome.