Abstract
Purpose: A checklist intervention was applied to the dental radiographic acquisition process to determine its effectiveness in improving the diagnostic value of bitewing images and reducing patient exposure to unnecessary radiation.
Methods: A mixed-method, nonrandomized research design using two dental hygiene program cohorts (n=33) as control and intervention groups was used to assess the effect of a radiographic checklist on the number and type of radiographic imaging errors. The relationship between perceived value of the checklist and willingness to continue its use was examined and analyzed using descriptive statistics. The effect of previous radiography experience, type and number of acquisition errors, and number of retake exposures were analyzed using Fisher's Exact test, chi-square analysis, Spearman's rho, ANOVA, Cronbach's alpha, Breslow-Day, and Pearson correlation coefficient test.
Results: Checklist use did not contribute to an improvement in the diagnostic value of radiographs, nor did the intervention reduce patient exposure to unnecessary radiation. Additionally, analysis did not reveal a statistically significant difference between types of radiographic errors and previous radiography experience. A positive correlation was found between perceived value and willingness to continue checklist use.
Conclusion: Similarities between the professions of medicine and dentistry suggest that dental procedures may also benefit from application of checklists. Although a positive association between checklist use and error and retake rate was not realized in this study, checklist use may help to improve the quality of radiographic exposures, thereby impacting patient safety by limiting unnecessary exposure to radiation. Additional research is needed to continue to evaluate the effects of checklist use on dental radiographs. As the body of knowledge related to checklist development and use continues to grow, dental hygienists can look for additional ways to incorporate checklists into practice.
Introduction
Checklists are used successfully in a number of professions to assist with safety protocol and job performance accuracy. Although checklists have been used as a procedural standardization tool in aviation since the 1930s and increasingly employed in medicine since the 1990s, their application to dentistry has been considered only recently.1-3 Advances in technology and its relevance to all aspects of health care have contributed to treatment innovations and improved patient outcomes. Yet in spite of increasingly specialized health care knowledge and practice, mistakes and failures persist and remain largely attributable to human error.2 Checklist use provides health care practitioners the opportunity to pause at specific procedural points, assess readiness, and address details that can lead to undesirable outcomes.
The medical profession has embraced checklist use on an increasingly broader scale since the early 2000s. Often credited as the impetus behind the medical checklist, Pronovost demonstrated that checklist use can realize dramatic improvements in patient treatment outcomes.2 Pronovost created a checklist to accompany the placement of central lines in patients in the intensive care units at Johns Hopkins Hospital with a goal of reducing the occurrence of infections. Over the course of one year, the 10-day line-infection rate went from 11% to 0%.4
In 2007, the World Health Organization (WHO) convened a group of international experts to develop a solution to the problem of unsafe surgery.5 Whatever the group devised had to be cost-effective, widely applicable, and measurable. The solution was a 19-item checklist known as the Surgical Safety Checklist designed for use by a surgical team at three key points: prior to induction of anesthesia, prior to skin incision, and before the surgical team left the operating room. Between October 2007 and September 2008, the checklist was piloted in 8 diverse hospitals throughout the world. A 36% reduction in postoperative complication rates and deaths was realized even when analysis was adjusted for case variables. Additionally, the effect was not restricted to high- or low-income sites nor was a single site responsible for the overall effect.6 In a follow-up review of data published through February 2012 on the Surgical Safety Checklist initiative and its effect on patient outcomes, Fudickar and colleagues found approximately 36% reduction in surgical complication rates and up to 62% reduction in mortality.7 These findings led the authors to suggest that a checklist intervention might improve outcomes in other medical areas as well.
Dentistry has been slow to adapt checklists to the profession. Pinsky et al. were among early advocates for adaptation of aviation checklists to dentistry. The authors note a number of similarities between the professions and suggest that checklists can help dental personnel organize their thoughts, identify errors, and increase situational awareness. Based on WHO surgical checklist guidelines, the 2007 American Dental Association guide for dental records, and personal experiences in dentistry and aviation, the authors proposed a dental checklist for outpatient dental visits. The checklist leads the dental team from the beginning to the end of the dental appointment ensuring that all pertinent safety issues are addressed. To date, the authors have not tested the checklist in a clinical trial.8
Radiographic images are among the essential diagnostic tools of dentistry. Dental hygienists typically expose a high volume of radiographic images in the clinic setting. The ability to acquire images that meet diagnostic criteria is highly advantageous. Early detection of disease, verification of the accompanying treatment required to address it, decreased risk of professional malpractice, and minimal patient exposure to radiation are key benefits of a reliable radiographic acquisition process.
The radiographer's goal is to obtain the highest quality images while exposing the patient to the lowest possible amount of radiation.9 Nondiagnostic images often necessitate a retake exposure to capture all areas of interest, effectively doubling the patient's radiation exposure. A radiography checklist can highlight those aspects of image production that are essential to attain diagnostic exposures, reduce the incidence of errors that require image re-exposure, and increase patient safety. However, the application of a checklist to the radiographic acquisition process had not been studied. Therefore, this study was designed to address the following three purposes.
Determine if a checklist intervention improves the diagnostic value of bitewing radiographs and reduces unnecessary radiographic exposure.
Determine if previous radiography experience affects the number of retake exposures.
Determine if students will value the use of a checklist intervention such that they will adopt it within and outside the educational setting.
Methods and Materials
A convenience sample of first-year dental hygiene students from two schools within the same community college district in Arizona was solicited for participation. Shared program admission requirements created a pool of study participants with a similar didactic foundation. Both dental hygiene programs are required to implement the same district-mandated curriculum and course competencies.
This study used a mixed-method research design incorporating both quantitative and qualitative assessments. A nonrandomized control group design was implemented with student participants during the spring semester of 2015 in two community college dental hygiene programs. One program cohort served as the control group and the other as the experimental group.
Participants in the control group used their customary supplies and equipment to acquire radiographic images on patients during regularly scheduled clinic sessions. Participants exposed bitewing images according to the usual protocol: inspect the patient's oral cavity, determine the number and type of exposures needed, choose and assemble supplies and exposure aids, and acquire images with consideration for the patient's specific oral conditions. There were no changes incorporated into the educational training and image acquisition process for the control group.
A checklist intervention implemented with the experimental group generated data for quantitative comparison. Participants in the experimental group used their customary supplies and equipment to acquire radiographic images on patients and, additionally, were instructed to follow the radiography checklist each time a four-image bitewing survey was exposed. The step-by-step, laminated checklist was hung on the wall in each treatment room and referenced as students prepared to expose bitewing images. Faculty in both the control and experimental group recorded the same evaluative data in the same manner for all bitewing images. Data collection concluded after 12 weeks when the semester came to an end. The goal was to collect data for five bitewing series for each participant in both the control and experimental group.
Checklist
Prior to the start of the checklist intervention, the PI met separately with the faculty who graded student images in both the control and experimental groups for study training and calibration. Evaluation criteria for diagnostic and nondiagnostic images as well as what constituted a failure and retake exposure for study purposes were reviewed. Program faculty evaluated all participant image exposures for technique and exposure errors, and the need for retakes. In addition to any programmatic evaluation and diagnosis requirements, evaluating faculty indicated on a data collection form when and how an image failed to meet minimum diagnostic criteria and whether a retake exposure was required to visualize all critical areas. Faculty recorded these data for all bitewing series exposures until each participant had exposed five four-image series or the semester concluded.
The intervention in the study was a radiography checklist that consisted of 8 procedures that a radiographer must perform in preparation for and during acquisition of bitewing images. The purpose of the checklist was to serve as a reminder of procedures that have already been learned; it did not include any instruction or information that the participants had not already received during the course of their education. The laminated checklist was hung on the wall in each radiography treatment room for the duration of the study. The checklist was optimally visible to the operator and sized and formatted for ease of use. The document served as a reference only; a physical checkmark as each task was addressed was not required. Creation of the checklist was based on the literature and incorporated those elements recommended for maximum impact including critical, actionable steps and a logical format2 (Figure 1).
Students in both the control and experimental groups completed a survey after the checklist intervention had concluded that addressed questions regarding individual demographics and prior radiography experience. Students in the experimental group also completed a perceived value survey and Radiography Checklist Intentions Survey that addressed their perceptions of the value of the checklist intervention and willingness to use the instrument within and outside of the educational setting. All study instruments were original tools, with the exception of the Radiography Checklist Intentions Survey, and developed based on the literature. The instruments were distributed to several currently licensed dental hygienists prior to use to evaluate content validity using a Content Validity Index.10 All instruments were also pilot-tested and evaluated for reliability using a test/retest method with a cohort of dental hygiene students and faculty in a third program housed within the same community college district as the control and intervention groups. Suggestions for improvements and modifications were incorporated into the instruments as deemed appropriate.
All data were analyzed using SPSS statistical software Version 22 (IBM Corp., Armonk, NY). A 95% confidence interval and Ɑ = 0.05 were adopted as criterion for two-tailed statistical significance. Descriptive statistics were used to report characteristics of subjects in both the control and intervention groups. Statistical tests used for data analysis included Fisher's Exact test, chi-square analysis, Spearman's rho, ANOVA, Cronbach's alpha, Breslow-Day, and Pearson correlation coefficient test.
Results
A total study sample of 33 study participants ranged from 21 to 43 years of age. The majority of study participants (54%) had some type of dental office experience prior to entering the dental hygiene program. Among those individuals, 79% had experience exposing radiographic images. Of the individuals with radiographic experience, approximately half received their radiography training on the job and half had formal training. Table I presents all participant demographic data.
The image fail rate in the control group was 23.2% and 31.4% in the intervention group. Fisher's Exact test revealed a significant difference (p=0.030) in the number of failing images between the intervention and control group. Data indicated checklist use did not contribute to an improvement in the diagnostic value of radiographs.
Radiographic errors documented for individual participants in both groups for the first 3 bitewing series were similar in both type and number. In the intervention group, there was a significant increase in the number of failing images in the fourth series of bitewings; however, the number of failing images rebounded back to the level observed in the control group in the fifth bitewing series. This data trend could not be explained by the natural course of study events. It is possible that unforeseen circumstances unrelated to the study influenced participant performance.
To determine whether a checklist intervention reduced patient exposure to unnecessary radiation, analysis was completed using the retake rate as the dependent variable and checklist use as the independent variable. The retake rate for the control group was 18% and 31.1% for the intervention group. Two-tailed Fisher's Exact test demonstrated a statistically significant (p<0.001) difference in the retake rate between the control group and intervention group. Data indicated a checklist intervention did not reduce patient exposure to unnecessary radiation.
It is noteworthy that all failing images do not necessarily require re-exposure as an adjacent image may adequately depict what is missing thus “saving” the failed image from a retake. However, this phenomenon accounted for just 16 total images (2.8%) in the study: 15 from the control group and one from the intervention group.
Also notable was the substantial number of failing images in series 4 in the intervention group. The intervention group experienced 23 failing images out of 48 compared to 3 failing images out of 52 in the control group. When considering only bitewing series 1, 2, 3, and 5, analysis demonstrated a 27.2% fail rate for the control group and a 28% fail rate for the intervention group. Two-tailed Fisher's Exact test (p=0.917) failed to demonstrate a significant difference. When excluding series 4 from data analysis regarding retake rate, the control group had a 21.1% rate, and the intervention group experienced a 27.6% rate. Fisher's Exact test failed to demonstrate a significant difference (p=0.130) when series 4 data was eliminated from analysis. Series 4 alone was the cause of the statistically significant difference between the study groups.
Years of Radiography Experience and Retake Percentage Rate
Participants with no prior radiography experience had a 28.4% retake rate, individuals with 1-3 years of experience had a 17% retake rate, and those with 4-9 years of experience had a 21.2% rate (Table II). Data revealed an overall retake rate of 24.5% for all groups combined. The number of retake exposures was compared between students who had and did not have previous radiography experience. Chisquare analysis revealed a statistically significant difference (p=0.032) between years of experience and number of retake exposures. A correlation between experience and retake percentage using Spearman's rho was not significant (rs=-.32, p=0.071).
In this study, more experience did not equate to fewer retakes. Also notable, upon closer analysis, within every category of experience, participants in the intervention group exposed the greatest percentage of retakes (Table III). However, a two-way ANOVA test of the interaction (years of experience x group assignment) did not yield significant results in predicting percentage retake (p=0.854).
Overall, film placement errors occurred with the greatest frequency (20.7%) followed by horizontal angle errors (4.8%) and then vertical angle errors (2.8%). When comparing types of radiographic acquisition errors with previous radiography experience, chi-square analysis revealed no statistical significance regarding film placement (p=0.077), horizontal angle (p=0.107), and vertical angle (p=0.755). When considering control and intervention groups separately to the variables of previous radiography experience and type of error, odds ratio tests (Breslow-Day 2-sided significance = 0.401) confirmed these findings.
Years of Radiography Experience and Mean Retake Percentage by Group
Years of Prior Radiography Experience by Percentage and Type of Radiography Errors
Within each study group, those with the least amount and greatest amount of experience had more errors than those with 1-3 years of experience (Table IV). A two-way ANOVA test of the interactions did not reveal any statistically significant differences regarding previous radiography experience and film placement (p=0.316) horizontal angle (p=0.304) and vertical angle, (p=0.850).
A two-tailed Pearson correlation coefficient was computed to assess the relationship between perceived value and willingness to use the checklist. A positive correlation was found between the two variables (r=0.562, n=18, p=0.019). Table V lists the mean scores of questions related to perceived checklist value.
Three questions on the Perceived Value of a Radiography Checklist survey solicited qualitative comments. Overall, students had a positive attitude toward checklist use and saw it as a useful aid to the radiographic acquisition process. Participants provided 11 comments regarding aspects of the checklist that caused it to be effective. Physical characteristics were cited 4 times and referred to document simplicity and size. Four participants noted the checklist served as a “reminder.” Several individuals mentioned using the checklist made them think about what they were doing. These comments confirm that the checklist met the researcher's intended criteria for appearance and use.
When asked what aspects of the checklist caused it to be ineffective, 11 comments were offered, including 3 that indicated no aspects were considered ineffective. Two comments were made regarding the physical characteristics of the checklist and one concerning the additional time it took to use. Two participants noted they forgot to use the checklist at times. One individual mentioned instructor pressure to use the checklist and another stated, “I already have experience.” One participant commented the checklist was implemented after habits were built.
Finally, participants were asked what they would change about the checklist to make it more useful. Ten comments were provided, including 3 that indicated no changes should be made and one that said it was very useful as currently written. The remaining 4 comments related to physical characteristics of the checklist (use of color, include images of teeth to be included on each bitewing) and locating additional checklists outside of the exposure room to assist in setting up for the procedure.
Discussion
This study examined whether use of a checklist application by dental hygiene students during radiographic exposure could improve outcomes related to radiographic imaging technique and patient exposure. Although improvements to the diagnostic value of images were not realized as a result of the checklist intervention, the study raised several notable areas of interest for discussion.
An unusual, unexpected study event was the type of bitewing orientation used by participants in each program cohort. The intervention group exposed vertical bitewings 91.5% of the time and the control group exposed horizontal images exclusively. The challenges of acquiring diagnostically acceptable vertical bitewings are known to dental radiographers. The discomfort experienced by patients due to the vertical orientation of the receptor and limitations imposed by normal oral anatomy can contribute to exposure errors.11,12 Although a search of the literature did not reveal any definitive guidelines regarding selection criteria for exposure of vertical versus horizontal bitewings, the dental radiographer must use clinical judgment to determine which are most appropriate. If a goal of bitewing imaging is to visualize the height of maxillary and mandibular alveolar bone, one must consider the need for vertical images based on the patient's medical, dental, and social history. The checklist was designed to be effective for both horizontal and vertical bitewing acquisition. Excluding the fourth series, error rates did not differ between the groups, so observed differences in bitewing film orientation did not seem to influence study outcomes.
Data indicate the intervention group participants had generally positive attitudes toward checklist use and found benefits to its application. However, the reported value for checklist use did not translate into improved radiographic outcomes. Although the difference in retake rate between the intervention and control groups was only statistically significant for series 4, the clinical significance of any radiographic exposure is more compelling. The effects of radiation are cumulative and every exposure must be carefully considered.
Perceived Value of the Radiography Checklist
The exposure of diagnostically acceptable radiographic images on the first attempt should be the goal of every dental radiographer. The hazards of ionizing radiation are well documented, as are the safety precautions that must be observed when working with radiation. Although dentistry generally incorporates low-level radiation exposures, there is no established lower threshold for safety.9,13 Based on the success of checklist applications to medical procedures, the researchers hypothesized that the dental radiographic acquisition process may benefit from a similar practice.
The use of digital imaging in dentistry continues to grow and is credited with a number of advantages over traditional film-based imaging including reduced radiation dose to the patient, fewer retakes due to density and contrast errors, and wider dynamic range and an associated reduction in retakes.14,15,16 When evaluating claims of reduced patient exposure to radiation as a byproduct of digital radiography, Wenzel, Møystad, and Berkhout et al. found the claims could not be substantiated.16,17 Although digital imaging systems require less radiation for exposure, retakes will minimize or even eliminate this benefit. This study supports the findings of these authors. The retake rate between both the control and intervention groups was compelling but even more so for the intervention group.
Limitations imposed by the type of radiography system used can impact the benefits that are realized as well as the need for retakes. Horner et al. and Thomson note the potential for mechanical damage to photo-stimulable phosphor plates (PSP) plates as well as the laser processing step they require can both contribute to poor image quality and need for retakes.18,19 Nearly all images in this study were acquired using PSP receptors, yet not one error and associated retake due to these factors was noted by evaluating faculty.
The literature contains relatively few examples of studies comparing radiographer experience with error and image retake rate. While many studies emphasize the clinician's diagnostic ability, this skill depends to a large degree on high-quality exposures. Some radiographers tend to overestimate their abilities, and others possess significant skills without specialized training.20,21 This study did not demonstrate that radiographers with more experience performed better than those with less experience, although they did perform better than those with no experience. Self-assessment as a form of quality assurance can assist the clinician with identifying strengths and weaknesses in their performance and contribute to high-quality patient care.
Several limitations were present in this study. Every image in the study was acquired using a paralleling aiming device. Results may vary for images acquired using tabs or another type of holding device. The majority of bitewing images in the study were acquired using PSP receptors. Results obtained using traditional film or digital sensors may vary. Participants in the intervention group acquired mostly vertical bitewing series, while control group participants acquired horizontal bitewing series exclusively. Further, it was not possible to determine if participants in the intervention group referenced the checklist for each exposure as a manipulation check of the dependent variable was not implemented as part of the study protocol. Participants were occasionally verbally reminded by faculty to use the checklist.
Checklist applications to a number of professions have resulted in improved safety and patient care outcomes. The similarities between medicine and dentistry suggest that the success seen in medical checklist applications demonstrate promise for dentistry as well. Patient-centered care compels the dental professions to search for innovations that improve the quality and safety of dental services. Checklists are a simple and cost-effective method to apply toward this goal. In light of the current research and growing uses for checklists, the following suggestions for future research are made:
Introduce the checklist during the initial preclinical radiology course. This may have the effect of preempting certain habits that may be acquired during the training phase as well as socializing students to routine checklist use thereby resulting in fewer errors, less retakes, greater perceived value, and checklist adoption.
Conduct the same research with one cohort of participants divided into control and intervention groups. This may have the effect of minimizing potential evaluator discrepancies in diagnosing type of bitewings to expose and assessment of radiographic images.
Investigate further the increased image retake rate among individuals with more radiographic experience compared to individuals with less radiography experience.
Conclusion
Checklist applications to medicine have demonstrated improved patient outcomes. Similarities between the professions of medicine and dentistry suggest that dental procedures may also benefit from application of checklists. Although a positive association between checklist use and error and retake rate was not realized in this study, checklist use may help to improve the quality of radiographic exposures thereby impacting patient safety by limiting unnecessary exposure to radiation. Additional research is needed to continue to evaluate the effects of checklist use on dental radiographs. As the body of knowledge related to checklist development and use continues to grow, dental hygienists can look for additional ways to incorporate checklists into the profession. The demonstrated success of checklist application to a variety of professions is too compelling to ignore.
Footnotes
Monica Williamson Nenad, RDH, DHEd is Director of Faculty Development, Accreditation, and Continuing Dental Education and Assistant Professor, A.T. Still University, at the Arizona School of Dentistry & Oral Health. Colleen Halupa, EdD, is Associate Professor, A.T. Still University, College of Graduate Health Studies. Ann Eshenaur Spolarich, RDH, PhD, is Director of Research, Professor, A.T. Still University, at the Arizona School of Dentistry & Oral Health. JoAnn R. Gurenlian, RDH, PhD, is Graduate Program Director, Professor, at Idaho State University.
This study supports the NDHRA priority area, Occupational Health and Safety: Investigate methods to decrease errors, risks and or hazards in health care and their harmful impact on patients.
- Copyright © 2016 The American Dental Hygienists’ Association
