Abstract
Purpose: Typically implemented as a safety measure, checklists can reduce risks and improve patient outcomes. Checklists have been widely used in medicine, but rarely applied to dentistry. The purpose of this replication study was to evaluate the effectiveness of a dental radiography checklist intervention for improving the diagnostic value of bitewing images and reducing retake exposures.
Methods: Two cohorts of dental hygiene students from programs in the same community college district participated in the mixed methods study; one as intervention group (n=22), the other as control group (n=23). The intervention group used a checklist each time bitewing images were acquired on manikins and live patients while the control group followed the usual protocol for image acquisition. Calibrated faculty evaluated all images and recorded whether images passed, failed, or required retakes. All participants completed a demographic survey at the study conclusion while the experimental group completed two additional surveys regarding perceived value of the checklist and intention to continue its use outside the educational setting. Descriptive and inferential statistics were used to analyze the data.
Results: Image failure and retake rates were significantly lower in the experimental group on both manikins and live patients (p<0.001). The control group experienced a lower failure rate on patients versus manikins; however, overall retake rates were higher than the experimental group. While the retake rate improved among both groups from manikin to human exposures, the magnitude of change across groups did not differ (p=0.992). Sensor placement was the most common cause for a failing image. Participants generally considered the checklist thorough and easy to use, however there was less agreement that it improved image quality or that they would continue its use outside the educational setting.
Conclusion: A radiography checklist used in an educational setting was successful in reducing bitewing image failure and retake rates, thus benefiting patient safety with reduced radiation exposure.
- checklists
- dental hygiene education
- dental radiography
- evidence-based practice
- dental radiation
- patient safety
Introduction
Checklists have a long history as a safety and standardization tool. First used in aviation in the 1930s, their use has expanded into a variety of professions including construction, finance, and medicine.1 Checklists are not intended to be instructional devices but serve as reminders of what the user is expected to know and do in a given situation.2 Particularly during a critical task or emergency procedure, checklists eliminate the need to rely on memory or intuition.3
One of the first and best-known examples of a checklist application in medicine was a study designed to reduce the incidence of catheter-related bloodstream infections among intensive care unit patients.4 When combined with additional measures including enhanced provider awareness and enforced adherence to infection control practices, results demonstrated near elimination of catheter-related infections. The checklist movement gained further momentum when the World Health Organization (WHO) published the Surgical Safety Checklist in 2008.2 Developed by a group of international experts with a goal of making surgery safer around the world, the resulting checklist has been lauded for its effectiveness.5-8
Checklist use in medicine expanded rapidly in the decade following the WHO initiative.9-11 Its use in dentistry however, remains the exception rather than the rule. A review of the literature suggests that when dental checklists are used, they are frequently applied to higher risk procedures such as implant placement and oral surgery.12-14 However, other procedures with less immediate risks may benefit from the procedural standardization that a checklist provides.
Dental radiographic imaging uses relatively low levels of ionizing radiation. Yet lack of a known “safe” threshold dose and the radiosensitivity of various tissues of the head and neck require that only essential images be exposed.15-17 Additionally, the need for ongoing radiographic exposures over a patients’ lifetime, challenges presented by intraoral image acquisition, and a goal of acquiring diagnostic images on the first attempt, mean that operator training and adherence to safety measures are essential. The range of checklist applications in medicine and similarities with dentistry suggest that additional exploration is warranted.
In a previous study evaluating the effectiveness of a dental radiography checklist on improving the diagnostic value of bitewing images and reducing retake exposures,18 the checklist intervention did not result in improved diagnostic value of images or a reduction in radiation exposure. However, in that study, only paralleling aiming devices were used, the majority (94%) of images were acquired using a photostimulable phosphor plate (PSP) system, and participants in the intervention group acquired mostly vertical bitewings (91%). Other limitations were the time frame (12-weeks) and the limited number of assessments (five sets of bitewings acquired on live patients per participant).18 Given the limitations of the original research and the potential impact of a checklist on patient safety, a replication study was warranted. The purpose of this study was to evaluate the effectiveness of a dental radiography checklist in improving the diagnostic value of bitewing images and reducing retake exposures on both manikin and live patients. User perception of the value of a radiography checklist and willingness to continue its use in clinical practice was also assessed.
Methods
The Institutional Review Board of the Maricopa County Community College District (2014-11-384) and A.T. Still University (2015-092) deemed this study exempt. A mixed-method design with quantitative and qualitative assessments was developed. Dental hygiene students from two programs within the Maricopa County Community College District in Arizona were solicited to participate. A sample size analysis was not conducted prior to study inception as the design called for participation of all students enrolled in each program, provided they agreed to participate. All participants shared a similar didactic foundation given that the programs utilized a shared admissions process and implemented the same district-mandated curriculum.
A nonrandomized control group design, using one program cohort as the control group (n=23) and the other as the experimental group (n=22) was implemented in fall of 2015 as the students matriculated into the four-semester program. The Principal Investigator (PI) met with each cohort during a regularly scheduled class period to introduce the study and obtain informed consent.
During semester one, both cohorts participated in a didactic radiology course and the associated laboratory course, in which only manikin images were exposed. From semester two through four, students made radiographic exposures on live patients during regularly scheduled clinic sessions. Both programs used their usual supplies and equipment throughout the study for the acquisition of radiographic images. All participants followed the same procedures when acquiring images: the oral cavity was inspected, the types and number of images needed were determined, necessary supplies and exposure aids were assembled, and the images were exposed with consideration for the patient’s specific oral conditions. Students elected to acquire either vertical or horizontal bitewings and whether to use a wired sensor or PSP system.
Didactic and image acquisition instructions remained the same for both the control and experimental groups with one exception. Participants in the experimental group were asked to reference an eight-step radiography checklist displayed in each radiography treatment room when acquiring bitewing images. The checklist highlighted the essential steps in the image acquisition process and was visible to the operator throughout the set-up and exposure procedures (Figure 1). The checklist was developed according to best practices identified in the literature and incorporated simple, minimal language, actionable steps, was sized and formatted for easy reference, and hung on the wall directly behind the patient chair for the duration of the study.1, 19-21 Participants were not expected to apply a physical checkmark to the document.
Dental radiography checklist for bitewing images
Faculty at each program randomly assigned an identification number to study participants; control group numbers ranged from 1-25 and intervention group numbers ranged from 26-50. This number was used by faculty when recording image data and by participants when completing study surveys. Participants were known to the PI by number only.
Faculty for both the control and experimental groups recorded the evaluative data on a collection form for all bitewing images acquired from semester one through semester four. All faculty who participated in evaluating and recording bitewing image data were calibrated by the PI in advance of the study. Faculty were asked to provide data from four-image bitewing series only. Any incomplete series or those consisting of fewer than four images were not evaluated for study purposes.
Evaluation criteria regarding diagnostic and nondiagnostic images was established as well as what constituted the need for a retake exposure. Faculty evaluated all exposures according to programmatic requirements but also indicated on the data collection form when and how images failed to meet minimum diagnostic criteria and whether a retake exposure was needed to visualize the targeted areas of interest. Failing images were noted with an “F” and the error causing the failure: sensor placement (SP), horizontal angle (HA), vertical angle (VA), and “other” (cone cut, reverse sensor, etc.). If an image needed to be retaken in order to visualize a specific area of interest, an “R” was also recorded. Not all failing images required a retake if the target information was visible on the adjacent image.
At the conclusion of the study, respondents in both the control and experimental groups completed a survey regarding individual demographics and previous radiography experience. Respondents in the experimental group completed two additional surveys: one that addressed perceived value of the checklist and a second survey that explored willingness to use the radiology checklist outside of the educational setting.
All instruments used in the study were created by the PI and based on the literature, with the exception of the Radiography Checklist Intentions Survey.22 The instruments were evaluated for content validity and then pilot tested for reliability using a test/retest method with a third dental hygiene program cohort in the same community college district as the control and experimental groups. Suggestions for improvement and modification were incorporated into the instruments as deemed appropriate.
Statistical analysis
Descriptive statistics, including counts and percentages for categorical variables and means and standard deviations for continuous variables, are provided. A generalized estimating equations approach was used to accommodate the multiple images exposed by each participant. Logit models with auto-regressive correlation matrices were specified. Sequential Bonferroni adjustments were used to interpret significance. Spearman’s rho was used to estimate monotonic relationships between variables and Cronbach’s α was calculated to estimate the internal consistency of scale items. Analyses were performed using a statistical software program (SPSS Ver. 25, IBM Corp., Armonk, NY, USA). The criterion for statistical significance was alpha = 0.05, two-tailed.
Results
A total of 45 dental hygiene students from the two programs consented to participate. All but one of the participants was female, and the mean participant age was 32 years (SD 7.34). Among all survey respondents (n=36), six participants from the control group and eight participants from the experimental group stated they had prior radiography experience. Most participants with prior radiography experience were trained through a formal, multi-session course as opposed to on-the-job training; sensor-based imaging was the most frequently identified radiographic acquisition system among participants with dental radiography experience. Sample demographic information collected in the post-intervention survey is shown in Table I.
Post-intervention survey sample demographics (n=36)
A total of 4,400 bitewing images were evaluated in the study. Images acquired in semester one were exposed on manikins, and from semester two through four, on live patients. Primary analysis was based on exposure of 2,160 bitewing images in the control group (n=23, M=94), and 2,240 images in the experimental group (n=22, M=102). The number of images exposed per student was not evenly distributed, i.e., some exposed more than the mean number of images and some fewer. Retrospectively, assuming only 50 replications per student, and an autocorrelation (AR1) of 0.60, analysis achieved 80% power (two-tailed) to detect an odds ratio as small as 1.7 (e.g., 20% incorrect responses for the experimental group versus 30% incorrect responses for the control group, alpha = 0.05).
Seventy percent of all images were horizontally oriented and 30% were vertically oriented while 72% of all images were acquired using a paralleling aiming device and 27% with tab holders. Thirty bitewing series were inadmissible as part of the data set (n=11 control group; n=19 experimental group). Reasons for rejection included missing data (participant number, type of holder, etc.), fewer than four images in the series, and students working together on image acquisition.
The failure rate was higher in the control group for both manikin and live patient exposures (p<0.001) (Figure 2). On average, the failure rate was slightly lower on patients (22.2%) versus manikins (25.5%), p<0.008. When considering the failure rate, interactions between the control and intervention groups and manikins versus live patients were statistically significant (Wald Chi-Square=0.000). Likewise, the retake rate was higher in the control group for both manikin and live patient exposures (p<0.001) (Figure 2). The retake rate among both groups was significantly lower on patients (11.5%) than on manikins (22.8%), p<0.001. While both groups improved from manikin to human exposures, the magnitude of change across the two groups did not differ (p=0.992).
Failure and retake rate by group and type
The most common error resulting in a failing image across both groups, all semesters, was sensor placement (16.9%), followed by horizontal angle (6.0%), and vertical angle (2.8%). A total of 27 images failed due to “other” causes. When considering only live patients, sensor placement remained the most common cause of failure for both the control and experimental groups (p<0.001). While the control group had a higher percentage of failures due to sensor placement (p<0.001) and horizontal angle (p<0.001), the experimental group experienced more vertical errors than the control group (p<0.001) (Figure 2). No significant differences in failure rate from all errors was identified across bitewing image views: right molar 22.4%, right premolar 22.5%, left premolar 23.7%, and left molar 22.2%.
Most participants (80%, n=36) completed the demographic survey at the completion of the study (4th semester). When asked about prior radiography experience, 61% indicated no experience, 8% had 1-3 years, 6% had 4-6 years, 6% had 7-9 years; 19% reported having 10 or more years of experience. Years of radiography experience were not correlated with either the number of failing (rs=0.11, p=0.55) or retake exposures (rs=−0.08, p=0.67). Participants with no experience and those with 10 or more years of experience demonstrated similar outcomes.
Experimental group participants (n=18) completed a survey designed to assess their perceived value of the checklist (Cronbach’s α=0.73). Respondents indicated that the checklist was simple to incorporate and use as part of the radiographic exposure process, but fewer agreed that it improved the quality of images (Table II). Three survey items solicited qualitative comments. When asked what aspects of the checklist caused it to be effective, 12 comments were provided. The physical characteristics of the checklist and its ease of use were mentioned by half of the respondents who provided comments (n=6) while the remaining comments indicated that the student either forgot about the checklist or didn’t use it at all. When asked to elaborate why the checklist was ineffective, participants stated that they forgot about the checklist or never used it (n=5); the location of the checklist hindered its use (n=2); the checklist didn’t provide enough detail on how to correct one’s errors (n=1). When asked if they would change something to make the checklist more useful, of the 15 comments provided, over half indicated that no changes were needed (n=8), while several felt the location of the checklist was a barrier (n=3), the remaining respondents were unsure (n=1) or felt the question was not applicable (n=3).
Perceived value of the radiography checklist (n=18)
Participants in the experimental group also completed a 12-item Radiography Checklist Intentions Survey (Cronbach’s α = 0.89) regarding their intentions to continue use of the checklist. Despite being considered easy to use, few respondents planned to use the checklist, or expected their classmates to, in the future. A modest correlation between perceived value of the checklist and intention to continue its use was found (rs = 0.184).
Discussion
High quality radiographs are an essential diagnostic tool for oral health care professionals. However, the dangers of ionizing radiation and radiosensitivity of head and neck tissues require the operator to be a skilled radiographer in order to minimize retakes. A radiography checklist designed to highlight the critical aspects of image acquisition can serve as an aid to the clinician in acquiring diagnostic images and reducing technique error. In this study, an experimental group used a radiography checklist throughout a four-semester program resulting in lower image failure and retake rates as compared to the control group.
While the failure rate of manikin images was considerably higher in the control group as compared to the intervention group, the percentage of failures among this group declined on live patients. However, the experimental group, who used the radiography checklist for all exposures, saw a small increase in failing images on live patients. Although the failure rate in the experimental group remained lower than the control group, the increase may be attributable to the challenges encountered when working in the oral cavity on a live patient.
It is noteworthy that not all failing images require reexposure. While an image may not meet minimum diagnostic criteria, if the areas of interest are evident on an adjacent image a retake exposure may not be necessary. In this study, the retake rate for both the control and experimental groups on manikin exposures was very similar to each group’s failure rate, suggesting that in the context of a four-image bitewing series, failing images were not “saved” by adjacent images in either group. The retake rate of live patient images decreased significantly for both groups, resulting in decreased radiation exposure to patients. A likely cause for the reduced retake rate could be due to gains in operator experience and learning.
When considering all errors that resulted in image failure, sensor placement occurred with the greatest frequency. While bitewing retakes frequently occur due to missing mesial or distal structures,23 challenges presented by tori, arch shape, and other anomalies may also contribute. Horizontal angle errors were second most prevalent among both groups, although the experimental group had a higher percentage of vertical angle errors than horizontal errors on live patients. The same causative factors related to sensor placement errors could also result in vertical angulation errors.
An interesting outcome regarding prior radiography experience and image failure and retake rates was evident. Although no correlation was found among these variables, participants with no radiography experience and those with the most experience demonstrated similar outcomes. It is not unusual for students with significant prior experience to initially struggle to succeed with dental radiography, especially if they received on-the-job training. Individuals who acquire their experience on-the-job often lack didactic and clinical instruction in radiographic principles and need to “unlearn” bad habits and poor technique. Strict attention to detail and familiarity with the grading criteria generally resolve this issue.
Even the best designed intervention will fail if it is not used as intended. In this study, participants generally agreed that the checklist was thorough and simple to use and was easy to incorporate without being disruptive. There was less agreement however, that the checklist improved the quality of the images. It is possible that participants in the experimental group, who were trained to use the checklist from the beginning of their radiography instruction, were not cognizant of the benefits it provided. A significantly lower image failure and retake rate as compared to the control group further supports this assumption.
The Radiography Checklist Intentions Survey indicated that although participants had a strong belief that they could use the checklist on their own, they did not intend to do so, nor did they believe that their classmates would. In this study, the radiography checklist was introduced into the academic setting as part of a research project designed to assess its effect on the diagnostic value of bitewing images. It is possible that students viewed the checklist as a temporary instructional tool rather than a permanent safety measure. Based on the significantly lower failure and retake rates attained by the experimental group as compared to the control group, it may be advantageous to promote the checklist as a standard component of the image acquisition process in the future.
Although this study saw significantly lower image failure and retake rates in the experimental group as compared to the control group, a previous study by Nenad et al.18 did not. In the earlier study, the intervention group experienced a higher failure and retake rate than the control group even though participants found it similarly helpful and easy to use. The larger number of images acquired on both manikins and live patients throughout the duration of the program improved confidence in trends observed in the current study.
Some important distinctions between the two studies should be noted. The current study was implemented over the course of a four-semester curriculum while the previous study took place during a 12-week period in the final semester of the program. It is possible that learning with the aid of the checklist for the duration of the program allowed its use to become a habit resulting in more successful images on the first attempt. Additionally, a greater percentage of images were horizontally oriented, exposed using a tab holder, and acquired with a digital sensor, than in the previous study. Each of these factors may have influenced the quality of the images thereby affecting the failure and retake rates.
This study had several limitations that may have influenced outcomes. Operator fatigue may have occurred as the study progressed resulting in participants no longer “seeing” the checklist. Participants were not asked to make a physical check mark on the document making it impossible to determine if each item was read and/or performed. If an image “failed”, was reexposed, and “failed” again, only data from the first “failed” image was recorded for purposes of the study. Suggestions for future research include adapting a radiography checklist to clinical dental hygiene practice outside the educational setting and further exploration of participant belief that although the checklist was readily adaptable to practice, it did not contribute to improved image quality.
Conclusion
Use of a radiography checklist in the educational setting can contribute to reduced radiographic image error and retake rates, thereby reducing patient exposure to ionizing radiation. Ease of implementation and participant acceptance of the checklist may further encourage dental and allied dental education programs as well as practitioners to consider adapting a radiography checklist to their image acquisition procedures.
Acknowledgements
Special thanks to Curt Bay, PhD, for his statistical expertise and guidance and Ann E. Spolarich, RDH, PhD, for manuscript development.
Footnotes
This manuscript supports the NDHRA priority area Professional development: Education (educational models).
- Received December 19, 2020.
- Accepted September 21, 2021.
- Copyright © 2022 The American Dental Hygienists’ Association
