Abstract
Purpose The use of manikin testing was recently introduced for dental hygiene licensure examinations. There is currently a lack of research about the efficacy and accuracy of manikin testing for clinical licensure. The purpose of this study is to identify perceptions of dental hygiene educators regarding the use of manikins for the dental hygiene clinical licensure exam.
Methods This qualitative study used an exploratory, online, focus group design with 20 dental hygiene educator participants recruited through purposive sampling. Pseudonyms were used to protect participants confidentiality. Two groups of educators were from institutions that did not use manikin testing during COVID-19, and two groups of educators were from institutions that did use manikin testing during the same time period. Themes were analyzed using the classic analysis strategy. Validity was established using investigator triangulation, member checks, and saturation.
Results Lack of knowledge including preconceived assumptions regarding manikin examinations; testing considerations with benefits and concerns regarding manikins versus live patients; and perceptions regarding the value of single point in time clinical licensure examinations were the three major themes identified by the participants.
Conclusion The manikin exam appeared to address ethical concerns, however, there were limitations in relation to assessing critical thinking and decision-making skills. Some participants expressed that graduation from an accredited dental hygiene program was considered sufficient for licensure. Future studies should include comparisons of recent graduates who complete a clinical licensure examination versus those who do not complete a clinical licensure examination.
INTRODUCTION
In order to receive a dental hygiene license in the United States (US), a candidate must graduate from an accredited dental hygiene program, achieve a passing score on the written National Board Dental Hygiene Examination, and receive a passing score on a clinical board examination for initial licensure.1 Prior to the COVID-19 pandemic in 2020, the only dental hygiene clinical board examinations were performed on live patients. For this examination, candidates must select a patient with an acceptable medical history, has diagnostic radiographs, and twelve surfaces of qualifying subgingival calculus.2,3 Candidates are evaluated on their ability to assess and select a qualifying patient. In addition, their clinical skills are assessed; these assessments include calculus detection and removal, ability to accurately measure periodontal pocket depths, and appropriate soft tissue management.2,3
Preparing for clinical licensure examinations and completion of program requirements during the COVID-19 pandemic created many issues for state licensure boards, as well as dental and dental hygiene education programs. Dental and dental hygiene students had limited capacity to treat live patients given the nature of the COVID-19 virus and the national guidance restrictions.4-6 To adapt to the situation, many schools needed additional clinic sessions to help students complete their necessary requirements for graduation.7 Live patient examinations for initial licensure could not be administered due to the pandemic which caused a delay in the students’ timeline for receiving their licenses to practice.7
Previous discussions have supported eliminating live patient examinations for initial licensure. In March 2014, the American Dental Education Association (ADEA) passed a resolution that created a task force to evaluate the elimination of live patients for dental and dental hygiene clinical licensure testing.8 In January 2016, the Task Force distributed their recommendations towards the elimination of live patients. In 2018, the American Dental Hygienists’ Association (ADHA) and the American Dental Association (ADA) passed resolutions supporting the elimination of live patients for clinical licensure.9,10
Other studies have demonstrated concerns regarding live patient examinations.8, 11-17 In 2016 ADEA surveyed dental hygiene program directors about ethical implications with human subjects for the clinical board testing and nearly all (93%) of the participants stated that their primary concern was incomplete treatment when only using the patient for the clinical board examination.8 Another ethical concern reported by most (92%) program directors were concerns that students would postpone treatment for the patient or provide monetary incentives to convince the patient to participate in the examination.8 Feil et al. expressed a concern that postponing treatment of patients to meet exam criteria may cause the patient to remain in a painful or diseased state for an unnecessary amount of time and also lead to compromising patient care for personal gain.11 Other researchers noted that dental hygiene students felt that delaying treatment was dependent on the actual amount of time elapsed and that delaying treatment is sometimes required.12 Another ethical consideration occurs when students feel pressure to find patients who meet certain requirements and share patient details with another student without patient consent violating patient privacy issues as defined by the Health Insurance Portability and Accountability Act.8,16 In addition, since board patients only sit for a specified examination with limited treatment, they are prone to patient abandonment by the student once the student passes the exam and graduates.17
Validity and reliability of both the dental and dental hygiene live patient single point in time clinical licensure examinations has been questioned. Studies conducted since 1999 have investigated these topics showing limited association between these parameters and academic performance.11,13-15 Gadbury-Amyot et al. examined the validity of a one-shot clinical licensure examination, Central Regional Dental Testing Service (CRDTS), as a predictor of success as defined by a canditate’s overall Grade Point Average, performance on the National Board Dental Hygiene Examination, and Portfolio assessments.13 Results indicated a lack of agreement between previously validated measures of student success and competency with successful performance on the CRDTS examination for clinical licensure.13
Although these concerns and discussions have occurred for over twenty years, no significant action was taken until the COVID-19 pandemic which required immediate action for identifying valid and reliable alternatives to live patient testing for clinical licensure.18-21 A manikin clinical licensure examination was offered as an alternative test in 2020; however, not all licensing boards accepted this option. These testing agencies offering a manikin clinical licensure examination also included a computer simulated Objective Structure Clinical Examination (OSCE) as part of the test.18-21 Candidates performed calculus detection, calculus removal, and measurement of periodontal pockets on typodonts provided by the agencies. Depending on the testing agency, the OSCE may include radiographs, photographs, model images, and laboratory data to provide background information regarding the patient scenarios.20 The use of OSCEs for clinical licensure is new to dentistry in the US, but the National Dental Examining Board of Canada (NDEB) has been using an OSCE since 1995 for clinical licensure testing in dentistry.22 Currently, many states have decided to indefinitely allow for a manikin examination, including OSCE component, for clinical licensure. However, several states only accepted manikin based examinations until the end of 2022 and some states do not accept manikin based examinations at all.23 In 2024, the Joint Commission on National Dental Examinations (JCNDE) will be launching the new Dental Hygiene Licensure Objective Structured Clinical Examination. This examination does not have a manikin component and it is not known which states will accept this examination for dental hygiene licensure.24
The use of manikin testing is new in the dental hygiene profession and there is limited research about the validity and reliability of manikin testing to assure competence for dental hygiene licensure. A recent study examining the scores between live patient and manikin examinations in students from a single dental hygiene program found no statistical significance in the passing rate and calculus removal scores of the candidates on either test. Further research is needed to verify the reliability of manikin-based examinations.25 Dental hygiene educators prepare students for clinical practice and regularly assess students’ clinical abilities while in school. Dental hygiene educators’ perceptions towards this type of examination and whether it is considered a valid assessment of clinical skills needs further investigation. The purpose of this study was to identify perceptions of dental hygiene educators regarding the use of manikins for the dental hygiene clinical licensure exam. The following questions guided the research process: What are dental hygiene educators’ perceptions of using manikins for the dental hygiene clinical licensure exam?, What are dental hygiene educators’ concerns about using manikins for the dental hygiene clinical licensure exam?, and Why do dental hygiene educators believe manikin testing is a valid or not valid assessment of students’ clinical skills?
METHODS
This exploratory qualitative study used virtual focus groups to gain insight to the different perspectives of dental hygiene educators on manikin-based examinations for clinical licensure and qualified as exempt by the Idaho State University Institutional Review Board (IRB-FY2022-180). The qualitative exploratory design was chosen due to a lack of data regarding dental hygiene educators’ perceptions on manikin testing for board exams and was considered appropriate due to the limited research available.26 Inclusion criteria were educators who are full-time faculty members in the US with three or more years of teaching experience. Adjunct or part-time faculty were excluded from the study as well as faculty with fewer than three years of teaching experience. Participants teaching outside of the US were also excluded.
This study used a small focus group design. Kruger and Casey define the small focus group as consisting of four to six participants and consider it is easier to recruit participants as they tend to feel more comfortable in interview sessions with fewer people.27 Three to four focus groups are recommended to determine if adequate saturation is reached.27 For this study, four groups of dental hygiene educators consisting of 5 participants per group were recruited for a total of 20 participants from 20 different dental hygiene programs. Two groups (10 participants) were from institutions that did not use manikin testing during COVID-19 and two groups (10 participants) were from institutions that used manikin testing during the COVID-19 pandemic.
Participants were recruited by emailing a random sample of thirty percent of the dental hygiene program directors from the entry level dental hygiene programs from every state in the US and asking for full-time faculty members with three or more years of teaching experience who were willing to participate in the study. Participation was fully voluntary and there were no incentives provided for participation. The first 10 respondents meeting the inclusion criteria that were from states allowing manikin testing and the first 10 respondents meeting the inclusion criteria from states not allowing manikin testing were included in the study. Focus group sessions were held using a virtual conferencing platform (Zoom; San Jose, CA, USA). Participants chose a pseudonym for the interview to maintain confidentiality and anonymity. They were also informed that their participation in the study was completely voluntary and they were allowed to withdraw at any point during the study without repercussions. All individuals participated; no one withdrew.
The interview guide was developed based on the research questions and the literature review. The guide followed the questioning route model27 which includes five components: opening, introduction, transition, key, and ending (Appendix A). The interview guide was reviewed for content validity by two qualitative researchers, prior to the start of the study. The primary investigator (PI) and a co-investigator pilot-tested the questions in a recorded interview with a volunteer educator who self-identified as being against manikin testing and a volunteer educator who self-identified as supporting manikin testing. The PI reviewed the recording with an experienced moderator for feedback on any adjustments in the presentation of the questions and reactions to the responses. Pilot testing also allowed for adjustments in the order of the questions and clarifications for improved understanding.27
The focus group interviews lasted between 45 to 60 minutes. Demographic questions included how long they have been a dental hygienist, years of teaching experience, and why they chose a career in education. Participants were then asked to answer open-ended questions regarding manikin examinations for clinical licensure. Participants answered all interview questions and there was no interaction among participants. The focus group sessions were conducted by the PI and recorded via the online platform and transcribed word for word from the recording. Once the transcription was completed, the PI and two research members worked together to perform an analysis of the data across groups to determine saturation. No new information was obtained; therefore, the researchers determined the four focus groups were sufficient for the study.
A qualitative data analysis software program (Dedoose; Los Angeles, CA, USA) was used to consolidate the data into parent and child codes. The classic analysis strategy was used to identify themes based on frequency, consistency, emotion and extensiveness.27 Validity was established through investigator triangulation,28 respondent validation, and member checks. The PI and co-investigator reviewed and interpreted the transcripts separately; agreement on differences was made through discussion. Transcripts were sent to participants to allow the opportunity to view and comment on the researchers’ interpretation of the data to verify accuracy. Saturation was also used to validate the study.26 When the same information was being repeated and no new information was being relayed during the interviews, the PI and co-investigator determined that saturation had been reached.
RESULTS
Twenty dental hygiene educators participated in the study; nearly all (95%, n=19) were female. Most had been dental hygienists for more than 20 years (65%, n=13) and nearly half (45%, n=9) had been teaching for 13 years or longer. Reasons for choosing a career in education included being influenced by dental hygiene educators from the programs they had attended or from the areas they resided in 30% (n=6) or having a desire to teach 35% (n=7). Demographic information is summarized in Table I.
Demographics (n=20)
Prior to addressing manikin use for clinical exams for licensure, participants were asked if they used manikins as part of the education experience for dental hygiene students. If so, they were asked to describe the advantages and disadvantages of manikin use. Many of the participants had used manikins in their programs to evaluate students’ preclinical abilities. Advantages of using manikins to evaluate clinical skills included being a more standardized method of evaluation, the ability to demonstrate while the student is working and provide immediate feedback when there is no live patient, and not being concerned about patient safety. Disadvantages included the lack of human characteristics in a manikin such as saliva, blood, cheeks, and a tongue, and that students are not experiencing realistic scenarios that they will encounter during instrumentation. Other disadvantages mentioned were that real-time patient management skills cannot be evaluated on a manikin and the expensive manikin costs and maintenance issues.
Three themes were identified from the focus groups: lack of knowledge; testing considerations; and perception of value (Figure 1). Participant quotes supporting the themes are shown in Table II.
Themes and subthemes
Selected focus group questions with representative responses
Lack of Knowledge
Focus group participants expressed an overwhelming concern about their personal lack of knowledge related to the manikin clinical examination. These concerns centered around lack of information and preparation. Participant S stated, “I actually feel less prepared with the manikin clinical exam because it is so new; I don’t feel like I have enough information to help [students] feel confident going into it.” CDRDH expressed lack of preparation concerns and commented on communication form the testing agencies.
“I have learned that they have an educators’ conference each year that I was not aware of that would be very helpful. I feel like personally the educators were not properly prepared to teach students the manikins; I wish we would have had better guidance.”
Participants who lacked experience with the manikin clinical examination had many preconceived assumptions related to the test including assessment procedures, characteristics of the manikin, and calculus qualities. These assumptions represented a subtheme. Evelyn stated, “The calculus is pre found for them,” while Thelma indicated:
“I know that they are graded on calculus detection plus calculus removal. I don’t think there’s radiographs involved, I guess that’s not something I know about. I think most of the periodontal aspects of that manikin clinical exam is non-existent.”
Testing Considerations
Five subthemes emerged related to testing considerations: ethics, standardization, operations, critical thinking, and outcomes. Participants had considerable concerns about the live patient clinical examination and the manikin removed some of those concerns. Ethical considerations regarding students and patients were discussed. Y Knot expressed, “[They] had a couple of students whose patients were extorting them for more money or they weren’t going to show up to the exam.” Regarding the manikin examination, Bellamuse stated, “students wouldn’t have to pay for patients to travel, and hotel rooms, and gas, and food as well.” Ethical concerns regarding patient care focused on completing treatment and preventing disease progression demonstrated in the following quote by Erin:
“I also believe that the manikin exam is more ethical because [in] the live patient exam, the student only removes calculus in part of the mouth, and therefore total patient care is not provided. And to me that is totally unethical, and I can’t believe that the licensing agencies have gotten away with that for years and years and years.”
Many participants felt the manikin examination provided testing standardization and created equity in testing conditions between the candidates. A representation of this subtheme was shown in the comment by CDRDH “I think it’s a fairer playing field that all students are taking the same board across the nation. And it’s more calibrated.” Wilma stated, “I think you can get a true assessment if you test everybody on the same manikin, the same typodont, the same calculus, the same teeth.” Liz Lemon discussed how the manikin test can help alleviate stress for the students, “I think the biggest thing is that standardizes it, and it eliminates a lot of the things that are out of the students’ control [with] their patient.”
Cost for live patient exam administration and infection control concerns were topics of discussion regarding operations. Participant M discussed how operating costs were decreased with manikin use because “you don’t have to buy gauze and suction. Less bloodborne pathogens, there’s no contamination, [and] no dentist needed to do anesthesia for the patient so overall less costs,” while Participant Me commented about the issue of contaminated instruments, “with a live patient, the candidates are having to take their used instruments out of that facility without being able to sterilize them.”
Concerns about decreased critical thinking regarding comprehensive care were articulated by a number of participants and best represented by Participant M:
“The reduction of critical thinking. It takes [away] the students’ ability to choose the correct patient, to calculus detect in advance, [because] the mountain of the exam is picking the right teeth. If we take that away [and] give the manikin, can you really detect calculus? There’s no blood, there’s no saliva, there’s no obstacles to see what you can do on a real person. You’re going to be treating people every day after graduation. So, does the manikin really tell me that you are a capable dental hygienist?”
Another concern mentioned was students’ general preparation for the manikin examination. Y Knot stated, “I think that the students feel that it’s going to be easier being on a manikin, so they do not prepare mentally as well for the exam.”
A final subtheme was many participants were concerned about the outcomes of clinical examinations for dental hygiene students. For example, Moonflower38 stated clinical exams “don’t truly demonstrate that the student is competent to manage a patient once they get out in the field” while LuLu expressed “I think it’s a false sense of passing, I just think it gives the false sense of skills.” Moonflower38 also noted “I just don’t feel that it’s a true representation of what’s truly going to be expected of them.” Other participants voiced concern about excellent student clinicians failing the manikin clinical examination and the outcome of the test result did not accurately reflect the ability of the clinician. Scenery shared, “we also had a student who was a very good clinician that was unsuccessful and it was just by half a point. We were discussing do you appeal it and pay $400 and not be sure that you’re going to pass.”
Perception of Value
Participants had mixed feelings regarding the validity of a manikin or a live patient high stakes clinical examination for licensure while other participants believed that there was value in the third-party assessment process. These sentiments are represented by comments from Wednesday Addams:
“We are the gatekeepers for our profession. And if somebody doesn’t pass the standards, unfortunately, they’re out of the program. Not all fields are for everyone. So sometimes it’s good that they don’t pass and end up with something that’s really not a good fit for them.”
On the other hand, some participants expressed fulfilling the accreditation standards of the entry-level program was sufficient to demonstrate competence for licensure. This discussion led to the subtheme of measuring competence. Participant Z stated, “The institution should be the final decision maker if the graduate is competent enough to practice in dental hygiene. That should be left up to the institution.” Another key quote from RDH supports this theme.
“I think graduating from a CODA accredited dental hygiene program is a valid measure of their skills. And this standardizes things because the Commission on Dental Accreditation requires the same standards for every program. If the program meets those standards, the student is competent, and they should be granted licensure.”
Furthermore, participants identified concerns about a one-time exam being relevant to measuring competence across time in the dental hygiene program. Y Knot explained, “In general, I am opposed to having a one-shot deal to get your licensure with an exam. I feel that fully accredited programs have proven themselves to graduate competent hygienists. A clinical exam is not needed.” Bellamuse expressed “I do not think the manikin exam is a valid assessment, but I would have to say the same for the human exam. I think the years that they’re in school and the competencies that they pass over 2 years is a valid assessment.”
Lastly, participants were asked what other options they would consider to be a valid evaluation of students’ clinical skills for licensure. Two participants recommended the use of e-portfolios to provide additional means of measuring student competence. Evelyn stated, “I think an e-portfolio could showcase the educational journey of the student and show the different competencies besides just calculus removal. You could see radiographic competencies, local anesthesia, restorative, [and] all the components of dental hygiene process of care,” while Liz Lemon indicated, “I feel like there could be some merit to portfolio system too, where we can demonstrate the clinical aspects of our programs and showcase those in a portfolio.”
DISCUSSION
Results from this study provided an insight into the opinions of dental hygiene educators towards the use of manikins for clinical licensure examinations. While eliminating the use of live patients for clinical licensure examinations has been a stated goal of both dental and dental hygiene professional associations, it was not until the spring of 2020 during the SARS-CoV 2 pandemic that licensing bodies began to accept manikin test results for dental hygiene licensure. Educators have voiced concern about the ethics of testing on live patients to obtain a license after graduation from an accredited dental or dental hygiene program for the past two decades.8,11,12,17 Manikin clinical licensure examinations appear to be a step forward in addressing these concerns. In this study, issues of cost, incomplete treatment, and preventing disease progression were examples of ethical considerations the participants reported as particularly noteworthy.
Although eliminating ethical concerns was an advantage of the manikin exam, a common disadvantage mentioned during the small focus group sessions was that the manikin examination only evaluates a limited skill set of abilities such as probing, calculus detection and hard deposit removal. Participants also mentioned that the manikin examination is another form of a point-in-time, high stakes exam and does not represent comprehensive dental hygiene care. Research has shown that this type of examination is situation reliant; the test assesses what the candidate has already accomplished.29 While the manikin examination may provide a standardized approach, it is only assessing a small portion of the skills that are required in clinical practice. This standardized approach to specific clinical skills does not evaluate the critical thinking, decision making, patient management, and other elements reflecting the broader scope of practice required for comprehensive dental hygiene care.
Many participants discussed concerns about the validity of manikin testing and high-stakes, single point in time clinical examinations for licensure. These participants believed that graduating from an accredited dental hygiene program should be sufficient for licensure, a perspective that has been addressed in previous research. A 2016 ADEA study reported that most (75%) dental hygiene program directors believed graduation from an accredited dental hygiene program was an appropriate measure of clinical competency without the additional need for a clinical licensure examination.8 Furthermore, 86% of the ADEA study participants supported using an alternative pathway for licensure and terminating the use of live patients in clinical licensure examinations.8 While these results supported graduation from an accredited program as a valid measure of competency for licensure,8 the authority to make this change lies in the state boards of dentistry whose overall mission is to protect the public.
Dental professions in the US are the only health professions that require a clinical licensure exam involving a live patient.30 There have been many discussions and recommendations regarding the elimination of live patient examinations dating back to 2005.13 According to the findings of the ADEA Task Force on Assessment of Readiness for Practice, using live patients for a single encounter is not in the best interest of the patient as it may result in a standard of care that is below acceptable levels.31 The task force has recommended that state dental boards eliminate single encounter exams on live patients, increase licensure portability, and support a standardized licensure compact between states.31
Alternative options for the live patient clinical examination such as OSCEs, the Buffalo Model, and the use of portfolios have been studied.22, 33-35 Currently OSCEs are being planned to be available in 2024 for dental and dental hygiene students in the US. However, these OSCE licensure examinations may not be recognized by individual state boards of dentistry.24
The Buffalo Model is an alternative option for the clinical licensure examination in dentistry that does not eliminate the use of human patients but revises the current examination to address ethical concerns. The Buffalo Model puts an emphasis on patient care by providing students with multiple exam dates that allow patients to be treated earlier in the academic calendar year rather than delaying treatment until the end of a school year for the traditional clinical exam date. Within this model the licensing examinations are conducted during a regular school session and scheduled on multiple days throughout the academic year. Calibrated faculty are available to observe candidates during the examination for any critical errors. If errors are made during the exam, the faculty are able to supervise the patient’s final treatment.33
Portfolios are another method of assessment that can be used in place of a live patient clinical examination. Portfolios are a collection of academic evidence throughout a student’s education that demonstrates knowledge, abilities, and a reflection of learning. Portfolios also provide examples of students’ critical thinking skills, the ability to problem solve, self-assess, and self-guide learning that helps express competency.32 Research has demonstrated external validity of portfolio assessments in dental hygiene programs in comparison to the National Board Dental Hygiene Examination, Grade Point Average, and CRDTS examination.36 In California, as of 2014, dental school graduates applying for licensure may submit a portfolio that is evaluated by calibrated examiners.37
The Dental Hygiene Board of California is moving towards the elimination of a clinical licensure examination for graduates of California dental hygiene programs. The state of California has the only self-regulating dental hygiene board in the US. The Licensing and Examination Committee has recently voted to recommend that the full board approve the statutory language amending the clinical examination requirement for graduates of California dental hygiene programs, provided that they apply within three years of graduation, have passing scores on the National Dental Hygiene Board Examination, and pass the California Law and Ethics Examination. Graduates from programs outside of California would still need to complete a board approved clinical examination. California licentiates moving out of state would be subject to the licensing requirements of the specific state.38
There were limitations to this study. The use of purposive sampling and qualitative data collection methods limits generalization to the entire population of dental hygiene educators in the US. Qualitative research is not intended to generalize, but rather provides the ability to learn in depth perceptions, opinions, and trends and patterns which may not be apparent through survey research.26 Another limitation was the PI served as the moderator for the focus group. However, steps were taken to control moderator bias including pilot-testing the questions, member checks, having an additional investigator in the interviews, and using an ending question that allows participants to restate their position on the matter.
Further investigation about clinical licensure examination is warranted. The perspectives of dental hygiene students and recent graduates who are practicing clinicians regarding use of the manikin exam or other licensure pathways should be considered in terms of testing considerations, patient safety, and quality of care. Future research should include outcome assessment comparisons of recent graduates who completed a clinical licensure examination versus those who did not complete a clinical licensure exam.
CONCLUSION
This qualitative study examined dental hygiene educators’ perspectives on the use of manikins for the clinical licensure exam. Lack of information and preparation for a manikin-based examination was challenging for educators as well as students. While many of the ethical concerns of the live patient examination were addressed with the use of a manikin there were limitations to the manikin exam in relation to assessing critical thinking and decision-making skills. Graduating from an accredited dental hygiene program was considered a valid measure for licensure rather than a clinical licensure examination however state licensing boards are responsible for determining candidates’ competency for licensure. Future studies should include comparisons of recent graduates with and without use of a clinical licensure examination to establish competence.
Appendix A. Interview guide
Footnotes
NDHRA priority area, Professional development: Regulation (scope of practice).
- Received November 11, 2022.
- Accepted April 21, 2023.
- Copyright © 2023 The American Dental Hygienists’ Association
