Abstract
Purpose: Concept mapping, as a teaching strategy, has been shown to promote critical thinking and problem solving in educational settings. Dental clinicians must distinguish between critical and irrelevant characteristics in the delivery of care, thus necessitating reasoning skills to do so. One of the aims of the American Dental Education Association Commission on Change and Innovation (ADEA-CCI) is to identify deficiencies in curriculum which were meant to improve critical thinking and problem solving skills necessary in clinical practice. The purpose of this study was to compare 2 teaching strategies, traditional lecture and lecture supported by concept mapping exercises within collaborative working groups, to determine if there is a beneficial effect on meaningful learning.
Methods: For this pilot study, the study population consisted of students from 2 geographically separated associate level dental hygiene programs in the southeastern U.S. A quasi-experimental control group pre- and post-test design was used. The degree of meaningful learning achieved by both programs was assessed by comparing pre- and post-test results.
Results: Both programs experienced a significant degree of meaningful learning from pre- to post-test. However, there was no statistically significant difference between the programs on the post-test. These results were in direct contrast to research in other disciplines on concept mapping and its effect on promoting meaningful learning. Further investigation into the study's outcome was obtained through a follow-up focus group.
Conclusion: In spite of careful attention to methodology in the development of this research project, the focus group illuminated methodological failings that potentially impacted the outcome of the study. Recommendations are underscored for future conduct of educational research of this kind.
Introduction
In 2006, repeated calls for change in the way dental hygiene education is delivered from both outside and within dentistry led to the formation of the American Dental Education Association Commission on Change and Innovation in Dental Education (ADEA CCI). ADEA CCI issued a white paper that outlined 8 core principles that the CCI felt should characterize dental education and inform curricula.1 Two of the ADEA CCI core principles are central to this research study: critical thinking as the cornerstone of the dental education experience, and faculty development.
The ADEA-CCI identified deficiencies in curriculum which were meant to improve critical thinking and problem solving skills necessary in clinical practice.2 This issue is not unique to dentistry, as educators across the country are struggling with how to promote critical thinking and problem solving in the educational environment where students often focus on memorization to learn, and fail to fully engage in critical thinking and problem solving for meaningful learning.3 Haden et al posit that a dental educational environment characterized by the discipline of critical thinking develops self-directed, self-disciplined and self-corrective learners.1 A teaching strategy, concept mapping, has been shown to promote critical thinking and problem solving in educational settings.4-5 This study compared 2 teaching strategies, traditional lecture and lecture supported by concept mapping exercises within collaborative work groups, to determine if there was a beneficial effect on meaningful learning and promotion of critical thinking and problem solving.
Dental educators acknowledge the need to improve critical thinking strategies to encourage good clinical judgment.6,7 Since the vast majority of dental hygiene students will spend their careers in clinical settings, it is important to consider how educational strategies can promote good clinical judgment. Williams et al explored the issue of critical thinking in dental hygiene education and found that the educational experience did little to promote critical thinking as assessed by a pre- and post-test.7 The authors emphasized the need for preparation in critical thinking practices to begin within the first semester of the students' pre-clinical curriculum. By employing teaching strategies that have been theorized to promote higher order thinking, educators can potentially foster skills required for critical thinking and problem solving at the onset of the students' specialized education. Both Wallace et al6 and Williams et al7 emphasized the need for faculty development and improvements in pedagogical processes to enhance critical thinking and problem solving.
Critical thinking and problem solving, though related, are distinct concepts. One main distinction between these processes is that critical thinking involves evaluation, whereas problem solving is goal oriented.8 Skills involved in critical thinking can be defined as locating information appropriate to a purpose, analyzing an argument, differentiating fact from beliefs, and weighing evidence or options. These are quite different from skills utilized in problem solving, which are strategic and organizational in nature. They include recognizing a problem exists, selecting a strategy, implementing a strategy and evaluating the results. Critical thinking and problem solving, though separate entities, coexist in support of each other and share a common foundational base in their development, meaningful learning. Meaningful learning, as opposed to rote memory, requires an in-depth understanding of material where facts are not isolated but rather interrelated and newly learned concepts are directly associated with those previously learned.9 However, rote learning, while not generally useful by itself, is essential to meaningful learning as it forms the basis/linkage of foundation knowledge to new material. The distinction between rote and meaningful learning is not a simple dichotomy, but rather a continuum of the learning process along with the student's willingness to achieve rich and meaningful learning outcomes.10 Two main goals in the learning process are the retention and the transfer of information. Retention is remembering previously learned information, whereas transferring involves the application of what one has learned in one situation to another situation. An obvious example would be how students transfer classroom learning to a clinical setting. The accomplishment of this kind of learning transfer has been defined as an indication that meaningful learning has occurred.11
Meaningful learning is not a new phenomenon in education. David Ausubel formulated the Assimilation Theory of Meaningful Learning in 1963, in which he hypothesized that complex knowledge requires understanding for conceptual meaning to be achieved and for meaningful learning to occur.12 His theory has aided in understanding how humans learn by linking old and new knowledge, and can be defined as a cognitive, goal oriented process that is active, constructive and cumulative over time. Knowledge gained through meaningful learning can then be applied or transferred, through the process of critical thinking or problem solving.10,11 Ausubel's contributions to conceptual learning and knowledge construction are supported by the work of developmental psychologists and constructivist theorists Jean Piaget13 and Lev Vygotsky.14 Their work emphasized the conceptual nature of the framework of learning, the need for pre-existing knowledge with which to construct one's own understanding and meaning of new knowledge (constructivism), and the influence of social interactions on the learning process (social constructivism).
Research has identified concept mapping as an effective educational strategy that is supported by cognitive constructivism and cognitive neuroscience theories for the promotion of meaningful learning.15-18 Concept mapping builds on the science of constructivism in that the act of mapping provides the venue for students to process information and organize knowledge. Further, by working within peer groups, the student not only presents their own pre-existing knowledge but is given the opportunity to hear their peers process the information by each bringing their own unique pre-existing knowledge to the exercise.15 The mapping exercise concentrates on a main subject or discipline and the linking of interrelated concepts establishes and/or builds a knowledge base for students to reflect upon. They are designed in two-dimensional formats to illustrate hierarchical and interconnection of concepts. The explicit mapping exercises involve strengthening links between various properties as ideas, text, etc., to create meaningful conceptual associations. Informed through the works of Vygotsky, Piaget, Ausubel and recent brain-based research in the neurosciences, McKay and Gibson conclude that curriculum development that proceeds from a constructivist and cognitive neuroscience perspective should recognize the centrality of the following 4 doctrines: visual representation, active learner involvement, the use of new and previous learned knowledge and social interactions to facilitate the learning process.15
This study was designed based on the intersection of educational theory, cognitive science and neuroscience research. The purpose of this study was to compare 2 teaching strategies, traditional lecture and lecture supported by concept mapping exercises within collaborative groups, to determine if there is a beneficial effect of concept mapping and social learning on meaningful learning. This study was approved by the institutional review boards of all participating institutions.
Methods and Materials
For this pilot study, the study population consisted of students from 2 geographically separated associate level dental hygiene programs in the southeastern U.S. A quasi-experimental control group pre- and post-test design was used to examine the effects of concept mapping on meaningful learning. In educational settings it is often impossible to have a true control group which would involve random assignment. The test group (Dental Hygiene Program 1) received the new method of instruction (for purposes of this study this included concept mapping in collaborative groups). The control group (Dental Hygiene Program 2), sometimes called the quasi-control group, received the traditional method of instruction (for purposes of this study lecture alone). Applying strict scientific methods to educational research which would involve randomization and a true control group is not easily accomplished. A quasi-experimental control group pre- and post-test design was used to examine the effects of concept mapping on meaningful learning, followed by a focus group of participants from Dental Hygiene Program 1.
Educational researchers must incorporate several steps to ensure the validity and reliability of outcomes. First, threats to internal validity were minimized by the following design measures. A control and test group study design using 2 different dental hygiene educational programs for data collection was employed to address the internal validity threat of diffusion of treatment. Using 2 geographically separated dental hygiene educational programs with no known relationship eliminated interaction of compensatory rivalry, compensatory equalization of treatment and/or resentful demoralization should this have occurred within the study. Second, internal threats to validity involving history and maturation were minimized by the use of a comparison group. Finally, testing threats were minimized by using different but equivalent forms of pre- and post-test instruments.
The statistical design reflected in this study is one of a two-factor, repeated measure. Analyses include comparisons of treatment and control groups at pre- and post-instruction periods as well as differences within each student group. The Ryan-Einot-Gabriel-Welsch Multiple Comparison Procedure was used with the appropriate error terms taken from the designs' repeated measures ANOVA. A Bonferonni adjustment for the 4 comparisons was made.
Subjects
First year, entry level dental hygiene students from Dental Hygiene Program 1 (test group) and Dental Hygiene Program 2 (control group) were recruited during the first week of the 2010 fall semester. The entering classes from Dental Hygiene Program 1 (test group, n=36) and Dental Hygiene Program 2 (control group, n=42) were offered the option to participate in the study while taking the course Dental Radiology. Course selection for this study was based on applying concept mapping to a foundational course within the dental hygiene curriculum to assist in the development of the students' knowledge base. A total of 33 students from Dental Hygiene Program 1 (test group) and 26 from Dental Hygiene Program 2 (control group) chose to participate in the study.
Demographics of Study Participants
Pre- and Post-Test Question Analysis
Participation in the study was completely voluntary. Demographics were gathered on all students participating in the study to include gender, ethnicity, age, years of undergraduate education prior to admission to dental hygiene, prior experience in the dental field and grade point average at the initiation of the study (Table I).
Procedures
The material and content taught in each course were independently analyzed by the primary and co-investigator to ensure comparability. Using the course syllabi, comparability was determined through a thorough examination of course similarities and content. A total of 4 subject categories were mutually agreed upon:
Subject Category 1: Foundational Knowledge (this included radiation basics, production and equipment, radiobiology, infection control, safety and protection, prescribing dental radiographs, patient education and patient management)
Subject Category 2: Application and Procedural Knowledge
Subject Category 3: Assessment Knowledge
Subject Category 4: Panoramic/Extraoral Imaging, Digital Radiology and Supplemental Procedures
The identified subject categories (1 through 4) were analyzed to determine amount of class time devoted to each (Table II). This information was then used when determining the proportion of test questions to include for each subject area on the pre- and post-test.
In an effort to categorize outcomes and, thereby, provide structure to objective evaluation by the expert reviewers in this study, a cognitive taxonomy was employed. One of the best recognized cognitive taxonomies is that of Bloom's.19 In this taxonomy, Bloom attempted to organize learning into levels according to the sophistication of mental effort necessary to meet a given goal. In 2001, a revision of Bloom's original taxonomy was published.20
Test questions were constructed by the primary investigator to represent levels of meaningful learning as defined by Bloom's Original Taxonomy of Learning Domains (application, analysis, synthesis and evaluation), and were similar in content and form to questions found in national dental hygiene board review books.19 To establish content validity of the test questions, 3 expert reviewers were selected to evaluate test questions for content accuracy and to independently determine levels of Bloom's taxonomy. All of the expert reviewers are authors of currently used dental radiology textbooks in dental hygiene programs across the U.S. Test questions determined by the content experts to be accurate and that were representative of Bloom's ratings of application and above were considered valid for capturing meaningful learning and thus used in the study.
For the pre- and post-test, 50 test questions were randomly selected for each from the pre- and post-test banks using a procedure that randomly selected a representative number of questions, as described above, from each subject category. Each test question was of equal weight when calculating scores and averages. This study assumed that test questions selected from the same set would produce equivalent instruments (pre- and post-test). During the first week of the 16 week fall semester 2010, all students from both participating institutions were given a sealed packet from their course instructor containing a consent form, a demographic survey and a pre-test.
Students at Dental Hygiene Program 1 (test group) who agreed to participate received traditional lecture with the use of PowerPoint assisted instruction and concept mapping teaching strategies designed to support corresponding lecture content. Prior knowledge in how to concept map is a requirement for successful mapping and consideration was taken within the study to introduce the fundamentals of concept mapping to students. A session was conducted at the beginning of the semester to provided information on the nature of concept mapping and instruction in how to construct concept maps through visual imaging, hands-on mapping exercises, open discussions and group activities for social learning. Four collaborative concept mapping sessions were scheduled and delivered throughout the semester. Collaborative learning groups were established for concept mapping exercises, and consisted of 4 randomly assigned students each. Student groups changed for each of the 4 concept map sessions to allow students to work with different members of their class. It was the design of this study to vary the groups to eliminate student pairings and maximize heterogeneity and diversity within the study. Each group was randomly assigned a facilitator from within each group of participating members to initiate discussions and concept mapping strategies. In contrast, students at Dental Hygiene Program 2 (control group) were taught via traditional lecture with the use of PowerPoint assisted instruction.
Each collaborative concept mapping exercise was initiated by a definitive task as a question obtained from course content presented during the 3 weeks of lectures held prior to each mapping session. For the first concept mapping exercise a template was provided by the primary investigator for students to complete in their assigned groups. This template guided the students through the initial mapping exercise by providing necessary nodes, links and phrases for map completion using pen and paper. For the second concept mapping exercise, a template was also provided, with fewer cues in the forms of nodes, links and phrases. Concept mapping requires an understanding in fundamental processes of linking information; these templates provided an illustration to assist students through the initial mapping exercises. Students worked collaboratively to complete this map within their groups. Subsequent concept mapping exercises required students to create free-formed concept maps within their collaborative groups without any assistance or cues from the investigator. After each concept map assignment, several student groups were selected to share their maps with the entire class describing their experiences regarding the exercises and use of concept mapping on meaningful learning. Each concept mapping session lasted approximately 25 to 30 minutes, and was held every 3 weeks over the 16 week semester. A relaxed, discussion friendly environment was established during these sessions. During the sixteenth week of the study, a post-test was administered by the course directors to all study participants (n=59).
Results
The demographic differences between the programs were limited to age, ethnicity and experience. However, when analyses were conducted only ethnicity was found to be statistically significant (p<0.001, Table I). Dental Hygiene Program 1 (test group) students were overall older, less ethnically diverse and possessed more experience in the dental field prior to entering the program than Dental Hygiene Program 2 (control group). There was a 9% dropout rate (three participants) for Dental Hygiene Program 1 (test group). Two of the subjects left the program prior to the end of the semester due to personal reasons, and 1 subject chose not to participate in the post-test. Pre-test scores of the 3 students who dropped out of the study ranged from 11 to 21. Dental Hygiene Program 2 (control group) did not experience any subjects dropping from the study, thus analyses were performed using pre- and post-test scores from those students who remained in the study throughout the entire duration of the semester (56 of 59, 95% participation rate).
There was a significant difference between the 2 programs on the pre-test (p<0.05). Dental Hygiene Program 1 (test group) demonstrated an average pre-test score of 19.33 with a 5.5 standard deviation compared to Dental Hygiene Program 2 (control group) which scored an average of 14.15 and a standard deviation of 3.0. The degree of meaningful learning achieved by those in Dental Hygiene Program 1 (test) and Dental Hygiene Program 2 (control) was assessed by comparing pre- and post-test results. Both programs experienced a significant degree of meaningful learning from pre- to post-test. However, there was no statistically significant difference between the programs on the post-test (Table III).
Final test results were analyzed for internal consistency reliability estimates within each of the subject categories using Cronbach's Alpha. Reliability estimates were as follows:
0.54 for Domain 1: Foundational Knowledge
0.65 for Domain 2: Application and Procedural Knowledge
0.39 for Domain 3: Assessment Knowledge
-0.01 for Domain 4: Panoramic/Extraoral Imaging, Digital Radiology, and Supplemental Procedures
As a result of the erratic reliability estimates on the pre- and post-test, and the considerably larger variability in the scores of Dental Hygiene Program 1's pre- and post-test scores, a small focus group consisting of 6 students from Dental Hygiene Program 1 (test group) were gathered in an attempt to gain additional insight into the study results. A series of questions were developed - student feedback to the questions can be found in Table IV. Of those, 5 reported being interested in the study; however, only 4 expressed an interest in taking the post-test. Five of the students felt the post-test was much more difficult than the final examination developed and administered by their course director. All 6 students felt the concept maps assignments should have counted toward the final course grade. All students expressed feelings of being overwhelmed, tired and rushed the day the post-test was administered. Five of the students felt strongly that concept mapping was a useful tool for meaningful learning, and 2 felt they would use it for future applications.
Discussion
Research has shown that constructing one's own knowledge by engaging in active learning (constructivism) and the positive role of social interaction through group work (social constructivism) results in deeper learning.15 Yet in this pilot study designed to empirically examine these relationships, the results did not support the literature. This study utilized concept mapping as an instructional strategy. Every 3 weeks within the test group throughout the 16 week semester, students participated in concept mapping exercises. A total of 4 mapping exercise sessions took place. Logic and previous research would suggest that students in the test group would have gained greater levels of meaningful learning than the control group as a result of implementation of the instructional strategy of concept mapping and group work.
Explanation of the pre-test difference between programs is problematic since academic and/or demographic factors (i.e. GPA, experience, education, age) that might have offered insight for the difference between programs actually did not discern between schools. Using the only academic/demographic factor (experience) that correlated substantially with pre-test scores for each program (0.77 for Program 1 and 0.35 for Program 2) in an exploratory covariance analysis resulted in essentially identical outcomes as compared to results without the use of this covariate. Even more confounding is the finding of no significant difference between the programs on the post-test, and the fact that Dental Hygiene Program 2 (control group) scored approximately a point higher on the post-test than Dental Hygiene Program 1 (test group). A difference in teaching style, years of experience and/or subject knowledge are a few inconsistencies that may exist among course directors, and would provide additional insight as to the varying results on the post-test. It is the recommendation of the authors that in future studies the course directors be assessed for similarities or differences in an effort to establish differences in subject delivery.
Pre- and Post-Test Mean Scores and Standard Deviations as a Function of Instructional Strategy
As previously reported, there is a large body of literature to suggest the positive impact of concept mapping and group work on meaningful learning.4,5 This prompted the researchers to gather additional data through the use of a focus group. What was learned from the focus group provides some insight into the outcomes of this study. The actual concept map development and subsequent class discussion served as a review of the course content and therefore was perceived as worthwhile. This did not prevail in the post-test results. Stress, lack of time or interest, and post-test difficulty were identified by the focus group as variables that may have played a role in their performance on the post-test. As a result of the focus group discussion, the investigators requested and were given a copy of the final examination by the course directors at each of the participating schools. In comparing the post-test to the final examination, the investigators noted that the post-test developed for this study was indeed more difficult than the instructor developed final exams. The design of the study was to capture meaningful learning, thereby requiring that post-test questions to be written at a level of application and above as determined by Bloom's taxonomy.20 The instructor developed final exams were written predominantly at the knowledge and comprehension levels of Bloom's taxonomy. These levels (knowledge and comprehension) provide the foundation for meaningful learning but are achieved through memorization and rote learning. This is an important finding given the ADEA-CCI identification of deficiencies in curriculum development when it comes to improving critical thinking and problem solving.1 To fully visualize the higher rankings of meaningful learning as defined by Bloom's Taxonomy within concept mapping, it is the recommendation of the authors that the use of concept mapping be employed more aggressively throughout the entire course. More mapping exercises would allow for additional mapping experience, which may provide a better illustration of the meaningful learning that has transpired through the linking of interrelated concepts by the development of more complex maps. This could potentially affect post-test results.
Dental Hygiene Program 1 (test group) Student Focus Group Responses (n=6)
Results of Dental Hygiene Program 1 (Test Group) Final Course Examination Scores Over a 3 Year Period
Perhaps even more impactful was the fact that both course directors chose to administer the post-test immediately following the course's 2 hour final examination. The researchers gained valuable insight through the focus group discussion where students acknowledged that time spent on the post-test ranged from 5 to 20 minutes. This provided greater understanding into the large degree of variability in the data for the post-test (36.77, SD 6.3) where students would have answered 50 questions in a time period of 20 minutes or less. Clearly there had to be a degree of students “guessing” to answer the questions in order to complete the post-test in the time frame indicated in the focus group. Based on what was determined in the focus group and the data results for Dental Hygiene Program 1 (test group), this leaves the investigators somewhat suspect of data from Dental Hygiene Program 2 (control group) where the degree of variability in both the pre- and post-test scores was half that of Dental Hygiene Program 1 (test group).
The majority of students within the focus group agreed that the use of concept mapping was a beneficial learning tool, which aided in the delivery of complex material for processing and retaining and ultimately meaningful learning. Of the students from Dental Hygiene Program 1 (test group) asked to take part in a small focus group, 2 stated how concept mapping was helpful to them in their other dental hygiene courses and felt they would use the concept mapping in the future to help them study for their National Board Dental Hygiene Exam (NBDHE). The course director for Dental Hygiene Program 1 (test group), which received concept map training, verified that overall grades on the final examination for the 2010 fall course were higher than the previous 2 years (Table V). Failures on the course final exam was reduced by 14% from the previous 2 years, accompanied by overall improvements in final exam scores by 17%. This is an important finding and suggests that concept mapping could have contributed to this outcome. The authors recommend the application of concept mapping in other dental hygiene courses to visualize the transfer of knowledge from one subject to another.
The use of experts for establishing the validity of the pre- and post-test, and an extensive analysis of class structure to ensure comparability across the 2 programs, ultimately did not overcome the lack of attention to a few design details that ultimately led to confounding outcomes. Because this was designed as a pilot study the authors are able to offer several recommendations for improving the research design for future studies:
Students need to be incentivized to participate in research given the high demands placed on them with their normal coursework. Lack of an adequate incentive resulted in student's giving limited attention to the study and to the concept mapping exercises. It is the recommendation of the authors that the concept mapping exercise carry some weight on the students' grade, either through extra credit or as an assignment towards their final course grade.
One methodological approach might have been to administer the post-test one week prior to the final exam under the guise of a review to avoid competing with the students impending final examination schedule.
The research design called for the course directors to administer the post-test during the last week of the semester. Unfortunately in both instances (Dental Hygiene Program 1 and 2) the course directors chose to administer the post-test following the course final examination. This resulted in student's giving limited time and attention to the post-test, as determined from the Dental Hygiene Program 1 (test group) focus group discussion, resulting in data that was unreliable. In retrospect, it would be the recommendation of the authors that the principle investigator or a co-investigator should have personally administered the pre- and post-tests as a measure of control. As stated above, the post-test should have been administered the week before the final examination. Minimizing contact between the course directors and the primary investigator was a part of this research design as a method for better ensuring the integrity (absence of investigator bias) of the “treatment” in each classroom. Particularly in the instance of Dental Hygiene Program 2 (control group), it was the intent of the investigators that this research be as non-invasive as possible. Students who were interviewed from Dental Hygiene Program 1 (test group) felt the course director needed a more active role in the study to support the use of concept mapping on a weekly basis. It is the recommendation of the authors that in future studies the course director be more actively engaged in the introduction to and application of concept mapping throughout the entire study. This may increase the test validity and reinforce concept mapping as an instructional method and teaching strategy.
Conclusion
Educational research is problematic in that the researcher is challenged with trying to minimize disruption to the delivery of the course for both faculty and students. However, this study illustrates the importance of balancing intrusiveness with detail. In this instance the primary investigator should have been the administrator of the pre- and post-test, included a means for reward or incentives for student participation and/or improved interest in the study, and involved the course instructor in the test group to reinforce concept mapping on a more consistent basis. Faculty development is necessary if the teaching and learning environment in dental education is to move beyond passive teaching strategies such as lecture and incorporate strategies that have been shown to promote critical thinking and problem solving.
Future studies are needed to empirically examine the educational strategy of concept mapping in collaborative working groups to determine what effect it has on critical thinking and problem solving. Examination of students' improvement in concept mapping over time would be interesting to explore. Findings from this study related to methodology should be helpful in the design and implementation of future research in this area.
Footnotes
-
Dina M. Canasi, MSDH, RDH, is an adjunct clinical and didactic instructor at Hillsborough Community College. Cynthia C. Gadbury-Amyot, MSDH, EdD, is the Associate Dean for Instructional Technology and Faculty Development, University of Missouri-Kansas City, School of Dentistry. Daniel Tira, PhD, is Professor Emeritus, University of Missouri-Kansas City.
-
This study supports the NDHRA priority area, Professional Education and Development: Validate and test measures that evaluate student critical thinking and decision-making skills.
- Copyright © 2014 The American Dental Hygienists’ Association
