Abstract
Purpose: The aim of this mixed-methods longitudinal study was to assess student perceptions of technology use, and to examine the relationship between technology use and performance as reflected by self-reported student grade point averages.
Methods: Students (n=351) enrolled in a dental hygiene program within a dental school located in the mid-western United States were surveyed in three courses from 2008 through 2012 to gather their perceptions regarding usage of a lecture recording system (LRS). Additionally, self-reported grade point averages were collected over the same period of time. Data were analyzed using a statistical software program (IBM SPSS; Armonk, NY).
Results: The response rate was 82%. Descriptive statistics demonstrated that students believed that the LRS increased their success and satisfaction in the course and would be useful in other courses. Students also reported they would not choose to miss class sessions based on the availability of the recorded lectures. Correlation statistics found no relationship between student GPA and students' perceptions regarding the LRS.
Conclusion: Students reported LRS use and availability did not impact their attendance. No relationship was found between students' self-reported GPA and evaluation of the LRS use within the limits of this study.
Introduction
Research on new teaching modalities supports the development and implementation of technology within the classroom setting as well as across all educational platforms. Higher education students have come to expect technology use within class formats; however due to the evolving nature of educational technologies, appropriateness of technology use is often overlooked.1,2 Implementing technologies primarily to satisfy student expectations is no longer adequate rational for use; selecting appropriate educational technologies is essential for student development and achieving learning goals.
The Internet, or worldwide web, has been a major contributor to educational technology; educational systems utilizing the Internet are frequently referred to as web-based technologies.3 Web-based deliveries vary in use ranging from courses offering exclusively online learning experiences to blended or hybrid delivery of online components, and face-to-face/on-campus experiences.2 Tegrity© (McGraw-Hill Education; New York, NY), a web-based lecture recording system (LRS), is capable of recording lectures and classroom activities, including camera-view events, audio, and media-based slides and quizzes, for later use. The platform integrates camera video, audio, and multimedia slides into a seamless viewing experience4 and may be used as an online only format or in combination with face-to-face class sessions. Students may choose to play, pause, fast forward, rewind, and increase or decrease the playback speed of these recordings.
LRS technologies may be offered for a variety of reasons including institutional policy, instructor choice, student expectations, support for absent or special needs students, support for non-English speakers, and as a supplemental learning and/or teaching method.2,5-6 While the integration of e-learning materials into the classroom experience may be expected by Millennial or Gen Next students, their acceptance and use frequently depends on students' perception of the specific technology options. Acceptance of a LRS is often dependent on personal experience with the format; multiple studies report that students view a LRS positively.5, 7-10
Previous studies have compared the effectiveness of web-based instruction to in-class instruction, however there are few studies reporting on blending digital education and traditional in-person classroom settings. Additionally, most studies have been short-term in nature, typically covering only one course or one class of students at a time. Little has been reported within the field of dental hygiene on blended educational technologies or on the relationship between LRS and student outcomes. The purpose of this study was to survey the perceptions of dental hygiene students over a five-year period regarding the use of a LRS with a focus on student satisfaction and content retention. Results of this study can serve to add to the body of knowledge regarding the use of LRS within dental hygiene education programs in addition to providing an aspect of program evaluation.
Methods
Study design
A descriptive, associational, and comparative study design was used to address the research questions. A descriptive model was used to summarize student demographic variables and the evaluation of the use of the LRS, Tegrity© (McGraw-Hill Education; New York, NY), within three separate dental hygiene courses utilizing both traditional and flipped classroom pedagogies. An associational approach was used to examine the relationships and possible predictors between the LRS use and self-reported student grade point averages (GPA) and a comparative approach was used to examine differences in the subgroups within analyzed data. Data collection methods and analysis were reviewed and determined exempt from the Social Sciences Institutional Review Board of the University of Missouri, Kansas City.
Study population
Program evaluation survey data, previously collected but not analyzed, was collected from a convenience sample of junior and senior students enrolled in the dental hygiene program at the University of Missouri, Kansas City, School of Dentistry. Each class consisted of approximately thirty dental hygiene students, totaling about sixty students per year. Inclusion criteria included all of the dental hygiene students enrolled in the program who were present for the final examination administered during the last on-campus session of selected courses dating from the summer of 2008 through 2012. Three courses, Oral Health, Dental Biomaterials, and Seminar in Dental Hygiene II, were assessed annually over the five-year period. Faculty, LRS use including availability, functions, and video and audio of the instructor during recordings, remained the same throughout the period of study. Two courses, Oral Health and Biomaterials, utilized flipped classroom pedagogies. Students prepared for class sessions by viewing pre-recorded lecture content independently, prior to the class session and were expected to put the newly acquired knowledge into practice during class through collaborative activities, case study evaluations, dental product reviews, and laboratory procedure preparations. The third course, Seminar in Dental Hygiene II, utilized a traditional faculty centered approach. Students attended class sessions in person and the lectures recorded during class were available for study and review following the sessions. Additionally, students in the seminar course were able to use the LRS to review pre-recorded course content to gain knowledge for the laboratory procedures planned for the following day including sealant application, intraoral camera utilization, and air powder polishing. It is possible that multiple exit surveys were collected from an individual student as they moved through the required courses during the two-year Bachelor of Science program. The surveys contained no individual identifiers therefore it is unknown how many surveys were completed per student; however, the surveys were numbered for quality assurance, and were linked to the responses. Student anonymity was taken into consideration in order to encourage honest, useful feedback. Students choosing to participate gave implied consent with their participation, completed the survey and placed it in an envelope placed at the back of the room. Students choosing not to participate were free to turn in their final examination and the blank survey prior to leaving the classroom. The surveys were sorted and stored by course, semester, and year.
Survey instrument
The exit survey was developed for use in a similar study; however, modifications were made to personalize it and include specific questions related to the issues of perceived retention of course material as a result of using the LRS and the perceived advantages of access to course materials when unable to attend class sessions. Faculty experts on survey development in the School of Dentistry provided input and revisions to the survey instrument, and further revisions were made following a pilot test of the modified survey.
The modified survey consisted of 26 multiple-choice and Likert-scale items with several opportunities for students to write comments about their use and perceptions of the LRS. Students were asked about number of opportunities of use, actual uses of, and reasons for the use of the LRS. If the student did not use LRS, written answers were requested asking for reasons and in what circumstances the student might find the LRS useful. Student perceived comparisons between courses in which the LRS was used and those in which the LRS was not used were requested, as well as direct comparison between the LRS and voice-narrated lecture slides only (no video) utilized in other courses taken by the student cohort. Additionally, questions regarding student perceptions of course content retention following individual LRS use and the video option within the LRS system were examined.
Data collection
The multiple-choice item answers included nominal, dichotomous, and ordinal; student perception items were considered ordinal. Medians and interquartile ranges were calculated as central tendency measures for findings ordinal in nature. The majority of answer options included broad categories, although several answers offered yes or no options only. Additional written comments were elicited in many cases.
Demographic variables included age range, race, gender, personal ownership of video-viewing device, and self-reported grade point average. Ownership of a personal video-viewing device, such as a smart phone, did not imply the device was utilized to view recorded lectures, merely that the student owned such a device. The remaining variables were collected to test the research questions regarding instructor use; student perceived advantages, disadvantages, and satisfaction level of LRS use; and, the relationship of LRS use to GPA.
Analysis
Data entered and analyzed in statistical software program (IBM SPSS; Armonk, NY) using descriptive statistics and correlation statistics to identify possible relationships between variables. Written comments were categorized and entered manually into a spreadsheet.
Results
Of the 429 students (n=429) estimated to be present during the period of study, a total of 351 surveys (n=351) were completed for the three courses over the five-year period for a response rate of 82%. The sample was predominately white, female, between the ages of twenty and twenty-two, with a self-reported GPA of 3.0 or higher, and owned some type of mobile recording device (Table I). This is comparable to the population demographic of eligible participants (i.e. dental hygiene students).
Over half of the students, 60%, reported instructors used the LRS in two or more courses in an online format. A majority, 81%, of the students reported using the LRS for review and study purposes. Students who used the LRS reported it aided in retention of course material (80%), increased their overall success (54%), and increased satisfaction with the course (53%). The majority of students believe the inclusion of the video of the instructor speaking in the LRS recordings was helpful (68%), and that they preferred the LRS recordings over voice narrated lecture slide sets (72%). Students reported rarely or never experiencing technical issues while using the LRS (67%). Seventy-five percent of students indicated the availability of the LRS recordings would not increase their likelihood to miss class and had no impact on their decision to attend class sessions (Table II). However, it is important to note that professional programs, such as dental hygiene, often have attendance policies, and influence of such policies may affect student responses to attendance-related questions.
Sample Demographics
In addition to describing the sample characteristics and item specifics, some variables were used to create sub-scales that were compared to student responses to the LRS use and evaluation items. Factor analysis (principal axis factor solution; varimax rotation; KMO = 0.80) was used to identify subgroups of items that represented underlying constructs. Two subscale scores emerged from the dataset. The first subscale score (eigenvalue = 3.31), representing student evaluative perceptions, was derived from four survey items seeking feedback on student perceptions of overall satisfaction, impact on success, usefulness in other courses, and overall satisfaction in the course. The Cronbach's alpha score for the evaluative subscale was 0.82. The second factor (eigenvalue = 1.20), representing student frequency of use, was derived from two survey items seeking student feedback on the frequency of use for review, and the frequency of use for study purposes. The Cronbach's alpha score for the frequency subscale was 0.78.
Subscale scores were calculated for survey items measuring student evaluation of the LRS and compared to self-reported student GPA. In the initial set of surveys, students were asked to denote their GPA based on the following ranges: below 2.4, 2.5 – 2.9, 3.0 – 3.4, 3.5 – 4.0. All GPA data was calculated in range values with no significant relationship found between the LRS evaluation items and GPA scores 2.5 and above. However, the statistical analysis suggested significant correlation between the LRS evaluation items and students with a GPA range below 2.4. For this category, Spearman's rho was .191 (p value .001) and may be misrepresentative due to small sample size (n=2) in this category. Following the first year of data collection, students were asked to indicate their exact GPA in writing on the survey. No significant relationships were found between the LRS evaluation score and students' hand-written GPA.
Thematic analyses of the written comments on all surveys was conducted and three major categories of themes were identified (Table III). Theme one represented students' reports on how they used the LRS, including using it to supplement the regular class sessions, review of class material, in lieu of attending class sessions in-person, and for emergency situations or due to illness. Theme two, effects on attendance, included the subcategories of student preference and program attendance requirements. Evaluation of the LRS use was the third theme and included positive and negative comments regarding use, and the impact of technical issues.
With regards to how instructors and students utilized the LRS, students overwhelmingly reported use was blended, with the majority of lectures available online. Students wrote, “All lectures were available online,” “Most lectures were prerecorded,” and “Teachers posted every lecture.” Additionally, students noted the LRS use was dependent on the specific instructor and was used predominantly “to review” following class or prior to an assessment. Students overwhelming commented that they would not be tempted to miss class sessions due to the LRS availability, often citing the program's attendance policy as a reason. Student comments included, “Can't miss or will fail the class,” “I don't like being counted absent,” “We can't miss class in the hygiene program,” and “I always go to class.”
Students perceived the LRS as positive overall, with many commenting on the usefulness of availability should a class be missed, increased repetition and retention of course materials, and increased attention and focus when viewing due to video of instructor. Survey comments included, “If I was ill and was unable to get out of bed I would consider making it up by reviewing the LRS recording,” “Only if I had to, I prefer in-class lectures, but knowing I could review exactly what everyone else heard would be helpful if I really needed to miss class,” “I would not intentionally miss but if I did have to, it is nice that it is there,” “With reading issues it helped me retain material faster and easier,” “It was helpful to hear the material repeated,” and the video of the instructor was “… more interactive” and “Helps me focus on what the instructor is saying.” Some students reported dislike of the particular LRS used due to technical issues, preferring alternate review resources, along with the amount of time needed to review recorded lectures outside of class time. Students wrote, “It freezes up occasionally mid-lecture,” Files that were very large could not be downloaded at home,” Certain browsers won't let me use it,” Takes a while to load,” “I enjoy voice narrated slide lectures more,” and “I don't like having both a lecture online and, in the classroom…I don't have time for both.”
LRS Use
Discussion
In courses where face-to-face attendance was mandatory or highly encouraged some students found no need to access the LRS, instead depending on in-class presentations and other course materials for learning.8, 11 Examples are found in students' comments regarding why they did not use the LRS such as, “I attended all class sessions”, “Got information from other sources”, and “I rarely used it because I never missed class.” The controversy surrounding attendance and web-based technologies continues when student perceptions are examined. Previously surveyed students have expressed that the availability of recorded lectures negatively affects their class attendance, tempting them to miss class.12-13 However, the findings of this study aligned with multiple other studies reporting that the availability of a LRS had no impact on students' decisions to attend class.8-9, 14-18
Within the literature reviewed for this study, the most common LRS barriers students report are technical issues, unfamiliarity of a LRS, and a lack of awareness of LRS benefits to the learning process.9 However, the students in this study reported rarely or never having technical issues with the particular LRS used. It is important to address technical issues encountered as this barrier may deter students from utilizing a LRS.11 Time limitations were also noted as a deterrent to LRS access by the students in this and previous studies.13,19
Regarding web-based educational technologies in general, student outcomes including final course grades, GPAs, and examination scores, are believed to be enhanced through the use of technology.20 Though previous studies on student achievements and lecture recordings are generally positive, they vary significantly in methodologies and field of study. Findings from this study did not identify a significant association between the students' evaluation of the LRS and a higher GPAs; however, a relationship was found between a higher evaluation of the LRS with students reporting GPA's of 2.4 or lower. This suggests students who are struggling overall have a more positive rating of the LRS. However, as previously discussed, the small sample size for this category decreases the validity of this finding. It is possible that struggling students relied more heavily on lecture recordings in an effort to improve their grades, which could be considered positive as students are often encouraged to utilize all available resources. Possible relationships between student GPAs and LRS warrants further study.
Themes of Written Survey Comments Regarding LRS
Options provided by the type of LRS may also alter study results. Statistically significant higher exam scores are found with the integrated recording systems currently used versus the older version LRS featuring a separate audio, video, and additional media files.21 Other types of online learning and review methods, such as online quizzes, have been shown to improve final course grades for those students who access the resource multiple times versus students who access them only once or not at all.22-23 Future studies should address the various individual online options for reinforcing and supplementing course content.
The majority of studies focusing on faculty and student perceptions are based on survey data. Survey methodology has innate flaws including participant recall/memory issues, traditionally low response rates, pressure of producing a desirable response, lack of focus or true desire to complete survey truthfully, and questionnaire item validity issues. Despite these shortcomings, questionnaires easily gather extensive data inexpensively as compared to alternate methods. Based on the known number of students in each course, the response rate in this study was optimal. Attempts were made to minimize other issues, including that the faculty members were not present during survey completion, no identifiers were included within the survey, and individual survey items were reviewed by unbiased field experts.
Because so many levels of technology integration exist, authors of previous studies suggest further research be completed to create a broader understanding of utilization of technology.24-25 The majority of previous studies have examined a small sample over a limited time frame creating future research opportunities that include larger and more diverse samples. 7,11,14,21,24-25 Furthermore, few studies report on associations between the use of a LRS and GPAs or other assessment outcomes. Future studies investigating possible relationships between LRS and examination scores, including national board scores, could be of value as well as examining the actual recorded cumulative GPAs of students upon graduation. While it was the intent to analyze associations between LRS use and National Dental Hygiene Board Examination (NBDHE) scores in this study, appropriate data was not available.
Conclusion
Students responded positively to the use of the LRS in the three courses surveyed with the majority believing that the LRS aided in retention of course material, and increased their success and course satisfaction levels. Students reported that LRS use and availability did not impact their attendance in the course; and technical issues rarely occurred during use. Results show faculty utilized LRS in a blended format in multiple courses. No relationship was found between student GPA and students' evaluation regarding the use of a LRS. This longitudinal study, supports previous similar research, adding to the body of evidence for informed decision making regarding the selection and implementation of web-based strategies in dental hygiene education and other related fields of study.
Footnotes
Carly J. Havner, RDH, MS is a registered dental hygienist and a graduate of the Master's degree program; Mary M. Gerkovich, PhD is an associate professor, Department of Biomedical & Health Informatics; Kimberly K. Bray, RDH, MS is a professor, Division of Dental Hygiene; Marsha A. Voelker, CDA, RDH, MS is an associate professor, Division of Dental Hygiene; all from the University of Missouri Kansas City School of Dentistry, Kansas City, MO.
This manuscript supports the NDHRA priority area: Professional development: Education (educational models).
- Received January 2, 2018.
- Accepted June 16, 2018.
- Copyright © 2018 The American Dental Hygienists’ Association
