Abstract
In the dental hygiene discipline, evidence-based practice serves as a cornerstone for delivering high quality patient care and moving professional standards forward. As practitioners delve deeper into research to inform clinical decision making, the integration of robust survey methodologies becomes imperative. However, the complexities of survey design, implementation, and analysis pose notable challenges, particularly in ensuring the reliability and validity of research outcomes. This short report provides brief practical guidance about the basics of survey research methodologies for dental hygiene professionals.
INTRODUCTION
Survey research is a strategy to gather information from a sample by asking a series of questions or items that may be used to make inferences more broadly in fields from social sciences to health care.1,2 Surveys have been widely used to gather information about many aspects of healthcare including practices of health care providers as well as attitudes and behaviors of patient populations.3,4 However, the intricacies of survey design, reliability and validity of survey instruments, implementation, analysis, and reporting of findings present notable hurdles to ensuring the quality of research outcomes.5,6 Surveys of lower quality may be due to poor reporting of some study criteria including validity and reliability of the survey instrument, lack of inclusion of the survey questions and issues with defining the response rate.7 By offering key insights and actionable strategies, this report aims to empower dental hygiene practitioners and researchers with the knowledge and tools needed to enhance survey research design.
SURVEY RESEARCH
Careful planning and reporting of study design, sample selection, data collection, and analysis are essential elements of all research, but in survey research there is the additional element of identifying a valid instrument or the creation and validation of an original survey.8 A crucial starting point in quality dental hygiene research is establishing a clear purpose and research questions or aims. While formulating research questions may appear simple in theory, it often proves challenging in practice to carefully identify specific variables that translate broad inquiries into specific outcomes.9 For instance, in a study investigating the effectiveness of oral hygiene education programs, researchers may pose broad questions like “How does the patient readiness to change impact outcomes of an oral hygiene education program?” However, breaking down this overarching question into specific components, such as the measuring the patient’s readiness to change, influence of delivery of educational content, impact of oral health literacy, and the outcomes being measured require careful consideration for survey design.
Survey Types
There are several types of surveys which consider the study’s time frame that include cross-sectional, longitudinal, and retrospective surveys. Retrospective surveys involve collecting data from individuals about their past experiences or behaviors. In dental hygiene, retrospective surveys offer dental hygiene researchers a valuable methodological approach to gather insights into past experiences and behaviors in both educational settings and clinical practice. By analyzing data from past cohorts or patients, researchers can identify trends, assess intervention effectiveness, and inform future research and practice improvements.10
Longitudinal surveys involve collecting data from the same sample of individuals over a period. These types of surveys are useful for assessing changes in behavior or attitudes over time. In dental hygiene, longitudinal surveys can be used to assess the effectiveness of oral health interventions or to identify risk factors for oral health issues over time.8,10
Finally, cross-sectional surveys involve collecting data from a sample of individuals at a single point in time. These types of surveys are useful for assessing the prevalence of certain behaviors or attitudes among a population. Questionnaire based data collection using a cross-sectional design has been the most common method used in dental hygiene survey research.8,10
Survey research has several advantages, including the ability to gather information from many individuals, including those who are difficult to reach; the ability to collect data on sensitive topics while maintaining anonymity or confidentiality; and the ability to provide accurate and reliable results if properly designed. However, there are also several limitations to survey research, including the potential for bias or error in the data collection process, difficulty in accurately measuring certain attitudes or behaviors based solely on self-report of the participant, and the potential for low response rates.11 Ensuring attention to the psychometrics or validity and reliability of an instrument is one way to mitigate some of these limitations.
Study Design
Establishing clear a research question(s) and/or aims is akin to charting the course for a journey. Well-defined research questions enable researchers to identify appropriate survey instruments and methodologies to enhance the quality of data collected and facilitate data analysis and interpretation of findings for more robust and insightful research outcomes.12
When planning the methodology to answer a research question, the first consideration is whether survey research is the best approach to answer the research question(s).2,13 What are the advantages and disadvantages of this approach for the research question(s)? What is the feasibility of accessing the population of interest?
If it has been determined that a survey approach is most appropriate, the survey must be constructed. This may consist of demographic questions along with one or more validated instruments.1 A conceptual framework or theory should underpin the research question(s) and be used as the basis for choosing survey instruments. For instance, an investigator may propose a study to examine the effect of anxiety on dental hygiene student success. There are validated measures of anxiety like the GAD (Generalized Anxiety Disorder), but caution is in order because this is intended to be a screening tool for anxiety disorder14 used in a clinical setting and may not be appropriate, so the researcher needs to be sure this is the variable they want to assess. If the GAD is used, are there mental health resources for those who might be experiencing GAD? This may be a question posed by the institutional review board tasked with protecting human subjects. Other options for variables might be perceived stress, psychological distress, worry, etc. Next the investigator needs to identify how they will measure student success. Is there a validated instrument to measure academic success? If there Is not an instrument, how does the investigator design and validate an instrument? Exploration of access to academic data such as grade or board outcomes might also be part of assessing academic success. The researcher should also consider other factors that need to be included in the demographic portion of a survey that might impact academic success?
There are several methods for collecting data in a survey approach including the use of questionnaires and interviews.1 Typically, respondents are asked to answer a series of questions or respond to items. For data collection the investigator needs to determine the method for distributing and gathering the surveys. Surveys may be self-reported using electronic or paper formats. However, surveys may also be administered by the researcher over the phone or in-person interviews.15
These are just a few of the design decisions to be considered for survey research to ensure the measure of outcomes are specific to the research questions(s) or aim(s).
In addition, to optimize the effectiveness and rigor of survey-based dental hygiene research, it is advisable to engage the expertise of statistical professionals early in the research process. Statistical experts can provide invaluable guidance and insights during the development of research questions and aims, ensuring that they are formulated in a manner conducive to data analysis. By involving statistical experts from the outset, researchers can align research questions/aims, survey design, and data analysis strategies, thereby maximizing the validity, reliability, and interpretability of study findings.16 This proactive approach fosters methodological rigor and enhances the overall quality of survey-based research in the discipline.
Survey Development
Once the research question(s) or aim(s) are clearly defined, the next phase is identifying a previously validated instrument(s) or development of an instrument to measure the outcome(s). Ideally instruments with previous validation in similar samples can be identified with a thorough review of the literature.2 As previously mentioned, these are often based on a theoretical or conceptual framework and have undergone validation and reliability or psychometric evaluation. Psychometrics are used to statistically analyze validity and reliability of a newly developed instrument to measure things like personality, attitudes, or behaviors.2,17 Validity and reliability will be discussed further in the next section.
When a validated instrument is not identified, the researcher may need to design an original survey. As previously noted, identifying a theory or conceptual framework for development of the items strengthens the validity. Once the theory or conceptual framework is identified, it is time to craft the questions or items. Write clear and straightforward questions using short, simple words appropriate for the audience.13 For instance, questions developed for dental hygienists might be written differently than a general audience with low oral health literacy. Carefully evaluate the order of the questions and avoid language that introduces bias and suggests a socially desirable response.13 If the theory or conceptual framework suggests multiple sub-scales, group these questions or items together and ensure questions cover the dimensions of the concept. For instance, if the instrument being developed was based on the Health Belief Model (HBM), items are needed for each of the six constructs (perceived susceptibility, perceived severity, perceived benefit, perceived barriers cue to action, and self-efficacy) of the model.18 Typically, many items will be generated initially and then reviewed to remove redundant items or those not pertinent to the topic to limit to the key questions or items. There is no ideal length for a survey, but the literature suggests 10 to 15 minutes for completion is best to improve completion or response rates.12,19 Next determine the response options to be used, i.e., dichotomous scales, multiple choice, and Likert scale. Response options should be mutually exclusive, meaning there is no overlap in the response choices.13
Ideally the research team should include a member with expertise in survey design to guide development and aid in fine tuning the wording for each question or item. A rigorous, systematic approach to survey design is essential to support measurement of outcomes relevant to the research question. Statistical analysis of the survey also must be considered throughout development to ensure outcomes can be measured in a way conducive to the planned analysis.
Validity and Reliability
In survey-based research, meticulous attention to the psychometric properties of questionnaires or instruments improves the methodological validity and replicability of study findings. However, common pitfalls such as selectively choosing items from validated subscales, modifying wording of items, or using a single subscale to represent a broader latent construct can significantly degrade the fidelity of the survey instrument. When researchers selectively pick items from validated subscales or alter the wording of items to fit their specific study context, they risk compromising the reliability and validity of the instrument.20 For example, consider a study assessing oral hygiene behaviors using a validated questionnaire with multiple subscales. If researchers only select items from one subscale or modify the wording of items, they may inadvertently introduce measurement error and bias into their study results. Measurement error refers to the discrepancy between the measured value of a variable and its true value and measurement bias refers to systematic errors or distortions introduced into research measurements or observations, leading to inaccuracies in the data collected.21 Unlike random errors, which occur by chance, measurement bias consistently skews the results in a particular direction.21 Bias can arise from various sources, such as poor study design, inadequate psychometrics for an instrument, respondent characteristics, or researcher subjectivity. It is crucial to recognize that degrading the psychometric properties of measurement instruments means that researchers are no longer measuring what they intend to measure, and this increases the risk of drawing erroneous conclusions based on flawed data.20
For an investigator designed survey, it is important to do a preliminary content validity with a panel of experts.22 Polit et al. describe the process for determining the relevance of each item with the research question(s) or aim(s) and a CVI (content validity index) can be calculated for the entire survey as well as for each item.22 The CVI aids in identifying items for removal. Following the CVI, piloting of the instrument may help in getting feedback from a group of individuals who meet inclusion criteria to ensure clarity and comprehension of items and is called face validity.12 Once data is collected the correlations between items and reliability of the instrument should be analyzed to generate a Cronbach alpha. The suggested acceptable level for the Cronbach alpha is .70 or greater.23 This is a rudimentary assessment of validity and reliability and more complex analysis such as factor analysis, confirmatory analysis (CFA), structural equation modeling and/or Rasch modelling would be necessary for rigorous psychometric evaluation of an instrument.12
Finally using reporting guidelines is another component of demonstrating the quality of survey research. There are several checklists available such as Consensus-Based Checklist for Reporting of Survey Studies (CROSS), Checklist for Reporting Results of Internet E-Surveys (CHERRIES), and SURGE (The SUrvey Reporting GuidelinE).24–26 Many guidelines for reporting research to improve quality and transparency can be found on Equator Network (equator-network.org) and increasingly journals are requiring these be completed and submitted with manuscripts seeking peer-reviewed publication.
CONCLUSION
Integrating best practice survey methodologies in dental hygiene research is essential to ensure the validity and reliability of study findings. Table I provides sources for best practices for survey research and Table II provides additional survey research resources. The key takeaway is that survey research is not simple; it is a complex process that is an art and science requiring careful planning of the research question(s) or aim(s), selection of psychometrically sound instruments, and working with statistical professionals early in the process to enhance the overall quality of survey research in oral health.
Best Practices for Survey Research
Survey Research Resources
DISCLOSURES
The authors have no conflicts of interest to disclose.
Footnotes
NDHRA priority area, Professional development: Education (evaluation).
- Received April 4, 2024.
- Accepted July 10, 2024.
- Copyright © 2024 The American Dental Hygienists’ Association







