Abstract
Confusion exists among some researchers and educators regarding the classification of research designs. Some classify quantitative research studies as mixed methods research when using and analyzing open-ended items on a survey. Therefore, the purpose of this short report is to describe the mixed methods research design and distinguish it from other research methods. Types of mixed methods designs and key questions to consider for this approach are discussed.
INTRODUCTION
Consider the following scenario. A researcher conducts a descriptive study surveying dental hygiene educators about their plans to retire from the workforce. The survey is well-designed with demographic questions, close-ended items and open-ended items. The survey is distributed among all full-time dental hygiene educators in the country and a forty percent response rate is obtained. Results are tabulated using frequencies and percentages for the close-ended items. Open-ended items are grouped, coded and categorized, and themes are determined. A manuscript is prepared for publication in a journal. The paper is described as a mixed methods study since there was a quantitative portion from the close-ended items on the survey and a qualitative portion represented by the open-ended questions on the survey. The paper undergoes peer review and is rejected. Comments returned to the author indicate that the study design is incorrect because the submission does not represent a mixed methods study. The author is confounded. Qualitative comments were represented in the open-ended portion of the survey. How could this research not be a mixed methods study?
Mixed Methods
Mixed methods research involves the integration of both quantitative and qualitative data within the same study.1,2 A multi-strategy approach is purposefully used for data collection, analysis and interpretation of evidence.1,3,4 A mixed methods design allows researchers to view their study from a broader perspective and integrate multiple data sources to study complex problems. This design offers an opportunity to answer research questions by combining the strengths of two types of research methods while compensating for the weaknesses of each method.5 For example, the quantitative research design brings breadth and generalizability to a study whereas the qualitative research method provides increased depth and greater understanding of the issues being investigated. In addition, qualitative data can be used to validate quantitative findings, and the quantitative data can be used to develop the qualitative sample.6
There are different types of mixed methods research designs. Merriam and Tisdell describe three primary types of mixed methods research: a convergent design, an explanatory sequential design, and an exploratory sequential design.7 Both quantitative and qualitative data are collected simultaneously or nearly simultaneously in the convergent design. In this type of design, a group of individuals can be surveyed on a given topic while another group can be interviewed on a related topic. Both sets of data are then analyzed and results are compared.7 Equal weight is given to both elements of the study and results can be presented in an interwoven manner.4
The explanatory sequential type of mixed method research occurs when the quantitative data are collected and analyzed first. Qualitative data is conducted following the quantitative portion of the research and is used to help explain the quantitative results.1,7 For example, a national study might be conducted to investigate infection control practices and vaccination status during the COVID-19 pandemic in federally qualified health centers. This study might then be followed with personal interviews of 15 to 20 respondents to learn more about their beliefs and experiences using protocols and prevention strategies.
In the exploratory sequential design, the qualitative data are collected and analyzed first. A survey is designed based on the analysis of the qualitative data. The qualitative data is used to define the topic typically when there is little known about the subject or the group being studied, and to further explore, develop and test the analysis of the qualitative data.4,7
In some cases, a study may have either a quantitative or qualitative design as the main focus of the study with an alternative component nested or embedded within it to improve the quality of the study or to answer a complementary question. For example, a project may begin as a qualitative study with observations and interviews being conducted. In the middle of the study, it may be important to conduct a survey with a larger group of participants to identify additional information. The study has evolved to a mixed methods study with the qualitative data remaining as primary and the quantitative portion being nested within the design.1,4,7,8
It should be noted that additional mixed methods research designs exist; however, the purpose of this paper is not to be exhaustive in describing each type of design. Palinkas et al describe additional mixed methods designs for evaluation research while Schoonenboom and Johnson offer multiple depictions of mixed methods design typology.9,10 These designs may be useful when considering innovative, extensive, or complex research studies.
When considering a mixed methods research approach, it is important to examine key questions to help justify the need for more than one method to address the study purpose. The following questions may serve as a useful tool.1,4,11
Does the research purpose and question(s) justify the use of a mixed methods design?
Is the method sequence clearly described and well aligned with the study aims?
Is data collection and analysis clearly described and aligned with the study aims? Is the data collection sequential, parallel, or convergent?
Does one method dominate the other or are they equally important? In some studies, the methods are equally weighted. In other designs, they are nested or embedded. Be explicit regarding the methodology.
Did the use of one method limit or confound the other method?
When, how and by whom is the data integration (mixing) achieved? Be clear about the extent of integration and how this is achieved in the methodology section.
Is the qualitative element explanatory or exploratory? The qualitative element may have a range of purposes including explaining previous findings or exploring a phenomenon.
To ensure rigorous research is being conducted, both the qualitative and the quantitative methods must be sound and of high quality. The Mixed Methods Appraisal Tool is a helpful guide for mixed methods studies.12
DISCUSSION
In general terms, quantitative research involves the collection and analysis of numerical data; frequency of information is the focus. Qualitative research attempts to understand how people interpret and experience their world; process, understanding and meaning are the focus. “The researcher is the primary instrument of data collection and analysis; the process is inductive; and the product is richly descriptive.”7 Mixed methods research integrates both quantitative and qualitative elements to describe a fuller understanding of the research problem.
Mixed methods research requires additional considerations for implementation. Practical considerations are resources, costs, time and management of data.13 Given that two types of data collection are being used, additional resources in personnel, content experts, and funding may be required to manage a successful project. Time can be a significant issue for this type of study as two data sets may need to be managed either simultaneously or sequentially. Further, more complex data sets require additional time and specialized skill sets for data management and analysis. Lastly, the researchers must demonstrate rigor throughout the study using clear audit trails to enhance validity.
In considering the scenario presented at the beginning of this paper, the researcher was disappointed that their manuscript was not accepted for publication having assumed that the use of open-ended items on a survey represented a mixed methods study design. In this case, there was no qualitative component to the study. No observations or interviews were conducted. The open-ended items should have been summarized and presented as frequencies and percentages since that information also represents numerical data.
CONCLUSION
Mixed methods research is a complex study design that offers opportunities to gain perspectives on phenomenon not readily achieved through the use of single research design approaches. Combining qualitative and quantitative research has the potential to lead to unanticipated yet rich outcomes. However, the use of open-ended items on survey instruments does not represent a mixed methods study design.
Footnotes
NDHRA priority area, Professional development: Education (evaluation).
- Received June 3, 2023.
- Received July 18, 2023.
- Copyright © 2023 The American Dental Hygienists’ Association
