Chapter 10: Mixed Methods Procedures
CHAPTER 10 MIXED METHODS PROCEDURES
How would you write a mixed methods procedure section for your proposal or study? Up until this point, we have considered collected quantitative data and qualitative data. We have not discussed “mixing” or combining the two forms of data in a study. We can start with the assumption that both forms of data provide different types of information (open-ended data in the case of qualitative and closed-ended data in the case of quantitative). If we further assume that each type of data collection has both limitations and strengths, we can consider how the strengths can be combined to develop a stronger understanding of the research problem or questions (and, as well, overcome the limitations of each). In a sense, more insight into a problem is to be gained from mixing or integration of the quantitative and qualitative data. This “mixing” or integrating of data, it can be argued, provides a stronger understanding of the problem or question than either by itself. Mixed methods research, therefore, is simply “mining” the databases more by integrating them. This idea is at the core of a new methodology called “mixed methods research.”
Conveying the nature of mixed methods research and its essential characteristics needs to begin a good mixed methods procedure. Start with the assumption that mixed methods is a methodology in research and that the readers need to be educated as to the basic intent and definition of the design, the reasons for choosing the procedure, and the value it will lend to a study. Then, decide on a mixed methods design to use. There are several from which to choose; consider the different possibilities and decide which one is best for your proposed study. With this choice in hand, discuss the data collection, the data analysis, and the data interpretation, discussion, and validation procedures within the context of the design. Finally, end with a discussion of potential ethical issues that need to be anticipated in the study, and suggest an outline for writing the final study. These are all standard methods procedures, and they are framed in this chapter as they apply to mixed methods research. Table 10.1 shows a checklist of the mixed methods procedures addressed in this chapter.
COMPONENTS OF MIXED METHODS PROCEDURES
Mixed methods research has evolved into a set of procedures that proposal developers and study designers can use in planning a mixed methods study. In 2003, the Handbook of Mixed Methods in the Social and Behavior Sciences (Tashakkori & Teddlie, 2003) was published (and later added to in a second edition, see Tashakkori & Teddlie, 2010), providing a comprehensive overview of this approach. Now several journals emphasize mixed methods research, such as the Journal of Mixed Methods Research, Quality and Quantity, Field Methods, and the International Journal of Multiple Research Approaches. Additional journals actively encourage this form of inquiry (e.g., International Journal of Social Research Methodology, Qualitative Health Research, Annals of Family Medicine). Numerous published research studies have incorporated mixed methods research in the social and human sciences in diverse fields such as occupational therapy (Lysack & Krefting, 1994), interpersonal communication (Boneva, Kraut, & Frohlich, 2001), AIDS prevention (Janz et al., 1996), dementia caregiving (Weitzman & Levkoff, 2000), occupational health (Ames, Duke, Moore, & Cunradi, 2009), mental health (Rogers, Day, Randall, & Bentall, 2003), and in middle school science (Houtz, 1995). New books arrive each year solely devoted to mixed methods research (Bryman, 2006; Creswell, 2015; Creswell & Plano Clark, 2018; Greene, 2007; Morse & Niehaus, 2009; Plano Clark & Creswell, 2008; Tashakkori & Teddlie, 1998, 2010; Teddlie & Tashakkori, 2009).
Table 10.1 A Checklist of Questions for Designing a Mixed Methods Procedure
Is a basic definition of mixed methods research provided?
Are the reasons (or justification) given for using both quantitative and qualitative data in your study?
Does the reader have a sense for the potential use of mixed methods research?
Are the criteria identified for choosing a mixed methods design?
Is the mixed methods design identified?
Is a visual model (a diagram) presented that illustrates the research strategy?
Are procedures of data collection and analysis mentioned as they relate to the chosen design?
Are the sampling strategies for both quantitative and qualitative data collection mentioned for the design?
Are specific data analysis procedures indicated for the design?
Are the procedures for validation mentioned for the design and for the quantitative and qualitative research?
Is the narrative structure of the final study or dissertation or thesis mentioned, and does it relate to the type of mixed methods design being used?
Describe Mixed Methods Research
Because mixed methods research is still somewhat unknown in the social and human sciences as a distinct research approach, it is useful to convey a basic definition and description of the approach in a method section of a proposal. This might include the following:
A definition. Begin by defining mixed methods. Recall the definition provided in Chapter 1. Elements in this definition can now be enumerated so that a reader has a complete set of core characteristics that describe mixed methods (see a more expanded view of defining mixed methods research in Johnson, Onwuegbuzie, & Turner, 2007):
It involves the collection of both qualitative (open-ended) and quantitative (closed-ended) data in response to research questions or hypotheses.
It includes the rigorous methods (i.e., data collection, data analysis, and interpretation) of both quantitative and qualitative data.
The two forms of data are integrated in the design analysis through merging the data, explaining the data, building from one database to another, or embedding the data within a larger framework.
These procedures are incorporated into a distinct mixed methods design that indicates the procedures to be used in a study.
These procedures are often informed by a philosophy (or worldview) and a theory (see Chapter 3).
Terminology. Explain that many different terms are used for this approach, such as integrating, synthesis, quantitative and qualitative methods, multimethod, mixed research, or mixed methodology but that recent writings, such as the SAGE Handbook of Mixed Methods in the Social & Behavioral Sciences and SAGE’s Journal of Mixed Methods Research, tend to use the term mixed methods (Bryman, 2006; Creswell, 2015; Tashakkori & Teddlie, 2010).
Background of methodology. Educate the reader about the background of mixed methods by reviewing briefly the history of this approach to research. It can be seen as a methodology originating around the late 1980s and early 1990s in its current form based on work from individuals in diverse fields such as evaluation, education, management, sociology, and health sciences. It has gone through several periods of development and growth, and it continues to evolve, especially in procedures. Several texts outline these developmental phases (e.g., Creswell & Plano Clark, 2011, 2018; Teddlie & Tashakkori, 2009). This section could also include a brief discussion about the importance or rise of mixed methods today through indicators such as federal funding initiatives, dissertations, and the discipline-specific discussions about mixed methods found in journals across the social and health sciences (see Creswell, 2010, 2011, 2015).
Reasons for choosing mixed methods research. Follow this section with statements about the value and rationale for the choice of mixed methods as an approach for your project. At a general level, mixed methods is chosen because of its strength of drawing on both qualitative and quantitative research and minimizing the limitations of both approaches. At a practical level, mixed methods provides a sophisticated, complex approach to research that appeals to those on the forefront of new research procedures. It also can be an ideal approach if the researcher has access to both quantitative and qualitative data. At a procedural level, it is a useful strategy to have a more complete understanding of research problems and questions, such as the following:
Comparing different perspectives drawn from quantitative and qualitative data
Explaining quantitative results with a qualitative follow-up data collection and analysis
Developing better contextualized measurement instruments by first collecting and analyzing qualitative data and then administrating the instruments to a sample
Augmenting experiments or trials by incorporating the perspectives of individuals
Developing cases (i.e., organizations, units, or programs) or documenting diverse cases for comparisons
Developing a more complete understanding of changes needed for a marginalized group through the combination of qualitative and quantitative data
Evaluating both the processes and the outcomes of a program, an experimental intervention, or a policy decision
Indicate the type of mixed methods design that will be used in the study and the rationale for choosing it. A detailed discussion of the primary strategies available will be discussed shortly. Include a figure or diagram of these procedures.
Challenges to design. Note the challenges this form of research poses for the inquirer. These include the need for extensive data collection, the time-intensive nature of analyzing both qualitative and quantitative data, and the requirement for the researcher to be familiar with both quantitative and qualitative forms of research. The complexity of the design also calls for clear, visual models to understand the details and the flow of research activities in this design.
There have been several typologies for classifying and identifying types of mixed methods strategies that proposal developers might use in their proposed mixed methods study. Creswell and Plano Clark (2018) identified several classification systems drawn from the fields of evaluation, nursing, public health, education policy and research, and social and behavioral research. In these classifications, authors used diverse terms for their types of designs, and a substantial amount of overlap of types existed in the typologies. For purposes of clarifying the design discussion in the mixed methods field, we will identify three core mixed methods designs (as shown in Figures 10.1 and 10.2)—the convergent design, the explanatory sequential design, and the exploratory sequential design—and then briefly mention more complex designs (i.e., the mixed methods experimental design, the mixed methods case study design, the mixed methods participatory–social justice design, and the mixed methods evaluation design) in which the core designs can be embedded. Each approach will be discussed in terms of a description of the design, the forms of data collection and data analysis and integration, interpretation, and validity challenges.
Convergent Mixed Methods Design
Description of the design. The convergent mixed methods design is probably the most familiar of the core and complex mixed methods approaches. Researchers new to mixed methods typically first think of this approach because they feel that mixed methods only consists of combining the quantitative and qualitative data. In this single-phase approach, a researcher collects both quantitative and qualitative data, analyzes them separately, and then compares the results to see if the findings confirm or disconfirm each other (see Figure 10.1). The key assumption of this approach is that both qualitative and quantitative data provide different types of information—often detailed views of participants qualitatively and scores on instruments quantitatively—and together they yield results that should be the same. It builds off the historic concept of the multimethod, multitrait idea from Campbell and Fiske (1959), who felt that a psychological trait could best be understood by gathering different forms of data. Although the Campbell and Fiske conceptualization included only quantitative data, the mixed methods researchers extended the idea to include the collection of both quantitative and qualitative data.
Figure 10.1 Three Core Mixed Methods Designs
Data collection. The qualitative data can assume any of the forms discussed in Chapter 9, such as interviews, observations, documents, and records. The quantitative data can be instrument data, observational checklists, or numeric records, such as census data, as discussed in Chapter 8. Ideally, the key idea with this design is to collect both forms of data using the same or parallel variables, constructs, or concepts. In other words, if the concept of self-esteem is being measured during quantitative data collection, the same concept is asked during the qualitative data collection process, such as in an open-ended interview. Some researchers will use this design to associate certain themes with statistical data using different forms of data for the quantitative and qualitative data collection. For instance, Shaw et al. (2013) compared quality improvement practices in family medicine clinics with colorectal cancer screening rates. Another data collection issue is the sample size for both the qualitative and quantitative data collection process. Unquestionably, the data for the qualitative data collection will be smaller than that for the quantitative data collection. This is because the intent of data collection for qualitative data is to locate and obtain information from a small sample but to gather extensive information from this sample; whereas, in quantitative research, a large N is needed in order to infer meaningful statistical results from samples to a population.
How is this inequality resolved in a convergent mixed methods design? Sometimes mixed methods researchers will collect information from the same number of individuals on both the qualitative and quantitative databases. This means that the qualitative sample will be increased, and it will limit the amount of data collected from any one individual. Another approach would be to weight the qualitative cases so that they equal the N in the quantitative database. One other approach taken by some mixed methods researchers is not to consider the unequal sample sizes a problem. They would argue that the intent of qualitative and quantitative research differs (one to gain an in-depth perspective and the other, to generalize to a population) and that each provides an adequate count. Another issue in sampling is whether the individuals for the sample of qualitative participants should also be individuals in the quantitative sample. Typically, mixed methods researchers would include the sample of qualitative participants in the larger quantitative sample, because ultimately researchers make a comparison between the two databases and the more they are similar, the better the comparison.
Data analysis and integration. Data analysis in a convergent design consists of three phases. First, analyze the qualitative database by coding the data and collapsing the codes into broad themes. Second, analyze the quantitative database in terms of statistical results. Third comes the mixed methods data analysis. This is the analysis that consists of integrating the two databases.
This integration consists of merging the results from both the qualitative and the quantitative findings. One challenge in this design is how to actually merge the two databases since bringing together a numeric quantitative database with a text qualitative database is not intuitive. There are several ways to merge the two databases:
The first approach is called a side-by-side comparison. These comparisons can be seen in the discussion sections of mixed methods studies. The researcher will first report the quantitative statistical results and then discuss the qualitative findings (e.g., themes) that either confirm or disconfirm the statistical results. Alternatively, the researcher might start with the qualitative findings and then compare them to the quantitative results. Mixed methods writers call this a side-by-side approach because the researcher makes the comparison within a discussion, presenting first one set of findings and then the other. A good example of this can be seen in the Classen and colleagues’ (2007) study.
Researchers can also merge the two databases by changing or transforming qualitative codes or themes into quantitative variables and then combining the two quantitative databases—a procedure in mixed methods research called data transformation. The researcher takes the qualitative themes or codes and counts them (and possibly groups them) to form quantitative measures. Some useful procedures that mixed methods researchers have used can be found in Onwuegbuzie and Leech (2006). This approach is popular among researchers trained in quantitative research who may not value or see the worth of an independent qualitative interpretive database.
A final procedure involves merging the two forms of data in a table or a graph. This is called a joint display of data, and it can take many different forms. It might be a table that arrays the themes on the horizontal axis and a categorical variable (e.g., different types of providers such as nurses, physician assistants, and doctors) on the vertical axis. It might be a table with key questions or concepts on the vertical axis and then two columns on the horizontal axis indicating qualitative responses and quantitative responses to the key questions or concepts (Li, Marquart, & Zercher, 2000). The basic idea is for the researcher to jointly display both forms of data—effectively merging them—in a single visual and then make an interpretation of the display (see Guetterman, Fetters, & Creswell, 2015).
Interpretation. The interpretation in the convergent approach is typically written into a discussion section of the study. Whereas the results section reports on the findings from the analysis of both the quantitative and qualitative databases, the discussion section includes a discussion comparing the results from the two databases and notes whether there is convergence or divergence between the two sources of information. Typically the comparison does not yield a clean convergent or divergent situation, and differences exist on a few concepts, themes, or scales. When divergence occurs, steps for follow-up need to be taken. The researcher can state divergence as a limitation in the study without further follow-up. This approach represents a weak solution. Alternatively, mixed methods researchers can return to the analyses and further explore the databases, collect additional information to resolve the differences, or discuss the results from one of the databases as possibly limited (e.g., the constructs were not valid quantitatively or the qualitative themes did not match the open-ended questions). Whatever approach the researcher takes, the key point in a convergent design is to further discuss and probe results when divergent findings exist.
Validity. Validity using the convergent approach should be based on establishing both quantitative validity (e.g., construct) and qualitative validity (e.g., triangulation) for each database. Is there a special form of mixed methods validity that needs to be addressed? There are certainly some potential threats to validity in using the convergent approach, and several of these have already been mentioned. Unequal sample sizes may provide less of a picture on the qualitative side than the larger N on the quantitative side. Generally we find the use of unequal sample sizes in a convergent design study, with the researcher acknowledging the different perspectives on size taken by quantitative and qualitative researchers. The use of different concepts or variables on both sides, quantitative and qualitative, may yield incomparable and difficult-to-merge findings. Our recommended approach is to use the same concepts for both the quantitative and qualitative arms of the research study, but we acknowledge that some researchers use the convergent design to associate different qualitative and quantitative concepts. A lack of follow-up on conclusions when the scores and themes diverge also represents an invalid strategy of inquiry. In this discussion we have recommended several ways to probe divergence in more detail and would recommend the use of one or more of these strategies in a convergent design project.
Explanatory Sequential Mixed Methods Design
Description of the design. The explanatory sequential mixed methods approach is a design in mixed methods that appeals to individuals with a strong quantitative background or from fields relatively new to qualitative approaches. It involves a two-phase data collection project in which the researcher collects quantitative data in the first phase, analyzes the results, and then uses the results to plan (or build on to) the second, qualitative phase. The quantitative results typically inform the types of participants to be purposefully selected for the qualitative phase and the types of questions that will be asked of the participants. The overall intent of this design is to have the qualitative data help explain in more detail the initial quantitative results, thus it is important to tie together or to connect the quantitative results to the qualitative data collection. A typical procedure might involve collecting survey data in the first phase, analyzing the data, and then following up with qualitative interviews to help explain confusing, contradictory, or unusual survey responses.
Data collection. The data collection proceeds in two distinct phases with rigorous quantitative sampling in the first phase and with purposeful sampling in the second, qualitative phase. One challenge in this strategy is to plan adequately what quantitative results to follow up on and what participants to gather qualitative data from in the second phase. The key idea is that the qualitative data collection builds directly on the quantitative results. The quantitative results that then are built on may be extreme or outlier cases, significant predictors, significant results relating variables, insignificant results, or even demographics. For example, when using demographics, the researcher could find in the initial quantitative phase that individuals in different socioeconomic levels respond differently to the dependent variables. Thus, the follow-up qualitatively may group respondents to the quantitative phase into different categories and conduct qualitative data collection with individuals representing each of the categories. Another challenge is whether the qualitative sample should be individuals that are in the initial quantitative sample. The answer to this question should be that they are the same individuals, because the intent of the design is to follow up the quantitative results and explore the results in more depth. The idea of explaining the mechanism—how the variables interact—in more depth through the qualitative follow-up is a key strength of this design.
Data analysis and integration. The quantitative and the qualitative databases are analyzed separately in this approach. Then the researcher combines the two databases by the form of integration called connecting the quantitative results to the qualitative data collection. This is the point of integration in an explanatory sequential design. Thus, the quantitative results are then used to plan the qualitative follow-up. One important area is that the quantitative results cannot only inform the sampling procedure but it can also point toward the types of qualitative questions to ask participants in the second phase. These questions, like all good qualitative research questions, are general and open-ended. Because analysis proceeds independently for each phase, this design is useful for student research and perhaps easier to accomplish (than the convergent design) because one database explains the other and the data collection can be spaced out over time.
Interpretation. The mixed methods researcher interprets the follow up results in a discussion section of the study. This interpretation follows the form of first reporting the quantitative, first-phase results and then the qualitative, second phase results. However, this design then employs a third form of interpretation: how the qualitative findings help to explain the quantitative results. A common misstep at this point by beginning researchers is to merge the two databases. While this approach may be helpful, the intent of the design is to have the qualitative data help to provide more depth, more insight into the quantitative results. Accordingly, in the interpretation section, after the researcher presents the general quantitative and then qualitative results, a discussion should follow that specifies how the qualitative results help to expand or explain the quantitative results. Because the qualitative database questions narrows the scope of the quantitative questions, a direct comparison of the overall results of the two databases is not recommended.
Validity. As with all mixed methods studies, the researcher needs to establish the validity of the scores from the quantitative measures and to discuss the validity of the qualitative findings. In the explanatory sequential mixed methods approach, additional validity concerns arise. The accuracy of the overall findings may be compromised because the researcher does not consider and weigh all of the options for following up on the quantitative results. We recommend that researchers consider all options for identifying results to follow up on before settling on one approach. Attention may focus only on personal demographics and overlook important explanations that need further understanding. The researcher may also contribute to invalidated results by drawing on different samples for each phase of the study. If explaining the quantitative results in more depth, then it makes sense to select the qualitative sample from individuals who participated in the quantitative sample. This maximizes the importance of one phase explaining the other. These are a few of the challenges that need to be built into the planning process for a good explanatory sequential mixed methods study.
Exploratory Sequential Mixed Methods Design
Description of the design. If we reverse the explanatory sequential approach and start with a qualitative phase first followed by a quantitative phase, we have an exploratory sequential approach. A three-phase exploratory sequential mixed methods is a design in which the researcher first begins by exploring with qualitative data and analysis, then builds a feature to be tested (e.g., a new survey instrument, experimental procedures, a website, or new variables) and tests this feature in a quantitative third phase. Like the explanatory sequential approach, the second feature builds on the results of the initial database. The intent of this design is to explore with a sample first so that a later quantitative phase can be tailored to meet the needs of the individuals being studied. Sometimes this quantitative feature will include developing a contextually sensitive measurement instrument and then testing it with a sample. Other times it may involve developing new variables not available in the literature or attuned to a specific population being studied, or designing a website or an Internet application shaped to the needs of the individuals being studied. This design is popular in global health research when, for example, investigators need to understand a community or population before administering English-language instruments.
In this design, the researcher would first collect focus group data, analyze the results, develop an instrument (or other quantitative feature such as a website for testing), and then administer it to a sample of a population. In this case, there may not be adequate instruments to measure the concepts with the sample the investigator wishes to study. In effect, the researcher employs a three-phase procedure with the first phase as exploratory, the second as instrument (or quantitative feature) development, and the third as administering and testing the instrument feature to a sample of a population.
Data collection. In this strategy, the data collection would occur at two points in the design: the initial qualitative data collection and the test of the quantitative feature in the third phase of the project. The challenge is how to use the information from the initial qualitative phase to build or identify the quantitative feature in the second phase. This is the integration point in an exploratory sequential design.
Several options exist, and we will use the approach of developing a culturally sensitive instrument as an illustration. The qualitative data analysis can be used to develop an instrument with good psychometric properties (i.e., validity, reliability). The qualitative data analysis will yield quotes, codes, and themes (see Chapter 9). The development of an instrument can proceed by using the quotes to write items for an instrument, the codes to develop variables that group the items, and themes that group the codes into scales. This is a useful procedure for moving from qualitative data analysis to scale development (the quantitative feature developed in the second phase). Scale development also needs to follow good procedures for instrument design, and the steps for this include ideas such as item discrimination, construct validity, and reliability estimates (see DeVellis, 2012).
Developing a good psychometric instrument that fits the sample and population under study is not the only use of this design. A researcher can analyze the qualitative data to develop new variables that may not be present in the literature, to identify the types of scales that might exist in current instruments or to form categories of information that will be explored further in a quantitative phase. The question arises if the sample for the qualitative phase is the same for the quantitative phase. This cannot be, because the qualitative sample is typically much smaller than a quantitative sample needed to generalize from a sample to a population. Sometimes mixed methods researchers will use entirely different samples for the qualitative (first phase) and quantitative components (third phase) of the study. However, a good procedure is to draw both samples from the same population but make sure that the individuals for both samples are not the same. To have individuals help develop an instrument and then to survey them in the quantitative phase would introduce confounding factors into the study.
Data analysis and integration. In this strategy the researcher analyzes the two datab
We are a professional custom writing website. If you have searched a question and bumped into our website just know you are in the right place to get help in your coursework.
Yes. We have posted over our previous orders to display our experience. Since we have done this question before, we can also do it for you. To make sure we do it perfectly, please fill our Order Form. Filling the order form correctly will assist our team in referencing, specifications and future communication.
2. Fill in your paper’s requirements in the "PAPER INFORMATION" section and click “PRICE CALCULATION” at the bottom to calculate your order price.
3. Fill in your paper’s academic level, deadline and the required number of pages from the drop-down menus.
4. Click “FINAL STEP” to enter your registration details and get an account with us for record keeping and then, click on “PROCEED TO CHECKOUT” at the bottom of the page.
5. From there, the payment sections will show, follow the guided payment process and your order will be available for our writing team to work on it.