Mixed Methods [COMB]
Evaluating a Mixed Methods Study Question 6
"Does the study identify qualitative and quantitative data collection techniques clearly?"
Evaluating a Mixed Methods Study Question 1
"Does the study include a rationale for using a mixed methods research design?"
Evaluating a Mixed Methods Study Question 2
"Is the correct type of mixed methods research design used?"
pragmatic worldview
-mixed methods approach goes along with it -arises out of actions, situations, and consequences -emphasize the problem and use all approaches available to to understand the problem
2 General Strategies
1. Convergent 2. Sequential
What 3 issues does qualitative researchers address?
1. Know about the CHARACTERISTICS of whatever phenomena you are addressing 2. Know about people's PERCEPTIONS of the causes of the phenomena you are addressing 3. What are the CONSEQUENCES of the phenomena you are addressing
Nomenclature
1. Orientation: Whether the research is qual or quan oriented. 2. Dominance: Which aspect of the mixed methods design is dominant; uppercase: dominant, lowercase: less dominant. 3. Sequence: Whether mixed methods designs are conducted simultaneously, as designated by + or ->
How is theory used in a QUALitative study
1. broad explanation for behaviors/attitudes 2. theoretical lens 3. as an endpoint
the 2 dimensions of mixed methods designs
1. time order 2. paradigm emphasis
Multi-phase Design
A series of phases or separate studies, each of which may use a combination of sequential ad/or concurrent phases
PCC mix design procedure
Choice of: Slump, max aggregate size selection, mixing water and air content selection, water-cement ratio, cement content, coarse aggregate content, fine aggregate content, and adjustments for aggregate moisture
QUAL + quan or QUAN + qual
Dominant status / concurrent
Purpose, priority, sequence, integration
Four mixed method design considerations
Operational Form Hypothesis
Hypothesis written so that it represent specific information about the variables in the study
Literary Form Hypothesis
Hypothesis written so that it represent variables in the abstract
Holism
Idea that a whole, such as a culture, is more than the sum of its individual parts
Case Study
In-depth description or analysis of a single unit
What is the arrow symbol?
Indicates a sequential form of data collection with one form of data building on the other
What does the plus sign indicate?
Indicates simultaneous or concurrent form of data collection
What does capitalization mean?
Indicates weight or priority data
How does mixed methods take into account the different disciplinary approaches discussed in your study's lit review?
Individual Response:
What design did you select for your POP and why? What are the pros and cons of the design you used (e.g., embedded, exploratory, etc.) in your particular study?
Individual Response:
Feedback
Information about service delivery system outputs, outcomes, or operations that can guide program input.
What is merging?
Merged can occur after completion of qualitative and quantitative data collection and analysis. The total findings are then interpreted to find complementary, convergence and divergence among data sets
Quantitative is beneficial because there is
Norm and generalizability
Confirmability
Reduction of researcher bias; is bias acknowledged and considered?; neutrality
What is step 2?
Select the contrasting complementary method
Credibility
Similar to internal validity; can you believe the results?
Convergent Parallel Design
Simultaneous collection of quantitative and qualitative data, then margining the two together
Sequential Validity
The degree researcher appropriately addresses and/or builds on findings from earlier quantitative and qualitative phases
Weakness Minimization Validity
The degree researcher combines quantitative and qualitative approaches with non-overlapping weaknesses
Accessing Participants' Life World
The research participant's inner world of subjective experience
The Multistage Evaluation Design (Definition)
The systemic process of collecting and analyzing data about the quality, effectiveness, merit or value of programs, products or practices.
Lifeworld
a person's subjective inner world of experience
Population
an entire unit or group; includes all units of interest in the study (i.e., all students, all clients, all employees)
Theory in quantitative research
an interrelated set of constructs formed into propositions, or hypotheses, that specify the relationship among variables.
Qualitative Research
an open approach to the study of social phenomena; some aspects include: emergent design, grounded in the lived experiences of individuals, and drawing on multiple methods. Data is in narrative form.
qualitative research
an open approach to the study of social phenomenon; some aspects include emergent designs, being grounded in the lived experiences of individuals, and drawing on multiple methods. data are in narrative form
researcher bias in qualitative research
can be reduced by: - reflexivity - negative case sampling
Intrinsic Case Study
case study in which the researcher is only interested in understanding the individual case, organization, or event
Instrumental Case Study
case study in which the researcher studies a case in order to understand something more general than the particular case or insight into an issue or to develop, refine, or alter some theoretical explanation
Cross case analysis
cases are compared and contrasted for similarities or patterns or differences
Shared values
culturally defined standards about what is good or bad or desirable or undesirable
concurrent time orientation
data are collected for the quantitative phase and qualitative phase of the study at the same or during approximately the same time period
research methods
data collection, analysis, and interpretation
Low-Inference Descriptors
descriptions that are very close to participants' words or are direct verbatim quotes
postpositivist
determination, reductionism, empirical observation and measurement, theory verification
mixed methods research
incorporates elements of both quantitative an qualitative approaches
Strategies for Achieving Descriptive Validity
investigator triangulation
substantive coding part 2
open codes are connected to core categories
Pragmatists
researchers emphasize the research problem and use all approaches available to understand the problem.
research approaches
the plans and procedures for research that span the steps from broad assumptions to detailed methods of data collection, analysis, and interpretation.
Phenomenology
Idiographic philosophy
Embedded Mixed Methods
Nests one or more forms of data within a larger design
External audit
Peer review by someone not involved in teh study
Ethnography
Purpose is to describe a culture from the perspective of an insider
Theoretical coding
rethink and revise how substantive codes relate to each other
Evaluating a Mixed Methods Study Question 7
"Does the study use appropriate data analysis techniques for both qualitative and quantitative data?"
Evaluating a Mixed Methods Study Question 3
"Does the study use both quantitative and qualitative data collection techniques appropriately?"
Evaluating a Mixed Methods Study Question 4
"Is the priority given to quantitative and qualitative data collection and the sequence of their use reasonable given the research question?"
Ethnography
"Its good if you can't tell if it is _______ or hanging out."
Evaluating a Mixed Methods Study Question 5
"Was the study feasible given the amount of data to be collected and concomitant issues of resources, time, and expertise?"
What are Pros & Cons of Exploratory MM Design?
+ Although this design typically emphasizes the qualitative aspect, the inclusion of a quantitative component can make the qualitative approach more acceptable to quantitative-biased audiences. _________________________________________________________ - Requires considerable time to implement - Researchers should discuss whether the same individuals will serve as participants in both the qualitative and quantitative phases, as this will affect internal review board decisions
What are Pros & Cons of Triangulation (Convergent) MM Design?
+ Data collection is concurrent; can lend to better understanding of POP + Same time frame for data collection ___________________________________________________________ - Separate analysis is required (time consuming) - Researchers need to consider the consequences of having different samples and different sample sizes when converging the two data sets. Different sample sizes are inherent in the design because quantitative and qualitative data are usually collected for different purposes (generalization vs. in-depth description, respectively).
What are Pros & Cons of Explanatory MM Design?
+ Methods implemented sequentially (1. quan 2. qual) so as to explain the quantitative results (single researcher can do this) + The final report can be written in two phases + This design appeals to quantitative researchers, because it often begins with strong quantitative orientation. ____________________________________________________ - This design requires a lengthy amount of time for implementing the two phases. - It can be difficult to secure internal review board approval for this design because researcher cannot specify how participants will be selected for second phase until the initial findings are obtained.
What are the Pros & Cons of Embedded MM Design?
+ This design may be logistically more manageable for graduate students because one method requires less data than the other method. +This design may be appealing to funding agencies because the primary focus of the design is traditionally quantitative _____________________________________________________________ - For during-intervention approaches, the qualitative data collection may introduce potential treatment bias that affects the outcomes of the experiment. - Researcher needs to ensure that the qualitative data collection is carefully designed to match the intent for including qualitative data, such as to develop an instrument or shape the intervention.
exploratory design
- 2 phase process -QUAL than QUAN -how QUAN results provide new and better instruments & intervention
explanatory design
- 2 phase process -QUAN than QUAL - how do QUAL results explain QUAN results
What is the 3 phase design used to show?
- Data connected qual analysis informs quantitative data collection - Use to administer to large sample of population -Allows high contextual relevance
What does the 2 phase design used to show?
- Data connected quantitative analysis informs qualitative data collection - Used to explain and interpret quantitative results by collecting and analyzing qualitative data
Challenges of Mixed Methods
- Needing to understand both quantitative and qualitative research procedures. - Assumption about how research questions cannot be answered by a single (quan or qual) method.
Explain concurrent transformative
- One data collection phase - No predominant weighting - Must be guided by theoretical orientation
Explain the rigour of qualitative data:
- Site - Permissions - Purposeful sampling - Reciprocity - Types of data - Protocol - Data preparation - Data analysis - Validity strategy - Reflectivity
Explain the rigour of quantitative data:
- Site - Permissions - Systematic sampling - Types of data - Instruments - Data cleaning - Statistics - Validity and reliability
methods for strengthening theoretical validity in qualitative research
- extended fieldwork - theory triangulation - pattern matching - peer review
methods for strengthening descriptive validity in qualitative research
- investigator triangulation
methods for strengthening interpretive validity in qualitative research
- participant feedback - low-inference descriptors
methods for strengthening internal validity in qualitative research
- researcher as detective - methods triangulation - data triangulation
multiphase iterative design
-3 or more phases -early ones provide foundational data on which later ones can build
What are advantages of using mixed methods research?
-A researcher can use the strengths of an additional method to overcome weaknesses in another method by (principle of complementarity) -Provides stronger evidence for conclusion through convergence/corroboration of findings (triangulation) -produces more complete knowledge necessary to inform theory and practice -Can be used to increase the generalizability of the results
What are disadvantages of using mixed methods research?
-The researcher has to learn about multiple methods and approaches and understand how to appropriately mix them -It is more expensive and more time consuming. -Need to work out issues ahead of time (e.g., problems of paradigm mixing, how to qualitatively analyze quantitative data, how to interpret conflicting results)
3 designs of mixed methods
-convergent (parallel/concurrent) -explanatory -exploratory -embedded -multiphase iterative
constructivist worldview
-goes along with qualitative research -individuals seek understanding of the world in which they live and work
postpositivist worldview
-goes along with quantitative research -represents the traditional form of research -scientific method or science research -hold a deterministic philosophy in which causes determine their outcome
quantitative
-numbers -close ended questions -experiments -collecting data on instruments
how do you gain a deeper understanding of a problem in mixed method research?
-uses both QUAN & QUAL techniques for data collection and analysis
qualitative
-words -open ended questions -case study -collecting data by observation
Five mixed-method designs
1) Convergent Parallel 2) Explanatory Sequential 3) Exploratory Sequential 4) Embedded Design 5) Multi-phase Design
Common Content Analysis Steps (recording unit and coding)
1) Define the recording unit (word, phrase, sentence, and paragraph). 2) Define the coding categories. 3) Test the coding by applying it to a sample of your collected data. 4) Assess the accuracy and reliability of this sample coding (perhaps using multiple coders to check for intercoder consistency). 5) Revise the coding rules and/or instructions to improve consistency and accuracy in coding. 6) Return to step 3 and run the cycle until you achieve sufficient consistency and accuracy in coding. 7) Code all data. 8) Assess overall consistency and accuracy of the codings
Reasons for Mixed-Methods
1) Enhancement: Building on findings from one method using evidence gathered from the other method 2) Triangulation: using findings from one method to verify or corroborate findings collected using the other method. The process of operationalizing and measuring constructs or variables in multiple ways to converge upon a more accurate observation or assessment. 3) Completeness: using both methods to more comprehensively examine an area or issue of interest than would be possible with one method used alone. 4) Illustration: using qualitative data to explain quantitative findings, or vice versa. 5) Sampling: using one method to help with the sampling of participants or cases in a targeted and focused fashion
Challenges of Qualitative and Mixed-Methods Research
1. A tendency to report only the qualitative or quantitative data, but not both. 2. A tendency to report findings for one method, followed by findings from the other, without any attempt at integration. 3. The perceived need to focus the presentation of results toward either a quantitatively or qualitatively focused audience. 4. A personal preference of comfort with one method over the other. 5. A research design that inhibits the integration of quantitative and qualitative data (the quantitative structure overly limits the qualitative follow-up. 6. The pressure to publish some results as quickly as possible, which might push the researcher to lean on one method more than the other.
Give 4 features of qualitative research?
1. Collects non-numerical data 2. Not concerned with mathematical probabilities or representativeness 3. It is an inductive approach, not a hypothesis-deductive approach (you don't start off with a hypothesis that you test, you build theories and try to understand why things happen/phenomena) 4. It can be combined with quantitative research methods. This would then be mixed method or multi-method research.
6 broad types of research situations that are especially well suited for MM research:
1. Concepts are new and poorly understood 2. Neither method by itself is adequate in addressing the complexity of the research problem 3. Findings from one method can be greatly enhanced with a second source of data 4. The quantitative results are puzzling and difficult to interpret, and qualitative data can help explain the results 5. A particular theoretical perspective might require both qualitative and quantitative data 6. A multiphase project is needed to attain key objectives, such as the development and assessment of an intervention
Give 3 qualitative methods of data collection which are commonly used in social sciences...
1. Ethnography (e.g. participant observation) 2. Life Histories 3. Direct Observation
Describe the 5 stages of the research cycle in order...
1. Identify research area 2. Design research study 3. Carry out research 4. Analyse research results 5. Publish research results.
Give 2 qualitative methods of data collection which are commonly used in health service research...
1. Interviews 2. Focus Groups
Name all 7 terms used for mixed/multi-method research...
1. Multi-methods 2. Mixed-methods 3. Multi-methodology 4. Mixed methodology 5. Multiple methods 6. Combined methods 7. Integrated methods
Social Science Theory Key points
1. Place at beginning to guide hypotheses in study 2. Write how it is used then describe how it informs the qual/quan components of the study 3. include diagram of theory 4. provide framework for data collection 5. return theory @end of study to inform how findings and results compared w/use of theory
Phases of Exploratory Sequential Design
1. Research begins by working inductively, collecting qualitative data (through observation/interview) from a purposive sample. Then coding/identifying themes and/or categories occurs. 2. Then quantitative data is collected with a large, randomly chosen sample.
Phases of Explanatory Sequential Design
1. Researcher formulates a hypothesis, collects quantitative data, and conducts data analysis. - Findings determine data collection method 2. Data collection, analysis, and interpretation of qualitative data.
Give 6 qualitative methods of data analysis...
1. Thematic Analysis 2. Framework Analysis 3. Interpretative Phenomenological Analysis (IPA) 4. Grounded Theory 5. Narrative Analysis 6. Discourse Analysis/Conversation Analysis
Identifying Studies Using Mixed Method Designs
1. Title has quan. and qual, mixed methods, convergent parallel, explan., explor., simult., and sequential. 2. Purpose statement or research questions indicate mixed methods 3. Combo of quan. and qual. were used to collect and analyze data.
Creswell and Plano Clark (2011) distinguished four main types of mixed methods research designs with associated rationales. What are they?
1. Triangulation, Concurrent, or Parallel Design - which entails separate quantitative and qualitative data collection and analysis within the same timeframe 2. Embedded Design - a concurrent design where a qualitative part is embedded in a quantitative study, or vice versa, so that the qualitative results are used to support or explain findings from the other method. 3. Exploratory Sequential Design - in which a qualitative study conducted in the first phase informs a quantitative study conducted in the second phase. 4. Explanatory Sequential Design - a sequential design in which a quantitative study conducted in the first phase informs a qualitative study conducted in the second phase.
types of validity in qualitative research
1. descriptive validity 2. interpretive validity 3. theoretical validity 4. internal validity 5. external validity
Steps of inductive logic of a qual. research study
1. gather info 2. ask open ended questions 3.analyze data to form themes/categories 4. looks for patterns/generalizations/theories 5. poses generalizations or theories from past experiences and literature
types of validity in mixed methods research
1. inside-outside validity 2. weakness-minimisation validity 3. sequential validity 4. sample integration validity 5. multiple validities
case study designs
1. intrinsic case study 2. instrumental case study 3. collective case study
methods of data analysis in grounded theory
1. open coding: exploratory stage of noting important ideas and concepts 2. axial coding: narrowing and ordering concepts so they follow from each other and better suit the theory 3. selective coding: finalising and 'grounding' the theory
4 major qualitative research methods
1. phenomenology 2. ethnography 3. grounded theory 4. case study approach
Transformative Theory -applicable to what studies? -Common themes? -steps for theory?
1. study of community health or marginalized groups 2. Themes: - underlying assump. that rely on ethical stances of inclusion challenging opressive social structures - entry process w/commun. to build trust and make goals -disssemination of findings that encourages use of results to enhance social justice/human rights 3. Steps: -define problem/lit. rev -identify research design -find data sources -identify istruments/methods -analyze, interpret & report
Mixed methods theory use
1. used for deductive in quant. theory for testings/validity 2. used for emerging qual. theory/pattern 2 frameworks: social science and transformative
Participant selection variant
1st stage quan data are in the service of the 2nd phase qual component info about the characteristics of the large group in the first phase is used to purposively select participants in the 2nd dominant phase
Ethnography
A "key contact" or "gate keeper" is used to gain access to a community
Abstract
A brief review of the literature that summarizes major elements to enable a reader to understand the basic features of the article.
Hypothesis
A declarative statement of the relationship between two or more variables.
MIXED METHODS PURPOSE STATEMENT:
A good mixed methods purpose statement contains: •The overall intent of the study from a content perspective •Information about both the quantitative and qualitative strands of the study •A rationale for combining both quantitative and qualitative data •Guidelines include: •Use words and phrases that signal intent such as: purpose, intent of, or objective •Indicate the overall purpose of the study from a content perspective •Indicate the type of mixed methods design
Conducting Mixed Methods Research (Literature Review)
A researcher needs to consider how and when to conduct a literature review, whether or not to let it guide hypotheses or other expectations for the study, and how to present it in a final report.
Needs assessment
A type of evaluation research that attempts to determine the needs of some population that might be met with a social program.
Cost-benefit analysis
A type of evaluation research that compares program costs with the economic value of program benefits.
What are the advantages and disadvantages of Qualitative Data?
ADVANTAGES - Detailed perspective of a few people - Hear voices of participants - Understand perspectives in context - Built from participants views DISADVANTAGES: - Limited generalizability - Few people studied - Interpretive
What are the advantages and disadvantages of Quantitative Data?
ADVANTAGES: - Draw conclusions for large number of people - Efficient data analysis - Demonstrate relationships - Bias control - Examine cause and effect DISADVANTAGES: - Impersonal - Limited understanding of context
Challenges
Ability to set aside investigator self to be open to the emergent theory. Charmaz (2006) said this is impossible Researcher may have difficulty knowing when saturation is reach or richness of categories is detailed. Remedy of this is discriminant sampling (Creswell 2007?) by testing theory on people similar to original group to see if it matches.
Action Research con't
Action research challenges traditional social science by moving beyond reflective knowledge created by outside experts sampling variables, to an active moment-to-moment theorizing, data collecting and inquiry occurring in the midst of emergent structure. "Knowledge is always gained through action and for action. From this starting point, to question the validity of social knowledge is to question, not how to develop a reflective science about action, but how to develop genuinely well-informed action - how to conduct an action science" (Torbert 2002 Why educational research has been so uneducational: the case for a new model of social science based on collaborative inquiry)[citation needed]. In this sense, performing action research is the same as performing an experiment, thus it is an empirical process
SupePave aggregate properties to consider
Aggregate source, gradation and size, and blending. Properties are selected based on anticipated traffic loading as well as depth below surface.
Conducting Mixed Methods Research (Procedures)
All procedures to be used in a study must be reviewed by an IRB (Institutional Review Board) prior to data collection for that phase.
Mixed Methods Designs
Allow researchers to use two types of data to verify findings, to more deeply develop their theoretical frameworks; and to produce a richer dataset; use both qualitative and quantitative data
PCC mix design method
American Concrete Institute (ACI) mix design method
What is mixed methods?
An approach to research in the social, behavioral and health sciences in which the investigator gather both quantitative and qualitative data and integrates the two then draws interpretations based on the combined strengths of both sets of data
Emic perspective
An emic is a type of research study in which the focus is one single culture. This type of research attempts to study the behaviors of interest though the lens of a member of the culture. It involves looking at behaviors of a group from the perspective of one member of that group. An emic research study is the opposite of an etic study in which the behavioral research approach is across various cultures to learn about the universality of behaviors. Emic studies are common in social sciences.
What occurs during typology development?
Analysis of qual or quant data reveals set of categories that is then used as a framework to analyze the complementary data - Factor analysis of the quan data results in categories that are used in the analysis of the qual data - Categorical coding of qual data results in typology for use in quant hypothesis testing
Impact evaluation (or analysis)
Analysis of the extent to which a treatment or other service has an effect; also known as summative evaluation.
It is common to converge/merge the data at the _________
Analysis phase
What occurs during data comparison in MM?
Analyze qualitative and quantitative data separately then compare findings
Pragmatic worldview
Arises out of actions, situations and consequences rather than antecedent conditions. Focus is on what works for problem resolution. Instead of focusing on methods, concentrate on problem and use all approaches available to understand it. Use mixed methods. Truth is what works at the time.
Forms of theories (qual)
As part of an : • interconnected hypotheses (if then logic) • If then statements that explain influence of independent variables on dependent variable • visual model (causal modeling)
Advocacy and participatory worldview
Based on belief that theories and laws do not fit marginalized individuals or issues of social justice. Research must be intertwined with politics. Researcher has action agenda for reform for change. Addresses specific issues. Researcher collaborates with participants to design questions, data collection or analysis.
HMA mix design process
Basic HMA mix design always involves aggregate selection, asphalt binder selection, and optimum asphalt binder content determination
When is sample selected?
Before study starts, some designs sample during
Quantitative Examples
Behavioral Observations: Counting Frequency, Rating Intensity Interviews: Rating specific responses of the interviewee to specific questions for quality and accuracy Survey: Gathering self-ratings of agreement or satisfaction with descriptive items and scales Case Study: Small-n or single participant study of changes in behaviors, attitudes, or other measurable variables
Qualitative Examples
Behavioral Observations: Describing process or actions. Highlighting perceived causes and effects Interviews: Recording and transcribing the entirety of the interview experience, including interviewer questions, interviewee responses, gaps in conversation, etc., viewing all as meaningful information and reflections of interviewees' own reality. Survey: Eliciting writing responses to open-ended questions or comment-request boxes Case Study: More in-depth review of individualizing cases, with goal of providing detail regarding context and process, rather than generalize quantified effects.
Benefits (advantages) of Mixed Methods Research
Benefits (advantages) • In many, dare we say most cases the researcher can learn the most about a phenomenon by using a combined, mixed-methods approach • The researcher can also successfully apply principles of data triangulation (The process of operationalizing and measuring constructs or variables in multiple ways to converge upon a more accurate observation or assessment) to ensure you are gaining the clearest possible understanding of the phenomenon you are studying.
What is depth vs breadth of info?
Both in different strands of the research design
Why and how does feasibility of using mixed methods need to be assessed beforehand?
Both the RESEARCH QUESTION and the REQUIRED DATA will be the main determinants of the methods used. To a lesser extent, the choice of methods may be influenced by feasibility, the research team's skills and experience, and time constraints. Secondly, priority of methods will relate to the emphasis placed on each method in the study.
Ethnography
Can be "critical ______," which advocates for marginalized people...by sharing their story you are theoretically advocating for them
The Social Justice Design (Definition and Purpose)
Can be based on any of the 3 basic mixed methods design. Purpose: to address one or more injustices faced by some group in society and ultimately evoke societal change.
Limitations of Case Studies
Can't generalize the results to the wider population. Researchers' own subjective feeling may influence the case study (researcher bias). Difficult to replicate. Time consuming.
What is convenience sampling?
Captive sample - you ask everyone in the waiting room to fill out your survey Volunteer sample - you advertise your study (e.g. internet) and people self-select to participate
Member checks
Checking in with participants to make sure that what you're finding is accurate...see if there is anything misstated or anything that should be added
Exploratory Sequential Design
Collect qualitative data in the first phase, analyze results, and then use the results to plan the second quantitative phase
Explanatory Sequential Design
Collect quantitative data in the first phase, analyze results, and then use the results to plan the second qualitative phase
Mixed methods
Combines both qual and quant approach. Involves philosophical assumptions as well . strategy of inquiry: Sequential mixed methods, concurrent mixed methods, transformative mixed methods.
Purposive
Common sampling method for qualitative design; involves deliberately choosing individuals, groups of individuals, or institutions towards the goal of best answering the research question
Reflexivity
Concept of acknowledge and being sensitive to assumptions/biases related to the research topic; ongoing process through all phases of a study--may do a paper or a journal; promotes transparency
Dependability
Consistency of results; would the results be the same if the study was repeated or is another researcher analyzed the data?; consistency
What is one of the key differences between qualitative and quantitative research?
Context- in quantitative research you strip out the context in order to be as objective as you possibly can whereas in qualitative research the context is really important.
What does creswell tell us about mixed methods?
Creswell tells us mixed methods 1. Collecting and analyzing data 2. Rigour 3. Integrate 4. Framing within a greater framework
Integration can occur at what stages of research process
Data collection, analysis, interpretation, knowledge translation
What is step 3?
Decide how the data will be used - This will help you to determine how to design the data collection aspect of your study Collect quantitative data, then qualitative data Collect qualitative data, then quantitative data Collect at the same time
Sequential Exploratory
Design type order qualitative, then quantitative; collection and analysis of qualitative data followed by collection and analysis of quantitative data
Sequential Explanatory
Design type order quantitative, then qualitative; collection and analysis or quantitative data followed by collection and analysis of qualitative data
What is the first step?
Determine whether research question is quantitative or qualitative
Slump
Determines how workable the mix will be. Ideal slump is a mix with the greatest stiffness that can still be placed adequately
Thematic Coding
Differs from other, more quantitatively oriented forms of content coding. The goal of thematic coding is typically to extract themes from a set of data. Typically, these data are verbal in nature, either collected via open-ended survey or interview questions, or extracted from communication records (e.g., e-mail, or Internet chat logs). Thematic coding also can be used with records of speeches or recorded dialogues between people. Or even video capturing a person's movements and nonverbal behaviors.
QUAL --> quan or QUAN --> qual
Dominant status / sequential
What is embedded?
Embedded occurs in studies with both primary and secondary research questions in which different methods are employed to address each question
QUAN + QUAL
Equal status / concurrent
QUAN --> QUAL or QUAL --> QUAN
Equal status / sequential
Process evaluation
Evaluation research that investigates the process of service delivery.
Mixed Methods
Example: healthcare-related interventions, which are implemented in a highly complex environments, answering the question of why some interventions work while others fail is not easily done within a quantitative framework. Such a question can be more comprehensively addressed with the use of a qualitative or combo of qualitative and quantitative (mixed methods) strategy.
Transformative Mixed Methods
Exploratory sequential, explanatory sequential, or convergent approaches are incorporated within a social justice framework
What is the generalizability of MMS?
External validity or transferability
Qualitative Data Techniques
Field Notes • Observation-focused • Comprehensive: complete; including all or nearly all elements or aspects of something. "A comprehensive list of sources" of or relating to understanding. Behavioral observation, interviews, Case studies, surveys
Conditional Matrix
Final and optional step of study process to create matrix of theory in context of historical, cultural, social and economic conditions which influence the central phenomenon. Theory from the smallest to broadest perspective. Grand theory level
Group
Finding ideally represent the experiences of each individual participant AND the _____ as a whole --> convergence/divergence & commonality/individuality
Ethnography
Findings describe the shared beliefs, practices, rituals, artifacts, and/or behaviors of a group
open coding
First step- segmenting collected data into categories. Each category has properties and look for data to dimensionalize properties.
Case Study
Focus is on trying to better understand a unique or exemplary unit or a standard typical unit
Multivariate Questions
Follow descriptive questions; compare or relate variables
Grounded theory, ethnography, phenomenology, & case study
Four categories of qualitative research
Credibility, transferability, dependability, confirmability
Four concepts of trustworthiness (AKA rigor)
Embedded Design
Four different types of Mixed Design; This design is used when researchers need to answer questions from a different perspective and their research has primarily focused on one type of data.
Explanatory Design
Four different types of Mixed Design; a two-phase design where qualitative results are obtained first then the quantitative in order to inform the qualitative data.
What is external validity?
Generalize findings to wider population
What is transferability?
Generalized or transferred to other contexts or settings. Researcher must do a thorough job of describing the research context and the assumptions that were central to the research
Reduce
Goal of data analysis is to ______ data from whole into themes, theories, and/or models
Superpave specimen compaction
Gyratory compactor works by heating each sample to the anticipated mixing temperature, aged for a short time, and then compacted. Device applies pressure to sample through hydraulically or mechanically operated load. Desired mixing and compacting temperatures are based on different asphalt binder properties to ensure compaction occurs at the same viscosity level for different mixes
Advanced Mixed Methods Research Designs
Have emerged in recent years because mixed methods research has become more common within many social and health science disciplines.
What does the mixed-method priority sequence model do?
Helps you to decide what methods to use.
3 HMA mix design methods
Hveem, Marshall and SuperPave methods
What is an extreme case analysis?
Identified from either qual or quan and further examined using complementary data collection and analysis to further explore the nature of the outliers and refine your interpretation of the results - quant outliers are identified and their qual data is explored - extreme cases identified in constant comparative analysis of qual data are further examined in quan analysis to refine interpretation of qual data
Mixed Methods Research can take many forms, and many decisions need to be made including:
Implementation sequence (timing) Priority (weighting) Integration
What is the difference between sampling for quantitative vs. qualitative studies?
In QUALITATIVE study, sampling involves the intentional selection of participants who have experienced a central phenomenon, condition, or concept. In QUANTITATIVE study, sampling requires that selection include a large number of individuals who are representative of the population or a segment of the population.
A meta-matrix
In a meta-matrix, researchers array information from qual and quan data sources; in a typical meta-matrix the rows correspond to cases (individual participants) and data from multiple sources are entered into the column
So when should you consider qualitative or mixed-methods research methods?
In general, when your research questions are more in line with developing understanding than identifying differences, some form of qualitative inquiry may be helpful.
The Experimental Design
In the experiment mixed methods design, the quantitative element of the study is ALWAYS an experimental design. Primary data is quantitative and secondary (supportive) data is qualitative.
Stakeholders
Individuals and groups who have some basis of concern with the program.
Social Constructivis
Individuals seek understanding of the world and develop subjective meanings of their experiences, directed towards certain objects of things. Toal of research is to rely as much as possible on the participants' views of the situation. Questions are broad and general so participants can construct meaning. Addresses the process of interaction among people and context. Researchers own background shapes interpretation is acknowledged. Researchers generate theory as they go along.
confounding variable
Influence cannot be directly detected
Can occur at various stages of research process
Integration
Theoretical Maximum Specific Gravity (Rice Specific Gravity)
It is Gmm and is the specific gravity excluding air voids. The rice density is obtained by multiplying Gmm by the density of water.
What are the uses of qualitative research in practice?
It is useful for exploring new topics without preconceptions, it facilitates the discovery of information relevant to the research topic that the researcher might never have considered, it enables greater insight into and understanding of phenomena than quantitative methods alone, and it is increasingly required by funders of Health Service Research (HSR).
Qualitative and Mixed-Methods Research
It is your research question that should drive the method you use, not the other way around. You may want to ask a question that requires less quantification and more qualification, fewer numbers and statistics and more detail and description. In these cases, a qualitative and/or combined qualitative and quantitative method for data collection and analysis may be useful to you.
Glaser critique of strauss and corbin
It relates the fractured codes at a much earlier level. This forces the data - it is too structured and deductive. Introduces preconceived ideas while pretending to be a grounded theory
Limited
Literature review is ________ a priori to minimize bias
Typical water/cement ratio for PCC mix design. Why is it important
Lower ratio's produce stronger, more durable PCC. Most state DOT's set max water-cement ratio at 0.40 to 0.50. It is important in determining the desired compressive strength. Used to calculate required cement content
Grand tour
Main, open-ended question that guides an interview and is followed by follow-up questions, prompts, and probes
Sequential Exploratory
Mixed method research design that is focused on discovery
Explain the philosophical stance of mixed methods:
Mixed method researchers have looked for one underlying philosophy that informs both quantitative and qualitative data collection
Concurrent triangulation
Mixed methods research design that is focused on cross-validation within a single study--looking at two forms of data and hopefully getting similar information/answers
Sequential Explanatory
Mixed methods research design that is focused on explaining or confirming quantitative results by exploring certain results in more detail or by helping to explain unexpected results
How do mixed methods strengthen the validity of research findings? (Onwuegbuzie & Johnson, 2006)
Mixed methods, can capitalize on the respective strengths of each approach. Pairing quantitative and qualitative components of a larger study can achieve various aims, including corroborating findings, generating more complete data, and using results from one method to enhance insights attained with the complementary method. Quantitative Validity: ensure scores or ratings on participants are meaningful and accurate indicators of the construct being measured; that scores are consistent and stable over time; and that multiple coders produce similar results during observations (fidelity of implementation on participants, fidelity of implementation on training of coders, and inter-rater reliability on coders).
Planning Mixed Methods Research: Integration
Mixing or purposeful integration of quantitative and qualitative components
Marshall HMA mix design method
Most widely used HMA mix design method. It was designed in US army and designed to stress the entire sampler rather than just a proportion of it, it facilitates rapid testing with minimal effort, compact, light and portable.
Basic Mixed Methods Design
Mostly used in education; usually used as a starting point for educational researchers.
What type of data is collected?
Numeric and narrative
Grounded Theory
Often provide results in the form of a visual representation of the theoretical model
Grounded Theory Questions Creswell 2007 Corbin & Strauss 1990
Open coding: What is the process? what are the steps in the process? How did X unfold? For Axial coding (more detail): What caused x to occur?- causal conditions/ What was central to the process? -core phenomenon/ What strategies were incurred through the process - strategies/ What effect occurred? - consequences
Strategies for Achieving Interpretive Validity
Participant Feedback and low inference descriptors
Explain the three phase sequential explanatory design:
Phase 1 is Qual data collection which builds into phase 2 of quan phase instrument/intervention design and phase 3 which is the same as phase 2 followed by inferences
Explain the two phase sequential explanatory design:
Phase 1 is quantitative data collection and analysis which is explained by phase 2 which is the qualitative data collection and analysis followed by inferences
Discussion
Portion of the research article where the integration of qualitative and quantitative data is seen most
Philosophical worldview (aka paradigms, epistemologies, ontologies).
Post-positivism, Soc. Constructivism, Advocacy/Participatory, Pragmatism
What is concurrent sampling strategy for MM?
Probability and purposive sampling used at the same time Allows researchers to integrate results from QUAL and QUAN parts of research which allows for confirmation & cross-validation of results Purposively drawn sample a subset of probability sample Data are collected at the same time
Formative evaluation
Process evaluation that is used to shape and refine program operations.
Strengths of Case Studies
Provides detailed (rich qualitative) information. Provides insight for further research. Permitting investigation of otherwise impractical (or unethical) situations.
Phenomenology
Purpose is to better describe and understand a phenomenon that can only be understood from the perspective of those that experience it (insider)
Grounded Theory
Purpose is to generate theory from the data that explains a process, action, or interaction
Exploratory Sequential Design
QUAL -> quan - Qualitative data are collected first and given more emphasis or attention than the quantitative data.
Convergent Parallel Design
QUAN + QUAL - Quantitative and qualitative data are given equal attention and emphasis and are collected concurrently throughout the same study.
Explanatory Sequential Design
QUAN -> qual - Quantitative data are collected first and are more heavily weighted than are qualitative data.
What is the sequence of MM sampling strategies?
QUAN->QUAL or QUAL->QUAN Information from the first sample needed to draw the second sample Commonly used in health and behavioural sciences research
What is exploratory?
Qual data and results -> building -> quan data and results -> compare/relate interpret
Explain relative timing:
Qualitative and quantitative data collection (concurrent of sequential)
Embedded mixed-methods design
Qualitative and quantitative methods are used concurrently in the research but one is given priority.
Integrated mixed-methods design
Qualitative and quantitative methods are used concurrently, and both are given equal importance
Staged mixed-methods design
Qualitative and quantitative methods are used in sequence in the research and one is given priority.
Interviews, focus groups, document analysis, participant observation
Qualitative data
What is purposive sampling?
Qualitative designs Selecting specific participants based on the purposes of the study Example: Research question aims to describe treatment experiences in younger and older depressed and non-depressed people Purposive sampling based on clinical diagnosis or cutoff on validated measure of depression
How are stakeholders' concerns addressed through decision of research methodology? (Mertens, 2007)
Qualitative research often answers questions of community perspective, whereas quantitative research provides specific outcomes. Combining both provide deeper/ more honest perspectives on an issue.
What is the context of qualitative research?
Qualitative researchers study things in their natural setting, attempting to make sense of, or to interpret, phenomena in terms of the meanings people bring to them.
Three types of research design (plan or proposal to conduct research, which involves intersection of philosophy, strategies of inquiry and specific methods).
Qualitative, Quantitative, Mixed Method
Explain what is meant by convergent design:
Quan and Qual data merge to form interpretation
What is explanatory?
Quan data & results -> follow up-> qual data& results -> compare/relate interpret
What is triangulation/convergent?
Quant (data and results) -> compare/relate interpret <- qual data and results
Concurrent Triangulation
Quantitative and qualitative research occurring simultaneously
Questionnaires, performance tests, heart rate, cytokines
Quantitative data
What is probability sampling?
Quantitative designs - Randomly select a large sample from a population so as to minimize the chance of drawing a representative sample
How can a researcher access or ensure trustworthiness of qualitative data? (Creswell & Plano, 2011)
Reliability and validity remain appropriate concepts for attaining rigor in qualitative research. Qualitative researchers must take responsibility for reliability and validity by implementing verification strategies integral and self-correcting during the conduct of inquiry itself. Qualitative Trustworthiness depends on the following (Nastasi et al., 2005): credibility (INTERNAL VALIDITY); transferability (EXTERNAL VALIDITY); dependability (RELIABILITY); and confirmability (OBJECTIVITY).
What is Reliability? Why is it important to report in your study?
Reliability is the degree to which an assessment tool produces stable and consistent results. While reliability is necessary, it alone is not sufficient. For a test to be reliable, it also needs to be valid.
Dictates Research Design
Research Question
Philosophical World Views Dictate the
Research Question
How do you link research questions to the mixed methods data analysis procedures? (Onwuebuzie & Leech, 2006)
Research goals/purpose shape research questions by taking these first steps of analysis : 1.) What is goal? 2.) What are objectives? 3.) What is your rationale for mixing methods per research questions? 4.) What sampling design will you use? 5.) Decision on design
What is a research paradigm?
Research paradigms associated with both sides of the debate, e.g. quantitative: positivism, qualitative: interpretivism or constructivism
Mixed methods
Research that combines qualitative and quantitative methods in an investigation of the same or related research question(s).
What is the epistemological stance of qualitative research?
Research using methods such as participant observation or case studies which result in a narrative, descriptive account of a setting or practice.
Convergent Parallel Design
Researcher collects quantitative and qualitative data at the same time, analyzes them separately, and then compares results to see if the findings confirm (or disconfirm)
Convergent
Researcher converges qualitative and quantitative data in order to provide a comprehensive analysis
Ethnography
Researcher immerses themselves in a culture by either being a participant observer or a total participant (researcher should remain detached enough to reflect accurately on experience)
How do multiple perspectives tie in with mixed methods?
Researcher needs to acknowledge the theoretical/philosophical views , identify their components, and then relate them to the mixed methods questions, as well as results.
Phenomenology
Researcher reflects on and identifies personal beliefs/biases about this phenomenon and "brackets" those beliefs so they will not interfere with understanding
Sequential
Researcher seeks to elaborate on or expand the findings of one method with the other
Multiphase Mixed Methods
Researchers conduct several mixed methods projects
How should a researcher determine which methods and data collection to use?
Researchers should aim for PERSUASIVE QUALITATIVE data procedures, and RIGOROUS QUANTITATIVE data analysis.
Quant/Qual research and theory
Researchers use theory in quant. res. to provide an explanation or prediction about the relationship among variables in the study. A theory explains how and why variables are related. in Qual res. theory is a broad explantation , may also be a theoretical lens that raises questions.
How is sample size selected in MMS?
Sample size calculation and/or expert judgment
Grounded Theory
Sample size is relatively large since you're trying to generate a theory (> 20 participants is not uncommon); looking for breadth vs. depth of knowledge
Conducting Mixed Methods Research (Sampling)
Sampling is a particularly important consideration when conducting a mixed methods study because sampling for each phase of the study (qualitative and quantitative) must be compatible with the assumptions belonging to that part of the design.
Transferability
Similar to external validity; can you relate the participants' experiences to your experiences?; can the findings be applied to other situations?
The Multistage Evaluation Design (Aim of study)
Stages can be done sequentially or concurrently; can be built off of any three basic designs. *Each stage can require qualitative, quantitative or both.
What is basic sampling strategy for MM?
Stratified purposive sampling Divide population of interest into strata (e.g. low income, middle income, high income) Use purposive sampling to select small number of people in each strata to describe similarities and differences across strata
Rheology
Study of deformation and flow of matter. HMA pavement that deforms and flows to much (not stiff enough) is susceptible to rutting and bleeding. If to stiff (does not deform much), pavement susceptible to fatigue cracking
Mixed Methods Research Design
The collection, analysis, and "mixing" of quantitative and qualitative research designs to understand a research problem.
Program process
The complete treatment or service delivered by the program.
Collecting case study data Analyzing case study data Reporting case study data Reflective reporting Analytic reporting
The data collected can be analyzed using different theories (e.g. grounded theory, interpretative phenomenological analysis, text interpretation, e.g. thematic coding) etc. All the approaches mentioned here use preconceived categories in the analysis and they are ideographic in their approach, i.e. they focus on the individual case without reference to a comparison group. The procedure used in a case study means that the researcher provides a description of the behavior. This comes from interviews and other sources, such as observation. The client also reports detail of events from his or her point of view. The researcher then writes up the information from both sources above as the case study, and interprets the information. Interpreting the information means the researcher decides what to include or leave out. A good case study should always make clear which information is factual description and which is an inference or the opinion of the researcher.
The Social Justice Design (Goal)
The goal of the researcher in implementing a social justice advanced design is to bring about positive change for an underrepresented group.
Content Analysis
The goal with content analysis is typically to quantify otherwise-qualitative data. Such quantification is often done to identify the qualitative content areas that are of highest importance or greatest prevalence.
Outcomes
The impact of the program process on the cases processed.
Emic Perspective
The insider's perspective
Substantive Level Theory & Components
The level of theory that is applied to a specific group or situation. Not grand theory. For a specific level of inquiry. Central phenomenon, causal conditions, strategies, conditions and context, & consequences
Qualitative research
The means for exploring and understanding the meaning individuals or groups ascribe to a social or human problem. Process involves emerging questions and procedures, data collected in the participant's setting, data analysis inductively building from particulars to general themes and researcher makes meaning of data. Report has flexible structure. Strategy of inquiry: Ethnography, Grounded theory, case study, phenomenological, narrative.
Literary Alternative Hypothesis
The more first-generation college students use student support services, the more they will persist academically.
Operational Alternative Hypothesis
The more that first-generation college students use the academic advising center, the more they will persist at college after their freshman year.
Theoretical Sampling Glazer & Strauss 1967
The process of developing sampling based on emerging theory of study. To select participants based on addressing areas of need from study and may increase sampling size according to need of direction of study growth. Also use deviant sampling to interview someone different from your sample that may challenge your theory not just support already made conclusions. Search for observational data is active and dynamic to emerging theory until saturation. Begins first with homogenous sampling then after theory move to heterogenous sampling to confirm or disconfirm theory.
Glaser Grounded Theory
The process: basic description, conceptual coding, theorising
Script for a quantitative purpose statement
The purpose of this ......................... [experiment? survey?] study is [was? will be?] to test the theory of ......................... that ......................... [describes outcomes] or [compares? relates?] the ......................... [independent variable] to ......................... [dependent variable], controlling for ......................... [control variables] for ......................... [participants] at ......................... [research site]. The independent variable(s) ......................... will be defined as ......................... [provide a definition]. The dependent variable(s) ......................... will be defined as .................... [provide a definition], and the control and intervening variable(s) ......................... [identify the control and intervening variables] will be defined as ......................... [provide a definition].
Script for a qualitative purpose statement
The purpose of this ......................... [strategy of inquiry, such as ethnography, case study, or other type] study is [was? will be?] to ......................... [understand? explore? develop? discover?] the ......................... [central phenomenon being studied] for ......................... [the participants, such as the individual, groups, organization] at ......................... [research site]. At this stage in the research, the ......................... [central phenomenon being studied] will be generally defined as ......................... [provide a general definition].
What is a sequential mixed methods design?
The researcher seeks to elaborate on or expand the findings of one method with another method Explanatory: priority is on QUAN data collection and analysis Triangulation: Priority is on both QUAN and QUAL data Exploratory: Priority is on QUAL data collection and analysis
Inputs
The resources, raw materials, clients, and staff that go into a program.
Outputs
The services delivered or new products produced by the program process.
Culture
The shared beliefs, values, practices, language, norms, rituals, and material things that the members of a group use to interpret and understand their world
Phenomenology
Theory created by philosophers
Ethnography
Theory introduced by sociologists
Grounded Theory
Theory introduced by sociologists
Study purpose according the creswell + Script
There is a need for a theory on this topic that explains this process because exisiting literature is incomplete, or in need of modification" The purpose of this grounded theory study is to develop/generate a substantive theory of the process of (ex developing health literacy) for (population) at (site). At this stage in the research, (health literacy) will be generally defined as (give general definition)
Literary Null Hypothesis
There is no relationship between student support services and the academic persistence of first generation college students.
Operational Null Hypothesis
There is no relationship between the number of hours first generation students use the academic advising center and their persistence at college after their freshman year.
Example of Mixed Methods Purpose Statement
This study addresses ___________ [content aim]. The purpose of this exploratory sequential design will be to first qualitatively explore with a small sample and then to determine if the qualitative findings generalize to a large sample. The first phase of the study will be a qualitative exploration of ___________ [the central phenomenon] in which ___________ [types of data] will be collected from ___________ [participants] at ___________ [research site]. From this initial exploration, the qualitative findings will be used to develop assessment measures that can be administered to a large sample. In the tentatively planned quantitative phase, ___________ [instrument data] will be collected from ___________ [participants] at ___________ [research site].
Challenges of Mixed Methods
Timelines and resources, sampling and analysis decisions, blending philosophical world views
What is the purpose of qualitative research?
To find out how people make sense of their world and the experiences they have in the world.
Post-positivism
Traditional research form, als called scientific method, or postitivist research, and empirical science. It challenges the notion of absolute truth of knowledge, we cannot be "positive" about claims of knowledge. Based on Comte, Mill, Newton, Locke and Durkheim. Problems studied reflect need to identify causes that influence outcome. It is reductionistic, meaning to reduce idea into small discrete set of ideas to test. Knowledge gathered is based on observations. As laws and theories govern the world, these need to be tested and verified. Begins with theory, data collection supporting or refuting theory. Instead of saying they proved a theory they "fail to reject the hypothesis".
What occurs during data transformation?
Transformation/conversion of one type of data into another so that both can be analyzed together Qualitative data are coded and proportion of people endorsing each theme calculated (qual-> quant) Quantitative data are transformed into narrative and included with qualitative data in thematic analysis (quant-> qual)
Strengths of Mixed Methods
Triangulation, comprehensiveness, enhancing generalization
Qualitative analysis
Two Core Approaches are: 1) Content Analysis 2) Thematic Coding
Mixed Methods
Type of research in which a researcher or team of researchers combines elements of qualitative or quantitative research approaches for the broad purposes of breadth and depth of understanding and collaboration
Typical
Type of sampling in which you look for typical characteristics of the groups that you're wanting
Unique/examplary/extreme
Type of sampling in which you may be looking for non-traditional students or students with a 4.0 in undergrad or something
Maximum variation
Type of sampling in which you mix it up in terms of education, location, undergrad, wait or list, for example...
Phenomenology
Typically a small sample size; depth of knowledge vs. breadth of knowledge
Qualitative Purpose Statements
Use action verbs such as describe, understand, develop, examine the meaning of, or discover to convey how learning will take place Use neutral words and phrases, nondirectional language such as "self-expression experiences of individuals" Provide a general working definition of the central phenomenon or idea, particularly those not typically understood by a broad audience Include words that indicate the strategy of inquiry to be used in data collection, analysis and the process of research Mention the participants in the study Identify the site for the research Delimit the scope of participation or research sites in the study
Main Purpose of Mixed Methods Research
Use advantages of both quantitative and qualitative research designs.
Triangulation
Use of multiple sources, bringing in more viewpoints
Asphalt binder with lower stiffness
Used to increase a mixture's fatigue life by providing greater flexibility.
What are the features of mixed methods?
Uses rigorous methods that are both qualitative and quantitative - Specific types of mixed method designs for the combination and interpretation of qualitative and quantitative data - Design may be framed within a theory/broader framework
Grounded Theory
Uses the constant comparative model method of data analysis, which is "iterative". It starts when first data is collected. New data continually informs data collection process and initial development of a theoretical model. Data collection continues until "saturation" is reached.
What is triangulation?
Using different methods to measure the same phenomenon and increase confidence in the conclusions reached.
What is validity? Why is it important to report in your study?
Validity refers to how well a test measures what it is purported to measure. More specifically, validity applies to both the design and the methods of your research. Validity in data collection means that your findings truly represent the phenomenon you are claiming to measure. Valid claims are solid claims. Validity is the best available approximation to the truth of a given proposition, inference, or conclusion. Measures, samples and designs don't 'have' validity -- only propositions can be said to be valid. Technically, we should say that a measure leads to valid conclusions or that a sample enables valid inferences. It is a proposition, inference or conclusion that can 'have' validity.
Quantitative theory
Variables (aka constructs) (characteristic of an individual or organization than can be measured or observed and varies among people). Differ in temporal order ( one variable precedes another in time..one variable causes another). Independent probably cause outcome. Dependent depend on independent variable and are the result aka criterion/outcome/effect. Intervening variable/mediating v. stands between dependent and independent v. Moderating variables are constructed by researcher by taking one variable and multiplying it by another to determine the joint impact of both: age X attitudes toward quality of life)
What is sample size of MMS?
Varies
Explain weighting:
Weight or priority given to quantitative or qualitative research (equal or one given greater emphasis)
The Multistage Evaluation Design (Use)
When a researcher wants to evaluate the impact of a program using both qualitative and quantitative data collection techniques.
What is expansion?
When different methods are used to assess different phenomena to expand scope/range of study.
What is complementarity?
When different methods are used to investigate different aspects/dimensions of the same phenomena to deepen/broaden interpretations and conclusions.
What is initiation?
When different methods are used to investigate different aspects/dimensions of the same phenomenon.
What is development?
When results from one method are used to inform development of other methods.
When to use grounded theory? Creswell 2007
When there is no theory yet to use at all, not far enough reaching, or was based on populations other than who you'd want tested. Also theory may be needed to explain how people experience a phenomenon. Creswell says phenonmenology is better for policy and practice needs however I would disagree and say that GT is better for creating content for policy vs phenom which helps give rise to need for the policy or the evaluation of a policy but does not provide information useful for policy development strategies or actors.
Concurrently
When two or more things happen at the same time, they occur concurrently. If the concerts you want to attend are happening concurrently, you have to choose one — unless you know how to be in two places at the same time.
Packer critique (Wittgenstien)
Wittgenstien - words have only applied meaning, we use patterns to give words very context-specific meanings. Cannot see words as distinct from activity and context
Conducting Mixed Methods Research (Research Question)
Your research question(s) dictate whether a mixed methods study is appropriate for your research, and the decision about whether one type of data will be primary and one will be secondary determines the basic design.
Abstract
a brief review of the literature that summarizes major elements to enable a reader to understand the basic features of the article.
instrumental case study
a case study in which a specific case is studied only to gain an understanding of a broader phenomenon or event
intrinsic case study
a case study that provides in-depth description of a particular individual, organisation or event
hypothesis
a declarative statement of the relationship between two or more variables
Transformative mixed methods
a design that uses a theoretical lens drawn from social justice or power as an overarching perspective within a design that contains both quantitative and qualitative data.
Convergent parallel mixed methods
a form of mixed methods design in which the researcher converges or merges quantitative and qualitative data in order to provide a comprehensive analysis of the research problem.
Fieldwork
a general term for data collection in ethnographic research
Purposeful Sample
a group of individuals who have experience with the phenomenon under study
purposeful sample
a group of individuals who have experience with the phenomenon under study
Triangulation
a major purpose of mixed methods research that seeks convergence, correspondence, and corroboration of results from different methods
Complementarity
a major purpose of mixed methods research that seeks elaboration, enhancement, illustration, and clarification of the results from one method with the results from the other method
Initiation
a major purpose of mixed methods research that seeks the discovery of paradox and contradiction, new perspectives and new frameworks, and the recasting of questions or results from one method with questions or results from the other method
Expansion
a major purpose of mixed methods research that seeks to extend the breadth and range of inquiry by using different methods for different inquiry components
Development
a major purpose of mixed methods research that seeks to use the results from one method to develop or inform the other method, where development is broadly construed to include sampling and implementation as well as measurement decisions
researcher as detective
a metaphor applied to researcher looking for the local cause of a single event
Grounded theory
a methodology of systematic but flexible guidelines for collecting and analysing data to construct theories 'grounded' in the data themselves
Multiphase timing
a multiphase project that includes several sequential and/or concurrent substudies over a program of study
life world
a person's subjective inner world of experience
Ethnography
a qualitative design of inquiry coming from anthropology and sociology in which the researcher studies the shared patterns of behaviors, language, and actions of an intact cultural group in a natural setting over a prolonged period of time.
Phenomenological Research
a qualitative design of inquiry coming from philosophy and psychology in which the researcher describes the lived experiences of individuals about a phenomenon as described by participants.
Case Study
a qualitative design of inquiry found in many fields, especially evaluation, in which the researcher develops an in-depth analysis of a case, often a program, event, activity, process, or one or more individuals.
Grounded Theory
a qualitative design of inquiry from sociology in which the researcher derives a general, abstract theory of a process, action, or interaction grounded in the views of participants.
Narrative Research
a qualitative design of inquiry from the humanities in which the researcher studies the lives of individuals and asks one or more individuals to provide stories about their lives.
phenomenology
a qualitative research method that attempts to describe a participant's "subjective" experience; e.g. their emotions, feelings, reactions or thoughts. Akin to getting inside their head. - data collection: interviews and open-ended questionnaires - analysis/reporting: detailed narrative
ethnography
a qualitative research method that attempts to discover and describe the "culture" of a person or group of people - data collection: in-depth interviews and participant observation - analysis/reporting: descriptive narrative
case-study research
a qualitative research method that involves an intensive and detailed description and analysis of one or more cases, and the environment in which they exist/occur. - data collection: multiple sources - interviews, documents, questionnaires, tests - analysis/reporting: each case is intensively analysed as a separate entity that is then integrated into the broader literature
grounded theory
a qualitative research method used specifically to discover or generate a theory from a set of empirical data. Used mainly in social, behavioural, and clinical sciences - data collection: interviews and observations - analysis/reporting: (analysis) 3 stage process of open coding, axial coding, selective coding (reporting) descriptive and detailed
Experimental Research
a quantitative method that seeks to determine if a specific treatment influences an outcome. The researchers assesses this by providing a specific treatment to one group and withholding it from another and then determining how both groups scored on an outcome.
mixed methods research
a research method that uses quantitative and qualitative techniques in the same study
Confounding Variables
a spurious variable that is not actually measured or observed in a study.
collective/comparative case study
a study of two or more cases for the purpose of comparison
Sample
a subset of the larger population
explanatory design
a two phase mixed methods design in which qualitative data help explain initial quantitative results. this design can be ideal for researchers who are looking for additional qualitative data to explain quantitative results
exploratory design
a two-phase design where qualitative results are obtained first, followed by the quantitative in order to inform the qualitative data. this design is used when there are no measurements or instruments available to guide the study, when the variables are unknown, and when the study has no guiding framework or theory in use.
qualitative research
a type of empirical research that relies on qualitative data such as words, pictures and images. Focuses on: - individual people - subjective, non-generalisable results - intensive case studies
naturalistic generalisation
a type of generalisation based on the context of a study and similarity of participants
theoretical generalization
a type of generalisation in which theoretical concepts derived from the study can be used to develop further theory
inside-outside validity
a type of mixed methods validity referring to the degree to which a researcher accurately understand and presents the participants' subjective view as well as the objective outsider view
weakness-minimisation validity
a type of mixed methods validity that refers to the degree to which the researcher combines qualitative and quantitative approaches to compensate for the weaknesses in each
Mixed Methods Research Design
a type of research design where the researcher employs both quantitative and qualitative research methods to collect and analyze data, and report research findings in a single study
nomological causation
a view of causation that seeks to determine generalised, non-specific relationships between variables
idiographic causation
a view of causation that seeks to determine what caused a particular event in a particular situation at a particular time
Literature Map
a visual picture of groupings of the literature on the topic that illustrates how your particular study will contribute to the literature, positioning your study within the larger body of research.
null hypothesis
a written statement that predicts that there is no difference, or no relationship, between groups related to a particular variable
Null Hypothesis
a written statement that predicts that there is no difference, or no relationship, between groups related to a particular variable.
Interpretive Validity
accurately portraying the participants' subjective viewpoints and meanings
Disability Inquiry
addresses the understanding this population's sociocultural perspectives allowing them to take control over their lives rather than a biological understanding of disability.
dialectical approach
advocates a dynamic back-and-forth listening to multiple perspectives and multiple forms of data
Independent variable
affect outcomes (treatment, etc)
Important characteristics in aggregate selection for HMA mix design
aggregate source, gradation and size, toughness and abrasion resistance, durability and soundness, shape and texture, and even cleanliness
Glaser and Strauss grounded theory
aimed to mix strengths of quantitative research with qualitative approaches.
Convergent design
aka: triangulation design obtains different but complementary data about the central phenomenon under study qualitative and quantitative data are collected simultaneously and with equal priority
reflexivity
an ability to think critically about one's interpretations and biases. Can reduce researcher bias in qualitative studies.
Idiographic Causation
an action for a particular person in a local situation with an observable result
qualitative
an approach for exploring and understanding the meaning individuals or groups ascribe to a social or human problem
Qualitative Research
an approach for exploring and understanding the meaning individuals or groups ascribe to a social or human problem.
quantitative
an approach for testing objective theories by examining the relationship among variables
Quantitative Research
an approach for testing objective theories by examining the relationship among variables.
Mixed Methods Research
an approach to inquiry involving collecting both quantitative and qualitative data, integrating the forms of data, and using distinct designs that may involve philosophical assumptions and theoretical frameworks.
Research focus (mixed methods research)
an assumption that includes a multilens focus
View of Thought and Behavior (mixed methods research)
an assumption that includes dynamic, complex, partially predictable, multiple influences (i.e. environment/nature, biology/nature, free will/agency, and chance/fortuity
Ontology- Nature of Reality/truth (mixed methods research)
an assumption that includes pluralism, appreciation of objective, subjectivism, and intersubjective realities with their interrelations
Epistemology - Theory of Knowledge (mixed methods research)
an assumption that includes pragmatism, pragmatic justification of what works for whom in specific contexts, and a mixture of universal and community-specific needs-based standards
incompatibility thesis
an either-or position claiming mixing research methods is not possible or allowable (a problem is its failure to recognize that creative and thoughtful mixing of assumptions, ideas, and methods can be very helpful and can often best address your research question(s).
Research Design
an outlined plan for data collection and analysis; the purpose of the study determines the design that will be used
research design
an outlined plan for data collection and analysis;the purpose of the study determines the design that will be used
Inference quality
an overarching criterion for evaluating the quality of conclusions and interpretations made on the basis of mixed methods findings; incorporates notions from both internal validity and statistical conclusion validity within a quant framework and credibility within a qual framework; refers to the believability and accuracy of the inductively and deductively derived conclusions from an MM stud
Descriptive Questions
are written for each independent and dependent variable and any significant mediating variables in the study.
Optimum asphalt binder content
asphalt binder content that results in 4 percent air voids.
Social Constructivists
believe that individuals seek understanding of the world in which they live and work. These individuals develop subjective meanings of their experience.
single-phase approach
both qualitative and quantitative questions can be identified as one set of questions that does not rely on the others
intramethod mixing
both quantitative and qualitative data are obtained through the creative use of a single method of data collection
Variable
characteristic of an individual/organization that can be measured or observed (aka also known as construct)
Multiphase mixed methods
common in the fields of evaluation and program interventions; concurrent or sequential strategies are used in tandem over time to best understand a long-term program goal.
multivariate questions
compare or relate variables
Critical Theory
concerned with empowering human beings to transcend the constraints placed on them by race, class, and gender.
Equal status vs dominant status
concerns whether a particular type of data will be emphasized more or equally to the other type of data
pragmatism
consequences of actions, problem centered, pluralistic, real-world practice oriented
Pattern Matching
construction and testing of a complex hypothesis
charmaz
constructivist grounded theory
mixed methods research design
convergent exploratory sequential transformative, embedded or multiphase
What is a concurrent mixed method design?
converges quantitative and qualitative data in order to provide a comprehensive analysis of the research problem collects both forms of data at the same time during the study and then integrates the information in the interpretation of the overall results nests one form of data within another, larger data collection procedure in order to analyze different questions
Quantitizing
converting qual data into various codes than can be analyzed quantitatively
Coding process of constructivist grounded theory
create descriptive codes, line by line. Raise codes to concepts through drawing on theoretical knowledge, rereading data and comparing and contrasting the codes.
Grounded theory v quantitative methods
critiques 'arm chair theorising', and also challenges assumption that theory can only come from theorising or from a deductive approach
Key features of Glaser and Strauss
data collection and analysis occur at the same time; codes and categories constructed from the data; constant comparison; memo-writing to elucidate categories; non-random theoretical sampling; theoretical saturation; completing the literature review after analysis
Embedded Design
data collection may be either concurrent or sequential. One form of data is embedded within another form, thus one form is supportive of the other.
sequential time orientation
data obtained from the sample during the first phase of the study are used to shape or structure the sample selection of the next phase of the study
Validity strategies used in research
data triangulation, extended fieldwork, external audit, investigator triangulation, low inference descriptors, methods triangulation, negative case sampling, participant feedback,
Theoretical Validity
degree to which theory or explanation fits data
qualitative sampling
depends on the purpose of the study and the information that will be most useful, the larger the number of participants, the less depth and detail that can go into the data collection with each, the number is based on this approach used in the study
Essence
description or structure of an individual(s) experience
Peer Review
discussing your interpretations with one's peers and colleagues
Summary: Glaser
emergent concepts
quantitative research questions
emphasizes measurement and variables, questions are directional, reflect the relationship variables, but place this information in the form of a question
Inference transferability
encompasses the quan term external validity and the qual term transferability; the degree to which the MM conclusions can be applied to other similar people, context, settings, time periods and theoretical representations of the phenomenon
benefits of mixed methods design
expand understanding, help clarify and assist in answering difficult questions, allows researcher to use two types of data to verify findings, produce richer data
Achieve Theoretical Validity
extended fieldwork; theory triangulation; pattern matching; peer review
Finding a sample size that is "just right"
finding the ideal sample size depends on the method of data collection in use.
Open Coding
first stage of data analysis in GT and its the most exploratory stage
Corbin and strauss grounded theory
focused more on verification with strict coding process. advocated going back and forward between inductive and deductive process.
Queer Theory
focuses on individuals calling themselves lesbians, gays, bisexuals, or transgendered people.
Ethnography
focuses on the discovery and description of the culture of a group of people
Tenant of constructivist grounded theory Charmez (2006)
from symbolic interactionist perspective, observer cannot be neutral. researcher is acknowledged as an active player in analysis and approaches field with theoretical perspective
Examples of variable in a study
gender, age, SES, atittudes, racism, political power etc.
Naturalistic Generalization
generalization based on similarity, made by the reader of a research report
Theoretical Generalization
generalization of a theoretical explanation beyond the particular research study
critique of grounded theory
glaser and strauss have strong positivist thread run through their work. it glosses over epistemological assumptions in minimising relationship to sociological theory
Gatekeepers
group members who control a researcher's access to the group
Postpositivists
hold a deterministic philosophy in which causes determine effects or outcomes. These problems reflect the need to identify and assess the causes that influence outcomes.
Transformatives
holds that research inquiry needs to be intertwined with politics and a political change agenda to confront social oppression at whatever level it occurs.
transformative worldview
holds that research inquiry needs to be intertwined with politics and political change agenda to confront social oppression at whatever level it occurs
3 types/forms of theories in quantitative research
hypotheses if-then logic statements visual models
Case studies are
in-depth investigations of a single person, group, event or community. Typically, data are gathered from a variety of sources and by using several different methods (e.g. observations & interviews). The research may also continue for an extended period of time, so processes and developments can be studied as they happen. The case study is not itself a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies. Most of this information is likely to be qualitative (i.e. verbal description rather than measurement) but the psychologist might collect numerical data as well.
qualitative research questions
includes broad, open ended questions that answer the how or what about a particular phenomenon. example; how does college counseling impact the coping mechanisms of college freshman?
Moderating Variables
independent variables that affect the direction and/or the strength of the relationship between independent and dependent variables. Typically found in experiments.
Onwuegbuzie and Johnson's Nine Mixed Research Validities
inside/outside, paradigmatic/philosophical, commensurability approximation, weakness minimization, sequential validity, conversion validity, sample integration, sociopolitical, and multiple
Qualitative methods
instead of relying on numbers, counts, and frequency-type data, qualitative research will often involve the collection and analysis of detailed observations, stories or narrative histories, sounds, picture or video. • Qualitative methods often bring a fresh or new perspective to existing research in fields of science that have been dominated by quantitative methods. • When combined with quantitative techniques, qualitative strategies can often help researchers to more strongly support their research design choices and final inferences.
meta-inferences
integrative inferences or conclusions based on qualitative and quantitative data and findings
Modified stem leaf plot
internality behaviors from unstructured data sources are listed on one side and the identification numbers from individuals who exhibited those behavior were listed on the right
charmaz
interpretive, flexible, acknowledge taken for granted processes and researchers as co-creating meaning.
Definition of a theory
interrelated set of constructs formed into propositions (hypotheses), that specify the relationship among variables.
Where are theories placed in quantitative studies
introduction lit review after hypotheses/research questions as a rationale in a seperate section
Action Research
investigation or analysis that is directed in the direction behind a useful objective, typically a growth in a specific function or system, in comparison to strictly trial and error studies. In company growth and advancement, it comprises not just methodically gathering information regarding a company but additionally supplying evaluations to the company, taking actions to improve the company mainly based on the evaluations, and consequently reviewing the feedback behind these behaviors, ACTION RESEARCH: "The action research methods would lead scientists to provide further proof of the drug's efficacy."
Embedded mixed methods
involves as well either the convergent or sequential use of data, but the core idea is that either quantitative or qualitative data is embedded within a larger design and the data sources play a supporting role in the overall design.
Iterative analysis
involves ongoing interpretive feedback loops
A multilevel relationship
involves selecting samples from different levels of hierarchy, usually this means sampling from different but related populations
Inductive reasoning (Qualitative Research)
is a logical process in which multiple premises, all believed true or found true most of the time, are combined to obtain a specific conclusion. Inductive reasoning is often used in applications that involve prediction, forecasting, or behavior.
Quantitative research
is also typically designed to test predetermined hypothesis that are formed on the based on existing theory (a deductive process), while qualitative research often functions to develop theory from the data that are collected (an inductive process). With these two distinctions in mind it is also often suggested that qualitative research tends to focus more on rich description of a phenomenon than on its qualifications.
A case study
is one type of observational data collection technique in which one individual is studied in-depth in order to identify behavioral, emotional, and/or cognitive qualities that are universally true, on average, of others. Case studies often include face-to-face interviews, paper and pencil tests, and more. Sigmund Freud conducted case studies.
Ethnocentric
judgment of people in other cultures based on the standards of your culture
Memo writing
key method of organizing thoughts and emergent theory progresses
Validity of qualitative research is often questioned because
lack of rigor findings are largely dependent on the researcher (researcher bias)
Further Packer critique
line by line is not how words are experienced; arbitrary to split line by line instead of by meaning; glaser's reliance on emergence underplay's researchers role in interpreting; bracketing assumptions problematic
Sequential Validity
making sure that the ordering of quantitative and qualitative components in a sequential design does not bias the results
Multiple Validities
making sure your mixed methods study meets appropriate quantitative, qualitative, and mixed methods validity types
Quantitative research
means for testing objective theories by examining the relationship among variables. Data is analyzed using stats. Final written report has a set structure consisting of introduction, literature and theory, methods, results and discussion. Strategy of inquiry: Survey, experimental
Participant Feedback
member checking to see if participants agree with the researcher's statements, interpretations, and conclusions
Researcher - as Detective
metaphor applied to researcher looking for the local cause of a single event
Case Study Research
method in which the researcher provides a detailed description and account of one or more cases
Grounded Theory
methodology for generating and developing a theory that is grounded in the particular data
Data Collection
most common methods of data collection are interviews and observations
axial coding
moving from abstraction to reconnection by connecting categories to other categories. focusing on one open code as a core phenomenon and creates coding paradigm/logic paradigm (visual model of core, cause, strategies, intervening conditions & consequences)
qualitative research design
narrative research phenomenology grounded theory ethnographies case study
Qualitative Research
non-numeric research such as words, pictures, images, etc. . .
Reactive Effect
non-typical behavior of participants because of the presence of the researcher
Fieldnotes
notes taken by the researcher during (or immediately after) one's observations in the field
Non interview forms of data collection
observations, documents, audiovisual
An identical relationship
occurs when exactly the same people are in both components of the study
Theoretical Saturation
occurs when no new information relevant to the GT is emerging from the data and the GT has been sufficiently validated
Explanatory sequential mixed methods
one in which the researcher first conducts quantitative research, analyzes the results, and then builds on the results to explain them in more detail with qualitative research.
paradigm emphasis
one of the two dimensions of a mixed methods design Paradigm emphasis has two levels: 1. equal status (the qual and quan parts are given equal emphasis; e.g. QUAL+QUAN) 2. dominant status (either the qual OR quan part is given primary emphasis; e.g. qual+QUAN)
time order
one of the two dimensions of a mixed methods design Time order has two levels: 1. concurrent (the qual and quan parts are conducted concurrently; e.g QUAL + QUAN) 2. sequential (the qual and quan parts occur one after the other; e.g. QUAL > QUAN)
Time Order
one of two dimensions used in MM design matrix and its levels are concurrent and sequential
Paradigm Emphasis
one of two dimensions used in MM matrix and its levels are equal status and dominant status
mediating variable
one that mediates the effect of the independent variable on the dependent variable
Temporal order
one variable affects/causes another variable
Researcher bias
only noticing data that support one's prior expectations
Substantive coding part 1
open coding or fracturing. immersion in the words of the data. line by line analysis generating as many codes as possible. constantly comparing until possible overarching theme emerges
Strauss and corbin process
open coding, axial coding, selective coding
Objectives of HMA mix design
optimize deformation resistance, fatigue resistance, low temperature cracking resistance, durability, moisture damage resistance, skid resistance, and workability
Dependent variable
outcomes/results of the influence of the independent variable
Going Native
over-identification with the group being studied so that one loses any possibility of objectivity
nested sample relationship criterion
participants selected for one phase represent a subset of participants selected for another phase
Significant statements
participants words, phrases, or sentence length that vividly communicates the participants experience
4 major qualitative research methods
phenomenology, ethnography, grounded theory, case study research
Pragmatism
philosophy focusing on what works as the criterion of what should be viewed as tentatively true and useful in research and practice
embedded design
places one type of data in the lead role with the other type of data acting as a "supporting" actor. is used when researchers need to answer questions from a different perspective and their research has primarily focused on one type of data. useful when the researcher needs to insert a qualitative component within a quantitative design
transformative
political, power and justice oriented, collaborative, change-oriented
Compatibility Thesis
position that quantitative and qualitative research methods and philosophies can be combined
inside-outside validity
present when the researcher both the inside and objective outsider perspectives
Weakness- Minimization Validity
present when the researcher compensates for the weaknesses of one approach through the use of an additional approach
Warranted assertibility
present when you have good evidence about your research claim (John Dewey)
aims of constructivist grounded theory
prioritises the studied phenomenon rather than the methods of studying it. Researcher looks at their own assumptions and implicit meanings. More interpretive, meaning co-created.
research designs
procedures of inquiry
Style manuals
provide guidelines for creating a scholarly style of a manuscript, such as a consistent format for citing references, creating headings, presenting tables and figures, and using nondiscriminatory language.
Survey Research
provides a quantitative or numeric description of trends, attitudes, or opinions of a population by studying a sample of that population.
survey research
provides a quantitative or numeric description of trends, attitudes, or opinions of a population by studying a sample of that population.
Mixed methods research
provides a way to investigate the quantitative and the qualitative aspects of human thought and behavior in its various contexts
What is the purpose of MMS?
purpose: generate sample that will address research question
What is incompatibility thesis?
qualitative and quantitative research paradigms, including their associated methods, cannot and should not be mixed Pragmatism has been suggested
Interpretive integration
quan data analyzed in quan form, qual data analyzed in qual form, interpretation developed by comparing the 2 sets of data, interpretation is simply at a narrative level
Parallel databases variant
quan data are collected and analyzed in parallel with the collection and analysis of qual data the 2 separate analyses are compared and contrasted goal is to develop internally confirmed conclusions about a single phenomenon
compatibility thesis
quantitative and qualitative approaches can be used together in a single research study as long as researchers respect the assumptions associated with quantitative and qualitative research and construct a thoughtful combination that will help to address their research question(s)
Mixed method research
quantitative and qualitative data techniques are combined in a single research study
Research Question
questions that the researcher seeks to answer.
Racialized Discourse
raises important questions about the control and production of knowledge, particularly about people and communities of color.
What does sand equivalent test measure
rapid field test that shows relative proportions of fine dust or clay like materials in fine aggregate or granular soils. Excessive dust or plastic fines (clay-like fine particles) in HMA aggregate can contribute to a lack of stability (rutting or shoving) or moisture damage and stripping. Higher sand equivalent values indicate cleaner or less fine dust or clay like aggregate.
paradigm/research-approach emphasis
refers to whether the qualitative and quantitative parts of the study are given approximately equal emphasis, or if one part is considered primary and more strongly emphasized
Delaying literature review
reinventing the wheel
operational form
represent specific information about the variables in the study
literary form
represent variables in the abstract
quantitative research
research that employs the use of predetermined instruments in order to measure and observe; some aspects include experiments, and the collection of survey data via predetermined instruments. data are in numerical form
Quantitative Research
research that employs the use of predetermined instruments in order to measure and observe; some aspects include: experiments and the collection of survey data via predetermined instruments. Data is in numerical form.
Phenomenology
researcher attempts to understand and describe how one or more participants experience a phenomenon
Participant Observation
researcher becomes an active participant in the group being investigated
convergent design
researcher collects both QUAN & QUAL data usually at the same time, see results of both, merge for hope of same conclusions about phenomenon
What is a mixed methods study?
researcher tends to base knowledge claims on pragmatic grounds (e.g. consequence oriented, problem-centered, and pluralistic). - It employs strategies of inquiry that involve collecting data either simultaneously or sequentially to best understand research problems. - The data collection also involves gathering both numeric information (e.g. on instruments) as well as text information (e.g. on interviews) so that the final database represents both quantitative and qualitative information
Sample Integration Validity
researchers must not treat the quantitative and qualitative samples as equals, but, instead, draw appropriate conclusions from each sample
Problems with to high asphalt binder content
results in mix that has greater tendency to deform elastically rather than fracture under repeated loading. However, needs to be high enough to prevent excessive fatigue cracking
identical sample relationship criterion
same people participate in both the quantitative and qualitative phases of investigation
quantitative sampling
sample needs to be large enough to perform the statistical procedures you use to make inferences about study findings
parallel sample relationship criterion
samples for quantitative and qualitative components are different, but drawn from the same population
negative case-sampling
searching for case studies that challenge or disprove one's expectations or general findings. Used to reduce researcher bias.
Negative Case Sampling
searching for cases that challenge one's expectations or one's current findings
Axiel coding
second stage of data analysis in GT its focus on making concepts more abstract and ordering them into the theory.
experimental research
seeks to determine if a specific treatment influences an outcome
Data transformation variant
separate but concurrent quan and qual data collection and then analysis transforms the qual data into quan form and then comparing and interrelating the data sets
Exploratory designs
sequential designs with qual data collected in the first phase usually the first phase is focused on detailed exploration of a little-researched phenomenon, and the second phase is focused on measuring or classifying it
Explanatory designs
sequential designs with quan data collected in the first phase, followed by qual data in the second phase either types of data can be given a stronger priority data from the 2nd phase are used to build on or describe data from the first phase
Theory
set of variables/constructs prevent a view of a phenomenon w/purpose of explaining it
embedded design
similar to convergent design but one general approach dominates
Control Variables
special type of independent variable that researchers measure because they potentially influence the dependent variable.
Extended Fieldwork
spending enough time in the field to fully understand what is being studied
extended fieldwork
spending enough time in the field to fully understand what is being studied
Mediating variables
stan between indepnd/depend variables
Intervening or Mediating Variables
stand between the undefended and dependent variables, and they mediate the effects of the independent variable on the dependent variable.
selective coding
starting from a core category, examine how it links to other categories. Storyline or hypothesis which explains relationships of paradigm.
Shared beliefs
statements or conventions that people sharing a culture hold to be true or false
Methodology
strategy, process or design lying behind the choice of particular methods and linking these chosen methods to a desired outcome
Collective Case Study
study of multiple cases for the purpose of comparison
Collective Case Study Example
study of three individuals with intellectual disabilities who are placed in a general education class.
validity, trustworthiness, legitimation
synonyms used in quantitative, qualitative, and mixed method research respectively
Constant Comparative Method Glazer & Strauss 1967
taking data from new collection and comparing to emergent categories
Problems with flat and elongated particles in mix
tend to break down during compaction, decrease workability, and lie flat after compaction
External Validity
the ability to generalize the findings to other people, settings, and times
Methodology
the application of particular procedures toward acquiring the answers to a wide variety of research questions
methodology
the application of particular procedures toward acquiring the answers to a wide variety of research questions
Etic perspective
the behavioral research approach is across various cultures to learn about the universality of behaviors
Meta-inference
the conclusion generated by integrating inferences obtained from the results of qualitative and quantitative strands of an MM study
Sociopolitical Validity
the degree a mixed researcher addresses the interests, values, and viewpoints of multiple stakeholders in the research process
internal validity
the degree to which a researcher can draw cause-and-effect inferences from a study
external validity
the degree to which a researcher can generalise the results of a study to a wider population. The least-used type of validity in qualitative research.
theoretical validity
the degree to which a researcher's explanations or theory about a phenomenon fits the data
Commensurability Approximation Validity
the degree to which meta-inferences made in a mixed research study reflect a mixed worldview
Conversion Validity
the degree to which quantitizing or qualitizing yields high-quality meta-inferences
Paradigmatic / Philosophical Validity
the degree to which the mixed researcher clearly explains his or her philosophical beliefs about research
Sample Integration Validity
the degree to which the mixed researcher makes appropriate conclusions, generalizations, and meta-inferences from mixed samples
Inside - Outside Validity
the degree to which the researcher accurately understands, uses, and presents the participants' subjective insider (emic) views and the researcher's objective outsider (etic) view
descriptive validity
the extent to which an account reported by a researcher is factually accurate
interpretive validity
the extent to which participants' viewpoints and meanings are accurately portrayed in a study
Multiple Validities
the extent to which the pertinent validities (quantitative, qualitative, and mixed) are addressed and resolved successfully
Descriptive Validity
the factual accuracy of the researcher's account
theoretical saturation
the final stage of grounded theory when no new concepts emerge from the data, the theory makes sense, and it is well validated
In thematic coding Example: The meaning of ethnic identity was explored, with an emphasis on how this was associated with student's adjustments to college in ethnically diverse college settings.
the first task for a researcher is typically to get familiar with the data that have been collected. By sorting and labeling responses into thematic categories, the qualitative research can begin to make sense of the rich information that has been provided. • The goal in thematic coding is not necessarily to quantify the number of responses per category (although that is always an option), but rather to identify which categories are most prevalent and/or important to respondents and then to dig into the responses in those categories to start to make sense of whatever phenomenon is under study.
compatibility thesis
the idea that quantitative and qualitative research methods are compatible and can be used together in a single study
research problem
the issue being addressed, the researcher's personal experiences, and the audiences for the study
Key Characteristics of Grounded Theory
the newly constructed grounded theory should fit the data the theory must provide understanding of the phenomenon the theory should have some generality the theory should contribute to some control of the phenomenon
Pragmatism
the paradigm most often associated with MM research, the belief that it is the research question that should drive the inquiry and that the question is more important than the methods used; "whatever works best to arrive at good evidence is appropriate"
A nested relationship
the participants in the qual strand are a subset of the participants in the quant strand
emic perspective
the perspective of the insider
etic perspective
the perspective of the outside observer
Mixed methods research
the planned integration of qualitative and quantitative data within a single study or a coordinated series of studies; "research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using both quantitative and qualitative approaches or methods in a single study or program of inquiry"
Follow-up explanations variant
the researcher collects qual data that can best help to explain the initial quan data emphasis on quan aspects of the study
Theory development variant
the researcher identifies important constructs and develops a theory, taxonomy or classification system grounded in the in-depth data gathered in the qual phase, then the quan phase is used to test or explore the taxonomy or theory with a broader group formal hypothesis generated in the 1st phase are tested in a subsequent phase
Etic Perspective
the researcher's external or "objective outsider" perspective
pragmatic knowledge
the result of research that is planned and conducted based on what will best help the researcher answer the research questions
Exploratory sequential mixed methods
the reverse sequence from the explanatory sequential design. The researcher first begins with a qualitative research phase and explores the views of the participants. The data are then analyzed, and the information used to build into a second, quantitative phase.
A parallel relationship
the samples in the 2 strands are completely different, although they are drawn from the same or similar population
two-phase approach
the second part of the approach elaborates on the first
Important characteristics in asphalt binder selection for HMA mix design
the source, durability, rheology, purity, and potential modifying agents that can be added to the asphalt
Nomological causation
the standard view of causation in science; refers to causal relationships among variables
triangulation
the use of multiple methods to ensure that the data collected are trustworthy and accurate
Triangulation
the use of multiple methods to ensure that the data collected is trustworthy and accurate
Theory Triangulation
the use of multiple theories or perspectives to aid in interpreting the data
Reflexivity
thinking critically about one's interpretations and biases
Selective Coding
third and final stage of data analysis in GT in which the theory is finalized
Independent Variables
those that (probably) cause, influence, or affect outcomes. Also called treatment, manipulated, antecedent, or predictor variables.
Dependent Variables
those that depend on the independent variables; they are outcomes or results of the influence of the independent variables. Other names are criterion, outcome, effect, and response variables.
purposeful sampling
those that you specifically choose and then contain a certain element that will highlight and inform a particular aspect of the study, the sample is not random and may become subject to research bias
Cynical nature of action research
those who are more driven by the researcher's agenda and those more driven by participants; those who are motivated primarily by instrumental goal attainment and those motivated primarily by the aim of personal, organizational or societal transformation; and 1st-, to 2nd-, to 3rd-person research, that is, my research on my own action, aimed primarily at personal change; our research on our group (family/team), aimed primarily at improving the group; and 'scholarly' research aimed primarily at theoretical generalization or large scale change.[1] .
fundamental principle of mixed methods research
thoughtful mixing of methods, procedures, and other paradigm characteristics resulting in high-quality research, researchers should mix in a way that provides multiple (divergent and convergent) and complementary strengths (viewed broadly) and nonoverlapping weaknesses
Superpave Method
ties asphalt binder and aggregate selection into the mix design process, and considers traffic and climate as well. The compaction device is a gyratory compactor and the compaction effort in mix design is simulates expected traffic.
triangulation design
to simultaneously collect both quantitative and qualitative data, merge the data, and use the results to understand a research problem, this is a one phase design and both types of research carry equal weight
Qualitizing
transforming quan data into qual form
intermethod mixing
two or more of the methods of data collection are used in a research study
Exploratory Sequential
two sequential phases but in opposite order. Qualitative data first followed by quantitative data to provide breadth for emergent relationships.
Explanatory Sequential
two sequential phases, quantitative data collected first followed by qualitative data
sequential validity
type of mixed methods validity that ensures the results of a study are not due to the sequence of its qualitative and quantitative components (e.g. interviews followed by testing giving different results than testing followed by interviews)
multiple validities
type of mixed methods validity that refers to the degree to which a researcher fulfills the relevant quantitative AND qualitative types of validity necessary to guarantee a strong study
sample integration validity
type of mixed methods validity that refers to the need to draw appropriate conclusions from the qualitative and quantitative samples in a study. (e.g. testing 1000 participants but interviewing only 10 of them. The interviews would be useful, but would not yield comparable/generalisable results)
constructivist
understanding, multiple participant meanings, social and historical construction, theory generation.
Category Strauss & Corbin 1990
unit of information containing events, happenings and instances
Investigator Triangulation
use of multiple investigators to collect and interpret the data
Methods Triangulation
use of multiple research methods or methods of data collection
Data Triangulation
use of multiple sources of data
quant. theories
use of theory is deductive and is at beginning of study, with objective of testing theory rather than developing it. It becomes the framework for the entire study.
Integration
use of two or more approaches, methods, and purposes being combined in complex mixed methods designs
Instrument development model
used when data from the qual phase are used in the development of quan instruments
investigator triangulation
using multiple investigators to collect and interpret data. Can aid descriptive validity in a study
methods triangulation
using multiple research methods or methods of data collection; e.g. multiple interviews or observations in different settings
theory triangulation
using multiple theories or perspectives to help interpret and understand the data
monomethod research
using only quantitative or qualitative methods - limiting
multilevel sample relationship criterion
using quantitative and qualitative samples obtained from different levels of population under study
Scatterplot
values along the vertical axis were internality scores, those along the horizontal axis were externality scores
Corbin and Strauss
verifiable coding
Feminist Perspective
view as problematic women's diverse situations and the institutions that frame those situations.
the order of the questions
what determines the priority of qualitative or quantitative methods?
determine the order, weight, how, and the theory
what do you do when you make decisions about a mixed method design
pragmatism
what is ultimately important and justified or "valid" is what works in particular situations in practice and what promotes social justice, focused on consequences and the ends that researchers value
participant feedback
when a researcher discusses their findings with participants to see if they agree with the researcher's statements, interpretations and conclusions
peer review
when a researcher discusses their interpretations with their peers and colleagues
pattern matching
when a researcher makes a complex hypothesis and then determines if it is supported
low-inference descriptors
when a researcher uses descriptions that are very similar to a participant's own responses and language, or uses direct verbatim quotes
Why use a mixed methods design?
when quantitative + qualitative data together provide a better understanding of the research problem than either type by itself to provide a more complete picture of the research problem quantitative: trends and describing population qualitative: participants' words when you want to build from one phase of a study to another explore qualitatively then develop an instrument follow-up a quantitative study to obtain more detailed information, a more in-depth understanding
Concurrent
when the two types of data are collected at the same time; simultaneous or parallel
Sequential
when the two types of data are not collected at the same time
Bacon's Idols:
when we carry preconceptions into the data collection or analysis process we may influence our findings and interpretations. All of these risks also exist in the case of qualitative research, and for this reason there is no way to ensure that the development of understanding can occur completely from the data and not the researcher's own experience or knowledge.
Theoretical Lens or Perspective in qualitative research
which provides an overall orienting lens for the study of questions of gender, class, and race.
significant statements
words, phrases or sentences that vividly communicate a participant's subjective experience
quantitative research questions
write either a research question or hypothesis, not both, measure and write about the independent and dependent variables separately, link questions and hypotheses to the theory
Norms
written and unwritten rules specifying how people in a group are supposed to think and act
descriptive questions
written for each dependent and independent variable and any significant mediating variables in the study
Purposeful Sampling
you specifically choose and that contain a certain element that will highlight and inform a particular aspect of the study.
Explain concurrent embedded
• Different weightings given to qual and quan data • Can integrate or compare data
Most common mixed-methods approaches include:
• Either self-administered survey or questionnaire, or a structured or semi-structured interview
Theoretical lenses in qual. research
• Feminist perspective • Racialised discourses • critical theory • Queer theory • Disability inquiry
Grounded Theory Methodology
• Perhaps the most well-known discussion of this inductivist approach to qualitative research is contained within explanations of the grounded theory methodology.. By definition, this orientation toward research implies that one develops the relevant theory for a study from the data as they are collected and analyzed in qualitative fashion.
Quantitative Research
• Pertains to numerical forms of data • Is focused on identifying the presence and magnitude of effects • Deductive Process
Explain concurrent triangulation
• Quan and Qual data collected at the same time • Used to validate findings generated by each method through evidence produced by the other
Qualitative Research
• Tends to include the "other" forms of data. • Emphasizes sense-making • Inductive process
QUALITATIVE PURPOSE STATEMENT:
● A good qualitative purpose statement contains: ○ The central phenomenon ○ The participants ○ The research site ● Include language drawn from qualitative inquiry ● Use words such as purpose, intent, or objective to signal attention to this statement as the central controlling idea ● Focus on a single phenomenon, narrow the study to one idea to be explored or understood. This statement should acknowledge that the study may evolve into an exploration of relationships or comparisons among ideas
Concurrent Nested
● Characterized by: A nested approach that gives priority to one of the methods and guides the project, while another is embedded or "nested." ● Purpose: The purpose of the nested method is to address a different question than the dominant or to seek information from different levels.
Sequential Exploratory
● Characterized by: An initial phase of qualitative data collection and analysis followed by a phase of quantitative data collection and analysis. ● Purpose: To explore a phenomenon. This strategy may also be useful when developing and testing a new instrument
Sequential Transformative
● Characterized by: Collection and analysis of either quantitative or qualitative data first. The results are integrated in the interpretation phase. ● Purpose: To employ the methods that best serve a theoretical perspective
Sequential Explanatory
● Characterized by: Collection and analysis of quantitative data followed by a collection and analysis of qualitative data. ● Purpose: To use qualitative results to assist in explaining and interpreting the findings of a quantitative study.
Concurrent Transformative
● Characterized by: The use of a theoretical perspective reflected in the purpose or research questions of the study to guide all methodological choices. ● Purpose: To evaluate a theoretical perspective at different levels of analysis.
Concurrent Triangulation
● Characterized by: Two or more methods used to confirm, cross-validate, or corroborate findings within a study. Data collection is concurrent. ● Purpose: Generally, both methods are used to overcome a weakness in using one method with the strengths of another.
QUANTITATIVE PURPOSE STATEMENT:
● This statement differs from the qualitative purpose statement in terms of language and focus ● Focus on relating or comparing the variables and how these relate to ○ The participants ○ The research site ● Identify the proposed major variables in a study ● A good quantitative purpose statement includes: ○ Words to signal the major intent ○ Identify the theory, model, or conceptual framework ○ Identify the independent and dependent variables as well as any mediating, moderating, or control variables ○ Use words that connect the independent and dependent variables ○ Position the independent before the dependent variable ○ Mention the specific strategy of inquiry used ○ Mention the participants ○ Use the literature to define
