11. Quantitative vs. Qualitative Research Methods
What are the strengths of quantitative research methods?
- All strengths from scientific method - Sensitive stats to identify very small patterns of associations - Falsification prevents wrong ideas or weak theories to thrive for too long - Allows for generalization and prediction - Ideally can control outcome of interventions
What are the limits of qualitative research?
- Conclusions aren't guaranteed to be true (induction, verification) - Less well suited to decide between theories - Analysis based on subjective content given by participants and interpreted by researchers (introspection - low validity) - Researcher's involvement may be a disadvantage in high-stakes situations where validity and reliability should be high
What is the hermeneutic critique for quantitative methods? + response
We cannot know what we cannot measure (though we do just that with qualia)
What's the key for descriptive quantitative research technique?
Using large samples and few data points per participant
What are the strengths of qualitative research?
- Directly focused on understanding situations and solving problems through verification (esp. for applied psych where you want a solution not a generalizable principle, e.g. therapy) - Generation of new ideas and elaboration of theories - More perceptive to the needs of participants (decreases risk that advice would be perceived as unhelpful, esp. since science is trial-and-error)
What is characteristic of the researchers who think quantitative and qualitative methods can complement each other?
- Less concerned with contradictions in the underlying philosophies - Focus on what the methods bring in terms of info
Outline the arguments that qualitative researchers make against quantitative research
- No actual claim on truth with scientific knowledge - Scientific method gives them high status in society, progress doesn't matter to them - Misguided in search of objective reality (if exists, would have made more progress, if not, then info is useless)
What are the limits of quantitative research methods?
- No interest in person as individual - Researchers as uninvolved, dispassionate observers (to avoid confounds) - How can you measure real-life, complex situations? - Falsification is better for destroying ideas than finding practical solutions
What are the underlying assumptions of qualitative research methods?
- Only reality that matters is the one perceived and constructed by people - Attempts to control situations make the setting aritificial and don't help understand real world - Research should be immersed in situation to understand it - Ideographic approach is better - Induction over deduction to not lose sight of big picture - Evidence-based research should be rooted in observations, not intuitions/opinions
What do qualitative argue for how their approach could entail false/biased conclusions?
- The risk is offset by the expected gains due to an understanding of the situation - All conclusions are relative (depend on paradigm) - Biases can be prevented by being aware of them and doing the analysis in a replicable way
List the hierarchy of evidence in medical science (from top of pyramid to bottom)
1. Meta-analysis 2. Randomized controlled studies 3. Follow-up studies 4. Case-control studies 5. Cross-sectional surveys 6. Case reports
What are the weaknesses in quantitative research that qualitative research can address?
1. Not enough interest in participants 2. Research too much driven by what can be measured and tested experimentally 3. Too much focus on falsification at the expense of a pragmatic solution to the problem
Describe the 4 phases in the procrastination study by Schraw et al, 2007. What coding technique was used? What was the purpose?
1. Open: identify codes within categories for further analysis 2. Axial: explore codes in detail, relate codes to one another to construct themes 3. Selective: construct paradigm model and discuss themes in relation to model; establish story line that integrates paradigm model 4. Selective: test, validate and explicate paradigm model until saturated; identify emergent principles consistent with paradigm model; conduct member checks
How does qualitative research address the weaknesses in quantitative research?
1. Participants are the focus of attention 2. Handles difficult to quantify and manipulate variables 3. Useful for theory building and offering practical solutions
What 3 slightly incorrect but strong statements are qualitative researchers making when they say that quantitative research is just a positivist search for physical laws?
1. Quantitative psychology defends a positivist view of the world 2. Quantitative psychology is looking for laws similar to Newton's laws of physics 3. Quantitative psychology is only interested in behavior in artificial settings *Make sure you know what the quantitative response is to these
List the checks that qualitative researchers make to prevent biases
1. Representativeness 2. Confirmability 3. Credibility 4. Comparisons of situations that differ on one critical aspect 5. Alternative explanations 6. Refutability
What are the assumptions underlying quantitative methods?
1. There's an outside reality that can be discovered through scientific method 2. Main aim of scientific research is to find universal causal relations 3. Important to avoid confounds and sources of noise 4. Humans (researchers) are fallible and can be sources of confounds/noise 5. Most scientific progress comes from falsification
What are the common misconceptions that quantitative researchers have about qualitative research methods according to Maracek?
1. They are complementary methods and lead to the same type of understanding 2. Qualitative research is an adjunct to quantitative research (first step, needing quant research after) 3. Qualitative research only consists of inductive reasoning 4. Qualitative research is nothing but psychology without numbers
How are interviews transcribed in the qualitative method?
Auditory/video recordings transcribed in written form, using elaborate coding system through specialized software to include non-verbal info (e.g. hesitations)
In the study investigating prostate cancer screening in GPs (Starks & Trinidad, 2007), what did grounded theory contribute?
A more practical, objective solution of discussions between patients and GPs
Describe a *nomothetic approach*
A search for universal principles that exceeds the confines of the study
Describe an *ideographic approach*
A study of what is relevant to the subject under study (e.g. the person)
Give an example of how falsification didn't help with progress in psychology
AI psych studies for language perception and understanding weren't making a lot of progress for 40-50 years, most progress done in past 15 years when engineers took a different approach
Define *case reports*
Anecdotal evidence related to individual patients
In which fields are qualitative methods particularly strong in?
Anthropology, sociology, social work, education, a bit of educational psychology, minority in psychology
Why are meta-analyses highest on the hierarchy of evidence?
Because it has RCTs and replication
Why does psychology more often use "effect" instead of "law" to describe its theories
Because it's stochastic
Outline the arguments that quantitative researchers make against qualitative research
Because qual research rejects the existence of an objective reality by falling in line with post-modernist and hermeneutic approaches, it throws away all the progress science has made. It is not the job of researchers to form relationships with people, that's the job of everyone. Don't provide researchers with new info and devalue research to pop psych
Why do some qualitative researchers think there can never be any reconciliation between the two methods?
Because you can't just pick certain forms of it that work - you have to adopt the underlying methods as well
What are the "Six Cs" of social processes and guiding principles for the analysis of transcripts?
Causes, contexts, contingencies, consequences, covariances, conditions of discussion
What are the hermeneutic critiques for grounded theory?
Close to main tenets of logical positivism (assumes existence of objective reality to be discovered, uses inductive reasoning and verification) and results are more of a social construction of the research than a description of the reality
How is data analyzed in qualitative research?
Data written as flow chart of core ideas, based on multiple closed readings and guided by the questions emphasized by the different approaches - cycle through this until saturation is reached
Name the 3 broad quantitative research techniques
Descriptive, relational, experimental
What are the two properties of science that made people realize you couldn't call all theories "Law of..." in the 20th C? Describe them
Deterministic - small variability, outcome that can be predicted with high probability Stochastic - lots of noise, outcome can only be predicted with some probability
In the study investigating prostate cancer screening in GPs (Starks & Trinidad, 2007), what did discourse analysis contribute?
Discourse of medicine pushes GPs into different roles that can put them in difficult positions when communicating with patients and when they may not be able to meet expectations of their roles
From who was phenomenological analysis inspired? Why?
Edmund Husserl - came up with phenomenology
What is a downside of how much quantitative researchers love falsification?
Everyone doubts findings and are constantly trying to prove each other wrong - Very easy to get far in the career by destroying another theory (often kick a theory when it's down)
What theory sort of accounts for scientific progress? How?
Evolutionary theory: scientific discoveries are born out of theories before them and selected as a function of how well they fit with the environment
What is characteristic of the researchers who think quantitative and qualitative methods are incompatible?
Exaggerate the differences between the approaches at the expense of commonalities
What method is used to examine correlation structure between a large number of variables?
Factor analysis
Define *follow-up studies*
Follows two matched groups (one with disease/treatment, one without) longitudinally
What does grounded theory make possible?
For a theory to emerge from data through inductive reasoning
Who came up with factor analysis?
Galton
What sort of relationships are we intuitively pretty decent at detecting? What are we bad at intuitively detecting?
Good = positive relationships, bad = negative relationships
Name the three approaches for qualitative data analysis
Grounded theory, phenomenological analysis, discourse analysis
In the study investigating prostate cancer screening in GPs (Starks & Trinidad, 2007), what purpose did each type of qualitative analysis have for the study?
Grounded theory: develop effective training and education for GPs about how to approach prostate cancer screening discussions Phenomenology: understand GPs' experience of decision making with patients under conditions of clinical uncertainty Discourse analysis: shed light on the reasons for the limited or incomplete adoption of IDM by GPs
According to evolutionary theory, how are new scientific theories selected?
How pragmatic it is
Define *phenomenology*
Idea that psychology should be reflective of consciousness as experienced from a first-person point of view
When is an effect "probably true" for quantitative researchers?
If there is converging evidence for it and if it resisted several falsification attempts
Describe *grounded theory*
Involves rewriting raw materials (interviews) on the basis of thematic/theoretical questions, coding the answers in themes and then grouping them into higher-order categories
What are meta-analyses good for?
Lets you look at effect size of all studies and see if evidence converges
Where does discourse analysis come from?
Linguistic turn in philosophy of science and critical psychology - reality is nothing but a social construction based on language and culture
Describe *bracketing*
Requirement in qualitative research to look at a phenomenon with an open mind and to free oneself from preconceptions
Define *cross-sectional surveys*
Looks for correlations between different health-related questions
What are quantitative methods based on and what do they almost always involve?
Magnitude, frequency; statistical analyses
In the study investigating prostate cancer screening in GPs (Starks & Trinidad, 2007), what did phenomenology contribute?
Many GPs expressed discomfort with the fact that they couldn't meet patient expectations because of no clear recommendations for these screenings
Which of the most prominent psychologists would be classified as qualitative researchers by present day standards?
Maslow, Piaget, Freud, Rogers
Define *case-control studies*
Matches patients on a number of variables with a control, comparing medical histories to find source of disease
Who are quantitative methods preferred by?
Natural-science oriented psychology researchers
Can we predict what governs scientific progress? Why or why not?
Not really because it's hard to isolate predictive factors and even then figure out if it has a causal influence
Why are semi-structured interviews preferred in qualitative research?
Open-ended, non-directive questions allow for and encourage elaboration, giving as much data as possible
Describe the descriptive quantitative research technique
Operationalizing a phenomenon by observing its numerical data, e.g. averages, frequencies, steers, etc.
Define *noise* in quantitative research
Other source of influence on the effect that are of no interest to researchers
Define *confounds* in quantitative research
Other things that could account for results that aren't controlled for
Define *illusory correlation*
Perception of correlation between events for which independent evidence can be found
Describe *Discourse Analysis*
Post-modernist way of investigating the ways in which language constructs social reality, focusing on understanding the possibilities and impossibilities of language/how it's used in real-world setting
What is the relational quantitative research technique good for?
Preventing against illusory correlation
How do qualitative researchers check for representativeness?
Provide criteria used for analysis to gauge representativeness of reported instances
Describe the experimental quantitative research technique
Uses experimental manipulations on a suspected cause to determine cause-effect relations
Define *randomized control studies*
Randomly assigns half of participants to a treatment and other half to a control that only lacks the critical therapeutic agent
How did the divide between qualitative and quantitative research arise?
Scientists kept trying to please the ever-changing definitions of science in philosophy, eventually leading to focusing on hypotheses to please falsification requirement, but post-modernists said they were too hypothesis-driven and science is nothing special, leading to the divide
What is the most frequently used technique in qualitative research?
Semi-structured interviews
In what field did *grounded theory* originate, by who, and around when?
Sociology, Glaser ans Strauss, late 60s
Define *meta-analysis*
Study of studies
How might quantitative and qualitative research methods complement each other?
They are each better suited to answering different questions, so researchers should pick the method that best fits their theoretical questions and analytic situations
Describe *Interpretative Phenomenological Analysis*
Transcribes data to themes, focusing on how people make sense of their personal and social world
How is qualitative research structured, thorough and systematic?
Transcripts made available to everyone; numerous checks for biases
What is the purpose of qualitative research methods?
Understanding phenomena in their historical and socio-cultural context
What's the point of the Schraw et al, 2007 procrastination study?
Used grounded theory to learn a lot of things that weren't quite self-evident, and would not have been found using quantitative methods. Used a lot of verification
Describe the relational quantitative research technique
Uses correlations to begin to uncover cause-effect relationships
When is an illusory correlation especially strong?
When the story makes sense