11. Quantitative vs. Qualitative Research Methods

What are the strengths of quantitative research methods?

- All strengths from scientific method - Sensitive stats to identify very small patterns of associations - Falsification prevents wrong ideas or weak theories to thrive for too long - Allows for generalization and prediction - Ideally can control outcome of interventions

What are the limits of qualitative research?

- Conclusions aren't guaranteed to be true (induction, verification) - Less well suited to decide between theories - Analysis based on subjective content given by participants and interpreted by researchers (introspection - low validity) - Researcher's involvement may be a disadvantage in high-stakes situations where validity and reliability should be high

What is the hermeneutic critique for quantitative methods? + response

We cannot know what we cannot measure (though we do just that with qualia)

What's the key for descriptive quantitative research technique?

Using large samples and few data points per participant

What are the strengths of qualitative research?

- Directly focused on understanding situations and solving problems through verification (esp. for applied psych where you want a solution not a generalizable principle, e.g. therapy) - Generation of new ideas and elaboration of theories - More perceptive to the needs of participants (decreases risk that advice would be perceived as unhelpful, esp. since science is trial-and-error)

What is characteristic of the researchers who think quantitative and qualitative methods can complement each other?

- Less concerned with contradictions in the underlying philosophies - Focus on what the methods bring in terms of info

Outline the arguments that qualitative researchers make against quantitative research

- No actual claim on truth with scientific knowledge - Scientific method gives them high status in society, progress doesn't matter to them - Misguided in search of objective reality (if exists, would have made more progress, if not, then info is useless)

What are the limits of quantitative research methods?

- No interest in person as individual - Researchers as uninvolved, dispassionate observers (to avoid confounds) - How can you measure real-life, complex situations? - Falsification is better for destroying ideas than finding practical solutions

What are the underlying assumptions of qualitative research methods?

- Only reality that matters is the one perceived and constructed by people - Attempts to control situations make the setting aritificial and don't help understand real world - Research should be immersed in situation to understand it - Ideographic approach is better - Induction over deduction to not lose sight of big picture - Evidence-based research should be rooted in observations, not intuitions/opinions

What do qualitative argue for how their approach could entail false/biased conclusions?

- The risk is offset by the expected gains due to an understanding of the situation - All conclusions are relative (depend on paradigm) - Biases can be prevented by being aware of them and doing the analysis in a replicable way

List the hierarchy of evidence in medical science (from top of pyramid to bottom)

1. Meta-analysis 2. Randomized controlled studies 3. Follow-up studies 4. Case-control studies 5. Cross-sectional surveys 6. Case reports

What are the weaknesses in quantitative research that qualitative research can address?

1. Not enough interest in participants 2. Research too much driven by what can be measured and tested experimentally 3. Too much focus on falsification at the expense of a pragmatic solution to the problem

Describe the 4 phases in the procrastination study by Schraw et al, 2007. What coding technique was used? What was the purpose?

1. Open: identify codes within categories for further analysis 2. Axial: explore codes in detail, relate codes to one another to construct themes 3. Selective: construct paradigm model and discuss themes in relation to model; establish story line that integrates paradigm model 4. Selective: test, validate and explicate paradigm model until saturated; identify emergent principles consistent with paradigm model; conduct member checks

How does qualitative research address the weaknesses in quantitative research?

1. Participants are the focus of attention 2. Handles difficult to quantify and manipulate variables 3. Useful for theory building and offering practical solutions

What 3 slightly incorrect but strong statements are qualitative researchers making when they say that quantitative research is just a positivist search for physical laws?

1. Quantitative psychology defends a positivist view of the world 2. Quantitative psychology is looking for laws similar to Newton's laws of physics 3. Quantitative psychology is only interested in behavior in artificial settings *Make sure you know what the quantitative response is to these

List the checks that qualitative researchers make to prevent biases

1. Representativeness 2. Confirmability 3. Credibility 4. Comparisons of situations that differ on one critical aspect 5. Alternative explanations 6. Refutability

What are the assumptions underlying quantitative methods?

1. There's an outside reality that can be discovered through scientific method 2. Main aim of scientific research is to find universal causal relations 3. Important to avoid confounds and sources of noise 4. Humans (researchers) are fallible and can be sources of confounds/noise 5. Most scientific progress comes from falsification

What are the common misconceptions that quantitative researchers have about qualitative research methods according to Maracek?

1. They are complementary methods and lead to the same type of understanding 2. Qualitative research is an adjunct to quantitative research (first step, needing quant research after) 3. Qualitative research only consists of inductive reasoning 4. Qualitative research is nothing but psychology without numbers

How are interviews transcribed in the qualitative method?

Auditory/video recordings transcribed in written form, using elaborate coding system through specialized software to include non-verbal info (e.g. hesitations)

In the study investigating prostate cancer screening in GPs (Starks & Trinidad, 2007), what did grounded theory contribute?

A more practical, objective solution of discussions between patients and GPs

Describe a *nomothetic approach*

A search for universal principles that exceeds the confines of the study

Describe an *ideographic approach*

A study of what is relevant to the subject under study (e.g. the person)

Give an example of how falsification didn't help with progress in psychology

AI psych studies for language perception and understanding weren't making a lot of progress for 40-50 years, most progress done in past 15 years when engineers took a different approach

Define *case reports*

Anecdotal evidence related to individual patients

In which fields are qualitative methods particularly strong in?

Anthropology, sociology, social work, education, a bit of educational psychology, minority in psychology

Why are meta-analyses highest on the hierarchy of evidence?

Because it has RCTs and replication

Why does psychology more often use "effect" instead of "law" to describe its theories

Because it's stochastic

Outline the arguments that quantitative researchers make against qualitative research

Because qual research rejects the existence of an objective reality by falling in line with post-modernist and hermeneutic approaches, it throws away all the progress science has made. It is not the job of researchers to form relationships with people, that's the job of everyone. Don't provide researchers with new info and devalue research to pop psych

Why do some qualitative researchers think there can never be any reconciliation between the two methods?

Because you can't just pick certain forms of it that work - you have to adopt the underlying methods as well

What are the "Six Cs" of social processes and guiding principles for the analysis of transcripts?

Causes, contexts, contingencies, consequences, covariances, conditions of discussion

What are the hermeneutic critiques for grounded theory?

Close to main tenets of logical positivism (assumes existence of objective reality to be discovered, uses inductive reasoning and verification) and results are more of a social construction of the research than a description of the reality

How is data analyzed in qualitative research?

Data written as flow chart of core ideas, based on multiple closed readings and guided by the questions emphasized by the different approaches - cycle through this until saturation is reached

Name the 3 broad quantitative research techniques

Descriptive, relational, experimental

What are the two properties of science that made people realize you couldn't call all theories "Law of..." in the 20th C? Describe them

Deterministic - small variability, outcome that can be predicted with high probability Stochastic - lots of noise, outcome can only be predicted with some probability

In the study investigating prostate cancer screening in GPs (Starks & Trinidad, 2007), what did discourse analysis contribute?

Discourse of medicine pushes GPs into different roles that can put them in difficult positions when communicating with patients and when they may not be able to meet expectations of their roles

From who was phenomenological analysis inspired? Why?

Edmund Husserl - came up with phenomenology

What is a downside of how much quantitative researchers love falsification?

Everyone doubts findings and are constantly trying to prove each other wrong - Very easy to get far in the career by destroying another theory (often kick a theory when it's down)

What theory sort of accounts for scientific progress? How?

Evolutionary theory: scientific discoveries are born out of theories before them and selected as a function of how well they fit with the environment

What is characteristic of the researchers who think quantitative and qualitative methods are incompatible?

Exaggerate the differences between the approaches at the expense of commonalities

What method is used to examine correlation structure between a large number of variables?

Factor analysis

Define *follow-up studies*

Follows two matched groups (one with disease/treatment, one without) longitudinally

What does grounded theory make possible?

For a theory to emerge from data through inductive reasoning

Who came up with factor analysis?

Galton

What sort of relationships are we intuitively pretty decent at detecting? What are we bad at intuitively detecting?

Good = positive relationships, bad = negative relationships

Name the three approaches for qualitative data analysis

Grounded theory, phenomenological analysis, discourse analysis

In the study investigating prostate cancer screening in GPs (Starks & Trinidad, 2007), what purpose did each type of qualitative analysis have for the study?

Grounded theory: develop effective training and education for GPs about how to approach prostate cancer screening discussions Phenomenology: understand GPs' experience of decision making with patients under conditions of clinical uncertainty Discourse analysis: shed light on the reasons for the limited or incomplete adoption of IDM by GPs

According to evolutionary theory, how are new scientific theories selected?

How pragmatic it is

Define *phenomenology*

Idea that psychology should be reflective of consciousness as experienced from a first-person point of view

When is an effect "probably true" for quantitative researchers?

If there is converging evidence for it and if it resisted several falsification attempts

Describe *grounded theory*

Involves rewriting raw materials (interviews) on the basis of thematic/theoretical questions, coding the answers in themes and then grouping them into higher-order categories

What are meta-analyses good for?

Lets you look at effect size of all studies and see if evidence converges

Where does discourse analysis come from?

Linguistic turn in philosophy of science and critical psychology - reality is nothing but a social construction based on language and culture

Describe *bracketing*

Requirement in qualitative research to look at a phenomenon with an open mind and to free oneself from preconceptions

Define *cross-sectional surveys*

Looks for correlations between different health-related questions

What are quantitative methods based on and what do they almost always involve?

Magnitude, frequency; statistical analyses

In the study investigating prostate cancer screening in GPs (Starks & Trinidad, 2007), what did phenomenology contribute?

Many GPs expressed discomfort with the fact that they couldn't meet patient expectations because of no clear recommendations for these screenings

Which of the most prominent psychologists would be classified as qualitative researchers by present day standards?

Maslow, Piaget, Freud, Rogers

Define *case-control studies*

Matches patients on a number of variables with a control, comparing medical histories to find source of disease

Who are quantitative methods preferred by?

Natural-science oriented psychology researchers

Can we predict what governs scientific progress? Why or why not?

Not really because it's hard to isolate predictive factors and even then figure out if it has a causal influence

Why are semi-structured interviews preferred in qualitative research?

Open-ended, non-directive questions allow for and encourage elaboration, giving as much data as possible

Describe the descriptive quantitative research technique

Operationalizing a phenomenon by observing its numerical data, e.g. averages, frequencies, steers, etc.

Define *noise* in quantitative research

Other source of influence on the effect that are of no interest to researchers

Define *confounds* in quantitative research

Other things that could account for results that aren't controlled for

Define *illusory correlation*

Perception of correlation between events for which independent evidence can be found

Describe *Discourse Analysis*

Post-modernist way of investigating the ways in which language constructs social reality, focusing on understanding the possibilities and impossibilities of language/how it's used in real-world setting

What is the relational quantitative research technique good for?

Preventing against illusory correlation

How do qualitative researchers check for representativeness?

Provide criteria used for analysis to gauge representativeness of reported instances

Describe the experimental quantitative research technique

Uses experimental manipulations on a suspected cause to determine cause-effect relations

Define *randomized control studies*

Randomly assigns half of participants to a treatment and other half to a control that only lacks the critical therapeutic agent

How did the divide between qualitative and quantitative research arise?

Scientists kept trying to please the ever-changing definitions of science in philosophy, eventually leading to focusing on hypotheses to please falsification requirement, but post-modernists said they were too hypothesis-driven and science is nothing special, leading to the divide

What is the most frequently used technique in qualitative research?

Semi-structured interviews

In what field did *grounded theory* originate, by who, and around when?

Sociology, Glaser ans Strauss, late 60s

Define *meta-analysis*

Study of studies

How might quantitative and qualitative research methods complement each other?

They are each better suited to answering different questions, so researchers should pick the method that best fits their theoretical questions and analytic situations

Describe *Interpretative Phenomenological Analysis*

Transcribes data to themes, focusing on how people make sense of their personal and social world

How is qualitative research structured, thorough and systematic?

Transcripts made available to everyone; numerous checks for biases

What is the purpose of qualitative research methods?

Understanding phenomena in their historical and socio-cultural context

What's the point of the Schraw et al, 2007 procrastination study?

Used grounded theory to learn a lot of things that weren't quite self-evident, and would not have been found using quantitative methods. Used a lot of verification

Describe the relational quantitative research technique

Uses correlations to begin to uncover cause-effect relationships

When is an illusory correlation especially strong?

When the story makes sense