SW 650, Research Methods
Use letters A: B: C: BC:
"A" is the baseline phase "B" is the first internvention you try "C" the second intervention you try "BC" a combinatikon of two interventions
Baseline Stage
---
Question wording: Use positive constructions, avoid double negatives
-A negative construction negates a verb and may result in a negative target object or double negative -a positive construction does not negate the verb -Bassili & Scott (1996): Longer RLs; 4xs repeat/clarification (16% vs. 4%) -use to create balanced scales Problematic question Policies that do not safeguard the environment are bad. Do you agree or disagree? Revision -Policies that safeguard the environment are good. Do you agree or disagree?
Advantages and disadvantages of unipolar items
-Advantages of unipolar items *do not have to find opposites *can discover whether dimensions are related *can begin with low or less desirable end -Disadvantages of unipolar items *some constructs, especially some comparisons, are inherently bipolar (e.g., "are things getting better or worse?" *need two items if want to measure both positive and negative dimensions
Example Child with Autism
-Applied Behavior Analysis -Intensive (25 to 40 hours a week) -1:1 with social worker/psychologist -Target specific behavior (language, stereotypical gestures such as rocking, hand flapping or aggressive behaviors, tantrums)
Response Categories Exhaustive
-By response categories being exhaustive we mean that they should include all of the possible responses that might be expected -For example, What is your marital status? (Married, divorced, separated, widowed, single)
Participant Observation Roles
-Covert participant -Participant as observer -Overt Observer -Covert Observer
Qualitative Paradigms, Traditions & Methods
-Ethnography -Phenomenology -Grounded theory -case study -participatory action research -qualitative research study-or other approaches based on pragmatic philosophy
Agree-disagree (AD) versus construct-specific (CS) items
-Example of an AD item I think most people are better informed about politics and government than I am. Do you strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree? -Advantages *easy to write *efficient to administer -Disadvantages *increased cognitive burden for respondents *increased administrative burden for interviewers *reduced data quality
Avoid response categories that might produce central tendency bias
-How satisifed are you with the Health Care Reform Act? 1. very satisfied 2. satisfied 3. neither satisfied nor dissatisfied 4. dissatisfied 5. very dissatisfied
Likert scaling
-How satisifed are you with the Health Care Reform Act? 1. very satisifed 2. satisifed 3. neither satisfied nor dissatisfied 4. dissatisfied 5. very dissatisfied
When Qualitative Research is Indicated?
-How? What? Why? -Understanding an experience? -understanding context *how economic, social, political, cultural, environmental factors influence... *what is or is not working in a program or service -Explaining/understanding a phenmenon -generating theory -quantiative findings don't make sense
Balancing Positive and Negatively Worded Items
-I am unhappy -I feel good about myself -I feel sad -Everything is an effort
General Thoughts on Construction of Survey
-Ordering of the questions: (responses don't affect later response, placement of sensitive questions, placement of background questions) -layout of the questionnaire-white space -transitions between sections, especially if response categories change -keep wording of questions simple, easy to understand -end on an upbeat tone -good cover letter
Response format: Write question to project answer in the format you want
-Problematic question Where do you live? CITY____ STATE______ -Revision In which city and state do you live? City_______ State____
Length of Baseline
-Stable Basline -Some say need at least 3 data points/others say at least 10. And still others should observe behavior for at least 7 days
Biased/Leading Questions
-The government should outlaw the killing of babies -Many people have said that there is a need for stricter laws on dangerous weapons. Do you agree? -Wouldn't you agree that people with schizophrenia should be cared for in the community? -How many children do you have?
Determined by Study Purpose
-What are you trying to understand? -What is the rationale? -What is already known/not known? -Personal, practical, intellectual (scholarly) goals?
Inclusive Language
-What is your race/ethnicity (check only one) -What is your marital status? (married, separated, divorce, widowed, single) -How old are you? 20-24, 25-30, 31-40, 40-49, 50+
What do you think causes schizophrenia
-Would you say: (circle all that apply) -genes -bad parenting -stress -family system pathology -birth trauma -virus
grounded theory
-a grounded theory is one that is inductively derived from the study of the phenomena it represents
Phenomenology
-a philosophy and a methodology -"a context-sensitive...form of inquiry into a lived experience"
construct or concepts
-abstract labels that describe properties of units *depression, quality of life *prejudice, oppression, deprivation *inherently unobservable -example: weight
Advantages of bipolar items
-appropriate for comparisons (e.g., "better, worse or the same") -may match some native concepts (e.g., "extremely unlikely" may match better than "not at all likely") -uses half as many items if dimensions are correlated -may be less irritating to ask one bipolar than two unipolar
Response Categories should be mutually exclusive
-by response categories being mutually exlusive we mean that the respondent should not feel compelled to answer more than one -how many years of social work experience do you have (none, 1-2 years, 3-5 years, 5 or more years) -Not mutually excluvie because for people with five years, it is unclear which category they should check
Constructs: example-weight*
-cannot observe (exactly or perfectly) weight* -measuring instrument (scale) provides a metric (pounds) for taking measurements (e.g., 120 pounds) -a measurement weight*--> scale -measurements are influenced by a construct's true value and random errors weight*--> scale <-- u1
Conceptualization and operationalization
-conceptual definition *process of defining, formulating, and clarifying concepts by --> specifying dimensions --> selecting indicators *example: health care use
Types of surveys
-cross sectional (needs assessment) -Longitudinal (following people over time) -General survey -typical election polls -cohort survey -birth/age cohorts -panel survey -follow same people over time
Response dimensions
-dimension that a question asks R to think about -has two aspects *structure--> categories built around structure *substance--> often an internal continuum with a label
Sordid Beginnings-Social Anthropology
-emerged as formal discipline during height of colonialism -asserted native peoples not capable of governing themselves; British colonialists would provide and maintain order -study of traditional peoples and their cultres: Ethnographic methods (late 19th early 20th C.)
Warnings!
-examples show problematic questions and suggested revision *revisions only address problem under consideration *revisions may still have "problematic" features -guidelines come from many sources -guidelines change as new research becomes available *UWSC docs are living documents
Main Types of Qualitative Notes
-field notes (Descriptive) -Personal -Methodological -Theoretical (Analytic memo)
Ambigious question
-how satifised are you with your field placement? (Are you referring to my suprvior, the types of skills I'm developing, the types of clients I see) -did you parents attend college? _Yes _No -overall how satisfied are you with the climate in Madison?
Qualitative Research Purpose
-identify -describe -understand -explain -generate
Participatory Action Research (PAR)
-involves community stakeholders in social change oriented research -who identify a community problem, develop research plan, collect data, and analyze findings -data inform recommendations for resolving primary concerns -to alleviate oppression or improve community or service quality
Question wording: Avoid words or phrases with a point of view
-leading or emotionally loaded words or phrases provide a positive or negative point of view or orientation toward an object
Other important considerations
-length of instrument -instructions for responding: new instructions for each different section/type of question (no matter how obvious it may seem to you....); wording of instructions is as important as wording of questions
Disadvantages of bipolar items
-may be difficult to label -structure relationships between dimensions -must consider whether to begin with negative end (more difficult to process) or positive end (more likelhy to get positive answer) -may be associated with lower data quality-acquiescence bias, less differentiation, etc.
Response categories: Unipolar scale labels
-most direct response dimensions --> importance, satisfaction, effectiveness *not at all *a little *somewhat *very *extremely How much --> More direct: how much help did you receive.... --> less direct: how much did setting goals help you... *not at all/none *a little *somewhat/some *quite a bit
The Nature of Qualitative Research
-not everything that can be counted counts, and not everything that counts can be counted
Parts of a survey question:
-personal characteristics -preamble -question stem -target question -target object -response dimension -response format
Question structure: Use parallel structures
-position reference periods and clauses in the same place -when items are repeated in lists, keep them in the same order and use the same wording Problematic questions -how old were you when you told that you had diabetes? -At what age did you start taking insulin? -When did you stop taking it? Revisions -How old were you when you were told that you had diabetes? -How old were you when you started taking insulin? -How old were you when you stopped taking insulin?
pilot testing
-provides insight into all areas of instrument-confusion, validity, annoying aspects, biases in measurement, wording, clarity, unncessary redunancy, instructions, structure, time required -often useful to discuss with participants and get feedback-focus groups -sometimes possible to include pilot tested group in full sample
Thoughts on a good cover letter
-purpose of study/survey -why important to participate? how will information be used that participant might find meaningful? -How will data be reported? are responses confidential or anonymous? -will participate be given any feedback on results? -Any incentive to participate? -Who is conducting the survey? Their affiliation? Who to contact if have questions or concerns? -put on agency letterhead and signed by person in authority (agency director)
"Chicago School"
-qualitative methods best suited for study of urban social phenomena -notion of "social Worlds" (complex inter-group patterns of social intractions) -"social disorganization" theory (neighborhood inability to solve problems together) -culture and conflict (how ethnic groups interact and compete) -ecologicaln theory: burgess-concentric zone model of urban growth
general rules
-question wording -question structure -response format -target object
Some common response formats: open
-record-verbatim open there have been a lot of national and world events and changes over the past 50 years-say from 1930 right up until today. Would you mention one or two such events or changes that seem to you to have been especially important
Question Wording: overview
-rules about actual words, terms, or phrases -avoid words or phrases that are difficult to understand, ambiguous, unclear, or use analytic vocabulary
The Chicago School 1920s
-sociologists adapted field research methods to study social groups in "modern" communities -1860-pop=10,000 -1910-pop=2 million -rapid increase produced homelessness, poor housing conditions, bad working conditions
Experimental design
-split-ballot, between-subjects design
Response format: overview
-structure of a question projects an answer -rules is about designing questions to project answers in the format you want
Negatively Worded Questions
-the community should not have a residential facility for the developmentally disabled 1. strong disagree 2. disagree 3. agree 4. strongly agree the ues of cognitive behavior therapy will not improve the well-being of people with PTSD 1. strong disagree 2. disagree 3. agree 4. strongly agree
Question structure:overview
-the following set of rules is about *how many subjects or verbs a question has *how clauses are joined *how phrases are repeated across questions *how multiple questions may be embedded in a single question -avoid using structures that *are compound *are complex *contain more than one subject, verb, or object *are not parallel
Transitions
-this first sections asks about your satisfaction with the courses that you are taking; How satifised are you with SW 650 1. very dissatisfied 2. dissatisfied 3. satisfied 4. very dissatisfied; please indicate how much you agree with the following statements: the social work building is accessible to persons with disabilities: 1. strongly agree 2. agree 3. disagree 4. strongly agree
core elements of grounded theory
-to discover social & psychological processes -analytic process prompts theory discovery and development (inductive) *create analytic codes and categories from the data *data collection and analysis proceed simultaneously
Some decisions for evaluations (Schaeffer & Dykema 2011)
-unipolar or bipolar *if bipolar: labeled midpoint -number of categories -category labels -filter questions -tailored response dimensions or statements
Some decisions for evlauations (Schaeffer & Dykema 2011)
-unipolar or bipolar *if bipolar: labeled midpoint -number of categories -cateogry labels -filter questions -tailored response dimensions or statements
tradeoffs
-using standardize questions to allow comparisons versus -rewording questions to correct problems
Some common response formats: closed
-yes-no response format -selection response format (from ordered categories) -discrete value response format
Problems with ABAB design
1. Carry-over effects-the effects obtained in one phase carry over to the next phase. Unable to retrieve the baseline phase. 2. May be unethical or unwise to try to return clients to pretreatment status 3. problems of internal validity-testing effects, dropout 4. problems of generalizability-importance of replication across individuals, settings, and measures
ABAB design
A phase (baseline) B phase (internvetion phase) A phase (return to baseline, withdrawal of treatment) Second B phase (reintroduce treatment)
Question wording: Avoid analytic vocabulary
Analytic vocabulary: languaged used by the analyst such as construct labels, technical terms, and jargon Native vocabulary: language used by respondents Analytic vocabulary: -occupants of this household -your responses to this questionnaire -work-related employment issues Revision: -people who live here -your answers -job concerns
Question structure: Avoid compound or complex question structures
Compound or complex question: includes two or more clauses Problematic question In the last 12 months, did you ever cut the size of your meals or skip meals because there wasn't enough money for food? Revision In the last 12 months, did you ever cut the size of your meals because there wasn't enough money for food?
Stable Baseline
Differences of opinion: Absence of a trend or presence of well defined slope
Improve standardization through question design
Goal: reduce random errors in measurement, focusing on the behavior of the interviewer (or respondent for SAQs) Interviewr-administered surveys *read questions as worded *in some surveys, provide neutral clarifications of survey concepts *record answers completely and accurately -self-administered surveys *read questions as worded *understand the meaning of words and phrases as investigators intended *record answers completely and accurately
Unipolar items
How satisfying is your marriage? How dissatisfying is your marriage? How interesting was that book? How boring was that book?
Bipolar items
How satisfying or dissatisfying is your marriage? How interesting or boring was that book?
Semantic Differential
My partner/spouse/boyfriend/girlfriend/lover is: 1 2 3 4 5 6 7 8 9 10 Boring Interesting 1 2 3 4 5 6 7 8 9 10 Unfriendly Friendly 1 2 3 4 5 6 7 8 9 10 Uncaring Caring 1 2 3 4 5 6 7 8 9 10 Cold Hot
Simplified wording example
Problematic question How much personal reflection have you done on your racial identity and how it impacts your relationships at work/school? Revision How much have you thought about how your racial identity influences your relationships at work?
Bipolar response scales: Use a labeled midpoint
Problematic question How satisfied or dissatisfied are you with your marriage: extremely satisfied, somewhat satisfied, somewhat dissatisfied, or extremely dissatisfied? Revision: How satisfied or dissatisfied are you with your marriage: extremely satisfied, somewhat satisfied, somewhat dissatisfied, extremely dissatisfied, or do you have mixed feelings? How satisfied or dissatisfied are you with your marriage: extremely satisfied, somewhat satisfied, somewhat dissatisfied, extremely dissatisfied, or are you in between?
Analytic vocabulary example
Problematic question: Are you a first generation college student? Revision: Which of the following best describes your parents: *neither of my parents graduated from college *one but not both of parents graduated from college *both of my parents graduated from college
Question wording: Use parallel wording
Problematic questions Q1: What competitive sports (school or club team) are you participating in now OR have you participated in within the last 12 months? Q2: What age did you start participating in organized, competitive sports? Revision Q1: The first questions are about organized, competitive sports that you play, either in school or as part of a club team. In the last 12 months, which of the following organized, competitive sports have you played? Q2: How old were when you started playing organized, competitive sports?
Questions with a point of view example
Problematic questions: Please rate the degree to which you believe each of the following factors contributes to the core problems of our state health care system: ---high cost of health care services ---shortage of primary care physicians ---patients' low level of health literacy Revision Please rate the degree to which each of these is a problem in our state's health care system: ---cost of health care services ---number of primar care physicians ---patients' level of health literacy
Response categories: Use 5 fully labeled categories for unipolar, 7 with bipolar
Problematic unipolar question: If 0 stands for not at all satisfied and 4 stands for extremely satisfied, what number from 0 to 4 stands for how satisfied you are with your marriage? Revision: How satisfied are you with your marriage: not at all satisfied, a little satisfied, somewhat satisfied, very satisfied, or extremely satisfied?
Contingency question
Q1. During the last 7 days, have you had 5 or more drinks on a single occassion? 1. yes 2. No (go to question 3) Q2. On how many of the last 7 days did you have 5 or more drinks 1. one to two days 2. three to four days 3. more than four days Q3. During the last 7 days, have you spent time listening to music?
Common Distinctions Made Between Qualitative and Quantitative Research
Qualitative (Inductive (discover; explore), naturalistic uncontrolled conditions, design may "emerge" as study unfolds;textual data (from audio, video, field notes); researcher IS instrument non-generalizable formative, earlier phases categories result from analysis data are "rich" and time-consuming analysis is ongoing many variables and few cases Quantiative (Deductive (test; confirm, explain) particularistic: scientific method --> controlled conditions --> design decided in advance numeric data standardized instruments -generalizable -tests hypotheses, latter phases -predetermine categories -data are more efficient, may miss contextual detail -analysis is end point -many cases and few variables
Number of response categories: Issues
Rs have to engage in a matching/mapping process Points on scale *be ordinal *need to cover range of possibilities * Rs must have precise and stable understanding *Across Rs must agree on interpretation
Single Subjects Design
a quasi-experimental design featuring continuous or nearly continuous measurements over a period of time on a single person
AB design
a single baseline phase followed by an intervention phase
Definitions of qualitative research
aim to gather in-depth understanding of human behavior and the reasons that govern such behavior -describe -understand -explain -identify -develop -generate -key is doing qualitative research is selecting the best method to answer your research question
Two Phases
baseline phase intervention/treatment phase
Question wording: use simplified wording
complex word: employment, exhausted, regarding, utilize revision: work, tired, about, use
close ended
fixed set of responses (yes/no; Likert scale; fixed set (e.g. Marital status) -reliable -easy to analyze
Approaches to Qualitative Research
key to doing qualitative research is selecting the best method to answer your research question
open ended
large range of possible responses -more likely to generate true and specific responses-validity -may be difficult to score/code and analyze due to larger variation -may not be properly answered -more likely to be left blank (in self-administered instruments)
Unstable Baseline
no discernable stable pattern
Types of questions
open ended closed ended
Baseline phase
prior to introduction of the intervention or treatment
measurement
process of assigning labels and numbers that represent levels of properties of constructs to units of analysis
waiting list control group
receive treatment after experimental group receives it
reliability and validity
reliability -extent repeated measurements with same instrument give the same results when value* of property has not changed validity -extent measuring instrument measures the concept it is supposed to measure
conceptualizing "health care use"
select indicators
conceptualizing "health care use"
specify dimensions *what kind of contact counts as a contact? *who qualifies as a health care provider? *frequency: what time is the metric? *what is the time frame? *self versus others? reason for use?
operationalization
statement of the exact procedures to be used to apply a measurement instrument to a unit of analysis to obtain a measurement of a value of a construct
Double Barreled Questions
the US should do everything necessary to prevent North Korea from developing a nuclear weapon and shoulds upport israel 1. strong disagree 2. disagree 3. agree 4. strongly agree -How satisfied are you with your pay and job conditions? -Should the government spend less money on the military and more on education?
Ethnography: Ethno-human culture; graphy-description of
the study of people in their 'natural' occurring settings
Baseline Stage
want a stable baseline prior to beginning treatment What is a stable baseline?
Intervention phase
when applying the treatment
Intervention Phase (B Phase)
when the treatment is applied