SOC 201 Exam 2

Content Analysis

• Systematic analysis of existing documents • Requires coding system - Unstructured text to variables - Coded by human, computer algorithm or both • Unstructured text data is abundant - Newspapers, television transcripts, books - Blogs and social media

Multiple Measures to Maximize Sensitivity

• Take multiple measures. - In other words, more than one DV. • Test a prison rehabilitation program. • Experimental design? • What does rehabilitation mean? • Recidivism • Arrests • Self-reported criminal behavior • Job-status

Stratified random sampling

divide the population into subgroups (i.e. strata) and then randomly sample within strata. - Example: The names of all 10,000 College X students are sorted by major, and a computer program randomly chooses students from each major.

Probability Sampling

each member of the population has a specific probability of being sampled. -Prob. sampling is better than non-prob. sampling.

Simple random sampling

every member of the population has an equal chance of being selected

Pretest-Posttest

experimental designs are useful because we measure the DV before and after treatment (the IV).

Content Analysis + Archival Research =Quantitative, Observational Study

i think we need to know this

Cluster Sampling

identify clusters, randomly sample clusters, then sample all individuals within the selected clusters. - Example: 200 clusters of sociology majors are identified at schools all over the US. Out of these 200 clusters, 10 clusters are chosen at random. All individuals in each cluster are sampled. • In other words, choose 10 schools at random, and survey all sociology majors at those 10 schools.

Anchoring

is a cognitive bias in which one relies too heavily on the first piece of information offered (the "anchor"). • Replicated many, many times. • Occurs in lab experiments even when number is demonstrated to be random • Effective in negotiations- E.g. a car's sticker price

panel study

is a research study in which the same people are surveyed at two or more points in time. - Can track individual-level changes over time - Each survey administration is called a wave. • The first survey administration is called Wave 1. • The Second Wave 2, the third Wave 3, and so on... - More complex to administer

Manipulating the Independent Variable staged minipulations

In staged manipulations, the difference between conditions exists in the events the participant witnesses or participates in. These events are orchestrated by the experimenters. - Deception likely involved - Used when your target of the study is not amenable to normal experimental procedures (e.g. pen and paper tests) - May use confederates

Floor effect

all scores are near the lowest possible value. - Example: Memory task is too difficult.

Population in sample

all the individuals of interest to the researcher

Choosing Between Between-Subject and within-Subject Designs

• Advantages of within-subject designs - Need fewer subjects - Removes group differences as each subject is his own control • But sometimes within-subject designs are impossible and impractical - Examples: smoking cessation programs, vaccine testing - Often, between-subject designs are easier to implement in practice • Example: it may be easier to recruit 40 people for a one-day experiment than 20 people for a two-day experiment

Archival Research

• Archival research involves using previously compiled information to answer research questions - Statistical Records - Survey Archives - Written and Mass Communication Records - Example: General Social Survey (GSS) • Employment, political views, family structure

Expectancy Effects

• Biased data can result from actions taken by the experimenter who has expectations about how the data will turn out. • Example: "smart" mice running mazes • Eliminate experimenter bias with: - Consistent delivery - Simultaneous treatment - Automated delivery of the experiment - Double-blind experiments • Double-blind - neither the participant nor the experimenter knows who is receiving the drug or the placebo.

Debriefing

• Debriefing is the period at the end of the study reserved for explaining the experiment to the participant. - Absolutely necessary when deception has been employed. • Debrief: 1. Thank the subject for their participation. 2. Ask the subject about their experience. 3. Explain the hypothesis being tested. 4. Ask the subject not to discuss the experiment with others.

Continuous Responses Maximize Sensitivity

• Prefer continuous responses over discrete responses. - Example: measuring health outcome. - Discrete: Obese/Not Obese - Continuous: • Weight • BMI • Body Fat %

Naturalistic Observation process/problems

• Process - Describe the setting, events, and persons - Analyze the categories that emerge - The researcher must interpret what occurred - Generate hypotheses that help explain what was observed - Write a final report of the results • Problems - Researcher participation can affect what is to be observed. - Research concealment poses ethical quandaries.

Rating Scales

• Rating scales ask people to provide "How much?" judgments. - E.g. agreement, liking, or confidence - The president should send ground troops to fight ISIS in Syria. • Strongly Agree ---------------------------- Strongly Disagree - How much did you enjoy the movie? • 1 (not at all) to 10 (very much) - How confident are you that the Cleveland Browns will win the Super Bowl this year?• 0% to 100%

Manipulation Check

• Researchers sometimes perform a manipulation check to ensure that manipulation has had the desired effect. • Example: emotion experiments

Selecting Research Participants

• Sample the population of interest - Probability sampling if necessary or practical - More representative sample greater external validity - Consider:• Representative samples are crucially necessary for surveys. • Representative samples are welcome, but convenience samples are OK for experiments. • Sample size - More is always better - Larger N leads to more precise estimates, higher statistical power

Confidence Intervals

• Sampling a population introduces sampling errors in our measurement. • We didn't measure the whole population, so we can't know the exact value of a variable; we can only estimate. • Thankfully, we can quantify our uncertainty.• Confidence Interval - an interval likely to contain the true value of a variable for a population.

Why conduct surveys?

• To find out what people feel, think, and do • To study relationships between variables • To study how attitudes, beliefs and behaviors change over time • To provide useful information for making public policy decisions • Because they are relatively easy and fast

Measuring the Dependent Variable

• Types of measures - Self-report • Agree/Disagree, Have you ever X? How often do you X? How much do you like X? • Likert Scales are useful self-report measures. • Continuous response options are better than yes/no options. - Behavioral • Direct observations of behavior- Rate- Duration- Accuracy- Reaction time - Physiological • Galvanic skin response (GSR)• Electromyogram (EMG)• Electroencephalogram (EEG)• Functional magnetic resonance imaging (fMRI)

Maximize the Strength of the Manipulation

• Weak manipulations lead to ambiguous results. • Maximize the strength to prove the existence of a relationship. • After the relationship is observed, vary the manipulation strength to characterize the full curve.

Latin Squares

• With many conditions in a repeated measures design, running all possible orders becomes impossible. • Instead, select orders with a Latin square. • If you have N conditions, the Latin square - Is size N x N - Ensures each condition appears once in each ordinal position - Ensures each condition precedes and follows each other condition one time

Case Study

• Describe an individual instance in detail. • Like an ethnography but for only one person • Very useful for rare cases • Advantages: better for story-telling, memory • Disadvantages: N=1, worst possible external validity, reactivity, and selection biases

Maximize the Sensitivity of the Measure

• Experiments are expensive and time-consuming. - You may only have one chance. - Make sure your measurement is sensitive enough to detect any change. - Maximize sensitivity by: • Taking multiple measures. • Preferring continuous response over yes/no discrete outcomes. • Avoiding ceiling and floor effects.

naturalistic observation

• Field Work • Researchers make observations in a natural setting over a period of time • Used to describe and understand how people in a social or cultural setting live, work, and experience the setting •"People-watching"

The most basic experiment has:

- 1 independent variable with 2 levels • Control versus Experimental - 1 dependent variable

Matched Pairs Design

Matched pairs designs compare pairs of subjects pre-matched on one or more variables. • Attempts to capture the benefits of a within-subjects design .• Only used when low N of subjects available .• Example: training sprinters

Minimize the Cost of the IV

Minimize the cost of the manipulation, so you can run many experiments. • Expensive - Confederates - Equipment - Computer programmers • Inexpensive - Undergraduates - Online experiments - Doing your own programming

Quantitative methods compared to qualitative

-less depth and nuance -often previous experience required -easier to provide objective results -methods are well defined and reproducible

Qualitative methods compared to Quantitative

-more depth and nuance -good place to start -more difficult to remain objective -methods may be impossible to reproduce

Ceiling effect

all scores are near the highest possible value. - Example: Memory task is too easy.

Convenience Sampling

"haphazard sampling" sample individuals in the most convenient fashion - Example: a researcher stands in front of the library and asks every student who walks by to take a survey.

Constructing Good Survey Items things to avoid

- Avoid technical terms and jargon. - Avoid ill-defined terms. • Avoid these pitfalls - Double-barreled questions • Do you believe in ghosts and the afterlife? • Please tell me whether you would vote for or against a candidate who supports reducing federal spending on education and welfare. - Loaded words in the item • Should Americans buy imported automobiles that take away American jobs? • Avoid these pitfalls - Accidentally introducing a double negative • Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened? • Are you for or against not allowing gays and lesbians to legally marry?

Independent Variable (IV)

- What the experimenter manipulates - The different experiences a person has depending upon if they are in the control or experimental condition

Dependent Variable (DV)

- What the experimenter measures - Measured to observe the difference in outcomes between the control and experimental treatments

Qualitative methods

-observe and report -describes people, their behavior, their environment -typically uses smaller samples -data are non-numerical -results are expressed as verbal descriptions and narratives

Quantitative methods

-requires variables outcomes must be countable -tests hypothesis -typically uses larger samples -data are numerical -results are expressed as comparisons of numbers

Placebo Effect

A placebo effect occurs when the participants expectation of the effect of an experimental manipulation causes the effect. -Eliminate placebo effect with blinding: - Single-blind - the participant does not know if they are receiving a drug or a placebo.

Repeated Measures

Advantages • Removes group differences as a source of random error • Requires fewer subjects • Another way of saying the above is that within subject designs have greater statistical power than between subjects designs. Disadvantages • Order effects - Practice effect - Fatigue effect - Carryover effect

Systematic Observation

Careful observation and counting of specific behaviors • In "the field," not the lab .• Systematic Obs = Naturalistic Obs + Counting • In all observation studies, we need to worry about - Reactivity - Reliability - Generalizability

Counterbalancing

Counterbalance condition orders to identify and compensate for order effects.

Constructing Good Survey Items

First, define the research question. - What do you want to know? - Do you want to describe how things are? - Do you want to look for a relationship between variables?• Define the target measure(s). - Facts and demographics -Attitudes and beliefs - Behaviors Keep items simple. - Use short sentences.

Minimize the Cost of the DV

Minimize the cost of the measure, so you can run many subjects. • Expensive - Equipment - Hard to find subjects - Human raters • Inexpensive - Pen and paper - Online experiments - Automated rating

Ethnography

The word 'Ethnography' is derived from- the Greek ἔθνος (ethnós), meaning "a company, later a people, nation" and - 'graphy meaning "field of study". - Ethnographic studies focus on large cultural groups of people who interact over time. - Ethnography is a qualitative method, where the researcher discusses their observations about learned patterns of values, behavior, beliefs, and language of a culture shared by a group of people

Sampling

To estimate the value of a variable for an entire population, we measure the value in a sample

Confounding Variable

Varies with the IV, but not part of the experiment

Pitfalls of Surveys

What people say, what people do, and what people say they do are all different things. • Be aware of the Lizardman Constant- A (hopefully small) percentage of respondents will give meaningless responses. - Some respondents are hostile, joking, or aren't paying attention.

Response Rate Bias

potential measurement error due to some respondents choosing not to answer. - When some respondents do not answer, we do not know which option they would have chosen. Even with perfect sampling, response rate bias can introduce error into your measurement. • Response rate - the percentage of sampled individuals who actually complete the survey. • Response rate bias can occur when different groups vary in their response rate. - Example: younger people are less likely to complete and mail in a survey sent by postal mail .- Can be corrected with statistical methods- Or younger people can be oversampled to make up for the response rate bias

Sampling Techniques

• A sample is best if it is representative of the population .• The sample should "look like" the population. • Bias results from a systematic difference between the sample and the population.

Survey Vocabulary

• A survey is made up of items and is administered to respondents. - A survey might also be referred to as a questionnaire or instrument. - Respondents are the individuals who take the survey - An item consists of prompt and response options • Items don't have to be questioned, but frequently they are.

Quota Sampling

take a haphazard sample of individuals until you reach a quota for each subgroup of interest - Example: a researcher stands in front of the library and asks the first 50 females and the first 50 males they see to complete their survey. - Advantage: sample reflects the numerical composition of various subgroups in the population

Purposive Sampling

take a haphazard sample of those who meet a criterion - Example: a marketer asks only young people exiting a theater what they thought of the movie. - Example: advertise a survey on Twitter to only those who follow FOX News - Advantage: sample includes only the types of people you are interested in.

Non-probability Sampling

the probability of any member of the population being sampled is unknown.

repeated survey

the same questions are asked of different people at recurring intervals. - Can track population-level changes over time - Easier to administer

Social Desirability Bias

the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others. - Over-reporting "good behavior" and/or under-reporting "bad", or undesirable behavior.

Sample

those individuals chosen to be in a study

Demand Characteristics

• A demand characteristic is any feature of the experiment that informs participants of the desired results from the experiment. • Avoid demand characteristics with - Deception - Filler items • Measure with - Post-experiment questionnaire

Pilot Study

• A pilot study is a small, quick version of a more elaborate planned study. • Useful for: - Training administrators - Clarifying instructions - Seeing the experiment from the participants' point of view - Previewing the results

Time Interval Between Treatments

• In addition to order effects, a researcher must be cognizant of the appropriate time interval to place between conditions in a repeated measures design. • A long interval can allow for rest, forgetting or metabolic changes that may or may not be desirable between conditions. • Multi-session experiments are more likely to suffer subject attrition / mortality.

Manipulating the Independent Variable straightforward manipulations

• In straightforward manipulations the difference between conditions exists in the instructions, stimuli or procedure. - No deception - Most research involves straightforward manipulations .- Example: In our in-class memory experiments, I just want you to follow every instruction I give you.

Manipulating the Independent Variable

• Is deception justified? - It may invalidate informed consent. - It may decrease the trust subjects have in experiments .- But some things are impossible to test in a straightforward way. - Use debriefing to mitigate the potential problems of deception.

Likert Scales

• Likert scales are rating scales. • Named for Rensis Likert, an American psychologist. • Likert scales are symmetric. • Responses are assumed to be of numerical value and equally spaced. • Example: Stony Brook University should increase tuition.- Strongly Disagree = -2 - Disagree = -1 - Neither Agree nor Disagree = 0 - Agree = 1 - Strongly Agree = 2

Basic Experiments

• Manipulate the independent variable. - Assign IV level to a subject with random assignment • A chance procedure (such as a random number generator or coin flip) determining an individual's participation condition. - Eliminate confounding variables • Confounding variables are factors that vary accidentally along with the independent variable. If present, they include a straightforward interpretation of the results. • Measure the dependent variable.

Lab-Based Observation

• Measurement of multiple variables in non-natural setting (i.e. lab). - Not experimental • Example: Red Brain, Blue Brain?

Sample Size

• Obviously, we'd like to minimize the size of our confidence interval. • Larger sample sizes decrease uncertainty. • Larger sample sizes decrease the width of the confidence interval. • Larger sample sizes are better. They allow us to estimate with more precision.

Experimental Design -Assignment to Condition

• Our anchoring demonstration used an independent groups design• Independent groups == between subjects • Subjects are randomly assigned to the Control condition or the Experimental condition • In repeated measures design every subject is in both the Experimental and the Control condition. • Repeated measures == within subjects • Random assignment is not necessary, because every subject is in all conditions. • Each subject is under his own control.