Research Methods Ch. 5-8

Snowball sampling

A variation on purposive sampling, a biased sampling technique in which participants are asked to recommend acquaintances for the study.

Observer bias

A bias that occurs when observers' expectations influence their interpretation of the participants' behaviors or the outcome of the study.

Quota sampling

A biased sampling technique in which a researcher identifies subsets of the population of interest, sets a target number for each category until the quotas are filled.

Purposive sampling

A biased sampling technique in which only certain kinds of people are included in a sample.

Reactivity

A change in behavior of study participants (such as acting less spontaneously) because they are aware they are being watched.

Observer effect

A change in behavior of study participants in the direction of an observer's expectation. (aka expectancy effect)

Statistical significance

A conclusion that a result from a sample (such as an association or a difference between groups) is so extreme that the sample is unlikely to have come from a population in which there is no association or no difference.

Cronbach's alpha

A correlation-based statistic that measures a scale's internal reliability.

Strength

A description of an association indicating how closely the data points in a scatterplot cluster along a line of best fit drawn in through them.

Stratified random sampling

A form of probability sampling; a random sampling technique in which the researcher identifies particular demographic categories of interest and then randomly selects individuals within each category.

Oversampling

A form of probability sampling; a variation of stratified random sampling in which the researcher intentionally overrepresents one or more groups.

Self-selection

A form of sampling bias that occurs when a sample contains only people who volunteer to participate

Population

A larger group from which a sample is drawn; the group to which a study's conclusions are intended to be applied.

Known-groups paradigm

A method for establishing criterion validity, in which a researcher tests two or more groups, who are known to differ on the variable of interest, to ensure that they score differently on a measure of that variable.

Poll

A method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via the Internet.

Survey

A method of posing questions to people on the telephone, in personal interviews, on written questionnaires, or via the internet.

Cluster sampling

A probability sampling technique in which clusters of participants within the population of interest are selected at random, followed by data collection from all individuals in each cluster.

Systematic sampling

A probability sampling technique in which the researcher counts off members of a population to achieve a sample, using randomly chosen interval.

Multistage sampling

A probability sampling technique involving at least two stages: a random sample of clusters followed by a random sample of people within the selected clusters

Ordinal scale

A quantitative measurement scale whose levels represent a ranked order, in which it is unclear whether the distances between levels are equal

Which of these samples is more likely to be representative of a population if 100,000?

A randomly selected sample of 100 people

Convenience sampling

Choosing a sample based on those who are easiest to access and readily available, a biased sampling technique.

Representative sample

A sample in which all members of the population of interest are equally likely to be included (usually through some random method) & therefore the results can generalize to the population of interest.

Biased sample

A sample in which some members of the population of interest are systematically left out, and as a consequence, the results from the sample cannot generalize to the population of interest (aka unrepresentative sample)

Census

A set of observations that contains all members of the population of interest.

Response set

A shortcut respondents may use to answer items in a long survey, rather than responding to the content of each item (aka nondifferentiation)

Correlation coefficient r

A single number, ranging from -1.0 to 1.0, that indicates the strength and direction of an association between two variables.

t test

A statistical test used to evaluate the size and significance of the difference between two means.

Describe which sampling techniques allow generalizing from a sample to a population of interest, and which ones do not.

Allow: Simple Stratified Proportionate Cluster, Multistage Not: Convenience sampling Purposive Snowball Self-selected

Mean

An arithmethic average, a measure of central tendency computed from the sum of all the scores in a set of data, divided by the total number of scores.

Bivariate correlation

An association that involves exactly two variables

Convergent validity

An empirical test of the extent to which a measure is associated with other measures of a theoretically similar construct.

Unobtrusive observation

An observation in a study made indirectly, through physical traces of behavior, or made by someone who is hidden or is posing as a bystander.

Acquiescence

Answering "yes" or "strongly agree" to every item in a survey or interview

Why are convenience, purposive, snowball, and quota sampling not examples of representative sampling?

Because they are not random samples so they do not generalize to the population

What is reactivity? What three approaches can researchers take to be sure people do not react to being observed?

Blend in Wait it out: get used to researchers presence Measure the behavior's result (ex: empty liquor bottles in a residential garbage cans indicates how much alcohol is being consumed in community)

Criterion validity

Both address how well a measure relates to a specific outcome. (Does it predict/correlate)

Explain why a variable will usually have only one conceptual definition but can have multiple operational definitions.

Conceptual definition is the researchers definition of the variable in question at a theoretical level. Operational definition represents a researcher's specific decision about how to measure or manipulate the conceptual variable.

Interrogate the construct validity of a study's variables.

Construct validity: a measurement of how well a variable was measured (or manipulated) in a study Measured variables: Variables whose levels are observed and recorded (with no manipulation)

What are four ways of selecting a biased sample of a population of interest? Which subsets are more likely to be selected in each case.

Convenience sampling: who is easiest to access Purposive sampling: Certain kinds of people they want to study Snowball sampling: Participants are asked to recommend few acquaintances Quota sampling: target a number for each category non randomly

Which requires stronger correlations for its evidence: convergent validity or discriminant validity?

Convergent validity

Content validity

Does it capture all parts of a defined construct?

Discriminant validity

Empirical test of the extent to which a measure does not associate strongly with measures of other, theoretically different constructs.

What are some ways to ensure that survey questions are answered accurately?

Ensure anonymity Use weed out items: Include "filler items" (Ex: Interested about racial attitudes, but also ask about politics, gender rolls, and education) Use implicit measures Ask about actions rather than attitudes

Experimenter expectations

Experimenter has certain expectations -> expectations alter experimenter's behavior toward participants -> expected response is more likely shown by participants

When will it be most important for a researcher to use a representative sample?

Frequency claim If you want to generalize beyond your sample, it matters. You can't always confirm this (you don't actually take a full census of an entire population that you've first sampled) Sometimes you can.... Election polling is confirmed (or not) by election results.

Socially desirable responding

Giving answers on a survey that make one look better than one really is.

Faking bad

Giving answers on a survey that make one look worse than one really is.

Internal reliability

In a measure that contains several items, the consistency in a pattern of answers, no matter how a question is phrased.

What types of sampling errors could occur through internet research/polling?

Internet research uses a nonrandom sample Participants are self-selected volunteers Participants know how to use computers Participants have access to computers Participants are Internet savvy (maybe)

Face validity

Is this a plausible measure of the variable? (does it make sense at a gut level)

Explain how carefully prepared questions improve the construct validity of a poll or survey.

It is crucial that each question be clear and straightfoward to answer so it does not confuse respondents or influence their answers.

Explain why a larger sample is not necessarily more representative than a smaller one.

Larger sample = smaller margin of error Statistical term quantifying the degree of error in the study. If 28% of people support a piece of legislation, with a margin of error of 3, then if you did the pool over and over, 95% of the time your result would be between 25-31%.

What are three potential problems related to the wording of survey questions? Can they be avoided?

Leading questions, double-barreled questions, negative wording. Avoided by testing different wording... If the results are the same no matter the wording, it clearly doesn't matter... If results differ, may need to report results differently for each question

Ethics of behavioral observation

Observing public behavior is considered ethical (no expectation of privacy) Researchers don't report on who they watched specifically Videotaping in public is usually okay too Using 1-way mirrors or private videotaping generally requires permission in advance

Describe the differences between ordinal, interval, and ratio scales.

Ordinal allows you to say "first, second, third" Interval has equal distance between level but no true zero so can't say "twice as much" etc. Ratio has an absolute zero

Self-report measure

People answer questions about themselves in a questionnaire or interview.

Fence sitting

Playing it safe by answering in the middle of the scale fir every question in a survey or interview.

Quantitative variable

Quantitative

Ratio scale

Quantitative measurement in which the numerals have equal intervals and the value of zero truly means "nothing"

Interval scale

Quantitative measurement scale that has no "true zero," and in which the numerals represent equal intervals (distances) between levels

Negatively worded question

Question in a survey or poll that contains negatively phrased statements, making its wording complicated or confusing and potentially weakening its construct validity.

Which of the following correlations is the strongest: r = .25, r = -.65, r = -.01, or r = .43?

R= -.65 because it's closest to -1.0

In your own words, describe the difference between random sampling and random assignment.

Random sampling: Researchers draw a sample randomly Random assignment: only used in experiments; researchers randomly assign participants to groups; helps ensure that the groups are the same at the start of the experiment

Physiological measure

Recording biological data.

Observational measure

Recording observable behaviors of physical traces of behaviors.

Describe the kinds of evidence that support the construct validity of a measured variable.

Reliability: how consistent is the measurement? Validity: is it measuring what it's supposed to measure?

Semantic differential format

Response scale whose numbers are anchored with contrasting adjectives.

Rosenthall effect

Rosenthal was one of the researchers for both the intellectual bloomers AND the maze bright/dull studies. Study showed that observers not only see what they expect to see; sometimes they even cause the behavior of those they are observing to conform to their expectations.

Faking good

Same as socially desirable responding.

Name three common ways in which researchers operationalize their variables.

Self report Observational Physiological

For each of the three common types of operationalizations (self-report, observtional, and physiological) indicate which types of reliability would be relevant.

Self-report: test-retest and internal reliability Observational: interrater reliability Physiological: interrater

What are five techniques for selecting a representative sample of a population of interest? Where does randomness enter into each of these five selection processes?

Simple Stratified Proportionate Cluster Multistage During random selection

To establish criterion validity, researchers make sure the scale or measure is correlated with

Some relevant behavior or outcome.

Explain why external validity often matters for a frequency claim.

The findings need to generalize to a larger population or to other settings

Masked design

Study design in which the observers are unaware of the experimental conditions to which participants have been assigned (aka blind design)

Forced-choice format

Survey question format in which respondents give their opinion by picking the best of two or more options.

Open-ended question

Survey question format that allows respondents to answer any way they like.

Likert scale

Survey question format; a rating scale containing multiple response options that are anchored by the terms 'strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. Likert-type scale does not follow this format exactly.

Reliability is about consistency. Define the three kinds of reliability, noting what kind of consistency each is designed to show.

Test-retest reliability: test is given twice, scores from the two are compared to see if same Interrater reliability: two observers rate some type of behavior and agree with each other Internal reliability: do the different items correlate well with each other

Validity

The appropriateness of a conclusion or decision.

Test-retest reliability

The consistency in results every time a measure is used.

Reliability

The consistency of the results of a measure

Interrater reliability

The degree to which two or more coders or observers give consistent ratings of a set of targets.

Sample

The group of people, animals, or cases used in a study; a subset of the population of interest.

Effect size

The magnitude of a relationship between two or more variables.

Simple random sampling

The most basic form of probability sampling, in which the sample is chosen completely at random from the population of interest.

Probability sampling

The process of drawing a sample from a population of interest in such a way that each member of the population has an equal chance of being included in the sample, usually via random selection.

Observational research

The process of watching people or animals and systematically recording how they behave or what they are doing.

Slope direction

The upward, downward, or neutral slope of the cluster of data points in a scatterplot.

Random assignment

The use of a random method to assign participants into different experimental groups.

What do face validity and content validity have in common?

They both are subjective ways to assess validity.

For which topics, and in what situations, are people most likely to answer accurately to survey questions?

They make an effort to think about each question, they don't worry about looking good or bad, simply because they are unable Self-report is the best way.

What is the difference between observer bias and observer effects? How can such biases be prevented?

Train observers well Create clear rating scales (codebooks) Multiple observers

Double-barreled question

Type of question in a survey or poll that is problematic because it asks two questions in one, thereby weakening its construct validity.

Leading question

Type of question in a survey or poll that is problematic because its wording encourages only one response, thereby weakening its construct validity.

Categorical variable

Variables whose levels are categorical

Why do you think researchers might decide to use an unrepresentative sample, even though a random sample would ensure external validity?

When studying association or causal claims. When it actually matters (reviews on shoes online)

Describe how researchers can make observations with good construct validity.

When they can avoid three problems: observer bias, observer effects, and reactivity