Research Methods Chapter #3

What are the three criteria that causal claims must satisfy?

1. Must establish that the two variables are correlated and cannot be zero 2. It must show that the causal variable came first and the outcome variable came later 3. It must establish that no other explanations exist for the relationship

Type I error

A "false positive" result from a statistical inference process, in which researchers conclude that there is an effect in a population when there really is none.

association claim

A claim about two variables, in which the value (level) of one variable is said to vary systematically with the value of another variable, such that when one variable changes, the other variable tends to change also.

causal claim

A claim arguing that a specific change in one variable is responsible for influencing the value of another variable.

frequency claim

A claim that describes a particular rate or degree of a single variable.

scatterplot

A graphical representation of an association, in which each dot represents one participant in the study measured on two variables.

conceptual definition

A researcher's definition of a variable at the theoretical level. Also called construct. See also conceptual variable.

margin of error estimate

A statistic, based in part on sample size for a poll, indicating the probable true value in the population.

experiment

A study in which one variable is manipulated and the other is measured.

positive association

An association in which high levels of one variable go with high levels of the other variable, and low levels of one variable go with low levels of the other variable. Also called positive correlation. See also curvilinear association, negative association, zero association.

How can the language used in a claim help you differentiate between association and causal claims?

Association claims: is linked to is at higher risk for is associated with is correlated with prefers are more/less likely to may predict is died to goes with Causal claims: causes affects may curb exacerbates changes may lead to makes sometimes makes hurts promotes reduces prevents distracts fights worsens increases trims adds

Which of the four big validities should you apply to a frequency claim? An association claim? A causal claim?

Frequency- construct, external, statistical Association- construct, external, statistical Causal- internal, construct, external, statistical

What question(s) would you ask to interrogate a study's construct validity?

How well the researchers measured their variables.

Define external validity, using the term generalize in your definition:

How well the results of the study generalize to or represent people or context besides those in the study, itself

dependent variable

In an experiment, the variable that is measured. In a multiple-regression analysis, the single outcome, or criterion variable, the researchers are most interested in understanding or predicting. Also called outcome variable. See also independent variable.

What is internal validity? Why is it mostly relevant for causal claims?

Internal validity means that a study should be able to eliminate alternative explanations for the association. Causal claims are used to express linked variables so they have to ensure no third variable interferes

Why don't researchers usually aim to achieve all four of the big validities at once?

It depends on research goals, it can be found to be impossible to conduct a study that satisfies all four validities at once.

level

One of the possible variations, or values, of a variable.

validity

The appropriateness of a conclusion or decision. See also construct validity, external validity, internal validity, statistical validity.

claim

The argument a journalist, researcher, or scientist is trying to make.

statistical validity

The extent to which statistical conclusions derived from a study are accurate and reasonable. Also called statistical conclusion validity.

operational variable

The specific way in which a concept of interest is measured or manipulated as a variable in a study. Also called operationalization or operational definition.

operational definition

The specific way in which a concept of interest is measured or manipulated as a variable in a study. Also called operationalization or operational variable.

random assignment

The use of a random method (e.g., flipping a coin) to assign participants into different experimental groups.

How are causal claims special, compared with the other two claims?

They make a stronger statement, they're held to a higher standard

correlate

To occur or vary together (covary) systematically, as in the case of two variables. See also correlational study, covariance.

operationalize

To turn a conceptual definition of a variable into a specific measured variable or manipulated variable in order to conduct a research study.

Which validity would you be interrogating by asking: how well did the researchers measure sensitivity to tastes in this study? a. construct validity b. statistical validity c. external validity d. internal validity

a. construct validity

Which of the following claims is an association claim? a. Tanning changes the brain. b. Depression in the elderly may predict dementia. c. A thoughtful gift makes the giver feel closer to the recipient. d. Teenage pot smoking affects intelligence.

b. Depression in the elderly may predict dementia. FEEDBACK: Table 3.3, Causal Claims — "Affects", "changes," and "makes" are causal claim verbs.

Which of the following headlines is a frequency claim? a. obese kids less sensitive to tastes b. exercise: 45% of you shake your booty in zumba c. feeling fat? Maybe Facebook is to blame d. daycare and behavior problems are not linked

b. exercise: 45% of you shake your booty in zumba

Which of the following is an example of a causal claim? a. Folic acid is tied to better toddler talk. b. Sweetened drinks are linked to depression risk. c. Ice applied to a twisted ankle prevents swelling. d. Women re-gift at a 50 percent greater rate than men.

c. Ice applied to a twisted ankle prevents swelling. FEEDBACK: Table 3.3, Causal Claims — "Is linked" and "tied to" are associative claims. "50 percent greater rate" is a frequency claim

Which of the following variables is manipulated rather than measured? a. number of pairs of shoes owned, in pairs b. a person's height, in cm c. amount of aspirin a researcher gives a person to take, either 325 mg or 500 mg d. degrees of happiness, rated on a scale from 1 to 10

c. amount of aspirin a researcher gives a person to take, either 325 mg or 500 mg

Which of the following questions addresses construct validity? a. To what populations, settings, and times can we generalize this claim? b. What is the effect size? c. Does the study control for alternative explanations? d. How well has the researcher measured or manipulated the variables in the study?

d. How well has the researcher measured or manipulated the variables in the study? FEEDBACK: Table 3.5, Interrogating the Three Claims Using the Four Big Validities — Interrogating construct validity means evaluating how well the variables were measured or manipulated in the study.

What kind of a claim is the following headline making? "You Gotta Have Friends? Most Have Just Two True Pals." a. anecdotal claim b. association claim c. causal claim d. frequency claim

d. frequency claim FEEDBACK: Table 3.3, Frequency Claims — This headline is claiming that most people have two friends.

For an association claim, you should interrogate all of the following validities EXCEPT __________. a. construct b. statistical c. external d. internal

d. internal FEEDBACK: Interrogating Association Claims — Association claims are not asserting causality so internal validity is not relevant to interrogate. Internal validity is relevant when you assert that one variable causes another to change.

How many variables are there in a frequency claim? An association claim? A Causal claim?

frequency - 1 association - at least 2 causal - 2

Marvin reads a journalistic report of a research study and asks, "How strong was the effect?" Which validity is Marvin asking about? a. internal validity b. construct validity c. statistical validity d. external validity

c. statistical validity FEEDBACK: Other Validities to Interrogate in Causal Claims — Statistical validity is concerned with the strength of effects.

manipulated variable

A variable in an experiment that a researcher controls, such as by assigning participants to its different levels (values). See also measured variable.

negative association

An association in which high levels of one variable go with low levels of the other variable, and vice versa. Also called inverse association, negative correlation. See also curvilinear association, positive association, zero association.

temporal precedence

One of three criteria for establishing a causal claim, stating that the proposed causal variable comes first in time, before the proposed outcome variable. See also covariance, internal validity.

In your own words, describe at least three things that statistical validity address:

What is the margin of error of the estimates? How strong the association is? What is the effect size?

zero association

A lack of systematic association between two variables. Also called zero correlation. See also curvilinear association, positive association, negative association.

correlational study

A study that includes two or more variables, in which all of the variables are measured; can support an association claim.

measured variable

A variable in an experiment whose levels (values) are observed and recorded. See also manipulated variable.

construct

A variable of interest, stated at an abstract level, usually defined as part of a formal statement of a psychological theory. See also conceptual variable.

conceptual variable

A variable of interest, stated at an abstract, or conversational, level. Also called construct. See also conceptual definition.

independent variable

A variable that is manipulated in an experiment. In a multiple-regression analysis, a predictor variable used to explain variance in the criterion variable. See also dependent variable.

constant

An attribute that could potentially vary but that has only one level in value in the study in question.

variable

An attribute that varies, having at least two levels, or values. See also dependent variable, independent variable, manipulated variable, measured variable.

construct validity

An indication of how well a variable was measured or manipulated in a study.

external validity

An indication of how well the results of a study generalize to, or represent, individuals or contexts besides those in the study itself. See also generalization.

covariance

One of three criteria for establishing a causal claim, which states that the proposed causal variable must vary systematically with changes in the proposed outcome variable. See also internal validity, temporal precedence.

internal validity

One of three criteria for establishing a causal claim; the ability to rule out alternative explanations for a causal relationship between two variables. See also covariance, temporal precedence.

generalizability

The extent to which the subjects in a study represent the populations they are intended to represent; how well the settings in a study represent other settings or contexts.

Type II error

a "miss" in the statistical inference process, in which researchers conclude that there is no effect in a population when there really is one.

There is a negative association between exercise and obesity. Which of the following would a researcher need to demonstrate in order to make the claim that a lack of exercise causes obesity? a. The lack of exercise came before the obesity. b. The association between exercise and obesity generalizes to the whole population. c. There is a positive relationship between exercise and being thin. d. There are no obese people who exercise.

a. The lack of exercise came before the obesity. FEEDBACK: Three Rules for Causation — This is the temporal precedence rule for causation. It is possible that because of the difficulty in exercising, obese people stop exercising after becoming obese.

Heather reads an article entitled, "Waist Circumference Associated with Life Expectancy." She wants to use the correlation reported in the article to predict her own life expectancy based on her waist circumference. What kind of correlation would be best for her to make this prediction? a. a strong positive correlation b. a weak negative correlation c. a weak positive correlation d. a zero correlation

a. a strong positive correlation FEEDBACK: Making Predictions Based on Associations — The stronger a correlation is the better it can be used to make predictions.

Which of the following headlines is a causal claim? a. holding a gun may make you think others are, too b .younger people can't read emotions on wrinkled faces c. strange but true: babies born in the autumn are more likely to live to 100 d. check the baby! Many new moms show signs of OCD

a. holding a gun may make you think others are, too

In most experiments, trade-offs are made between validities because it is not possible to achieve all four at once. What is the most common trade-off? a. internal and external validity b. construct and statistical validity c. statistical and internal validity d. external and statistical validity

a. internal and external validity

Professor Nakum designs a memory experiment to test the effect of word familiarity on memory. Three lists of words are created: common words, uncommon words, and made-up words. Participants are randomly assigned to study one of the lists of 30 words for 5 minutes, do math problems for 5 minutes, then write all the words they recall from the list. Their score is the number of words correctly recalled. The use of random assignment of participants increases which of the following? a. internal validity of the study b. the strength of association of the study c. the temporal precedence of the study d. external validity of the study

a. internal validity of the study FEEDBACK: Experiments Can Test Causal Claims — Random assignment controlled for possible alternative explanations.

Jenny reads the headline, "Tea Party Supporters Have Slipped To 20 Percent." She asks, "What is the margin of error of the estimate?" What validity is she interrogating? a. statistical validity b. construct validity c. external validity d. internal validity

a. statistical validity FEEDBACK: Statistical Validity of Frequency Claims — The margin of error of the estimate is a statistical figure that indicates where the true value in the population probably lies.

Phuong is conducting an experiment on proofreading. All participants are asked to read a passage with several typos and correct the errors. However, one group proofreads by reading silently, a second group proofreads by reading aloud, and a third group proofreads by reading the passage silently backward. The researcher calculates the average number of errors found in each group and finds that both the reading silently and reading aloud groups found more errors than the reading silently backward group. What are the variables in this study? a. type of proofreading and number of errors found b. number of errors found c. reading silently, reading aloud, or reading silently backward d. reading forward and reading backward

a. type of proofreading and number of errors found FEEDBACK: Variables — There are two variables in this study. The three types of proofreading (silent, aloud, and backward) are all levels of one variable — type of proofreading.

In most experiments, tradeoffs are made between validities because it is not possible to achieve all four at once. What is the most common tradeoff? a. Increased external validity results in decreased statistical validity. b. Increased internal validity results in decreased external validity. c. Increased statistical validity results in decreased internal validity. d. Increased construct validity results in decreased statistical validity.

b. Increased internal validity results in decreased external validity. FEEDBACK: Producers of Information: Prioritizing Validities — Experiments are studies designed to show causality, which depends on internal validity. In order to emphasize causality, the generalizability of the results is reduced.

Which of the following is NOT an operational definition of stress? a. length of time a participant submerges one hand in ice water b. feelings of anxiousness and pressure c. responses to a standardized stress questionnaire d. a measurement of the amount of a "fight or flight" hormone in saliva

b. feelings of anxiousness and pressure FEEDBACK: From Conceptual Variable to Operational Definition — "Feelings of anxiousness and pressure" is a conceptual definition and is not operationalized.

A recent headline read, "School Lunches Make Kids Fat." The article cited a study comparing children at two schools. The school with the lower obesity rate had few children who ate school lunches (lunch provided by the school). The other school had a greater percentage of obese children and most of these children ate school lunches. No mention was made that many of the children who eat school lunches are low-income children who receive free lunches. Poverty is a major factor in obesity. Which validity is threatened by the potential confound, poverty? a. statistical validity b. internal validity c. external validity d. construct validity

b. internal validity FEEDBACK: Interrogating Causal Claims — Confounds, or third variables, are an example of an internal validity problem — an alternative explanation for the finding.

Professor Lee is curious if there is any pattern to who earns extra credit in her course. She examines this question by graphing a scatterplot of her students' exam grades and the number of extra credit points earned. She finds that the students with the lowest exam grades tend to have the most extra credit points. What type of association is this? a. curvilinear b. negative c. positive d. causal

b. negative FEEDBACK: Association Claims — The lower the grade, the more extra credit points is an example of a negative association.

Reading the news on the Internet, Johan comes across the headline, "When Stress is Increased, Men Rush Ahead, Women More Cautious." (This headline is based on a study conducted by Lighthall et al., 2011.) In this study, men and women were asked to perform a decision task as many times as possible in a set period, in either a stressed or unstressed condition. In the unstressed condition, men and women performed similarly. However, in the stressed condition, the number of decision tasks performed by men increased while the number performed by women decreased. Which of the following is a variable in this study? a. the cause of the stress b. the amount of stress c. the decision task type d. the length of the set period

b. the amount of stress FEEDBACK: Variables — The amount of stress is varied between stressed and unstressed.

Professor Nakum designs a memory experiment to test the effect of word familiarity on memory. Three lists of words are created: common words, uncommon words, and made-up words. Participants study one of the lists of 30 words for 5 minutes, do math problems for 5 minutes, then write all the words they recall from the list. Their score is the number of words correctly recalled. Which of the following is the dependent variable in this study? a. the type of word list b. the number of words correctly recalled c. the length of the list studied d. the time spent studying the list

b. the number of words correctly recalled FEEDBACK: Experiments Can Test Causal Claims — The number of words recalled is what is being measured.

Professor Nakum designs a memory experiment to test the effect of word familiarity on memory. Three lists of words are created: common words, uncommon words, and made-up words. Participants study one of the lists of 30 words for 5 minutes, do math problems for 5 minutes, then write all the words they recall from the list. Their score is the number of words correctly recalled. Which of the following is the independent variable in this study? a. the number of words correctly recalled b. the type of word list c. the length of the list studied d. the time spent studying the list

b. the type of word list FEEDBACK: Experiments Can Test Causal Claims — The type of word list is what is being manipulated.

Professor Nakum designs a memory experiment to test the effect of word familiarity on memory. Three lists of words are created: common words, uncommon words, and made-up words. Participants study one of the lists of 30 words for 5 minutes, do math problems for 5 minutes, then write all the words they recall from the list. Their score is the number of words correctly recalled. How were the words for each list selected? a. to operationalize different levels of the independent variable "difficulty" b. to operationalize different levels of the independent variable "familiarity" c. to operationalize different levels of the dependent variable "memory" d. to confuse participants

b. to operationalize different levels of the independent variable "familiarity" FEEDBACK: Other Validities to Interrogate in Causal Claim — The lists vary in terms of their familiarity.

Which of the following criteria is NOT required for a causal claim? a. The independent variable came first and the dependent variable came later. b. There is no other explanation for the relationship. c. The correlation between the independent variable and dependent variable is zero. d. The independent variables and dependent variable are correlated.

c. The correlation between the independent variable and dependent variable is zero. FEEDBACK: Causal Claims — In the case that the correlation is zero, a causal claim cannot be made.

It has been reported that half of Americans show road rage. For this claim to have strong external validity, which of the following would have been the best sample for the researcher to have used? a. young men from ages 20 to 30; since they are the group most frequently charged with road rage, they should be the focus b. students from Intro to Psych courses who are earning extra credit c. a cross-section of American drivers representing men and women; people from the city, suburbs, and farms; of all ages d. a random sample of people from a Los Angeles telephone book

c. a cross-section of American drivers representing men and women; people from the city, suburbs, and farms; of all ages FEEDBACK: Interrogating Frequency Claims — The other choices represent only specific subsets of Americans.

Reading the news on the Internet, Johan comes across the headline, "When Stress is Increased, Men Rush Ahead, Women More Cautious." (This headline is based on a study conducted by Lighthall et al., 2011.) In this study, men and women were asked to perform a decision task as many times as possible in a set period, in either a stressed or unstressed condition. In the unstressed condition, men and women performed similarly. However, in the stressed condition, the number of decision tasks performed by men increased while the number performed by women decreased. In this study, the number of decision tasks performed is best described as which of the following? a. a constant b. a level of a variable c. a measured variable d. a manipulated variable

c. a measured variable FEEDBACK: Measured Versus Manipulated Variables — The number of decision tasks performed is measured.

What kind of a claim is the following headline making? "Former NFLers at Higher Risk for Brain, Mood Problems." a. frequency claim b. causal claim c. association claim d. anecdotal claim

c. association claim FEEDBACK: Table 3.3, Causal Claims — "At higher risk for" is an association claim verb.

Which validity would you be interrogating by asking: how did the researchers get their sample of people for this survey? a. construct validity b. statistical validity c. external validity d. internal validity

c. external validity

Which of the following headlines is an association claim? a. chewing gum can improve your mood and focus b. want to cheer up? cheer harder. Hard-core sports fans show less depressions c. swine flu shot tied to narcolepsy, study finds d. eating kiwis may help you fall asleep

c. swine flu shot tied to narcolepsy, study finds

Reading the news on the Internet, Johan comes across the headline, "When Stress is Increased, Men Rush Ahead, Women More Cautious." (This headline is based on a study conducted by Lighthall et al., 2011.) In this study, men and women were asked to perform a decision task as many times as possible in a set period, in either a stressed or unstressed condition. In the unstressed condition, men and women performed similarly. However, in the stressed condition, the number of decision tasks performed by men increased while the number performed by women decreased. Which of the following is a constant in this study? a. number of decision tasks performed b. sex of the participants c. the amount of stress d. the amount of time to perform the decision task

d. the amount of time to perform the decision task FEEDBACK: Constant — The time period is set. All the other choices vary in the study.

What is the name for the measured variable in an experiment? a. the testing variable b. the independent variable c. the constant d. the dependent variable

d. the dependent variable FEEDBACK: Experiments Can Test Causal Claims — In an experiment, the experimenter manipulates one variable, the independent variable, and measures the other, the dependent variable.