CH 6: Research Strategies and Validity

curvilinear relationship

non-linear, but consistent, similarity between two variables

Mila studies social behavior in nonhuman primates and generalizes her results to humans. Source of threat:

participants

assignment bias

threat to internal validity resulting from groups with noticeably different characteristics

research strategy

A general approach/plan to research determined by the kind of question that the research study hopes to answer.

research design

A general plan for implementing a research strategy. A research design specifies whether the study will involve groups or individual subjects, will make comparisons within a group or between groups, or specifies how many variables will be included in the study.

the novelty effect

A threat to external validity that occurs when individuals participating in a research study (a novel situation) perceive and respond differently than they would in the normal, real world.

multiple treatment interference

A threat to external validity that occurs when participants are exposed to more than one treatment and their responses are affected by an earlier treatment. ex: fatigue, practice

practice

A threat to internal validity that occurs when prior participation in a treatment condition or measurement procedure provides participants with additional skills that influence their performance on subsequent measurements. An example of an order effect.

fatigue

A threat to internal validity that occurs when prior participation in a treatment condition or measurement procedure tires the participants and influences their performance on subsequent measurements; an example of an order effect.

extraneous variable

Any measured construct that exists within a study other than the variables being studied. In an experiment, any variable other than the independent and dependent variables.

Consider the following scenarios and decide whether each is an example of the correlational or the nonexperimental strategy. A survey of 200 people showed that the more they exercised, the less they weighed. A recent study found that male college students exercise more than females.

Correlational; nonexperimental

time-related variables

Environmental or participant variables or factors like practice and fatigue that change over time. A threat to the internal validity of studies that compare measures of the same individuals taken at different times.

Lennie develops a therapeutic intervention for children who have witnessed domestic violence incidents. Aware that her own passion about the issue might affect outcomes, she trains her most senior graduate students to perform the intervention when it is tested. Lennie is trying to avoid:

Experimenter bias

Results from a research study suggest that a stop-smoking program is very successful. However, the participants who volunteered for the study were all highly motivated to quit smoking and the researcher is concerned that the same results may not be obtained for smokers who are not as motivated. What kind of validity is being questioned?

External validity

Demand characteristics will be minimized in which setting?

Field study

A researcher measures mood for a group of participants who have listened to happy music for 20 minutes and for a second group who have listened to sad music for 20 minutes. If different mood scores are obtained for the two groups, the researcher would like to conclude that music influences mood. However, the happy music group was tested in a room painted yellow and the sad music group was in a room painted dark brown and the researcher is concerned that the room color and not the music may influence mood scores. What kind of validity is being questioned?

Internal validity

What aspect of a study is threatened if the participants are tested in one treatment condition at one time and then tested in a second treatment condition at a different time?

Internal validity

Is the following scenario an example of an experiment? A researcher measures the amount of exercise each adult in a group gets and how much each adult weighs. The study demonstrates that the more adults exercise, the less they weigh.

No

A research study attempts to describe the relationship between self-esteem and birth order position by measuring self-esteem for each individual in a group of first-born boys, and then comparing the results with self-esteem scores for a group of later-born boys. Which research strategy is being used?

Nonexperimental

William is interested in the relationship between caloric intake and emotional arousal. He has a series of in-depth discussions with a morbidly obese individual who identifies himself as an emotional overeater and then describes the relationship between emotional arousal and caloric intake using illustrations from the individual's life.

Nonexperimental

individual differences

Personal characteristics that differ from one participant to another. Individual differences are part of every study. Individual differences can produce high variability in the scores and can, for studies that use different groups for each treatment condition, if there are consistent differences between the groups, individual differences can become a confound.

Fergie is studying how children whose siblings have been diagnosed with cancer deal with the anxiety that comes with that diagnosis. She has developed an arts and crafts-based intervention that she believes will help the siblings process their feelings. Before the program starts, she administers an assessment of illness-associated anxiety and concerns. In doing so, she has introduced the possibility that the questions she asks will cause some of the youngsters to become more aware of their concerns. This is referred to as:

Pretest sensitization

External validity (generalizability)

The extent to which we can generalize the results of a research study to people, settings, times, measures, and characteristics other than those used in that study.

A journal article reports that a new teaching strategy is very effective for first-grade students. A teacher wonders if the same strategy would be effective for a class of third-grade students. What is the teacher questioning?

The external validity of the report

How can sensitization threaten external validity of a study?

The results may be limited to individuals who have experienced a pretest.

Which of the following questions can be addressed with the descriptive strategy?

What is the average number of text messages that a typical adolescent sends in a month?

Vonda studies children's reading skills and how parents can help their children become better readers. She asks parents to estimate the amount of time they read to their children per week and then relates the parental estimates to the children's word recognition scores. This is best described as

a correlational design.

Chloe believes that various cognitive techniques like "thought stopping," in which individuals consciously tell themselves to "stop" when they think negative thoughts, may help students who have test anxiety. She recruits 50 students with test anxiety, has everyone complete a test-anxiety assessment, randomly assigns participants to a treatment group that is taught thought stopping or a control group that meets for a discussion of student life, and after midterm exams, readministers the test-anxiety assessment. This is best described as

an experimental design.

quasi-experimental research strategy

attempt to limit threats to internal validity and produce cause-and-effect conclusions

A study examining the relationship between humor and memory compares memory performance scores for one group presented with humorous sentences and a second group presented with nonhumorous sentences. The participants in one group are primarily 8-year-old students and those in the second group are primarily 10-year-old students. In this study, age is potentially a(n) ________ variable.

confounding

Brittney is interested in the relationship between caloric intake and emotional arousal. She creates a carefully measured array of attractive snack foods. Brittney allows participants to eat as much of various snacks as they want while completing a written measure of emotional arousal. She then calculates the calories they consumed based on the amount of food that is left and looks at the relationship between the amount of calories each participant consumed and his or her emotional arousal score.

correlational

Identify the source of the threat being described by each scenario in the following table. Gordon is conducting a memory study that consists of two treatment conditions. All goes well on the first day when he is working with the first treatment condition. However, on the second day, violent thunderstorms descend when he is working with the second treatment condition.

environment

Sebastian is interested in the relationship between caloric intake and emotional arousal. He randomly assigns his research participants to one of three treatment conditions—high, medium, or low emotional arousal. Sebastian has all participants play a simple arcade-style computer game and varies emotional arousal by having frequent distracting interruptions (high), occasional distracting interruptions (medium), or no distracting interruptions (low). He then has them wait alone for five minutes with access to carefully premeasured snacks before thanking them for their participation and allowing them to leave. He calculates the calories consumed based on the amount of food left and compares the amount of food consumed by each of the groups.

experimental

A study of attitudes toward organic foods conducted with a wealthy, well-educated urban sample

external

A study of memory function conducted with introductory psychology students

external

correlational research strategy

general approach to research that involves assessing the relationship between two variables

Experimental research studies tend to have very _______ internal validity but often have relatively _______ external validity.

high; low

Identify the source of the threat being described by each scenario in the following table. Donna assigns her research participants to groups based on their scores on a pretest. However, she finds that in addition to differing on their pretest scores, the groups differ in their age and educational attainment as well.

individual differences

A study of emotional arousal in which research assistants are sloppy about how they implement the treatment conditions

internal

An observational study of children's reactions to an individual who is upset, in which the observers are not well-trained on the use of the rating scale

internal

Ruth has developed an intervention for overweight children and their families. She assesses her intervention at the end of its 12 weeks of classes and finds that the children and their parents have increased knowledge of nutrition and report healthier eating habits. Source of threat:

measures

Tai is conducting research on creativity and has selected one of the newer ways of assessing creativity that has been widely criticized. Source of threat:

measures

Jon conducts his studies of child-rearing practices at the onsite daycare for office workers at a software-development company and generalizes his results to all parents. Source of threat:

participants

Malorie is interested in the relationship between caloric intake and emotional arousal. She has participants complete a questionnaire while waiting alone with a carefully premeasured buffet of attractive snacks and then calculates caloric intake by the food that is left. One of the questions asks participants to indicate, with a yes or no response, whether they consider themselves to be emotional. Malorie compares the amount of calories consumed by people who consider themselves to be emotional to the amount of food consumed by people who do not consider themselves to be emotional.

quasi-experimental

linear relationship

similarity between two variables that approximates a one to one association

Charley wants to be as time- and cost-effective as possible in his study of learning, so rather than bringing his participants back for several, shorter sessions, he puts his participants through an 11-hour day packed with tasks and tests. Source of threat:

study

Quincy adores doing research with children and often chats and plays with the children in ways unrelated to the research while data are being collected. Source of threat:

study

Identify the source of the threat being described by each scenario in the following table. Tina is studying the effects of paint color on motivation. She has a single group of participants who come back on five consecutive days and spend an hour in a room painted a specific color while completing a rather dull task. On Monday, the participants are in a pale pink room in the morning, on Tuesday they are in an aqua room around lunchtime, on Wednesday they are in an emerald green room in the early afternoon, on Thursday they are in a red room in the late afternoon, and on Friday they are in a beige room in the evening.

time-related

correlation

value that measures and describes the direction and degree of relationship between two variables

volunteer bias

A threat to external validity that occurs because volunteers are not perfectly representative of the general population.

threat to internal validity

Any factor that allows for an alternative explanation for the results of a study.

The descriptive strategy is intended to answer questions about the current state of individual variables for a specific group of individuals. Can the following question be addressed with the descriptive strategy? What is the average income level of a recent college graduate?

Yes

Lee is interested in the development of children's social skills and believes that being in daycare may help children develop their social skills. He has observers rate children's social behavior on playgrounds and finds that children who are in daycare differ from children who are not in daycare. This is best described as

a nonexperimental design.

In an effort to reduce underage drinking, Student Life offers no-cost, alcohol-free weekend events in two of the university's four undergraduate dormitories. At the end of the semester, the number of incidents related to underage drinking has gone down at both dorms, but the decrease was much greater at the dorms with the special events. This is best described as

a quasi-experimental design.

selection bias

a threat to external validity that occurs when the sampling procedure favors the selection of some individuals over others if a sample does not accurately represent the population, then there are serious concerns that the results obtained from the sample will not generalize to the population.

internal validity

extent to which a study produces a single explanation for the relationship between two variables

confounding variable

An extraneous variable that changes systematically along with the independent variable AND has the potential to influence the dependent variable. Provides an alternative explanation for the observed relationship and, therefore, is a threat to internal validity

threats to external validity

Any characteristic of a study that limits the ability to generalize the results.

research procedure

The exact, step-by-step description of a specific research study.

descriptive research strategy

act of quantifying variables for the purpose of summary

nonexperimental research strategy

attempt to demonstrate a relationship between two variables by comparing groups of scores

experimental research strategy

attempt to establish the existence of a cause-and-effect relationship between two variables