Chapter 11 - Within-Subjects Experimental Designs

What example in the text demonstrates one-way within-subjects design ANOVA?

Does price of wine affect perceived taste or quality of the wine?

What example in the text demonstrates related-samples t test?

Does reading comprehension vary between ebooks and print books?

What equation is associated with one-way within-subjects analysis of variance?

F = variability between groups / variability attributed to error

True or False: The between-subjects design has greater power to detect an effect than the within-subjects design.

False. The within-subjects design has greater power.

Demonstrate the use of counterbalancing and control for timing using a within-subjects experimental design.

for the counterbalancing procedure, the order in which participants receive different treatments or participate in different groups is balanced or offset in an experiment; two types of counterbalancing are as follows: 1. complete counterbalancing, in which all possible order sequences are included in an experiment 2. partial counterbalancing, in which some, but not all, possible order sequences are included in an experiment - one example of partial counterbalancing is the latin square we can also control for order effects by controlling timing; we can control the interval between treatments or groups, and the total duration of an experiment; as a general rule, increasing the interval between treatments and minimizing the total duration of an experiment is often the most effective strategy to minimize threats to internal validity that are associated with observing the same participants over time

List 3 sources of variability.

1. between groups variability 2. within groups variability 3. between-persons variability

List the two types of counterbalancing.

1. complete counterbalancing 2. partial counterbalancing

List the 2 types of matched sample designs.

1. naturally occurring matched pairs 2. experimentally matched pairs

List 4 assumptions of within-subjects ANOVA. Describe.

1. normality - values on dependent variable are distributed normally 2. independence - participants are observed independently within groups 3. homogeneity of variance - variance within each group are equal 4. homogeneity of covariance - the associations between scores in the different groups are similarly related

List 2 assumptions of paired or related samples t test. Describe.

1. normality - values on the dependent variable are distributed normally 2. independence - individuals sampled within each group are independent and difference scores obtained from different individuals

List three terms and processes of within-subjects experimental designs.

1. order effects 2. carryover effects 3. counterbalancing

List the 2 sources of control in experimental designs.

1. quasi-experimentation 2. experimental

What are the 3 rules to a latin squares matrix?

1. the number of order sequences equals the number of treatments 2. each treatment appears equally often in each position 3. each treatment precedes and follows each others treatment one time

In a within-subjects design, why do we assume that the value of the between-persons variability is equal to zero?

Because we assume that individual differences are the same in each group (i.e., there is no difference)

Which source of variation is computed and removed from the denominator of the test statistic for the one-way within-subjects ANOVA?

Between-persons variability

Which type of design, the between-subjects or the within-subjects design, allows for the random assignment of participants to groups?

Between-subjects design

How can researchers ensure that order sequences are representative using partial counterbalancing?

By ensuring that each treatment or group appears equally often in each position, and each treatment or group precedes and follows each treatment or group one time

A Solomon four-group design is an experimental design that includes four groups. Describe the procedures for each group.

In one group, participants are given a pretest, a treatment, then a posttest. In a second group, participants are given only a treatment, then a posttest. In a third group, participants are given a pretest and a posttest without a treatment, In a fourth group, participants are given only a posttest.

State two strategies used to minimize the possible ethical concern of participant fatigue in experiments that use a within-subjects design.

Minimize the duration needed to complete an experiment, and allow for a reasonable time interval or rest period between treatment presentations

For a within-subjects experimental design, the same participants are observed in each group or at each level of an independent variable. Which requirement of an experiment (manipulation, randomization, or comparison/control) is not met using this design?

Randomization (random assignment)

Which sampling design, repeated measures or matched samples, can include the manipulation of an independent variable and is therefore more appropriate for use in an experiment?

Repeated-measures

State two ways that researchers control for order effects.

Researchers control order and control timing

What source of variation is equal to zero using a within-subjects design? How does the related-samples t test eliminate this source of variation, thereby reducing its value to zero?

The between-persons source of variation is equal to zero using a within-subjects design. A related-samples t test eliminates the between-persons variability by reducing the data to one column of different scores.

What test statistic is used to compare differences in interval or ratio data between two or more groups to which the same participants were observed in each group?

The one-way within-subjects ANOVA

When we observe the same participants in each group, do we assume that the individual differences of participants are the same or different in each group?

The same

For a within-subjects design, are the same or different participants observed at each level of an independent variable?

The same participants

Why do researchers compute post hoc tests?

To determine which pairs of group means significantly differ following a significant ANOVA

Which type of design, the between-subjects or the within-subjects design, is associated with a smaller value of error variance, assuming the data are the same?

Within-subjects design

Contrast the use of a between-subjects design and a within-subjects design for an experiment.

a between-subjects design allows for the use of random assignment; a within-subjects design does not the advantage of using random assignment is that we do not need to control for order effects using a between-subjects design; however, we do need to control for order effects using a within-subjects design an advantage of using a within-subjects design is that fewer participants are required overall (economizing) because the same participants are observed in each group; a second advantage of using a within-subjects design is that the test statistic for this design has greater power to detect an effect between groups

Describe partial counterbalancing.

a procedure in which some, but not all, possible order sequences in which participants receive different treatments, or participate in different groups are balanced or offset when number of treatments or groups is >3

Describe a one time within-subjects design.

a study involving favorability ratings of different restaurants (iv=type of restaurant; levels=each restaurant; dv=favorability)

Delineate the within-subjects design and the within-subjects experimental design.

a within-subjects design is a research design in which the same participants are observed one time in each group of a research study a within-subjects experimental design is an experimental research design in which the levels of within-subjects factor are manipulated, the the same participants are observed in each group or at each level of the factor - to qualify as an experiment, the researcher must (1) manipulate the levels of the factor and include a comparison or control group and (2) make added efforts to control for order and time-related factors

Describe matched sample designs.

a within-subjects research design in which participants are matched experimentally or naturally based on shared or preexisting characteristics cannot be used in an experiment because groups are created based on preexisting characteristics and not on a manipulation made by the researcher

Describe complete counterbalancing.

all possible order sequences in which participants receive different treatments, or participate in different groups are balanced and offset used when number of treatments or groups is 2 or 3

What is a related-samples t test?

also called a paired-sample t test or dependent groups t test statistical procedure used to test difference in interval or ratio scale data for two related samples

Which source of variability is attributed to the following? The manipulation

between-groups variability

Which source of variability is attributed to the following? Individual differences

between-persons variability

Describe latin squares matrix.

design in which a limited number of order sequences are constructed

What is a within-subjects design?

design in which the same participants are observed one time in each group, or more than one time at each level of the factor

The levels of a within-subjects factor are manipulated, then the same participants are observed in each group or at each level of the factor. Describe or apply to a scenario.

is participation in distinct PE activities (soccer vs. volleyball) associated with variations in mood? each activity is observed either in the fall or the spring (e.g., soccer in the fall, volleyball in the spring) happiness is assessed during each season

Describe a two time within-subjects design.

iv=time; levels=each time; pretest (1 level) to posttest (2nd level) e.g. - knowledge, acquisition, changes in mood, attitudes, behavior

Because groups and assessments occur at different times, other things that covary with time can influence the dependent variable and threaten internal validity. What are some examples?

maturation, testing, regression toward the mean, historical factors, attrition (heterogeneous) participant fatigue (especially in intensive studies)

Describe experimental.

methodological (randomization; control order and timing) and statistical

Describe naturally occurring matched pairs.

participants are matched based on preexisting characteristics inherent to each individual

Describe experimentally matched pairs.

participants are matched based on their scores on a dependent measure for some characteristic of interest

Describe carryover effects.

participation in one group carries over, or causes changes in the performance of a second group this is similar to testing effect

Describe counterbalancing.

researchers control order and carryover effects by balancing, or off setting the order in which participants receive different treatments in an experiment control order and carryovers effect this

In an experiment, why are efforts made to control for time factors (i.e., making them the same across groups) that covary with the manipulation?

so that only levels of the independent variable differ between groups

Describe between groups variability.

source of variance in a DV that is caused by or associated with the manipulation of the levels of an IV

Describe between-personas variability.

source of variance in a dependent measure (variable) that is caused by or associated with individual differences in responses across groups assumed to be zero in a within-subjects design

Describe within groups variability.

source of variance in a dependent measure that is caused by or associated with observing different participants within each group

Describe quasi-experimentation.

statistical (compute test statistic)

What is the equation associated with a related-samples t test?

t = mean differences between groups / mean differences attributed to error

Apply a Solomon four-group design for the within-subjects experimental design.

the Solomon four-group design is a combination of the one-group pretest-posttest design and the one-group posttest-only design, with two added control groups that can account for the possibility that confounding variables and extraneous factors have influenced the results to apply the Solomon four-group design, two groups are given a treatment, and two groups are given no treatment; in one treatment group, participants are given a pretest, a treatment, and a posttest and in the other treatment group participants are given a treatment and a posttest; in one no-tretament group, participants are given a pretest and a posttest without a treatment and in the other no-treatment group, participants are given a posttest only without a treatment; in all, we make five comparisons to demonstrate evidence that a treatment caused changes in a posttest measure because the Solomon four-group design includes appropriate control groups, it is regarded as an experimental design, so long as the researcher manipulates the treatment and randomly assigns participants to the different groups; the strength of the design is that it is capable of controlling for threats to internal validity but its main limitation is the complexity of the design itself

Describe order effects.

the order of exposure influences dependent variable; these are controlled in an experiment

Identify the appropriate sampling method and test statistic for related samples to compare differences among two or more group means.

the repeated-measures design is used to observe participants in more than two groups; using a repeated-measures design, participants are selected from a single population and observed in multiple groups in an experiment the appropriate test statistic for comparing differences between two or more group means using a within-subjects design is the one-way within-subjects ANOVA; if the ANOVA is significant, then we conduct post hoc tests to determine which pairs of group means are significantly different; using this test, we divide the variability between groups by the value for within-groups error; because individual differences are the same in each group, the between-persons error is measured and removed from the value for error placed in the denominator; the larger the value of the test statistic, the more likely we are to conclude the manipulation, not error, caused a mean difference between groups

What reason would you want to compute a within-subjects design?

to compare responses of participants to different exposure or treatments to limit sample size to limit error variance

What is the goal to experimentation?

to minimize the possibility that individual differences or something other than a manipulation caused differences between groups

Describe the within-subjects factor.

type of factor in which the same participants are observed in each group, or at each level of the factor

What is a one-way within-subjects analysis of variance?

used to test for differences in group means for two or more levels or groups the results inform us that at least one group is different from another group - it does not tell us which pairs of groups differ post hoc tests are used when more than two groups exist

Identify the appropriate sampling method and test statistic for related samples to compare differences between two group means.

using a within-subjects design, the same participants are observed in each group; when participants are observed in this way, the sample is called a related sample; there are two ways to select related samples: 1. the repeated measures design, in which we select a sample from one population and observe that one sample of participants in each group; this type of sampling can be used in an experiment when researchers control for order effects and manipulate the levels of the independent variable 2. the matched-samples design, in which participants are matched, experimentally or naturally, based on characteristics or traits they share; this type of sampling cannot be used in an experiment because groups are created based on preexisting characteristics of the participants and not on a manipulation made by the researcher the appropriate test statistic for comparing differences between two group means using the within-subjects design is the related-samples t test; we divide the mean difference between groups by the value for within-groups error; because individual differences are the same in each group, the between-persons error is removed by reducing the data to difference scores before computing the test statistic; the larger the value of the test statistic, the more likely we are to conclude that the manipulation, nor error, caused a mean difference between the two groups

Identify 3 sources of variation and explain why one source is removed using the within-subjects design.

we can measure individual differences as between-persons variability we measure the mean difference caused by manipulation as between-groups variability we measure variability in participant responding within each group as within-groups variability when the same participants are observed in each group, we assume that the individual differences of participants are also the same in each group; we therefore measure and remove the between-persons variability because we will assume that the value of the between-persons variability is zero - that there is no difference in the individual differences in each group; the individual differences between persons are the same

In terms of within-subjects experimental designs, describe individual differences and variability.

when the same participants are observed in each group, the individual differences of participants are also the same in each group you don't need as many participants because you have the same people receiving or being exposed to different levels of the independent variable with will cause less error variance within-subjects design produces more statistical power

Explain why it is important to control for time-related factors using a within-subjects experimental design.

when we observe the same participants over time, factors related to observing participants over time can also vary between groups; when time-related factors covary with the levels of the independent variable (the manipulation), it can threaten the internal validity of an experiment time-related factors include maturation, testing effects, regression toward the mean, attrition, participant fatigue, and carryover effects

Which source of variability is attributed to the following? Participant responding within each group

within-groups variability