Experimental psych exam 4
between-groups variance has two sources:
(1) the influence of the manipulated independent variables (experimental variance), and (2) the influence of extraneous, uncontrolled variables (extraneous variance).
Experimental design has two basic purposes:
(1) to provide answers to questions by testing causal hypotheses and (2) to control variance.
Solomon's Four-Group Design
-A way to deal with a possible pretest-manipulation interaction
Randomized, Pretest-Posttest,Control-Group Design
-Adding a pretest allows us to quantify the amount of change following treatment -Also allows us to verify that the groups were equal initially -A strong basic research design, with excellent control over confounding
Single-Group, Posttest-Only Design
-Even with the manipulation, virtually no control over confounding variables -We tend to use an implicit control group (what we think would have happened if there had been no manipulation)
Single-Subject Designs
-Extensions of within-subjects designs -Single participant tested under all conditions, with the researcher actively manipulating the independent variable -Variation on time-series designs, with repeated measurement of the dependent variable
Factorial Designs
-Includes two or more independent variables -Essentially two (or more) studies in one -By testing more than one independent variable at a time, we can look at the interactive effects of independent variables -Most independent variables in psychology interact with other independent variables
Pretest-Posttest, Natural Control-Group Design
-Like an experiment, except that participants are not randomly assigned to the groups -A reasonably strong design except that it does not control for selection • Selection could be a powerful confounding factor in many studies
Multilevel, Randomized,Between-Subjects Design
-May or may not include a pretest -Multi-group extension of the basic experimental designs -Controls virtually all sources of confounding variables
Correlated-Groups Designs
-More sensitive than independent-groups designs -Introduces a correlation between groups in the way groups are formed
Within-Subjects Strengths
-More sensitive to small group differences -Fewer participants are needed -Instructions may take less time
Randomized, Posttest-Only,Control-Group Design
-Random assignment controls for selection -Other confounding variables are controlled by comparing the treatment and no treatment groups • For example, history and maturation should be the same in both groups
main effects of factorial design
-The effect of each of the independent variables on the dependent variable is the main effect of that variable -We must always interpret main effects in light of interaction effects when they are present!
Mixed Designs
-The independent variables do not have to be the same • Mixed (within-subjects & between-subjects) • Mixed (manipulated & nonmanipulated) • Mixed in both senses is also possible
Evaluating Interactions
-The interaction is best seen by graphing the results -The fact that the lines are not parallel suggests an interaction, which is confirmed by the ANOVA
Single-Group, Pretest-Posttest Design
-The pretest documents that change occurred, but factors other than the treatment could have accounted for the change • History, maturation, regression to the mean, etc.
There are many possible outcomes for factorial studies
-There may be main effects for one or more of the factors, but no interaction; -there may be an interaction, but no main effects; -or there may be both interactions and main effects; and finally, there may be no effects at all
Variance
-Variance is necessary in any research • Without variance, there is nothing to test
Repeated-Measures Factorials
-Within-subjects design (also called repeated measures design) -As with all within-subjects designs, sequence effects must be controlled -The ANOVA will have to take into account that the same participants appear in all of the conditions
Experimental Design
1. Tests hypotheses about causal effects of the independent variable (IV) 2. Includes at least two levels of the IV 3. Randomly assigns participants to conditions 4. Includes specific procedures for testing hypotheses 5. Includes control for the major threats to internal validity
In a factorial design, the notation "2 X 3 X 2" tells us that the design has ________ independent variables.
3
There are two basic ways of introducing the correlation among participants in correlated-groups designs:
:(1) by having a single group of participants exposed to all of the conditions (within-subjects or repeated measures designs); and (2) by matching participants on important variables and then randomly assigning these matched sets of participants so that one participant of each set is assigned to each condition (matched-subjects designs).
negative practice effects
A decrease in performance on a dependent measure that results from previous exposure of the participant to the measurement procedures.
counterbalancing
A method of controlling for order effects in a repeated measure design by either including all orders of treatment or by randomly determining the order for each subject
Ex Post Facto Design
A very weak design • What we do when we try to figure out, after the fact, what caused something to happen • Not good science • Does not control confounding variables
design notation definition
A way of indicating the number of factors and how many levels of each factor there are. For example, a 2 X 4 X 3 design has three factors with the first factor having two levels, the second having four levels, and the third having three levels.
practice effects
Any change in performance on a dependent measure that results from previous exposure to the measurement procedure.
Statistical Analysis
Appropriate Statistical Analyses • Correlated t-test (for 2 groups only) • Repeated measures ANOVA
Within-Subjects Weaknesses
Because participants experience all conditions, they may figure out the hypothesis (potential subject effects) -Major issue is sequence effects • Practice and carry-over effects
Non-experimental Designs are weak
Do not include the critical controls of experimental designs May still be used, but caution is necessary
Which of the following is true for within-subjects designs?
Each participant serves as his or her own control.
positive practice effects
Enhancement of performance on a dependent measure that results from previous exposure to the measurement procedure.
What is the design notation for the following study?
IV #1 - Weather; 2 levels (Rainy, Clear) IV #2 - Vehicle Speed; 3 levels (Slow, Moderate, Fast)
Mixed (within-subjects & between-subjects):
In the type of mixed design involving between-subjects and within-subjects variables, the critical issue is a statistical one. That is, the statistical formulas used in the analysis of variance will differ depending on which factors are within-subjects factors and which are between-subjects factors.
Mixed (manipulated & nonmanipulated):
In the type of mixed design involving manipulated and nonmanipulated variables, the essential issue is one of interpretation of results. Manipulated factors are part of true experiments; therefore, causal inferences can safely be drawn. But this cannot be done with the nonmanipulated variables in the study. In a mixed design, interpreting main effects and interactions involving nonmanipulated factors must be done cautiously and with careful attention to possible confounding variables.
non-manipulated factors
Independent variables in a factorial design in participants are assigned to groups on the basis of some preexisting factor.
manipulated factors
Independent variables in a factorial design in which the levels of the factors are determined by active manipulation by the experimenter.
What is the major strength of the within-subjects design?
It guarantees that the participants in the various conditions are equivalent at the start of the study.
Mixed in Both Ways.
It is also possible to develop a design that is mixed in both of the ways described (between- and within-subjects and manipulated and nonmanipulated factors are included in one study). Here both problems (the need to use the appropriate statistical procedure and the problems of interpretation of the nonmanipulated component) arise, and particular care is needed.
Experimental Designs
Meet all criteria for an experiment -Provide more powerful tests of hypotheses
__________ designs control unwanted sources of variance in order to evaluate the effects of the independent variable
Research
What does an A X B interaction mean in a two-way ANOVA?
The affect of factor A depends on the level of factor B.
Error variance
The amount of variability among the scores is caused by chance or uncontrolled variables. This is bad and we want to limit it
interaction in factorial design
The combined effect of two or more independent variables on the dependent variable (i.e., more than just a sum of the main effects) is an interaction
What information is given in the factorial design notation, 2 X 3 X 2?
The design has three independent variables, two levels of A, three levels of B, and two levels of C.
carry-over effects
These effects result from of a participant's involvement in one condition affecting his or her performance in all subsequent conditions. Carry-over effects occur only in within-subjects designs.
Which of the following statements is correct about interactions?
They are enhancements of the effect.
What is the major advantage of within-subjects designs over between-subjects design?
They are more sensitive.
What makes within-subjects designs more sensitive than between-subjects designs?
They reduce error variance.
random order of presentation.
This is a way of controlling for carry-over effects in within-subjects designs. Each participant is tested under all conditions, but the order of the conditions is randomly determined for each participant
Example: Children's Dark Fears Study
Two factors • Level of illumination (lighted or dark) • Images (frightening or neutral) -Test hypothesis that fear of the dark in children is really a fear of darkness and frightening thoughts or images
Systematic between-groups variance
Variability between groups that is brought about by either the experimental manipulation or by a confounding variable.
A manipulation check is
appropriate only at the experimental level.
Careful planning and general research design characteristics
are important at all levels of research.
Compared with between-subjects designs, correlated-groups designs
are more sensitive to the effects of the independent variable.
In experimental research, extraneous variables are always
between-group variables
F is a ratio of
between-groups variation divided by within-groups variation.
in within-subjects designs, the unwanted effects due to the influence of one condition on the following conditions is called
carry-over effects.
Experiments attempt to answer _______ _______
causal questions.
Experimental variance, extraneous variance, and sampling error
contribute to the between-groups variance.
Careful planning for experimental design can build in the _______ necessary to have confidence in the conclusions that we draw from the results.
controls
The main disadvantage is sequence effects, which can be controlled with ______________.
counterbalancing
Which of the following is a major control for sequence effects?
counterbalancing
Nonsystematic within-groups variance
due to chance factors and individual differences
Extraneous variance
effects of uncontrolled confounding variables. Bad we want as little of this as possible
Nonsystematic, within-groups variability is also called
error variance.
Which level of research includes unbiased assignment of participants to conditions and the prediction of causal relationships?
experimental
Between-groups variance is a function of
experimental effects and confounding variables
Which of the following is due to the effects of the independent variable?
experimental variance
when selecting similar participants to control for extraneous variance limits what?
external validity and generalization
Experimental design answers questions by controlling many sources of ___________ __________.
extraneous variation
In factorial designs, the independent variables are called _________
factors
Nonsystematic, within-groups variability
has random effects.
Ex post facto studies
have value in generating causal hypotheses.
The notation for a factorial design shows
how many independent variables there are and how many levels of each variable are included.
The single-group, posttest-only design
includes a manipulation of an independent variable.
What is the major contributor to error variance?
individual differences
Within-groups designs eliminates the typically largest source of error variance, which is...?
individual differences
What term refers to the situation where two independent variables have an effect when they are in combination?
interaction
always interpret the ________ effects first
main
What is the term for the unwanted enhancement of performance on subsequent conditions in within-subjects designs?
positive practice effect
what is the best way to control for extraneous variance?
random assignment
Which of the following is NOT used in correlated-groups designs?
random assignment to conditions
Which of the following is accomplished in a within-subjects design?
reduction of error variance
Which of the following is an appropriate statistical test for a within-subjects experiment with two experimental conditions and a dependent variable that produces score data?
repeated measures ANOVA
A 2 X 2 factorial design
results in a four-cell matrix.
Which of the following is a potential confounding factor in within-subjects designs, but not in a between-subjects design?
sequence effects
Factorial studies are considerably more complicated than single-variable experiments. Because they are essentially
several designs combined into one study, factorial experiments contain more than one hypothesis.
Experimental variance
the effect of the independent variable. Is good and want a lot of it
If strong carry-over effects are expected in an experiment,
the within-subjects design is not recommended.
For example, a 2 X 3 factorial design includes_________ independent variables, where there are _______ levels of the first and _________ levels of the second.
two ; two; three
Control in research is control of __________
variance
The most universally-used measure of variation is the
variance
If sequence effects are expected to be very strong, ____________ designs are not recommended.
within-subjects
In which design are all participants exposed to all experimental conditions?
within-subjects
What are the two types of designs used to introduce the correlation in correlated-groups designs?
within-subjects designs and matched-subjects designs
Many factorial designs are either _____________ factorials, in which each participant is tested under all conditions, or ________ designs, that blend different types of factors into a single study.
within-subjects; mixed
why would you run a solomon's 4 group design?
worried about pretest measurement confounding my study
when you want to know which main affect is significant or if neither are
you look at the mean for each main affect and if one is bigger than the other it is significant but if the means are the same then neither are significant
ANOVA can analyze any factorial design -The number of effects will depend on the number of independent variables (IVs)
• 2 IVs: A & B main effects; AB interaction • 3 IVs: A, B, & C main effects; AB, AC, BC, & ABC interactions • 4 IVs: A, B, C, & D main effects; AB, AC, AD, BC, BD, CD, ABC, ABD, BCD, & ABCD interactions • 5 IVs: You DON'T want to know!
Minimizing error variance
• Careful measurement •Control over setting •Reliable measures • Special designs (e.g., correlated-group designs)
sequence effects CONTROLS
• Control PPE with prior training • Control NPE with rest intervals • Control carryover effects by varying the order of conditions
Analyze as if it were a within-subjects study
• Data from matched participants are organized as if the data came from a single participant -Act as if the number of participants was equal to the actual number of participants divided by the number of conditions (e.g., for 40 participants and 4 conditions, tell the program that you had 10 participants and 4 conditions in a within-subjects design)
Fewer participants are needed (with-in design strengths)
• Each participant appears in each condition
Four non-experimental designs covered in this section
• Ex post facto design • Single-group, posttest-only design • Single-group, pretest-posttest design • Pretest-posttest, natural control-group design
matched pairs design Weaknesses
• Extra work of matching participants • Participants without appropriate matches cannot be used in the study - Attrition
Controlling extraneous variance
• Groups as similar as possible at the start of the study •Random assignment •Select participants who are similar -Limits external validity •Build a confound into the study as another IV •Matched assignment • The only difference is the independent variable manipulation
matched pairs design strengths
• Increased sensitivity to group differences • No sequence effects
Evaluating main effects involves
• Looking at all the people tested under each level a Factor A regardless of the level of Factor B • Doing the same for Factor B, as shown here
The relationship of the various sources of variance to the F ratio is shown here. You want to
• Maximize experimental variance • Minimize error variance • Control extraneous variance
Instructions may take less time (with-in design strengths)
• Participants were already instructed on the task in previous conditions
sequence effects SOURCES
• Positive Practice Effects (PPE) • Negative Practice Effects (NPE) • Carryover Effects
Solomon's 4 group design combines two basic experimental designs
• Randomized, posttest-only, control-group design • Randomized, pretest-posttest, control-group design
Experimental Designs discussed in this chapter
• Randomized, posttest-only, control-group design • Randomized, pretest-posttest, control-group design • Multilevel, completely randomized, between-subjects designs
Maximizing experimental variance
• Real differences between the groups on the independent variable •Manipulation Check •Multiple Levels •Increase Differences between Groups
Graphing Factorial Designs for two independent variables
• Select one independent variable, and label the X-axis with the levels of that variable • Label the Y-axis with enough range to graph the mean scores of each cell • Graph and label the means from the first level of the second independent variable and label that line • Repeat that process for each level of the other independent variables, labeling each line
Match participants in sets
• Set size is equal to the number of conditions -Once sets are matched, randomly assign participants in the set to the conditions
Matching gets more difficult as:
• The number of matching variables increases • Matching is done on continuous variables • The number of conditions increase
More sensitive to small group differences (with-in design strengths)
• The variability due to individual differences is statistically eliminated • No group differences due to sampling error • Selection cannot be present since groups are guaranteed to be equal
Within-subjects design:
•Same participants in each group -All participants are exposed to all experimental conditions -Need to control for sequence effects • The experience with one condition affecting performance in subsequent conditions • Controlled by varying the order of presentation (counterbalancing)
Matched-groups design
•Uses matched random assignment -Introduces correlation through matched random assignment -Should match on "relevant" variables • Variables that affect the dependent variable • Variables that show considerable natural variation in the population sampled