Experimental psych exam 4

between-groups variance has two sources:

(1) the influence of the manipulated independent variables (experimental variance), and (2) the influence of extraneous, uncontrolled variables (extraneous variance).

Experimental design has two basic purposes:

(1) to provide answers to questions by testing causal hypotheses and (2) to control variance.

Solomon's Four-Group Design

-A way to deal with a possible pretest-manipulation interaction

Randomized, Pretest-Posttest,Control-Group Design

-Adding a pretest allows us to quantify the amount of change following treatment -Also allows us to verify that the groups were equal initially -A strong basic research design, with excellent control over confounding

Single-Group, Posttest-Only Design

-Even with the manipulation, virtually no control over confounding variables -We tend to use an implicit control group (what we think would have happened if there had been no manipulation)

Single-Subject Designs

-Extensions of within-subjects designs -Single participant tested under all conditions, with the researcher actively manipulating the independent variable -Variation on time-series designs, with repeated measurement of the dependent variable

Factorial Designs

-Includes two or more independent variables -Essentially two (or more) studies in one -By testing more than one independent variable at a time, we can look at the interactive effects of independent variables -Most independent variables in psychology interact with other independent variables

Pretest-Posttest, Natural Control-Group Design

-Like an experiment, except that participants are not randomly assigned to the groups -A reasonably strong design except that it does not control for selection • Selection could be a powerful confounding factor in many studies

Multilevel, Randomized,Between-Subjects Design

-May or may not include a pretest -Multi-group extension of the basic experimental designs -Controls virtually all sources of confounding variables

Correlated-Groups Designs

-More sensitive than independent-groups designs -Introduces a correlation between groups in the way groups are formed

Within-Subjects Strengths

-More sensitive to small group differences -Fewer participants are needed -Instructions may take less time

Randomized, Posttest-Only,Control-Group Design

-Random assignment controls for selection -Other confounding variables are controlled by comparing the treatment and no treatment groups • For example, history and maturation should be the same in both groups

main effects of factorial design

-The effect of each of the independent variables on the dependent variable is the main effect of that variable -We must always interpret main effects in light of interaction effects when they are present!

Mixed Designs

-The independent variables do not have to be the same • Mixed (within-subjects & between-subjects) • Mixed (manipulated & nonmanipulated) • Mixed in both senses is also possible

Evaluating Interactions

-The interaction is best seen by graphing the results -The fact that the lines are not parallel suggests an interaction, which is confirmed by the ANOVA

Single-Group, Pretest-Posttest Design

-The pretest documents that change occurred, but factors other than the treatment could have accounted for the change • History, maturation, regression to the mean, etc.

There are many possible outcomes for factorial studies

-There may be main effects for one or more of the factors, but no interaction; -there may be an interaction, but no main effects; -or there may be both interactions and main effects; and finally, there may be no effects at all

Variance

-Variance is necessary in any research • Without variance, there is nothing to test

Repeated-Measures Factorials

-Within-subjects design (also called repeated measures design) -As with all within-subjects designs, sequence effects must be controlled -The ANOVA will have to take into account that the same participants appear in all of the conditions

Experimental Design

1. Tests hypotheses about causal effects of the independent variable (IV) 2. Includes at least two levels of the IV 3. Randomly assigns participants to conditions 4. Includes specific procedures for testing hypotheses 5. Includes control for the major threats to internal validity

In a factorial design, the notation "2 X 3 X 2" tells us that the design has ________ independent variables.

3

There are two basic ways of introducing the correlation among participants in correlated-groups designs:

:(1) by having a single group of participants exposed to all of the conditions (within-subjects or repeated measures designs); and (2) by matching participants on important variables and then randomly assigning these matched sets of participants so that one participant of each set is assigned to each condition (matched-subjects designs).

negative practice effects

A decrease in performance on a dependent measure that results from previous exposure of the participant to the measurement procedures.

counterbalancing

A method of controlling for order effects in a repeated measure design by either including all orders of treatment or by randomly determining the order for each subject

Ex Post Facto Design

A very weak design • What we do when we try to figure out, after the fact, what caused something to happen • Not good science • Does not control confounding variables

design notation definition

A way of indicating the number of factors and how many levels of each factor there are. For example, a 2 X 4 X 3 design has three factors with the first factor having two levels, the second having four levels, and the third having three levels.

practice effects

Any change in performance on a dependent measure that results from previous exposure to the measurement procedure.

Statistical Analysis

Appropriate Statistical Analyses • Correlated t-test (for 2 groups only) • Repeated measures ANOVA

Within-Subjects Weaknesses

Because participants experience all conditions, they may figure out the hypothesis (potential subject effects) -Major issue is sequence effects • Practice and carry-over effects

Non-experimental Designs are weak

Do not include the critical controls of experimental designs May still be used, but caution is necessary

Which of the following is true for within-subjects designs?

Each participant serves as his or her own control.

positive practice effects

Enhancement of performance on a dependent measure that results from previous exposure to the measurement procedure.

What is the design notation for the following study?

IV #1 - Weather; 2 levels (Rainy, Clear) IV #2 - Vehicle Speed; 3 levels (Slow, Moderate, Fast)

Mixed (within-subjects & between-subjects):

In the type of mixed design involving between-subjects and within-subjects variables, the critical issue is a statistical one. That is, the statistical formulas used in the analysis of variance will differ depending on which factors are within-subjects factors and which are between-subjects factors.

Mixed (manipulated & nonmanipulated):

In the type of mixed design involving manipulated and nonmanipulated variables, the essential issue is one of interpretation of results. Manipulated factors are part of true experiments; therefore, causal inferences can safely be drawn. But this cannot be done with the nonmanipulated variables in the study. In a mixed design, interpreting main effects and interactions involving nonmanipulated factors must be done cautiously and with careful attention to possible confounding variables.

non-manipulated factors

Independent variables in a factorial design in participants are assigned to groups on the basis of some preexisting factor.

manipulated factors

Independent variables in a factorial design in which the levels of the factors are determined by active manipulation by the experimenter.

What is the major strength of the within-subjects design?

It guarantees that the participants in the various conditions are equivalent at the start of the study.

Mixed in Both Ways.

It is also possible to develop a design that is mixed in both of the ways described (between- and within-subjects and manipulated and nonmanipulated factors are included in one study). Here both problems (the need to use the appropriate statistical procedure and the problems of interpretation of the nonmanipulated component) arise, and particular care is needed.

Experimental Designs

Meet all criteria for an experiment -Provide more powerful tests of hypotheses

__________ designs control unwanted sources of variance in order to evaluate the effects of the independent variable

Research

What does an A X B interaction mean in a two-way ANOVA?

The affect of factor A depends on the level of factor B.

Error variance

The amount of variability among the scores is caused by chance or uncontrolled variables. This is bad and we want to limit it

interaction in factorial design

The combined effect of two or more independent variables on the dependent variable (i.e., more than just a sum of the main effects) is an interaction

What information is given in the factorial design notation, 2 X 3 X 2?

The design has three independent variables, two levels of A, three levels of B, and two levels of C.

carry-over effects

These effects result from of a participant's involvement in one condition affecting his or her performance in all subsequent conditions. Carry-over effects occur only in within-subjects designs.

Which of the following statements is correct about interactions?

They are enhancements of the effect.

What is the major advantage of within-subjects designs over between-subjects design?

They are more sensitive.

What makes within-subjects designs more sensitive than between-subjects designs?

They reduce error variance.

random order of presentation.

This is a way of controlling for carry-over effects in within-subjects designs. Each participant is tested under all conditions, but the order of the conditions is randomly determined for each participant

Example: Children's Dark Fears Study

Two factors • Level of illumination (lighted or dark) • Images (frightening or neutral) -Test hypothesis that fear of the dark in children is really a fear of darkness and frightening thoughts or images

Systematic between-groups variance

Variability between groups that is brought about by either the experimental manipulation or by a confounding variable.

A manipulation check is

appropriate only at the experimental level.

Careful planning and general research design characteristics

are important at all levels of research.

Compared with between-subjects designs, correlated-groups designs

are more sensitive to the effects of the independent variable.

In experimental research, extraneous variables are always

between-group variables

F is a ratio of

between-groups variation divided by within-groups variation.

in within-subjects designs, the unwanted effects due to the influence of one condition on the following conditions is called

carry-over effects.

Experiments attempt to answer _______ _______

causal questions.

Experimental variance, extraneous variance, and sampling error

contribute to the between-groups variance.

Careful planning for experimental design can build in the _______ necessary to have confidence in the conclusions that we draw from the results.

controls

The main disadvantage is sequence effects, which can be controlled with ______________.

counterbalancing

Which of the following is a major control for sequence effects?

counterbalancing

Nonsystematic within-groups variance

due to chance factors and individual differences

Extraneous variance

effects of uncontrolled confounding variables. Bad we want as little of this as possible

Nonsystematic, within-groups variability is also called

error variance.

Which level of research includes unbiased assignment of participants to conditions and the prediction of causal relationships?

experimental

Between-groups variance is a function of

experimental effects and confounding variables

Which of the following is due to the effects of the independent variable?

experimental variance

when selecting similar participants to control for extraneous variance limits what?

external validity and generalization

Experimental design answers questions by controlling many sources of ___________ __________.

extraneous variation

In factorial designs, the independent variables are called _________

factors

Nonsystematic, within-groups variability

has random effects.

Ex post facto studies

have value in generating causal hypotheses.

The notation for a factorial design shows

how many independent variables there are and how many levels of each variable are included.

The single-group, posttest-only design

includes a manipulation of an independent variable.

What is the major contributor to error variance?

individual differences

Within-groups designs eliminates the typically largest source of error variance, which is...?

individual differences

What term refers to the situation where two independent variables have an effect when they are in combination?

interaction

always interpret the ________ effects first

main

What is the term for the unwanted enhancement of performance on subsequent conditions in within-subjects designs?

positive practice effect

what is the best way to control for extraneous variance?

random assignment

Which of the following is NOT used in correlated-groups designs?

random assignment to conditions

Which of the following is accomplished in a within-subjects design?

reduction of error variance

Which of the following is an appropriate statistical test for a within-subjects experiment with two experimental conditions and a dependent variable that produces score data?

repeated measures ANOVA

A 2 X 2 factorial design

results in a four-cell matrix.

Which of the following is a potential confounding factor in within-subjects designs, but not in a between-subjects design?

sequence effects

Factorial studies are considerably more complicated than single-variable experiments. Because they are essentially

several designs combined into one study, factorial experiments contain more than one hypothesis.

Experimental variance

the effect of the independent variable. Is good and want a lot of it

If strong carry-over effects are expected in an experiment,

the within-subjects design is not recommended.

For example, a 2 X 3 factorial design includes_________ independent variables, where there are _______ levels of the first and _________ levels of the second.

two ; two; three

Control in research is control of __________

variance

The most universally-used measure of variation is the

variance

If sequence effects are expected to be very strong, ____________ designs are not recommended.

within-subjects

In which design are all participants exposed to all experimental conditions?

within-subjects

What are the two types of designs used to introduce the correlation in correlated-groups designs?

within-subjects designs and matched-subjects designs

Many factorial designs are either _____________ factorials, in which each participant is tested under all conditions, or ________ designs, that blend different types of factors into a single study.

within-subjects; mixed

why would you run a solomon's 4 group design?

worried about pretest measurement confounding my study

when you want to know which main affect is significant or if neither are

you look at the mean for each main affect and if one is bigger than the other it is significant but if the means are the same then neither are significant

ANOVA can analyze any factorial design -The number of effects will depend on the number of independent variables (IVs)

• 2 IVs: A & B main effects; AB interaction • 3 IVs: A, B, & C main effects; AB, AC, BC, & ABC interactions • 4 IVs: A, B, C, & D main effects; AB, AC, AD, BC, BD, CD, ABC, ABD, BCD, & ABCD interactions • 5 IVs: You DON'T want to know!

Minimizing error variance

• Careful measurement •Control over setting •Reliable measures • Special designs (e.g., correlated-group designs)

sequence effects CONTROLS

• Control PPE with prior training • Control NPE with rest intervals • Control carryover effects by varying the order of conditions

Analyze as if it were a within-subjects study

• Data from matched participants are organized as if the data came from a single participant -Act as if the number of participants was equal to the actual number of participants divided by the number of conditions (e.g., for 40 participants and 4 conditions, tell the program that you had 10 participants and 4 conditions in a within-subjects design)

Fewer participants are needed (with-in design strengths)

• Each participant appears in each condition

Four non-experimental designs covered in this section

• Ex post facto design • Single-group, posttest-only design • Single-group, pretest-posttest design • Pretest-posttest, natural control-group design

matched pairs design Weaknesses

• Extra work of matching participants • Participants without appropriate matches cannot be used in the study - Attrition

Controlling extraneous variance

• Groups as similar as possible at the start of the study •Random assignment •Select participants who are similar -Limits external validity •Build a confound into the study as another IV •Matched assignment • The only difference is the independent variable manipulation

matched pairs design strengths

• Increased sensitivity to group differences • No sequence effects

Evaluating main effects involves

• Looking at all the people tested under each level a Factor A regardless of the level of Factor B • Doing the same for Factor B, as shown here

The relationship of the various sources of variance to the F ratio is shown here. You want to

• Maximize experimental variance • Minimize error variance • Control extraneous variance

Instructions may take less time (with-in design strengths)

• Participants were already instructed on the task in previous conditions

sequence effects SOURCES

• Positive Practice Effects (PPE) • Negative Practice Effects (NPE) • Carryover Effects

Solomon's 4 group design combines two basic experimental designs

• Randomized, posttest-only, control-group design • Randomized, pretest-posttest, control-group design

Experimental Designs discussed in this chapter

• Randomized, posttest-only, control-group design • Randomized, pretest-posttest, control-group design • Multilevel, completely randomized, between-subjects designs

Maximizing experimental variance

• Real differences between the groups on the independent variable •Manipulation Check •Multiple Levels •Increase Differences between Groups

Graphing Factorial Designs for two independent variables

• Select one independent variable, and label the X-axis with the levels of that variable • Label the Y-axis with enough range to graph the mean scores of each cell • Graph and label the means from the first level of the second independent variable and label that line • Repeat that process for each level of the other independent variables, labeling each line

Match participants in sets

• Set size is equal to the number of conditions -Once sets are matched, randomly assign participants in the set to the conditions

Matching gets more difficult as:

• The number of matching variables increases • Matching is done on continuous variables • The number of conditions increase

More sensitive to small group differences (with-in design strengths)

• The variability due to individual differences is statistically eliminated • No group differences due to sampling error • Selection cannot be present since groups are guaranteed to be equal

Within-subjects design:

•Same participants in each group -All participants are exposed to all experimental conditions -Need to control for sequence effects • The experience with one condition affecting performance in subsequent conditions • Controlled by varying the order of presentation (counterbalancing)

Matched-groups design

•Uses matched random assignment -Introduces correlation through matched random assignment -Should match on "relevant" variables • Variables that affect the dependent variable • Variables that show considerable natural variation in the population sampled