Exam 2

Carryover with counterbalancing (i.e., half the sample does A and then B; the other half does B then A) that effectively controls for symmetrical carryover effects

If we assume that the A → B carryover is the same size and direction as the B → A carryover, they have to be the same The observed difference is the same as the true score difference Counterbalancing has canceled out these carryover effects

Cell means

Means corresponding to combinations of specific levels on the IVs (e.g., men in the swimsuit condition)

What defines an ANOVA?

There are 2 or more IV

Generally speaking, mere correlational data...

don't reliably establish causal relationships.

What are the two ways of achieving experimental control?

--> Balancing --> Holding constant

Selected orders (i.e., partial counterbalancing)

ABC BCA CAB As shaughnessy et al. note, Latin Squares can be used to generate subsets of possible orders.

What is a double-blind manipulation?

Neither the participant nor the experimenter knows which condition a given participant has been assigned to.

Primary purpose of experiments

Test for casual relationships

In a factorial design there are...

cell means and marginal means

Generally, you want to avoid trying to interpret __________ (e.g., male-neg mood vs. female-pos mood).

confounded comparisons

Non-manipulated IVs

- usually participant variables -- by definition between-subjects (you are either high or low in self-esteem, can't be both) - still an experiment if at least one IV is manipulated - causal conclusions can only be drawn about the manipulated independent variable - can have a non-experimental study with factorial design (only non-manipulated IVs --> so it's not an experiment)

Defining Features of true experiments

1) Controlled manipulation of at least one independent variable 2) Equivalence of participants across the levels of the IV

What does counterbalancing accomplish?

1) It controls the order of conditions so that it is no longer a confounding variable. 2) If there are carryover effects, it makes it possible to detect them.

What two ways can you avoid confounding variables?

1) One way to avoid confounding variables is by holding extraneous variables constant. 2) A second and much more general approach is random assignment to conditions.

What techniques are used to assign participants when using counterbalancing?

1) Random assignment - complete counterbalancing 2) Latin Squares design (more efficient - used for selected orders counterbalancing) (which randomizes through having equal rows and columns) 3) Random counterbalancing - used for large number of conditions, the order of the conditions is randomly determined for each participant. Will result in more random error.

JCCL: What are the two fundamental features of an experiment?

1) The first is that the researchers manipulate, or systematically vary, the level of the independent variable. 2) The second fundamental feature of an experiment is that the researcher exerts control over, or minimizes the variability in, variables other than the independent and dependent variable. (extraneous variables)

Why does one choose to use a within-subjects design?

1) conduct an experiment when few participants are available 2) conduct the experiment more efficiently 3) increase the sensitivity of the experiment 4) study changes in participants behavior over time

Problems with simple randomization

1)Not guaranteed equal number of participants in each condition 2)Time could be a confounding variable

Researcher strategies for supporting external validity

1)Random sampling 2)Conceptual replication (replicating a study, but systematically altering some feature) a) Remember that research is often programmatic (i.e., consisting of multiple studies on a topic, over time)

What are the advantages of within subjects design (repeated measures)?

1. All else being equal, fewer participants are needed. 2. Greater statistical power 3. Complete assurance that there are pre-existing differences between people in the different treatment conditions (i.e., at the different levels of the IV), because they are the same people. Controls for extraneous participant variables. 4. More sensitive This type of experiment is usually preferred if you can minimize carryover effects and counterbalance properly.

Two forms of counterbalancing in the context of incomplete designs

1. All possible orders (i.e., complete counterbalancing) 2. Selected orders (i.e., partial counterbalancing)

What are the advantages of a between-subjects design?

1. Conceptually simpler 2. Require less time per participant 3. Avoid carryover effects (don't need to counterbalance)

What are the disadvantages of within-subjects design?

1. Vulnerability to order effects (e.g., practice effects or fatigue effects) - These are effects due to performance later in a sequence than earlier (e.g., you are more tired on the second task you do than the first) 2. Vulnerability to carryover effects (e.g., interference from a prior task) - These are effects of having done a specific task earlier, such that it affects how you perform on a different task later. - E.g., if someone is slower on the Black+good IAT trials, not simply because they are tired but because they are having difficulty switching from the White+good key configuration - Referred to as "transfer effects" in Shaughnessy et al. -- Can often be controlled for in counterbalancing with all possible orders 3. Demand characteristics that arise, due to participants seeing all treatments - E.g., a participant who, upon encountering the second race-word parking, infers they are expected to respond more or less quickly - Demand characteristics = subtle cues that reveal how the researcher expects participants to behave

What is a carryover effect? What are three types?

A carryover effect is an effect of being tested in one condition on participants' behavior in later conditions. 1) One type of carryover effect is a practice effect, where participants perform a task better in later conditions because they have had a chance to practice it. 2) Another type is a fatigue effect, where participants perform a task worse in later conditions because they become tired or bored. 3) Being tested in one condition can also change how participants perceive stimuli or interpret their task in later conditions. This type of effect is called a context effect (or contrast effect).

Three ways of thinking about a confound

A confounding variable is an extraneous variable that differs on average across levels of the independent variable A confounding variable is an extraneous variable that varies systematically with the independent variable A confounding variable is an extraneous variable that a researcher has allowed to become correlated with an independent variable

What is a confound? (JCCL)

A confounding variable is an extraneous variable that differs on average across levels of the independent variable (i.e., it is an extraneous variable that varies systematically with the independent variable). A confound is correlated with the independent variable. Because they differ systematically across conditions—just like the independent variable—they provide an alternative explanation for any observed difference in the dependent variable. (variation between levels)

Main effects

A main effect of an IV describes its effect on the DV, collapsing across all other IVs. Involve a comparison of marginal means.

Manipulation check

A manipulation check is a separate measure of the construct the researcher is trying to manipulate. The purpose of a manipulation check is to confirm that the independent variable was, in fact, successfully manipulated.

Within subjects design

A manipulation designed so that each participant is exposed to all levels of that independent variable - E.g., an IAT participant completes both the White+good and Black+good pairings - A synonym is a repeated measures

Between subjects design

A manipulation designed so that each participant is exposed to only one level of that independent variable - E.g., a participant in a mood study might be assigned to only the positive or negative mood condition - A synonym for between-subjects is independent groups

Placebo Effect

A placebo is a simulated treatment that lacks any active ingredient or element that should make it effective, and a placebo effect is a positive effect of such a treatment. Although placebo effects are not well understood, they are probably driven primarily by people's expectations that they will improve. Having the expectation to improve can result in reduced stress, anxiety, and depression, which can alter perceptions and even improve immune system functioning. Solutions: - placebo control (sugar pill) - wait-list control - leave out the control entirely and compare to alternatives

Simple main effects

A simple main effect of an IV describes its effect on the DV, holding constant other IVs at specific levels. Involve comparisons of cell means. (e.g., sweater-male vs. swimsuit-male).

JCCL experiment definition

An experiment is a type of empirical study that features the manipulation of an independent variable, the measurement of a dependent variable, and control of extraneous variables.

What is an extraneous variable?

An extraneous variable is a variable other than a/an independent variable that could affect the dependent variable. An extraneous variable is any variable other than the independent and dependent variables.

Interactions

An interaction indicates that the effect of one IV on the DV differs depending on the levels of another IV. (look at cell means) The effect of clothing condition on DV, differs upon gender For women, clothing condition makes a big difference on test performance

What is an order effect?

An order effect occurs when participants' responses in the various conditions are affected by the order of conditions to which they were exposed. A carryover effect is an order effect

Two ways of manipulating Independent Variables (two ways to conduct experiments)

Between-subjects manipulation-Each participant (P) is exposed to only one level of the IV (i.e., each P is in only one experimental condition) [Synonym for "between-subjects" is independent groups] Within-subjects manipulation-Each participant (P) is exposed to all levels of the IV [Synonym for "within-subjects" is repeated measures] Experiments can be conducted using either between-subjects or within-subjects designs. Deciding which to use in a particular situation requires careful consideration of the pros and cons of each approach.

Random sampling process

By giving each member of the target population, regardless of their individual characteristics, an equal probability of selection into the sample, in the long run, the sample tends to be similar to the population

Random assignment process

By giving each participant (in our sample), regardless of their individual characteristics, an equal probability of assignment to each group, in the long run, the groups tend to be equivalent

Extraneous Variables (variables other than our IV that could affect the DV) can be problematic...

Can be problematic.... .... As confounds/confounding variables - Confounds threaten internal validity (the extent to which one is justified in inferring causation; i.e., the extent to which one can be confident that differences on the DV are at least in part a product of differences on the IV) - Confounds are a result of failing to have effective control over extraneous variables ...extraneous variables as "noise" - Add variability to the data - Makes it harder to tell what is causing the DV -reduces statistical power

What does the order of conditions sometimes result in (in a repeated measures design)?

Confounding variable

What is the best means of controlling for order effects and carryover effects?

Counterbalancing Which means testing different participants in different orders. Systematically varying the order of conditions With counterbalancing, participants are assigned to orders randomly, using the techniques we have already discussed. The best method of counterbalancing is complete counterbalancing in which an equal number of participants complete each possible order of conditions.

Independent variables in factorial designs

Each independent variable can be manipulated between-subjects or within-subjects. In a factorial experiment, the decision to take the between-subjects or within-subjects approach must be made separately for each independent variable. Non-manipulated independent variables (gender) can be included in factorial designs, however, they limit the causal conclusions that can be made about the effects of the non-manipulated variable on the dependent variable.

Complete design

Each participant is exposed to each level of the IV multiple times, in varying orders. E.g., ABBA design (which is the simplest possible complete design) is one in which each participant is exposed to the conditions in one randomized order, and then again in the reverse order. Practice effects are balanced in complete designs within each participant using block randomization or ABBA counterbalancing.

Incomplete design

Each participant is exposed to each level of the IV only once (i.e., each participant is given each treatment only once) E.g., the IAT is an incomplete design (one exposure to White+good and one exposure to Black+good) The rule for balancing practice effects in the incomplete design is that each condition of the experiment must be presented in each ordinal position (first, second, etc) equally often. Can have all possible orders or selected orders to balance out practice effects

What does experimental research on the effectiveness of a treatment require?

Experimental research on the effectiveness of a treatment requires both a treatment condition and a control condition, which can be a no-treatment control condition, a placebo control condition, or a wait-list control condition. Experimental treatments can also be compared with the best available alternative.

Single factor two-level design vs single factor multi level design

Experiments involving a single independent variable with two conditions are often referred to as a single factor two-level design. When an experiment has one independent variable that is manipulated to produce more than two conditions it is referred to as a single factor multi level design.

All possible orders (complete counterbalancing)

For example, with three conditions A, B, and C: ABC ACB BCA CAB BAC CBA As the number of sample size increases, all possible orders becomes impractical Each condition comes before the other conditions the same number of times that they come after the other conditions With k conditions, there are k! possible orders For example 4! Is 4x3x2x1 With three conditions... 3 x 2 x 1 = 6 possible orders Condition A is going to appear first for ⅓ of our sample, condition B is going to appear first for ⅓ of our sample, and so forth... The fatigue is evenly balanced The best we can do is balancing

More on confounded comparisons based on example: Suppose a researcher is testing the effects of behavioral priming. They think that priming thoughts of the elderly will lead people to behave similarly to their stereotype of the elderly. In particular, they think that priming thoughts of the elderly will cause people to walk more slowly as they leave the lab. To test this idea, the researcher brings in participants, one at a time to the lab. Each participant is randomly assigned to one of two conditions. In the "elderly prime" condition, they unscramble 20 sentences, about half of which have a word embedded in them that refers to the elderly stereotype (e.g., Florida, knit, retire). In the control condition, participants unscramble 20 sentences, none of which contain such elderly-stereotypic words. After completing the task, the researcher tells participants they are done, and dismisses them.

For height to a be a confounding variable, it would need to be a correlation between height and condition (more tall people in one condition for example) We are giving every individual an equal probability of being assigned to either condition, regardless of height Experimenter expectancy effect - Solution: double blind - The person who is timing it, they need to be blind If you have two people timing, there could be differences in their timing - Timer identity (gender) is confounded with condition - Confounded comparison - As you go from the control condition to the elderly condition, you are also going from a female timer to a male timer.

How is IQ a confounding variable?

If IQ is a confounding variable—with participants in the positive mood condition having higher IQs on average than participants in the negative mood condition—then it is unclear whether it was the positive moods or the higher IQs that caused participants in the first condition to score higher.

JCCL Notes on main effects, interactions and simple main effects

In a factorial design, the main effect of an independent variable is its overall effect averaged across all other independent variables. There is one main effect for each independent variable. There is an interaction between two independent variables when the effect of one depends on the level of the other. Some of the most interesting research questions and results in psychology are specifically about interactions. A simple effects analysis provides a means for researchers to break down interactions by examining the effect of each independent variable at each level of the other independent variable.

How do we achieve equivalence of participants across levels of the IV in a repeated measures design?

In a repeated measures design, we achieve equivalence of participants across levels of the IV because we use the same people in all levels! We don't need to rely on random assignment

Block randomization

In block randomization, all the conditions occur once in the sequence before any of them is repeated. Then they all occur again before any of them is repeated again. Within each of these "blocks," the conditions occur in a random order. Again, the sequence of conditions is usually generated before any participants are tested, and each new participant is assigned to the next condition in the sequence.

What are the two types of Within-Subjects Designs?

Incomplete design Complete design

Validity concerns in experiments

Internal validity-the extent to which one is justified in inferring causation; i.e., the extent to which one can be confident that differences on the DV are at least in part a product of differences on the IV External validity-the extent to which one's findings and conclusions can be generalized to the broader population, or other populations, or other contexts (e.g., other types of tasks).

When is random assignment most effective

Larger sample size The process is going to achieve the goal in the statistical long run

1) Controlled manipulation of at least one IV

Manipulation involves research assignment of participants to specific levels of the IV Experimental control is a way of ensuring that the only differences between conditions are on the IV (without allowing confounds) Two ways of achieving experimental control --> Balancing --> Holding constant

Marginal means

Means corresponding to levels of one IV, collapsing across all other IVs (i.e., ignoring all other IVs) 83.14 for example, we are ignoring gender 73.54, you are collapsing across clothing condition

Experimenter Expectancy Effect

One important source of such variation is the experimenter's expectations about how participants "should" behave in the experiment. This outcome is referred to as an experimenter expectancy effect. For example, if an experimenter expects participants in a treatment group to perform better on a task than participants in a control group, then they might unintentionally give the treatment group participants clearer instructions or more encouragement or allow them more time to complete the task. The idea is to minimize experimenter expectancy effects by minimizing the experimenters' expectations. It is important to standardize experimental procedures to minimize extraneous variables, including experimenter expectancy effects.

One-way vs factorial design

One-way experimental design: There is only one IV Factorial design: Two or more IVs are crossed

Matching/matched groups

Participants in the various conditions are matched on the dependent variable or on some extraneous variable(s) prior to the manipulation of the independent variable. This guarantees that these variables will not be confounded across the experimental conditions. Example: We could then use that information to rank-order participants according to how healthy or unhealthy they are.

2) Equivalence of participants across levels of each IV.

Participants should be the same prior to exposure of the IV (eliminate pre-existing differences between experimental groups prior to manipulation) Usually achieved in between-subjects designs through random assignment But can also be pursued through matching/matched groups

How to eliminate practice effects?

Practice effects cannot be eliminated in a repeated measures design but they can be balanced or averaged across the conditions When balanced across the conditions, practice effects are not confounded with the independent variable and the results of the experiment are interpretable

What are fundamental elements of experimental research for between and within-subjects experiments? What is the purpose of these techniques?

Random assignment to conditions in between-subjects experiments or counterbalancing of orders of conditions in within-subjects experiments is a fundamental element of experimental research. The purpose of these techniques is to control extraneous variables so that they do not become confounding variables.

More notes on "noise"

Reduces our ability to detect an effect even if it is truly there Noise makes it harder to see a real difference between the two conditions Although the mean difference between the two groups is the same as in the idealized data, this difference is much less obvious in the context of the greater variability in the data. Thus one reason researchers try to control extraneous variables is so their data look more like the idealized data, which makes the effect of the independent variable easier to detect (although real data never look quite that good).

What is conceptual replication?

Replicating a study, but systematically altering some feature

Factorial Design (JCCL)

Researchers often include multiple independent variables in their experiments. The most common approach is the factorial design, in which each level of one independent variable is combined with each level of the others to create all possible conditions.

What is another approach called, that is often used when participants make multiple responses in each condition.

Simultaneous Within Subjects Design

Mixed factorial design

Since factorial designs have more than one independent variable, it is also possible to manipulate one independent variable between subjects and another within subjects.

When are studies high in external validity?

Studies are high in external validity to the extent that the result can be generalized to people and situations beyond those actually studied. Although experiments can seem "artificial"—and low in external validity—it is important to consider whether the psychological processes under study are likely to operate in other people and situations.

What is high internal validity in an experiment? When are experiments high in internal validity?

Studies are high in internal validity to the extent that the way they are conducted supports the conclusion that the independent variable caused any observed differences in the dependent variable. Experiments are generally high in internal validity because of the manipulation of the independent variable and control of extraneous variables.

Counterbalancing (i.e., half the sample does A and then B; the other half does B then A) does not effectively control for asymmetrical carryover effects (aka differential carryover or differential transfer)

The A condition is harder to break than the B condition Starting with the A condition makes it harder to switch to the B condition, more so than switching from the B condition to the A condition Carryover effects aren't equal in magnitude

Condition vs level

The condition is what the experimenter uses to create the different levels of the IV

Example of main effect of format

The marginal mean for oral format (17.10) and written format (17.58) are not significantly different from each other.

Example of main effect of mood

The marginal mean for positive mood (20.05) is significantly different from the marginal mean for negative mood (14.63). We can reject the null hypothesis that people in the positive and negative mood conditions perform the same.

What is the primary advantage of between-subjects experiments?

The primary advantage of this approach is that it provides maximum control of extraneous participant variables. Participants in all conditions have the same mean IQ, same socioeconomic status, same number of siblings, and so on—because they are the very same people. Makes the data less noisy

Balancing

The proportion of individuals in each of the two conditions is the same. Random assignment is one way of achieving balance Matching is another way of achieving balance

Significant interaction between mood and format on performance

The simple effect (or simple main effect) of mood is different for the two formats. In other words, the effect of mood on performance in the oral condition is different from the effect of mood on performance in the written condition.

Non-significant interaction between mood and format on confidence

The simple main effect of mood is the same in the two formats. In other words, the effect of mood on confidence in the oral condition is the same as the effect of mood on confidence in the written condition.

What methods are used to recruit participants?

There are several effective methods you can use to recruit research participants for your experiment, including through formal subject pools, advertisements, and personal appeals. Field experiments require well-defined participant selection procedures.

If a participant in an experiment is blind to condition, what does that mean?

They don't know which level they've been assigned to.

Holding Constant

This technique can mean holding situation, task variables or participant variables constant. If we hold study environment constant at loud, we eliminate noise and the confounding effects Downsides/giving up: - sacrificing external validity - the ability to compare

What kind of design was the IAT? How many levels of the IV were there?

This was an experiment → within-subjects (repeated measures) manipulation Each participant is exposed to all levels of the IV (there are 2) IV: Race-word pairing Level 1: White + Good/Black + Bad Level 2: White + Bad/ Black + Good The levels correspond to the two blocks that are measuring the IV DV: Reaction time

In between-subjects experimental designs (also known as independent groups), how is equivalence of participants across levels of an independent variable is usually achieved?

Through random assignment but one alternative is matching/matched groups

Random sampling goal

To eliminate differences between your sample and the population (from which it came); i.e., to maximize the representativeness of the sample (in order to maximize external validity)

Random assignment goal

To eliminate preexisting differences between experimental groups (in order to maximize internal validity)

What does it mean to manipulate an IV?

To manipulate an independent variable means to change its level systematically so that different groups of participants are exposed to different levels of that variable, or the same group of participants is exposed to different levels at different times. must involve active intervention of the researcher

Random assignment

Using a random process to decide which participants are tested in which conditions. In its strictest sense, random assignment should meet two criteria. One is that each participant has an equal chance of being assigned to each condition (e.g., a 50% chance of being assigned to each of two conditions). The second is that each participant is assigned to a condition independently of other participants.

Why was lab 1 not an experiment?

We didn't manipulate an independent variable

What is a confounded comparison

When the levels of both IVs are changing simultaneously Ex: comparing female/negative mood to male/positive mood This is a problem because we don't know which IV caused the change in the DV Simple main effect of gender on the negative mood level The confounded comparisons are between: - female/negative & male/positive - female/positive & male/negative

Whats a danger sign that differential carryover effects may be present?

When you observe a difference in repeated measures that disappears with independent groups, that's a danger sign that differential carryover effects may be present. Usually in such a case, you should trust the results from independent-groups design.

What other factors could be preventing us from drawing the conclusion?

Why can't we just say that there is a previous mental association that white people associate black with bad... We can't rule out... - Order effects - - Participants may become fatigued - Task-switching -- Interference effect

Researchers (in psychology) often rely on __________ samples (e.g., undergrad PSYC101 participant pools), rather than ______ samples.

convenience random

Research is often ________

programmatic i.e. consisting of multiple studies on a topic, over time