PSY:2812 (Research Methods and Data Analysis in Psych II) Exam #2

What is the purpose of mixed-factor designs?

"Classic" pre-test/post-test, two group design Pre-test vs. post-test is (by definition) within-subjects Construct reason is because we want to examine the change Group is (by definition) between-subjects Started as a one-group design (treatment only), but we then added a control group because we cannot counterbalance the order of pre-test vs. post-test Our key test is whether the change is larger for the treatment group A difference in differences is (technically) an interaction, and this will have the power of a within-subject design (via pre-test vs. post-test)

The error-term (i.e., denominator) for the F-test for one-way within-subjects ANOVA has ___ degrees of freedom A. k - 1 B. N - 1 C. (N x k) - 1 D. (N - 1) x (k - 1)

(N - 1) x (k - 1)

What is the relationship between the three means?

(X - X⁼) = (X - X̄) + (X̄ + X⁼) X - X⁼ → total deviation (original values vs. grand mean) X - X̄ → deviation within group (original values vs. local group mean) X̄ + X⁼ → deviation between group (local group mean vs. grand mean) We can now apply a version of the additive variance rule by squaring each part of the above and then summing SStotal = SSwithin-groups + SSbetween-groups Because SS values are directly related to variance, we have analyzed the total variance into (two) pieces Recall that S² = SS/(N - 1), (variance = SS/df) With k groups: Between-group variance = SSbetween / (k - 1) Within-group variance = SSwithin / (N - k)

What are the consequences of within-subject experimental studies?

1. "Order effects" are a major threat to internal validity 2. Practice/fatigue effects 3. Carry-over effects (proactive interference) 4. Counter-balance (order) is the default, but concurrent conditions should be done if possible

What are the important things to consider for a 2-way analysis?

1. Assume two factors: A..B and 1..3 (a 2x3 design) 2. There are three null hypotheses: H₀: No main effect of Factor 1 (μA = μB) H₀: No main effect of Factor 2 (μ₁ = μ₂ = μ₃) H₀: No interaction Note that null hypotheses are simpler than the alternative In papers, they are reported in this order, but that is not the order in which they are examined 3. Check for a significant interaction (no → 2a, yes → 2i) No significant interaction (2a): 2a: Look at the main effects (in original ANOVA) 3a: If either main effect is significant and has 3+ levels, conduct pairwise comparisons within this main effect Significant interaction (2i): 2i: Choosing a parsing and check the simple main effects 3i: If any simple main effect is significant and has 3+ levels, conduct pairwise comparisons within this SME

How can you express the observed violation of the H₀ in a single number for a 1-way ANOVA? Hint #1: What else besides (three) differences can you calculate from three numbers? Hint #2: If the three sample means are the same, what single number will be 0?

1. Assume, also, that σ₁² = σ₂² = σ₃² and that the shapes are the same (normal) 2. Therefore, all three samples are from the same distribution 3. If sₚ² is the pooled estimate of the common true variance and there are Ng subjects in each group, the expected variance across group means = sₚ²/Ng (assuming that H₀ is true)

What are the consequences of between-subject and within-subject experimental studies?

1. Between-subjects: Different subjects in different conditions 2. Within-subjects: All subjects in all conditions

What are the tradeoffs of between-subject and within-subject experimental studies?

1. Between-subjects: Easier to have high construct validity 2. Within-subjects: Easier to have high stats conclusion validity

What are the threats to a pre-test/post-test, two-group design?

1. Biased attrition effects 2. Regression effects These threats are not addressed by adding a control group

What are the three main options for experimental design types?

1. Both IVs between-subjects → (pure) between-subjects design/factorial 2. Both IVs within-subjects → (pure) within-subjects design 3. One IV between- and one IV within-subjects → mixed-factor design

What are the four validities?

1. Construct Validity (extent to which the test or measure accurately assesses what it is supposed to) 2. Statistical (Conclusion) Validity (extent to which statistical inferences about the population, based on a sample, are accurate) 3. External Validity (extent to which findings can be generalized to other situations, people, settings, and measures) 4. Internal Validity (extent to which the causal relationship being tested is not influenced by other factors or variables) 5. Ethics, practicality, and efficiency are also very important

What are the steps for conducting a 1-way between-subjects ANOVA?

1. Decide which test to use (ANOVA) 2. Get the means and standard errors for a plot 3. Test the null hypothesis 4. Conduct pairwise tests (if and only if) needed

What are the follow-up tests for a 1-way design?

1. Even if you are only interested in some of the pairs, the "main effect" (of the IV) across all of the conditions must be significant (before you can look) 2. A significant main effect both justifies and obligates the follow-up pairwise comparisons Question: Does alcohol cause aggression? Conditions: Nothing, 36 oz water, 3 beers Follow-Up Question: Does the effect of drink condition depend on context (moderation)? What drunk → context (moderator) → aggression

What are the steps for a 2-way mixed-factor ANOVA?

1. Get the means and standard errors for a plot 2. Conduct the main analysis (testing the three null hypotheses) 3. Decide what to do next (if anything) If no interaction, for each ME that is significant and has 3+ levels, do pairwise comparisons If interaction is significant, ignore MEs, choosing parsing and plan to do SMEs (this is when mixed-factors might make a difference) 4. Must use "repeated-measures ANOVA" if there is at least one within-subjects factor 5. For the SEs, we pool across levels of the between while keeping the levels of the within separate 6. The appropriateness of the SEs from the stats package depends on which follow-up tests (if any) are conducted 7. The test for an interaction uses a within-subjects error-term 8. If in doubt, conduct SMEs on the within-subjects factor, separately for each group (for much more power)

What are the initial steps for conducting a 2-way ANOVA?

1. Get the means and standard errors for a plot 2. Conduct the main analysis (testing the three null hypotheses) 3. Decide what to do next (if anything) If the interaction is significant, choose parsing and do SMEs If no interaction (so we are taking the MEs seriously), do the following: 4. For each ME, if significant and 3+ levels, do pairwise comparisons (therefore, might not need to do anything more)

What are the steps for conducting a 1-way within-subjects ANOVA?

1. Get the means and standard errors for a plot Can be requested inside the analysis Can be requested using exploration Either way, these will be based on point estimation (they will still include the between-subject variance) 2. Check for a violation of sphericity Sphericity = all possible correlations (between measures) are equal (not zero) Tested using Mauchly's Test (H₀ data are spherical) If Mauchly's p-value < .05, we use Huynh-Feldt correction 3. Test the null hypothesis (maybe correcting for the above) 4. Conduct pairwise tests if and only if needed Could use Tukey's HSD as is used for between-subject, but Bonferroni is preferred for within-subjects 5. Get a better standard error for the plot (optional)

What are the steps for conducting a 2-way analysis?

1. If you see a different pattern of results for alcohol for the two different contexts (an "interaction"), you might be tempted to conduct separate 1-way ANOVAs (examining the effect of alcohol) for each of the contexts "hoping" for only one to be significant 2. You cannot do this (immediately) Like pairwise comparisons, these are follow-up tests; therefore, they require a previous significant finding 3. First, conduct a test of whether the effect of one factor depends on the level of the other factor This is a test of H₀ (no interaction; the lines are parallel) This is done automatically when you conduct an ANOVA with more than one factor 4. If and only if the interaction is significant should you conduct separate tests of one factor at each level of the other These "simple main effects" are not done automatically because, like pairwise tests, they might not be done at all

How can you get MS from SS and df?

1. In general, MS = SS/df (when df = N - 1, MS is also variance) MSbetween = SSbetween / dfbetween MSwithin = SSwithin / dfwithin SStotal = SSbetween + SSwithin dftotal = dfbetween + dfwithin However, MStotal ≠ MSbetween + MSwithin This is the reason why we do not calculate MStotal

What should you consider when choosing a parsing?

1. Interactions should only be "parsed" in one way Ex: Examine the SME of what-drunk at each level of others vs. examine the SMA of others at each level of what-drunk 2. The best way to choose a parsing is to draw diagrams Ex: What drunk → others (moderator) → aggression If this is what you believe is happening, then examine the SMEs of what-drunk (at each level of others) Ex: Others → what drunk (moderator) → aggression If this is what you believe is happening, then examine the SMEs of others (at each level of what-drunk)

After getting the means and standard errors, how do you interpret the results of a 2-way ANOVA?

1. Is there an interaction? Yes (present interaction) → choose parsing for simple main effects → get simple main effects → done No (absent interaction) → more things to consider 2. Is there a main effect for one or more factors? Yes (present main effect) → two things If there is a factor with 3 levels → get pairwise comparisons for that factor If there is not a factor with three levels → done No (absent main effect) → done

What are the four general "rules" for analysis of variance (ANOVA)?

1. No follow-up analysis without justification from previous significant results 2. Difference in significance is not a significant difference 3. When there is a significant interaction, main effects are ignored 4. Error terms should always include every subject

What is the difference between spurious and mediated in quasi-IVs?

1. Some "confounds" are part of or caused by the construct Ex: Quasi-IV = handedness; DV = spatial-compatibility effect; potential "confound" = experience with incompatibility Left-handed people have more experience with incompatible situations by definition (because they are lefties), so there is no threat to internal validity A mediated relationship is still causal Ex: Quasi-IV = cis-sex; DV = respect-worthiness; potential "confound" = height If it is sex → height → respect-worthiness, then height is not a confound, it is part of the explanation 2. Other confounds are threats to internal validity Ex: Quasi-IV = (categorized) age; DV = (political) conservativeness; confound = generation (decade when born) Even worse is when the confounding is "perfect", meaning that there is nothing you can do because the partial regression will remove all of the effect

What is the purpose of quasi-experiments?

1. Sometimes you are interested in the "effect" of the quasi-IV Ex: Do older people remember fewer items? Ex: Are lefties faster than righties? 2. More often, you are interested in whether the quasi-IV is a moderator of the effect of some other IV Ex: Do the elderly show larger Stroop Effects? Ex: Do lefties show smaller Simon Effects? 3. When the other IV is being manipulated within-subjects, you have a mixed-factor design

What are the consequences of between-subject experimental studies?

1. Subject confounds are a major threat to internal validity ("selection effects") 2. Random assignment is easier to do, but "matching" pre-test/post-test covariates are more effective

What are the four main threats to pre-test/post-test, one-group designs?

1. Testing effects 2. Instrumentation effects 3. Maturation effects 4. History effects The standard solution to all four of these problems (testing, instrumentation, maturation, and history) is to add a control group We cannot avoid these four problems, but we can measure their effects on the data and then remove these effects (by subtraction)

What are good examples of quasi-experiments?

1. The quasi-IV must be "older" and more stable than the DV 2. The quasi-IV should be set by a random process Ex: Cis-sex and agreeableness Ex: Handedness and Simon Effect

How can the common variance (σ²) be estimated?

1. This could be done by calculating three separate estimates (one from each group) and then averaging s₁², s₂², and s₃² 2. This becomes complicated when the Ngs are unequal 3. We use MSwithin instead, which is "pooled" across groups and works for unequal Ngs

What are the deviation equations for a 1-way between-subjects ANOVA?

1. Total deviation = deviation within groups + deviation between groups (X - X⁼) = (X - X̄) + (X̄ + X⁼) 2. Total sum of squares = within-groups sum of squares + between-groups sum of squares ∑(X - X⁼)² = ∑(X - X̄)² + ∑(X̄ - X⁼)² 3. Total df = within-groups df + between-groups df (N - 1) = (n - k) + (k - 1) "k" is the standard symbol for levels to a factor, which equal the number of groups for a 1-way ANOVA

What if you have a covariate in a 1-way ANOVA?

1. What if you are controlling for some EV that might not be equal-on-average across groups? 2. A loose explanation would be to change the values of the DV to be what they would be if EV was constant (easy to do) 3. ANCOVA can remove the effects of subject confounds while also increasing statistical power by removing between-subject variance 4. ANCOVA has additional assumptions

What are the symbols used for each type of mean in a 1-way ANOVA analysis?

1. When an analysis requires the use of local means, use X⁼ for the grand mean and sets of X̄ values (with a subscript) for the local means Ex: X̄₁ and X̄₂ for the means of Condition 1 and Condition 2 or Group 1 and Group 2 2. When you have only one set of values, the grand mean is the same as the one and only local mean 3. In this case, we use X̄ (without a subscript) for the mean

What does it mean when H₀ is rejected for a 1-way ANOVA?

1. When we reject H₀, we only know that the true means are not all the same (we do not know yet which means are different) 2. Pairwise comparisons are used if and only if H₀ is rejected 3. Each of these tests has risk (a 5% chance of a Type I error), so we usually apply a correction to keep the overall rate of at least one Type I error at 5% 4. For 1-way between-subject designs, we use Tukey's HSD 5. If the variances are not equal, switch to Welch's F-test 6. If the shapes are not the same (or not normal), switch to a non-parametric test (Kruskal-Wallis)

What are the rules for a 2-way ANOVA?

1. You may not conduct any follow-up analysis unless it is justified by a previous significant result 2. A difference in significances is not a significant difference 3. We only "reject" the idea that two things are the same when a test of this idea produced a p-value < .05 4. Main effects are ignored when there is an interaction (over-additive, under-additive, crossed) Ex: Does alcohol cause aggression? Many people: p < .05 Few people: p ≥ .05 Interaction: p < .05 Main effect: p < .05 SMEs: what drunk | X others; SME others | what drunk

How can a quasi-experiment establish causation?

A true experiment establishes causation because a cause is manipulated (IV), then the effect is measured (DV) The IV is the only difference between conditions A quasi-experiment can also establish causation only if we are sure that the quasi-IV was determined or set long before the experiment was run and if we are sure that the quasi-IV is the only difference between the groups defined by the quasi-IV Temporal Precedence: If the quasi-IV was set long before the experiment was run and is less likely to change than the DV, then the quasi-IV → DV is a more reasonable direction (this should be viewed as a firm requirement) Internal Validity: In a true experiment, the level of the IV is the only difference between conditions If the quasi-IV was set by an independent random process, then there should be no confounds If the quasi-IV is not random (and so there might be confounds), then additional measures or conditions will be needed

What type of values does additivity not hold for in a 1-way between-subjects ANOVA?

Additivity does not hold for MS values because neither MSwithin-groups nor MSbetween-groups is a variance due to neither having a df = N - 1 MStotal ≠ MSwithin-groups + MSbetween-groups

In general, how do you get the simple main effects for a 2-way between-subjects design? How do you do this for a 2-way within-subjects design?

After getting a significant interaction, you choose a parsing and test the simple main effect For a 2-way within-subjects, you use the same procedure with slight differences

What part of the output from a 2-way (or larger) analysis should you look at first, and why?

After the main analysis, you should look at the test of the interaction for significance If it is significant, you can choose a parsing and conduct SMEs If it is not significant, you can treat it as two 1-ways and do pairwise comparisons for any significant main effect

Assume that the experiment used a two-way design and that, when the data were analyzed, the interaction is significant. Which of the following is true? A. You should probably ignore the two main effects, as they can be very misleading B. You must choose a parsing before moving on to the next step of the analysis C. You should probably draw some causal diagrams (if you haven't already). D. All of the above

All of the above

What is a quasi-experiment?

An experiment in which subjects were not randomly assigned to a between-subjects factor A correlational study in which a qualitative measure is treated as a between-subjects factor Ex: Young vs. old; left-handed vs. right-handed In most cases, the quasi-IV is either a subject variable or something that cannot be manipulated (ethically)

Demand Characteristics

Any aspect of the experiment that provides subjects with information as to what is being studied and/or what results are expected Not a problem (on its own), but often causes reactivity, which is a huge threat to construct validity

Order Effect

Any change in the pattern of results due to when some data were collected relative to other data

Subject Confound

Any systematic, pre-existing difference between the subjects assigned to different conditions Opposite of equivalent groups (equivalent groups have the same mean value on all attributes that could influence the dependent variable)

Design Confounds

Aspects of the environment that covary with the independent variable Ex: A buzzing noise in dim-lighting conditions (only) Design confounds do not depend on the subjects They are built into the environment

When and why is an overall/common/global error term used for every test?

Between-subject design It is the only way to include every subject in error term

Variability in a 1-Way ANOVA (From Between to Within)

Between-subjects: SStotal = SSbetween-groups (explained) + SSwithin-groups (unexplained) Within-subjects: SStotal = SSbetween-conditions + SSwithin-conditions SStotal = SSbetween-conditions (explained) + SSbetween-subjects (also explained) + SSerror (unexplained) Variability is "explained" if it can be associated with (credited to and/or blamed on) a particular source, even if we are not interested in this particular source As with t-tests, we get to remove the overall differences between subjects from the error term Smaller error → more power = higher stats con validity

What is the difference between a 1-way between-subjects ANOVA and a 1-way within-subjects ANOVA?

Between: 1. # of conditions = # of groups = k 2. # of subjects = # obs (data) = N Within: 1. # of conditions = k, # of groups = 1 2. # of subjects = N, # obs (data) = N x K

In order for the results from a quasi-experiment to be used to establish causation (without the need for additional conditions or measures), ___. (Note: some people refer to quasi-experiments that don't need additional conditions or measures as "classic quasi-experiments") A. The value of the quasi-IV should be determined or set long before the experiment is run B. The value of the quasi-IV should be determined or set after the experiment starts C. The value of the quasi-IV should be set by a random process D. Both A and C E. Both B and C

Both A and C

A "mixed-factor" design is best defined as an experiment that has ___ A. At least one between-subjects factor B. At least one within-subjects factor C. Both of the above

Both of the above

What is being excluded from any particular error term for within-subject tests?

Differences between the subjects are removed from the error term

What does it mean when the interaction is not significant?

Different levels of one variable do not influence the relationship of the other two variables

Note that this question concerns SS values. The first step to making the transition from between-subjects ANOVA to within-subjects ANOVA is to change the detailed label for the "good" variance from SSbetween-groups to SSbetween-conditions. The next step is to change the detailed label for the "bad" variance from SSwithin-groups to SSwithin-conditions. The third step is to ___ A. Divide SSbetween-conditions into two pieces, both of which will be used later B. Divide SSbetween-conditions into two pieces, only one of which will be used later C. Divide SSwithin-conditions into two pieces, both of which will be used later D. Divide SSwithin-conditions into two pieces, only one of which will be used later

Divide SSwithin-conditions into two pieces, only one of which will be used later

Which of the four "rules" plays a key role in separate error terms?

Error terms should always include every subject

What are bad examples of quasi-experiments?

Ex: High/low depression and anxiety Ex: College major and any ability or personality trait

Regression Effects

Extreme values are not likely to repeat Ex: When only low-scoring subjects are given intervention due to random assignment not being used Threatens internal validity Without intervention = score - regression With intervention = score + regression The solution is to ensure equivalent groups at pre-test

What is the F-ratio for a 1-way between-subjects ANOVA?

F = observed variance across group means / expected variance across group means via H₀ Expected variance across groups (H₀ = σ²/Ng), which we estimate using sₚ²/Ng F = explained variance ("explained" by group/condition) / unexplained variance MSbetween = SSbetween / dfbetween MSwithin = SSwithin / dfwithin F = MSbetween / MSwithin with (k - 1), (N - k) degrees of freedom H₀: μ₁ = μ₂ = μ₃ (or more) The observed violation of this H₀ can be expressed in a single number by focusing on the variances of the X̄s

What is the expected t-value assuming H₀?

H₀: μ₁ = μ₂ The evidence against H₀ comes from X̄₁ - X̄₂ When H₀ is being obeyed, this value will be zero t = observed violation of H₀ / associated error The expected value of t when H₀ is true = 0

What is the expected variance of (X̄) assuming H₀?

H₀: μ₁ = μ₂ = μ₃ Assume, also, that σ₁² = σ₂² = σ₃² and that the shapes are the same (normal) Therefore, all three samples are from the same population H₀ (plus assumptions): all samples are from the same population Get a "pooled" estimate of the variance of the population (sₚ²) If all three samples are the same size (Ng), then the expected variance across the means is sₚ²/Ng because if sx(bar) = s/√N, then sx(bar)² = s²/N

What is the F-ratio for a 1-way within-subjects ANOVA?

H₀: μ₁ = μ₂ = μ₃ F = observed variance across condition means / expected variance across condition means via H₀ F = explained variability ("explained" by which condition) / unexplained variability (left "unexplained" by any measurable source of variability) The between-subjects variance is ignored

What is the null hypothesis for the F-ratio of a 1-way ANOVA?

H₀: μ₁ = μ₂ = μ₃ The expected variance across group means = sₚ²/Ng (assuming that H₀ is true) F = observed variance across group means / expected variance across group means via H₀ F = between-group variance / within-group variance F = MSbetween / MSwithin with k - 1, N - k degrees of freedom As always, reject H₀ if the associated p-value is < .05 If F is sufficiently above 1, reject H₀

What is assumed for a three-level factor?

H₀: μ₁ = μ₂ = μ₃ We assume that σ₁² = σ₂² = σ₃² (variance in each of the conditions are the same), and the shape of the distributions are the same (normal) Therefore, according to H₀, all three samples are from the same population, and the expected Var(X̄) = σ²/Ng

What is the null hypothesis for a 1-way ANOVA?

H₀: μ₁ = μ₂ = μ₃ (or more) Equal Variance: σ₁² = σ₂² = σ₃² = σ² Equal Shape: all are normal Therefore, according to the null hypothesis, all samples are being taken from the same distribution, but the three sample means will rarely (if ever) be the same

What is the expected F-ratio assuming H₀?

H₀: μ₁ = μ₂ = μ₃ (or more) The evidence against H₀ comes from the variance across the X̄s Even when H₀ is obeyed, this value will be greater than zero F = observed variability across condition means / expected variability across condition means assuming H₀ The expected value of F when H₀ is true = 1.00 We reject H₀ when it is larger than Fcrit Both of the above ratios are true, but the two numerators and the two denominators are not the same F = MSbetween / MSwithin, with dfbetween, dfwithin From these (three values), we can get the p-value The expected value of F | H₀ is 1.00, not zero

What is the null hypothesis for a t-test and t-ratio?

H₀: μ₁ = μ₂ or μ₁ - μ₂ = 0 t = observed violation of H₀ / standard error of mean violation t = X̄₁ - X̄₂ / sX̄-X̄ The mean violation of H₀ can be expressed in a single number (after subtraction)

What is the difference between a quasi-experiment and a non-equivalent groups design?

In a quasi-experiment, the attribute that defines the groups is the same as the IV of interest, such as age or sex In a non-EQ groups design, the attribute that defines the groups is not the IV, such as when grouping is done by class, but the IV of interest is lighting

Which of the following equations is not true for one-way between-subjects ANOVA? A. SStotal = SSbetween + SSwithin B. dftotal = dfbetween + dfwithin C. MStotal = MSbetween + MSwithin D. None, those are all correct

MStotal = MSbetween + MSwithin

When planning an experiment that will have more than one factor (a two-way design), you should ___ A. Make separate decisions with regard to between- vs within-subjects for each factor B. Make a decision with regard to between- vs within-subjects for the first factor (alone) and then apply this decision to both factors C. Make a decision with regard to between- vs within-subjects for the second factor (alone) and then apply this decision to both factors D. Make a single decision with regard to between- vs within-subjects for both factors (at the same time)

Make separate decisions with regard to between- vs within-subjects for each factor

What is the local mean?

Mean of a subset of the data; there can be several

Grand Mean

Mean of all pieces of data, regardless of condition When each piece of data comes from a particular group, it can deviate from the "grand mean" (x⁼; double x bar), and/or it can deviate from its group's mean, X̄ Group means, X̄, can also deviate from the grand mean

What is the grand mean?

Mean of all the data; there can only be one

What is the purpose of a non-equivalent groups design?

Most often used in "mass intervention" experiments Ex: Testing a new method of teaching some topic (new method used at one school and old method/control used at another school) It is all about convenience (the fifth thing that trades off) Due to the name of the design, we have no reason to believe that the groups are equal The best tests of an intervention using non-EQ groups use a pre-test/post-test design and also include one or more control measures 1. Group 1 and Group 2 are determined 2. Subjects enter experiment 3. IV = a or b, IV = b or a (random) → DV

One way to write the formula for the F-ratio for one-way between-subjects ANOVA is this: F = observed variance across group means / expected variance across group means via H₀. This helps make it clear why, when the null hypothesis is being (exactly) obeyed, the value of F will be ___ A. Zero B. One C. Equal to the degrees of freedom D. Infinity

One

What does it mean when the interaction is significant?

One variable has an effect on the relationship of something else (another variable)

What is an example of a 2-way design?

Question: Does the effect of alcohol on aggression depend on the context (or setting)? Note how this question can turn a lack of external validity into something positive and interesting The general method is to repeat the 3-condition experiment in two or more different contexts (few vs. many people) The different contexts few vs. many people) may show an interaction (over-additive, under-additive, or crossed) Over-additive is when the lines intersect at the beginning and separate at the end on a graph Under-additive is when the lines are separate at the beginning and intersect at the end on a graph Crossed is when the lines intersect in the middle on a graph H₀: μMany people = μFew people H₀: μNothing = μ36 oz water = μ3 beers H₀ = no interaction (parallel lines)

When designing an experiment, what are the important factors to consider?

Recall that one of the biggest decisions one makes when designing an experiment is whether to manipulate the IV(s) within- or between-subjects (stats validity vs. construct validity) 1. Demand: If an IV is very likely to cause demand, then that IV should probably be between-subjects If both IVs cause demand, run fully between 2. Power: All tests (including interactions) will have more power if they involve a within-subjects IV For interactions, only one within-subjects IV is needed 3. Expandability: If you might add more levels to a factor, then that IV should be between-subjects 4. Construct: If comparison(s) within a factor = measure, then the IV must be within-subjects 5. Extension: If one factor has been used many times in a certain way, then keep it that way

What is the univariate equation for the sum of squared deviations from the mean?

SS = Σ(X - X̄)²

What are the types of SS values?

SS = ∑(A - B)² A is usually X B is usually X̄ 1. An SS can be calculated for any pair of elements as long as A is a subset of B 2. If A is X and B is X⁼ (the grand mean), you are calculating SStotal 3. If A is X and B is X̄g (the group mean), you are getting SSwithin-groups 4. If A is X̄g (the group mean) and B is X⁼ (the grand mean), you are calculating SSbetween-groups

What is the following equation used for: ∑(X - X⁼)²? A. SSbetween B. SSwithin C. SStotal

SStotal

What are the deviation equations for a 1-way within-subjects ANOVA?

SStotal = SSbetween-conditions + SSbetween-subjects + SSerror ∑(X - X⁼)² = SStotal ∑(X̄cond = X⁼)² = SSbetween-conditions ∑(X̄sub - X⁼)² = SSbetween-subjects SSerror = SStotal - (SSbetween-conditions + SSbetween-subjects) dftotal = dfbetween-conditions + dfbetween-subjects + df error ((N x k) - 1) - ((k - 1) + (N - 1)) MSbetween-conditions = SSbetween-conditions / dfbetween-conditions MSerror = SSerror / dferror F = MSbetween-conditions / MSerror, with df = (k - 1), (N - 1) x (k - 1)

What is additivity in 1-way between-subjects ANOVA?

SStotal = SSwithin-groups + SSbetween-groups ∑(X - X⁼)² = ∑(X - X̄g)² + ∑(X̄g - X⁼)² (X - X⁼) = (X - X̄g) + (X̄g + X⁼) Additivity also holds for df values dftotal = dfwithin-groups + dfbetween-groups (N - 1) = (N - k) + (k - 1)

How can the four main threats to a pre-test/post-test, one-group design be organized?

Testing = subject and order (first/second) Instrumentation = other and order (first/second) Maturation = subject and time (earlier/later) History = other and time (earlier/later)

The "grand mean" is ___ A. Fabulous B. The mean of all of the data C. The mean of a subset of the data

The mean of all of the data

What is the sample variance?

s² = Σ(X - X̄)² / (N - 1) Σ(X - X̄)² = sum of squared deviations (from the mean), SS (N - 1) = degrees of freedom, df

History Effects

A change in behavior due to an external event (that occurs during an experiment) Threatens internal validity Without intervention = without event With intervention = with event

Reactivity

A change in behavior due to being studied

Testing Effects

A change in behavior due to previous testing Threatens internal validity Without intervention = first time subject tested With intervention = second time subject tested

Maturation Effects

A change in behavior that emerges over time (regardless of details of experiment) Threatens internal validity Without intervention = subject age = X With intervention = subject age = X + lag

Instrumentation Effects

A change in measurement due to previous use Threatens internal validity Without intervention = first time msr used With intervention = second time msr used

Biased Attrition Effects

A change in the data due to systematic (unequal) loss of participants The solution is to do (what you can) to avoid attrition Show that lost participants were typical Omit all of the lost participants' data

Which of the following is not an accurate definition of "quasi-experiment"? A. A correlational study in which one variable is measured more than once, like a within-subjects factor B. An experiment with a between-subjects factor with values (or conditions) that are attributes of the subjects C. A correlational study in which a qualitative measure is treated as if it were a between-subjects factor D. An experiment in which subjects were not randomly assigned to the conditions of a between-subjects factor

A correlational study in which one variable is measured more than once, like a within-subjects factor

Mixed-Factor Design

A design with at least one between-subjects factor plus at least one within-subjects factor Ex: Two groups (Group 1 tested in AM and Group 2 tested in PM; three within-subjects conditions (Stroop: C, N, and I)

What is the difference between a main effect and a simple main effect? Under what circumstances are each of these needed?

A main effect is the effect of a single factor when any other factor is ignored, and a simple main effect is the effect of one factor on the dependent variable at a single level of another factor A main effect is the first step necessary in a 1-way or 2-way ANOVA, and simple main effects are only necessary under 2-way ANOVAs if the main effects and interactions are significant

Here is the formula for sample variance (for a single set of values): s² = ∑(X - X̄)²/(N - 1) The numerator (i.e., upper part) of the right side of this equation is ___ A. The sum of the squared deviations from the mean B. The square of the summed deviations from mean C. Impossible to calculate D. Incorrect

The sum of the squared deviations from the mean

What is the relationship between three sets of deviation scores for a 1-way ANOVA?

Total deviation = X - X⁼ Within-group deviation = X - X̄ Between-group deviation = X̄ + X⁼ (X - X⁼) = (X - X̄) + (X̄ + X⁼) 1. We can square each of these and then sum (across subjects) ∑(X - X⁼)² = ∑(X - X̄g)² + ∑(X̄g - X⁼)² 2. This makes them SS values SStotal = SSwithin + SSbetween 3. We can also apply the "how many of each" rule to get the df values (N - 1) = (N - k) + (k - 1) 4. This also obeys additivity (by canceling the k's) dftotal = dfwithin + dfbetween 5. We now have three sets of SS and df values, and we'll keep two of them SStotal = SSwithin + SSbetween dftotal = dfwithin + dfbetween 6. In general, MS = SS/df MSbetween = SSbetween / dfbetween MSwithin = SSwithin / dfwithin 7. From these, we can calculate the F-ratio F = MSbetween / MSwithin

What is the difference between a true experiment and a quasi-experiment?

True experiment: 1. All subjects enter the experiment 2. Random assignment to Group 1 (IV = a → DV) or Group 2 (IV = b → DV) Quasi-experiment: 1. Long-term and stable process 2. Group 1 (IV = a) and Group 2 (IV = b) 3. Subjects enter the experiment 4. DV for each group = labile (liable to change)

What is the sum of squared deviations used for?

Used to get sample variance (and therefore, standard deviation): S² = SS/(N - 1) Note that N - 1 is the degrees of freedom

When and why do you need to conduct pairwise comparisons?

When a main effect is significant and has 3+ levels, then you conduct pairwise comparisons

What is another formula that is used for sample variance when the degrees of freedom is not equal to N - 1?

When df ≠ N - 1, this formula should be used: MS = SS/df MS = s², only when df = N - 1

Non-Equivalent Groups Design

When pre-existing groups are assigned to the different levels of a between-subjects IV These are similar to quasi-experiments because regular random assignment is not being used

When are separate error terms used for each test?

Within-subject design

Assume that the experiment used a two-way design and that, when the data were analyzed, the interaction was not significant. Which of the following is true? A. You now have to conduct some simple main effects B. You have the option of conducting some simple main effects C. You aren't allowed to conduct any simple main effects, but you might need to run some pairwise comparisons D. The analysis ends immediately... you never do anything else when the interaction is not significant

You aren't allowed to conduct any simple main effects, but you might need to run some pairwise comparisons

The (true) mean of the population is µ and the (true) variance of the population is σ². You take a random sample of 10 values from this population and calculate the mean. Then, you take a new random sample 10 values from this population and calculate the mean. Then, you take another new random sample 10 values from this population and calculate the mean. What is the expected variance across the three means? A. Zero B. σ² / µ C. σ² / 3 D. σ² / 10

σ² / 10