PSYC3010 - Stats II

Explain the difference between mediation and moderation analyses with respect to: o the research question that is addressed o the relationship between the main predictor and the second predictor (i.e., the mediator)

"Why does a relationship exist between predictors and a criterion?" e.g. therapy → lower negative thoughts → lower depression THUS the original predicter has an indirect relationship with the criterion VIA the mediator.

When ANCOVA adjusts the treatment means: o explain what research question is being addressed o explain how ANCOVA does this

"Would the focal IV have an effect on the DV if all participants were equivalent on the covariate?"

2-way ANOVA. Identify the degrees of freedom formulae for: the effect of each factor

# of levels of the factor - 1 (dfB = b - 1), (dfA = a -1)

How to test overall model variance of significance for regression (R²)

(variance accounted for / df) / (variance not accounted for / df) = MSregression/MSresidual(error)

List the key assumptions of mixed ANOVA

- DV is normally distributed For between participant: - assume homogeneity of variance within levels of BP factor For within-participant: - homogeneity of variance (WPF x P interactions constant at all levels of between participant factor) - variance-covariance matrix same at all levels of WPF -pooled variance-covariance in matrix exhibits compound symmetry (sphericity) - usual epsilon adjustments apply when within-participants assumptions are violated.

List the (3) types of research questions that can be addressed in a two-way (A x B) factorial design

- Is there a main effect of factor 1? - Is there a main effect of factor 2? - Is there an interaction?

Identify the assumptions of ANCOVA

- Linear relationship between covariate and DV - linear within each group - equal across treatment groups (homogeneity of regression slopes)

Identify the different omnibus effects that may be found in 3-way factorial ANOVA, and explain what research questions are addressed with each

- main effects (differences between marginal means of one factor) →there will be 3 - two-way interactions (the effect of one factor changes depending on the level of another) → e.g. age by alcohol interaction collapsed across gender.

How to conduct bootstrapping?

- take any sample - assume that it is sampled from the population of interest - create simulated new samples based on those participants (1000s of new samples) - report results with this much larger set of data - bootstrapping can be done for any analysis (particularly common with indirect effects)

ANCOVA should not be conducted when...

- the covariate i not known to be associated with the DV - no interaction between covariate and focal IV - is meaningfully related to the focal IV

Assumptions of multiple regression

-Distribution of residuals (normal distribution of Y values around the regression line; homoscedasticity; no linear relationship between Yhat and errors of prediction; independence of errors) -scales (Predictor and criterion scores) -- normally distrbuted, linear, not singular (extremely highly correlated), measured using a continuous scale.

Optimal level of power

.80 power (80% chance that you will find a significant effect if it exists in the population) → but now people are saying 90/95%

How many predictors are in correlation and bivariate regression?

1

When sphericity is met, epsilon = __

1 (minor/no differences)

Explain the relationship between the coefficient of determination and error/residual variance

1 - r² = everything that isn't accounted for by the IV.

Tolerance

1-variance shared amongst IVs -- high tolerance = good.

True/False: We regress our DV onto our IVs

TRUE: e.g. regress stress (DV) onto sleep deprivation (IV) = I believe sleep deprivation CAUSES stress.

Explain the three caveats (i.e., cautionary tales/qualifications) of power analyses

1. An effect MUST exist for you to find it (increase power can help you detect small effects. - But if your theory and predictions to not reflect what's actually going on, the null hypothesis will be right. 2. Large samples can be bewitching (a large sample can detect very small effects that may be relatively unimportant). 3. Error variance is also important (high error variance means that a large effect may still turn out non-significant)

When testing simple effects in a 2-way ANOVA: o Identify the research question (and hypothesis) being tested o Identify the type of statistical test (e.g. F, t, z) that is used o Identify the error term that is used o Identify how degrees of freedom are calculated for SSeffect / treatment and SSerror

1. Calculate SStreatment for Factor A at 1st level of Factor B (SStreatment for simple effects = variability of cell means) 2. Calculate MStreatment using the degrees of freedom (df) for the omnibus main effect of Factor A 3. Use MSerror from omnibus tests 4. Calculate F ration (F = MStreatment/MSerror e.g. "What is the effect of distraction at each level of consumption?" --> compare distracted and control participants.

Steps for testing moderation

1. Centre X and Z (calculate interaction term) 2. Test for significance of interaction 3. if sig, test for simple slopes (kinda like simple effects) 4. plot in a graph

What are the three omnibus tests in a two-way design? What is the difference between the omnibus tests and the follow-up tests?

1. Estimate between groups variability (main effects and interaction) 2. Estimate within groups variability 3. Weight each variability estimate by # of observations used to generate the estimate (df) Then, compare ratios of between groups variability among levels of A to error, B to error, and ABcells to error.

Identify the three situations in which you might care about power

1. I have done a study and need to report power of my significant effect (observed power → post hoc) 2. I have done a study and did not find a significant effect BUT I know the mean difference exists in the population. (predicted power → a priori) 3. I am designing a study and I want to be sure I have enough power to detect my predicted effects (predicted power → a priori)

Identify the strategies you can use to reduce error variance

1. Improve operationalisation of variables (increases validity) 2. improve measurement of variables (increases internal validity) 3. improve design of your study (e.g. using a blocking design) -- account for variance from other sources. 4. improve methods of analysis (e.g. ANCOVA) -- control for variance from other sources.

List the research questions that can be addressed in a 2-way factorial ANOVA

1. Is there a main effect of factor A? 2. Is there a main effect of factor B? 3. Is there an A x B interaction?

List the three types of epsilon and indicate which one is: rather liberal/lax, rather conservative, and just right

1. Lower-bound epsilon: always assumes worst case scenario. Too conservative and therefore increases chance of Type 2 error. Acts as if you only have 2 treatment levels. 2. Greenhouse-Geisser epsilon: most recommended (not too stringent, not too lax) → size of ε depends on the degree to which sphericity is violated. 3. Huynh-Feldt epsilon: results in epsilon exceeding 1 and is too lax - adjustment is applied to the Greenhouse-Geisser epsilon.

List the advantages of blocking

1. May equate treatment groups better than a completely randomised design 2. increased power because error term is reduced 3. Can check interactions of treatments and blocks (do effects of treatments generalise).

Explain factorial designs' two main advantages

1. More economical in terms of participants (cheaper) -- a two-way factoral design requires fewer participants because we average over the levels of the other factor. 2. It allows us to examine the interaction of independent variables. -- also the generalisability of results can be assessed.

List the three assumptions of the mixed-model approach

1. Sample is randomly drawn from population 2. DV scores are normally distributed in the population 3. Compound symmetry

List the three steps in testing for mediation, and for each step explain: o what analysis you would use o what you would report

1. Show path A: IV and mediator are related; SMR analysis: regress mediator on IV; Report: R² and β 2. Show path C: IV → DV; HMR analysis: block 1 predict DV from IV; report show path B: mediator → DV, controlling for IV; HMR analysis: block 2 predict DV from IV + mediator; report 3. conduct Sobel test/bootstrapping: if sig. there is an indirect effect of IV via mediator on DV.

Identify the four factors that affect power, and what these effects are (i.e., under what conditions does statistical power increase)

1. Significance level (a relaxed/generous α = more power) You can increase power by increasing chance of type I error, or reduce it but reduce your chance. BUT Dodgy if post hoc. 2. Sample size (more N = more power) BUT No control over the effect size. 3. Mean differences (larger differences = more power) 4. Error variance (less error variance = more power)

Questions in moderated regression

1. does the XZ interaction contribute significantly to the prediction of Y? (HMR) 2. how do we interpret the effect Z has on the X-Y relationship? (simple slopes only if there is a sig. interaction)

Identify the strategies you can use to maximise power

1. focus on studying large effects 2. Increase sample size 3. Increase α level (dodgy!) 4. decrease error variance

benefits of mean-centring

1. reduces multicollinearity 2. gives easier-to-interpret direct effects (easier to interpret coefficients in presence of interaction).

Tests in multiple regression

1. strength of overall relationship using F test (R²) 2. Important of individual predictors using t test (b, β, sr)

List disadvantages of blocking

1. time/$ consuming 2. loss of power if blocking variable is poorly correlated with the DV (because fewer df error) 3. blocking factor is treated as having discrete levels (artificial grouping may be necessary) → could lose some information

How to test for significance of an interaction in MMR?

1st block: enter centred IVs as predictors of DV (called 'additive effects') 2nd block: enter interaction term to see if it accounts for additional variance in DV

In a 3-way factorial ANOVA, identify the circumstances in which you would follow up: o an omnibus two-way interaction

2-way: if a significant omnibus 2-way interaction is foundm follow up. We then test simple effects (F test). Only follow up with simple comparisons if you find a simple effect for a factor with more than 2 levels.

In a 3-way factorial ANOVA, identify the circumstances in which you would follow up: o an omnibus three-way interaction

3-way: - simple interaction effects (F tests) to follow up significant 3-way interaction - if simple interaction effects are significant, follow up with simple simple effects (F tests) - if simple simple effects are significant with more than 2 levels, follow up with simple simple comparisons (t-tests and linear contrasts).

It's common to have ___ blocks in the within-participants studies.

4

2-way ANOVA. Identify the degrees of freedom formulae for: error

= total # of observations - # of treatments = N - ba = df for each cell x # of cells = (n - 1) ba

What does a significant Block x IV interaction show

A failure of treatment IV effect to generalise across the levels of a blocking variable.

Explain what is meant by the following: o one factor is nested under another factor

A participants factor is said to be NESTED under levels of the between-participants factor (i.e. each participant is tested in only one group).

List the key similarity and difference between ANCOVA and blocking

ANCOVA the same challenge as blocking of increasing power to test focal IV by introducing control variable. The difference is that the control variable in ANCOVA is CONTINUOUS (as opposed to categorical in blocking designs).

What method of analysis is used when you have CATEGORICAL predictors (IV) and CONTINUOUS criterion (DV)

ANOVA and Multiple Regression

List the first application of blocking

Adding a 2nd factor increases power by REDUCING error variance. - (and when we use a continuous variable e.g. IQ we make it into a categorical e.g. low, middle, high)

ANCOVA refines treatment effect to...

Adjust for any systematic group differences on covariate that existed before experimental treatment

Explain why the sphericity assumption is important - what implications does it have for research?

Adjusting for sphericity matters because the within-subjects tests become too liberal otherwise (probability of Type I error increases).

When ANCOVA adjusts the treatment means: o explain how ANCOVA does this

Adjusting the treatment means accounts for differences in the covariate (if any) in the levels of the IV. (removing confounds) If the covariate differs across groups, ANCOVA effectively partials out the effects of the covariate from the focal IV as well as the error term.

Within-participants

Aka: repeated measures Same participants serving in each treatment

How to get high power and low type I error in repeated-measures designs?

Always use Greenhouse Geisser convention and mixed-measures approach.

Define covariance (in words)

An assessment of the association between 2 variables --> average cross-product of the deviation scores.

Cronbach's α

An index of internal consistency (reliability) for a continuous scale

What does ANCOVA stand for?

Analysis of Covariance

Explain what an omnibus test is, and list the effects in a 2-way ANOVA that can be categorized as omnibus tests

Any test resulting from the preliminary partitioning of variance in ANOVA --> main effect of factor 1, 2 and interaction effect.

Explain covariance's main limitations

Aside from the direction, it isn't easy to tell the STRENGTH -- it is scale-dependent.

Explain how the F ratio is calculated in a 2-way mixed factorial design for: o the effect of the between-participants factor o the effect of the within-participants factor o the interaction of the between-participants factor and the within-participants factor

BP: take differences between group mean and grand mean. If all the same, no effect. If different, there is an effect. Error = Within groups variability. WP: comparing the marginal means. If different, there is a learning effect. Error = inconsistencies in the Block effect across participants Interaction: do we have parallel slopes or is it different for certain groups. Error = inconsistencies in the block effect across P's

Low parsimony =

Bad - IVs don't explain much unique variance and so appear to be redundant. (high collinearities)

Explain what types of factors are included in mixed ANOVA, and what types of research questions can be addressed with this analysis

Both within and between participants types of factors.

In a graph of a 2-way factorial ANOVA, explain how you would identify: o the main effects of each factor

By averaging across the lines (halfway between) - if the lines are the same height there is no main effect.

When following up a significant two-way interaction in a 2-way ANOVA: o Identify the two (possible) steps in the analyses o Identify the types of means that are being compared.

Comparing cell means (simple effects) --> 1. simple effects of A for every level of B and 2. simple effects of B for every level of A

Problems with follow-up tests in 3-way designs

Conducting an exhaustive set of follow-up tests can inflate familywise error rate, increasing type 1 error.

Confound variable in blocking designs

Confound variable: has a significant effect, which is NOT wanted (additional systematic variance).

List the disadvantages of within-participants/repeated-measures designs

Confounds!!! e.g. Fatigue, Habituation, learning/practice, sensitisation, contrast, adaptation, direct carry-over

The lower the epsilon value (closer to 0, further from 1), the more ____ the test becomes

Conservative → greater violation

Identify the kinds of variables that can serve as covariates

Continuous (ordered) variables - the covariate is used to remove error from both the error term and treatment effect.

Control variable in blocking designs

Control variable: has a significant effect, which is GOOD (reduces error variance) but not novel.

Define correlation in relation to covariance (in words) and explain how it addresses the limitations of covariance

Correlation = standardised covariance. (expresses the relationship between two variables in terms of standard deviations). It always ranges from +1 to -1 (standardised) so it is comparable across studies and scales.

When following up a significant 2-way interaction in a mixed ANOVA with the simple effects of the between-participants factor: o explain what scores you are comparing o list the two approaches that you could use o explain the strengths and weaknesses of these two approaches

Could also examine simple effects of group for each of the 4 blocks. You could: 1. use a separate error term for each simple effect (because we are decomposing an interaction effect with a repeated-measures variable), or 2. use a new pooled error term (estimate of the average error variance within the cells) -- less power for the last block and more power for the first block.

Explain the methodology that can be employed to reduce one of the disadvantages of within-participants designs

Counterbalance to reduce sequencing effects (half participants receive order A1 then A2, the remained receive A2 then A1) BUT you can still get treatment x order interaction.

Multicollinearity

Creates unstable regression coefficients (Type I and Type II error)

In a two-by three mixed ANOVA in which gender (male/female) serves as a between-participants variable and time of test (start of semester, mid-semester, end of semester) served as a repeated measures variable, participant is CROSSED with ______ and NESTED within...

Crossed with: TIME OF TEST (RM) Nested within: GENDER (BP)

If theory is wrong in regression

DV is causing IV or both are caused by a 3rd variable.

2-way ANOVA: Identify the circumstances in which simple effects would need to be followed up

Danger of reanalysing variability is type 1 error (false positives): 1. only do follow-up tests if omnibus is significant. 2. we focus on the focal theoretically relevant variable.

Omega-Squared (ω²)

Describes the proportion of variance in the populations' DV score that is accounted for by the effect. Less biased estimate of effect size. More conservative.

Explain what higher-order designs are, what research questions they can address, and why they are useful

Design with more than 2 independent variables (factors) → which allows for designs with higher external validity.

Explain the distinction between design issues and statistical issues, and why the two should not be confused

Design: correlational vs experimental Statistical: ANOVA vs regression Use of ANOVA does not automatically mean experimental data, and use of regression does not automatically mean data is correlational.

List the second application of blocking

Detecting potential confounds (factors that offer alternative explanations for systematic results) - e.g. each condition in your study is run by a different experimenter (2nd factor) -- the experimental effects may be due to the experimenter rather than treatment conditions. E x T interaction.

When following up a significant 2-way interaction in a mixed ANOVA with the simple effects of the within-participants factor, explain: o explain what scores you are comparing o indicate the type of test you would use o indicate which error term you would use

Determine focal IV based on theory. Run a 1-way within-participants anova on block separately for each group. (each has own error, do not use pooled error). Using the pooled error term is NOT appropriate.

Experimental designs

Determines causation through MANIPULATION of IVs in controlled setting, assessing the effect on DV (but some factors are impossible to manipulate)

Explain how within-participants designs can overcome the limitations of between-participants designs.

Each participant serves in each treatment (violating the assumption of independence in ANOVA) We can calculate and remove any variance due to dependence (thus reducing our error term and increasing power) → removes individual difference variation from the error term.

Between-participants

Each person only serves in one treatment

Explain the two sets of limitations associated with between-participants designs

Each person only serves in one treatment We assume that any difference is due to our experimental manipulation and the rest of the variance is residual error

Hypothetical: If studying accounts for 40% and IQ accounts for 30% of variance in test score, the model r² (shared variance) will

Equal LESS THAN 70%

In a 2-way mixed factorial ANOVA, indicate the sources of systematic and error variance for each effect

Error = For BP: SSpswithinG → deviations of mean for each P from group mean For WP: B x Group and B x PswithinG → interaction of the repeated measures factor → inconsistencies in the effect of the WSF across Ps, adjusted for group differences. 2 errors terms (1 for BP main effect, 1 for RP IV and interaction).

Explain how between-participants variance and within-participants variance are used in within-participants ANOVA

Error in between-participants lumps in all inconsistencies in the effect of the focal IV with individual differences. Error in within-participants compares the deviation that individuals have from their own average to the effect of the overall sample.

Explain how blocking reduces error variance

Error variance is ALL unmeasured variance -- therefore every time we include another variable we reduce error.

Error variance

Error variance is variance that cannot be explained (random or due to unmeasured influences) Error IS NOT just chance --> it is also unmeasured influences.

Explain how blocking differs from experimental designs

Experimental designs are FULLY randomised (participants are randomly assigned to 1 level of each factor) Blocking designs are NOT randomised (participants are categorised into levels of the blocking factor).

When the F ratio is calculated for each omnibus test in a 2-way ANOVA, identify which MS terms are used in the numerator and denominator in each case

F = MStreat / MSerror

In a 2-way ANOVA, explain how the variance is re-partitioned when testing: o The simple effects of Factor A o The simple effects of Factor B

Factor A: Add the SS of each level of factor A e.g. SSdistraction at D1 + SSdistraction at D2 + SS distraction at D3 Factor B: Add the SS of each level of factor B

Type 2 Error

False Negative = finding no significant difference in the sample when one actually exists in the population (β)

Type 1 Error

False Positive = finding a significant difference in the sample that actually doesn't exist in the population (α)

Explain the relationship between bivariate correlation and bivariate regression

From correlation we can only get an index describing the linear relationship between two variables; in regression we can predict the relationship between more than two variables and can use it to identify which variables x can predict the outcome variable y.

Indicate the general and specific notations for a factorial design with Factor A (e.g., with 2 groups/levels) and Factor B (e.g., with 3 groups/levels)

General: the number of factors involved 'two-way between participants factorial design' Specific: the number of levels of each factor involved '2 x 3 between-participants factorial design' Factor A = example type (a=2 levels) Factor B = lecturer preparation (b=3 levels).

High parsimony =

Good - because IVs explain unique variance

Explain the main strength of mixed ANOVA

Great for Power! can also manipulate a variable between-participants to exclude potential carryover effects.

Which is the best epsilon adjustment?

Greenhouse-Geisser

All my predictors are entered sequentially in a pre-specified order based on logic or theory. Each predictor is evaluated in terms of what it adds to prediction AT its point of entry (independently of other predictors in the model). What test am I?

Hierarchical regression

Compound symmetry

Homogeneity of variances in levels of repeated-measures factor AND homogeneity of covariances (equal correlations between pairs of levels) → a very restrictive assumption often violated

Path c.b aka path c' (mediation)

IV is NO LONGER related to the DV when the effect of the mediator is controlled for (full mediation)

Path a (mediation)

IV is related to (causes) the mediator

Path c (mediation)

IV is related to the DV

IV in blocking designs

IV: has a significant effect, which is of great interest to you (systematic variance - predicted)

Example of significant effect in bootstrapping

If BOTH are -ve or both are +ve

Example of non-significant effect in bootstrapping

If one is +ve and one is -ve (because then they include 0)

Identify the circumstances in which follow-up tests are required in 2-way factorial ANOVA

If there is a significant main effect: interpret it to establish the direction. If there is a significant interaction: interpret it to establish the direction(s) of the simple effect(s).

Explain how blocking can help you detect potential confounds

If there is an interaction between blocking factor and IV = this is a sign of a confound. Increases power to detect focal IV main effect... - BUT that positive outcome is OUTWEIGHED by negative outcome: interaction means that the effect of focal IV CHANGES depending on blocking factor.

Identify the pieces of information you need to calculate power estimates, and post hoc analyses

If you want to argue you didn't have sufficient power or did to help strengthen your argument Using dataset: 1. Effect size 2. Error (MSerror) 3. N in your study

Explain the similarities and differences in following up a significant main effect in 1-way between-participants ANOVA and 1-way within-participants ANOVA

In between-participants anova, MSerror is the term we would use to test any effect including simple comparisons. In within-participants ANOVA, we partition out and IGNORE the main effect of participants and compute an error term estimating inconsistency as participants change over levels.

Explain when violations of sphericity do not matter, and when they do

In between-participants designs (because treatments are unrelated) or When within-participant factors have two levels (because only ONE estimate of covariance is computed).

Identify the key limitations of partial eta-squared

In factorial ANOVA, error + effect is less than total, so partial eta squared is more LIBERAL or INFLATED. Your IV looks more important but becomes uninterpretable because it adds up to more than 100%. ∴ it is hard to make meaningful comparisons.

Unequal n - consequences (3 : 1)

Instability - more type 1 and type 2 errors

Explain what it means to say that interactions and main effects in a factorial design are independent

Interactions and main effects can occur in any combination.

Explain what question is being addressed when we test r for significance, and which statistical test is used for this purpose

Is r large enough to conclude that there is a non-zero correlation in the population. Use t-test (to analyse correlation). t = systematic variance divided by error variance, df = N-2

Explain what question is being addressed when we test the regression slope (i.e., b or β) for significance, and which test is used for this purpose

Is there a significant association between IV and DV? What is the direction? Using a t-test for significance.

Explain what simple simple comparisons are in a 3-way factorial ANOVA, including: o what question they answer o how they differ from simple comparisons o what error term is used o how to determine how many simple simple comparisons to test o what a significant t test for a simple simple comparison means

Like simple comparisons (contrasts) except we compute for each level of a third factor. e.g., significant simple simple comparison of Factor A at first level of Factor B at first level of Factor C

When ANCOVA adjusts the treatment means: o explain why we would want to do this

Logic: we are interested in the association between the focal IV and the DV → if theres a difference between the levels. BUT, if the covariate also differs, then we wouldn't have strong confidence whether it is our focal variable or our covariate creating the impact on the DV.

In conceptual terms (i.e., in words), explain what is meant by MStreatment, and MSerror in 1-way ANOVA

MStreat = index of variability among treatment means MSerror = index of variability among participants within a cell

State how the F ratio is calculated in a 1-way ANOVA

MStreatment/MSerror (treatment/participant error) If F = 1, not meaninful or significant If F > 1, significant (reject null hypothesis). Estimate between-groups variability, estimate within-groups variability, weight each variability estimate by number of observations --> compare ratio.

In a 3-way factorial ANOVA, identify the circumstances in which you would follow up: o a main effect

Main effect: if a significant omnibus effect is found, so follow up. Because if the factor has more than 2 levels we don't know where the differences are. Use main effect comparisons.

Explain how error terms are calculated in 2-way within-participants ANOVA follow-up tests

Main effects: A separate error term is created for each comparison undertaken (MS blockCOMPxP) Comparison x Participant Simple effects: MS AatB1xP (interaction between the A treatment and participants at B1) Simple comparisons: MS ACOMPatB1xP (interaction between the A treatment (only the data contributing to the comparison, ACOMP) and participants, at B1)

In a 3-way factorial ANOVA, explain what each of the following tell you: o a significant F test for the main effect of Factor A o a significant F test for the A x B 2-way interaction o a significant F test for the A x B x C 3-way interaction

Main effects: differences between marginal means of one factor (averaging over levels of other factors) 2-way interactions: examines whether the effect of one factor is the same at every level of a second factor (averaging over a third factor) 3-way interaction: examines whether the 2-way interaction between 2 factors is the same at every level of the third factor OR whether the cell means differ more than you would expect given the main effects and the two-way interactions.

In a 2-way factorial ANOVA, explain what the null hypothesis represents in tests of the main effects and the 2-way interaction

Main effects: no differences among means across levels of the factor Interaction: if there are differences between factor means, they are constant at each level of the other factor (parallel). or The differences between the means of factors B and the grand mean are the same for the 2 levels of factor A.

Explain how variance is partitioned in a 3-way ANOVA

Main effects: variance due to α, β, and γ 2-way interactions: variance due to αβ, βγ, and αγ 3-way interaction: variance due to αβγ Error/residual: variance due to e

Explain what epsilon adjustments are, what they do, and why they are important/useful

Makes the test more conservative by adjusting the degrees of freedom (Change critical F by adjusting degrees of freedom),

Define marginal means, and indicate the types of effects they are used to test in a 2-way factorial ANOVA

Marginal = in the margins of the table (a significant main effect of example type tells us that the marginal means are different). Main effects.

2-way ANOVA: In conceptual terms (i.e., in words), explain what is represented by each of the following: o SS for each main effect

Marginal means are represented by the levels of each factor collapsing across the other factor (e.g. average of good notes, bad notes, little study, much study).

Correlational designs

Measures IVs (Predictors) and assesses level of association with DV (Criterion) --> uses bivariate (simple) regression and multiple regression.

Explain how mixed ANOVA is different from mixed model within-participants ANOVA

Mixed Model ANOVA -- repeated measures design (standard) with evaluating sphericity adjustments, Greenhouse Geiser (mixed model or MANOVA --> but this weights the DV so they better fit the levels of the focal either) Mixed ANOVA -- has one of each IV (between and repeated)

For Mauchley's test of sphericity: o explain what question it tests o identify the statistic it uses o explain what a significant result means o indicate whether it is a robust test or not

More broad/less restrictive assumption for within-participants studies that compound symmetry. It tests whether the covariances and variances are ROUGHLY equal. Evaluated as a chi-square →if significant, we conclude we have a problem Almost NEVER significant (not robust at all) →therefore abandoned.

List the advantages of within-participants designs

More efficient Increase in power Reduced error term (individual differences removed in error term)

Steps for follow-up tests presented in the flow-chart

Omnibus tests → Main effect comparisons → Simple interaction effects → Simple simple effects → Simple simple comparisons If 3-way interaction is significant → Calculate simple interaction effects for each level of the least important moderator (e.g. A x B at C1) → Conduct simple simple effects of the key IV (A or B) (e.g. A at B1 at C1) → Conduct simple simple comparisons.

What method of analysis is used when you have CATEGORICAL/CONTINUOUS predictors and CONTINUOUS criterion?

Multiple Regression

What method of analysis is used when you have CONTINUOUS predictors and CONTINUOUS criterion

Multiple Regression

R

Multiple correlation: relationship between criterion Y and a set of predictors

2-way ANOVA. Identify the degrees of freedom formulae for: total effects

N - 1

Do you do main effect comparisons (3-way) if there is 2 levels in the factor?

No → only with more than 2 levels. The main effect can be directly interpreted if there's only 2 levels (looking at the means).

Define a control or concomitant variable

Often (blocking designs) variance in the DV can be explained by additional factors which are less novel or interesting -- known as control or concomitant variables.

R² in multiple regression with correlated represents

Often, predictors share overlapping variance with each other, as well as the DV (therefore R² = need to follow up importance of the individual predictors)

Two estimates of effect size are eta-squared and omega-squared. Explain which estimate is more biased than the other (and why)

Omega-Squared removes the inflation by just looking at a sample.

What's the difference between a simple 2-way interaction and an omnibus 2-way in a 3-level design?

Omnibus 2-way collapses across the third factor, and is an omnibus test (is always done). Simple 2-way is a follow-up test (only done if there is a significant 3-way), and deals with the treatment means within a level of a moderator. The error term is the SAME. (pooled error term) But the difference is that if you do 2 seperate 2-way graphs, in those designs your error term is different (seperate).

In a graph of a 2-way factorial ANOVA, explain how you would identify: o whether there is an interaction of the two factors

Parallel lines indicate NO interaction, Non Parallel lines indicate interaction present. Disordinal = lines cross Ordinal interaction = lines do not cross

The error term for BETWEEN main effects is

Participants within groups

Explain what simple simple effects are in a 3-way factorial ANOVA, including: o how to determine how many simple simple effects to test o what a significant F test for a simple simple effect means (e.g. significant simple simple effect of Factor A at first level of Factor B at first level of Factor C)

Pick the simple simple effects of the focal variable. Significant F test for a simple simple effect means that it needs to be followed-up with linear contrasts or t-tests.

List the main assumptions of ANOVA

Population = normal distributed Population = have the same variance (homogeneity of variance) Sample = independent (between-participants design) Sample = independently randomly sampled Sample = has at least 2 observations and equal n Data = measured on a continuous scale, using averages. (e.g. Scale 1-7)

Power's relationship with effect size

Power has a close relationship to effect size. d = how many SD's the means are apart.

Technical definition of Power

Probability of correctly rejecting a false null hypothesis. 1 - β (probability of accepting false Ho)

When following up main effects in a 2-way ANOVA: o Identify the type of means that are compared o Identify the type of statistical test (e.g. F, t, z) that is used o Explain how you would determine the number of follow-up tests that are needed o Explain what a significant main effect comparison tells you

Protected t-test or linear contrasts - comparing marginal means.

Other terms for 'blocking'

Randomised block design Stratification Matched samples design

Identify the problems (and solutions) associated with simple comparisons

Redundancy: explaining the same mean difference more than once. Solution = orthogonal linear contrasts Increases in family-wise error rates: type 1 error rate is higher. Solution = Bonferroni Adjustment for critical t, and conducting contrasts defined a priori.

Dodgy factors that affect power

Relaxed alpha & increasing sample size.

More powerful: Repeated measures designs or between-participants designs?

Repeated measures. (Reduces type II error).

If the sr values for the predictors are known, explain how to work out the unique variance explained by each predictor and the variance shared between all predictors

R² - (unique contribution 1 + unique contribution 2 + unique contribution 3) e.g. if srA = 21.9%, srB = 16.9%, srC = 5% (21.9 + 16.9 + 5 = 44%) 65 - 44 = 21 → shared variance

Eta-Squared (η²)

SSeffect/SStotal Describes the proportion of variance in the sample's DV scores that is accounted for by the effect. Considered a biased estimate of the true magnitude of the effect. The most commonly reported effect size measure (easily interpretable)

In the omnibus summary table for 3-way factorial ANOVA, explain what the SS and MS values represent, and how F is calculated for each effect

SSsource = values for the effects SSerror = taken from the omnibus 2 x 2 x 3 ANOVA

Explain what SSy, SSregression, and SSresidual represent

SSy = SSregression + SSresiducal SSregression = deviation of predicted score from the average (treatment) SSresidual = deviation of score from predicted score (error) dftotal = dfregression + dfresidual

What does S.A.L.E stand for?

Sample size Alpha level Larger effects Error variance

Better factors that affect power

Sample size & error variance (preferred but not much in our control).

Multiple regression

Scores on criterion Y are predicted using more than 1 predictor. Report 2 separate research questions: 1. Do the predictors jointly account for significant variance in DV? 2. Does EACH IV uniqely account for variance in the DV?

Explain what simple simple effects are in a 3-way factorial ANOVA, including: o what question they answer o how they differ from simple effects o what error term is used

Simple effects are follow-ups after an omnibus 2-way interaction which examine the effect of factor A at each level of factor B. Simple simple effects are like simple effect except they examine the effect of factor A at each level of factor B, at each level of factor C. Simple simple effects differ from one-way ANOVAs because they use the MSerror from the omnibus anova table as the error term.

2-way ANOVA: In conceptual terms (i.e., in words), explain what is represented by each of the following: o SS for the interaction

Simple effects of study time for bad notes compares cell means. If the simple effects are not the same, there is an interaction.

In a 3-way factorial ANOVA, explain what simple interaction effects are, including: o what they test o why we need them o how they are tested o how they are different from omnibus 2-way interactions in a 3-way factorial design o how they are different from omnibus 2-way interactions in a 2-way factorial design

Simple interaction effects break down 3-way interaction into a series of 2-way interactions at each level of the 3rd factor. Why? This gives a first look at where the differences between cell means might be. Once we know this, we can follow up these simple 2-way interactions further to figure out where the differences are (using simple simple effects and simple simple comparisons) They are like interaction follow-ups in 2-way, except their are three potential follow-up steps.

Explain disadvantage of mixed ANOVA

Some variables can be tricky or unethical to manipulate within-participants (e.g. gender, brain injury)

Mixed ANOVA can also be called

Split-plot ANOVA

All my predictors are entered simultaneously Each predictor is evaluated in terms of what it adds to prediction beyond that afforded by others (unique variance). What test am I?

Standard regression

ANCOVA refines error term by...

Subtracting variation that is predictable from covariate

Which is underestimated in regression

Sy.x (error) (for small samples)

Explain what the standard error of the estimate is (in words) and what it tells us (in bivariate regression)

Sy.x: reflects the amount of variability around the regression slope and is an important statistic in correlation and regression. It tells us that we are able to predict the variance e.g. 68% of individuals will score within + or - .9892 units of the predicted score (Yhat)

Treatment variance

Systematic differences due to our IV (e.g. experimental manipulation).

MANOVA (multivariate) approach

Takes all observations and applies coefficients so it maximises the effect you're looking for. Problem = instead of adapting model to observed DVs, it selectively weights or discounts DVs based on how they fit the model.

Mediation

Test of an indirect effect → where IV and moderator and DV are inter-related causally. IV and mediator = correlated

What is bootstrapping?

Test whether the indirect effect is RELIABLE (not just a 'blip' -- there really is an effect). It confirms that the indirect effect of the IV via the mediator is significant.

Moderation

Testing interactions → whether the relationship between the IV and DV changes. IV and moderator = uncorrelated

In a 2-way factorial ANOVA, explain what each of the following tell you: o a significant F test (p < .05) for the main effect of Factor A

That the effect is significant --> e.g. significant main effect of pints consumed. Need to follow up.

Explain the difference between fixed and random factors in mixed ANOVA

The between-participants factor is fixed (e.g. drug/lesion in rats) → chosen by us. The within-participants factor (blocks) is random → random rats assigned.

Useful definition of Power

The degree to which we can detect treatment effects (including main effects, interactions, simple effects etc) when they exist in the population.

Two estimates of effect size are eta-squared and omega-squared. Identify the factors that influence the difference between the two (and how)

The difference between the two estimates depends on sample size and error variance. The smaller your sample, the more biased your eta-squared.

The error term for WITHIN effects is

The effect being examined in INTERACTION with the random factor participants within groups.

Define cell means, and indicate the types of effects they are used to test in a 2-way factorial ANOVA

The effect of one factor at one level of the other factor Simple effects.

Explain how ANCOVA reduces error variance

The error term is adjusted statistically - by measuring another variable and estimating its parameters. If the variable affects the DV and is not part of the model, the variable is in the unmeasured 'error'

Simple comparison

The idea that it is within the level of the moderator.

Compare the sources of error variance in within-participants ANOVA and between-participants ANOVA

The interaction of Factor A x Participant as error (the changes in the effects of A across participants) Error in RM designs: When the participants react differently to the focal IV In between-participants designs, individual differences are inseparable from error (hence contribute to the error term) In within-participants designs, it is possible to partial out individual differences from the error term.

Random Factors

The levels of the IV are chosen at random (have different error terms).

Explain what is meant by the following: o one factor is crossed with another factor

The participants factor is said to be CROSSED with the within-participants factor block (i.e. each participant participates in each block).

Explain what the least squares criterion is (in bivariate regression)

The regression line is fitted according to the least squares criterion. (errors of prediction are minimum).

Moderator: enhances

The relationship strengthens

Moderator: attenuates

The relationship weakens

Principle of parsimony

The simplest model/theory with the least assumptions and variables but with greatest explanatory power.

In a 2-way factorial ANOVA, explain what each of the following tell you: o a significant F test (p < .05) for the A x B 2-way interaction

The the interaction is significant--> e.g. consumption x distraction interaction. Need to follow up.

R² in multiple regression with uncorrelated represents

The variance in the DV accounted for by linear model including all predictors. Can unambiguously identify proportion of variance accounted for by each predictor.

Explain what happens if the blocking variable does not reduce error variance

Then you have chosen the WRONG blocking variable. → it should be based on theory/prior research!

Identify the pieces of information you need to calculate power estimates, in a priori analyses

To know how many participants you need to find the effects you're looking for -the minimum you would aim for is .80. Using previous research: 1. Estimate of error (MSerror) 2. Estimate of effect size

Within-participants ANOVA - Explain the following in words (i.e., not using the formula): total variability, variability due to factor, variability due to participants, and error

Total variability: deviation of each observation from the grand mean Variability due to factor: deviation of factor group means from grand mean Variability due to participants: deviation of each participant's mean from the grand mean Error: changes (inconsistencies) in the effect of factor across participants (TR x P interaction)

Describe the sources of systematic variance and error/unexplained/residual variance in within-participants designs

Total variance = Between participants + within participants (between participants variance due to individual differences is partitioned out of error AND treatment). Residual error => inconsistencies in the treatment effect (TR x Participant Effect)

Fixed Factors

Treatment --> you chose the levels of the IV (e.g. you have sampled all the levels of the IV or you have selected particular levels based on a theoretical reason).

Explain how the blocking variable fits in with the predictions made in a research study and the reporting of findings

Unlike the effect of the focal IV, the effect of the blocking factor is NOT usually of interest. The blocking variable is only factored in to REDUCE ERROR and INCREASE the POWER of the test.

Explain why significance tests are not that helpful when we want to determine the importance of findings

Use of an arbitrary acceptance criterion (α) results in a binary outcome (significant or non-significant) There is no information about the practical significance of findings. A large p-value will eventually slip under the acceptance criterion as the sample size increases.

When following up the main effect of a between-participants factor in a mixed ANOVA: o explain what scores you are comparing o indicate the type of test you would use o indicate which error term you would use

Use original error term from the test of between-participants main effect!

2-way ANOVA: When testing simple comparisons... o Identify the research question (and hypothesis) being tested o Identify the type of statistical test (e.g. F, t, z) that is used o Explain what a significant simple comparison tells you

Use t tests or linear contrast to compare cell means. Simple comparisons show which cell means of the factor are different.

Define effect sizes and explain why they are useful

Using effect size has been proposed as an accompaniment to significance testing ('the magnitude of experimental effect'). They can compare size of effects within a factorial design (i.e. how much variance is explained by factor 1, 2, interaction etc)

Explain how variance is partitioned in a 2-way factorial ANOVA

Variability comes from: - Variance due to Factor A - Variance due to factor B - Variance due to A x B interaction

R² in bivariate regression represents

Variance in DV accounted for by all predictors

Explain the sources of variance in a one-way ANOVA, and how these are related to error variance and treatment variance

Variance is the dispersion or spread of score around a point of central tendency (the mean) In one way ANOVA: - Between groups variance: systematic variance due to membership in different groups. Distribution of GROUP means. - Within-groups variance: error variance (due to random chance or unmeasured influence). Distribution of INDIVIDUAL DV scores.

Homoscedasticity

Variance of Y values are constant across different values of Yhat (homogeneity of variance) -- e.g. sometimes it's possible to predict how well people do in a class, but not how poorly: this is a violation of homoscedasticity.

Multiple regression

Variation as a function of multiple predictors usually acting simultaneously (thus achieving better prediction).

To maximise R² we

Want low collinearities and high validities.

Explain how this helps increase statistical power for the test of the focal IV (i.e. the one we really care about in the study)

We are reducing Type 2 Error --> a smaller error term gives us more power compared to using an ANOVA.

Explain how three-way interactions are graphed

We show two graphs side by side. They have the same X-axis factor and line moderator, but the difference is the variable (e.g. gender - one for male one for female). 1. plot 2-way interactions within each level of the third factor. 2. check if pattern for 1st graph (simple interaction of AB at C1) is difference from 2nd graph (simple interaction of AB at C2). If graphs are not the same pattern there is a 3 way interaction.

Why do some people not like ANCOVA?

We're testing hypothetical means that remove the effect of the covariate.

Moderates/Qualifies

When one variable moderates/qualifies another variable, both indicate the presence of an interaction. A significant interaction may qualify significant main effects: the simple effects of one IV depend on the level of the other IV.

What is a suppression model?

When there are indirect effects for the same IV to the same DV in opposite directions - there can be a non-significant correlation. There are significant indirect effects in opposite directions. e.g. IV: Success of Collective Action DV: future intentions This is not significant correlation. However, there are positive indirect effects of pride and negative effects of anger.

Partial mediation

When you put IV and mediator in together, it becomes LESS significant. -- part of the relationship is explained by the mediator

Full mediation

When you put IV and mediator in together, the IV becomes non-significant (was sig. before!) -- the entire relationship is explained by the mediator

Generalisability

Whether the difference described by a main effect is the same across levels of the other factor.

In a graph of a 2-way factorial ANOVA, explain how you would identify: o the simple effects of each factor.

Whether the points on the lines of each factor differ

Why do we use mediation

While in HMR we consider the direct relationships between predictors and a criterion, we still don't know much about the underlying processes. A third variable (mediator) can explain or account for the relationship between an IV and a DV.

Explain how systematic/treatment variance and error/residual variance is partitioned in 2-way within-participants ANOVA. Explain the similarities and differences to 2-way between participants ANOVA

Within-participants design: each effect has a separate error term (corresponds to an interaction between the effect due to participants and the treatment effect). A x P B x P A x B x P (interaction)

Explain the structural model (also known as the linear or conceptual model) of 1-way ANOVA

Xij = u. + tj + eij Xij in any DV score is a combination of: the grand mean + the effect of the j'th treatment + error of individual and the group (chance and ummeasured factors)

Identify the structural model of ANCOVA

Xij = µ + αj + βZij + εij X for participant I in the J'th group = grand mean + 1st IV (factor A in group j) + 2nd IV (score on variable Z multiplied by a fixed weight β) + error

Identify the structural model for 1-way within-participants ANOVA

Xij = µ + πi + ιj + eij = the grand mean + variation due to j'th person + variation due to j'th treatment + (error - variation associated with the i'th cases in the j'th treatment)

Explain the structural model of 2-way factorial ANOVA

Xijk = u. + aj + Bk + aBjk + eijk Xijk: individual within group j of factor a, within a level of factor b. = the grand mean + the effect of the j-th treatment of factor A + the effect of the k-th treatment of factor B + the effect of differences in factor A treatments at different levels of factor B treatments + error for i person in the j-th and k-th treatments.

Explain the structural model in 3-way factorial ANOVA

Xijkl = µ. + βk + γl + αβjk + βγkl + αγjl + αβγjkl + eijkl grand mean + the effect of a + effect of b + effect of c + the interaction of a/c + the interaction of a/b + the interaction of b/c + three-way interaction + error

Structural model for moderation

Y ← X Moderated by Z Moderator Z has an effect on the relationship between Y and X.

Can the Model R² be significant even if indiviual βs are not?

Yes (and vice versa)

Define a factorial design

You can combine two one-way experiments using a factorial design. It has at least two factors (IVs), each with at least two levels. The two IVs can be examined simultaneously (crossed).

Rationales for order of entry

You can partial out the effect of control variables you can build a sequential model according to theory

When following up a significant simple effect of a between-participants factor with simple comparisons: o explain what scores you are comparing o indicate the type of test you would use o indicate which error term you would use

You could conduct follow-up comparisons for the 3 groups at block 4 (identical for the main effect follow up).

Partial Eta-squared

You divide the variability of your effect by the residual variance (i.e the variance left over to be explained not accounted for by any other IV in the model).

Explain how to use blocking in between-participants designs

You have a focal variable and you know you have a problem with power. Blocking introduces control variables into your design (reflect ADDITIONAL sources of variation or pre-existing differences on the DV score). (e.g. DV = problem solving, IV = teaching style, control = IQ).

Explain the relationship between b and β (beta)

b = unstandardised β = standardised (Z-score change in Y predicted from a 1 SD increase in X)

Order is...

crucial to outcome and interpretation of HMR.

List the formulae for various degrees of freedom in 1-way within-participants ANOVA

dftotal = N (number of observations) - 1 dfP = n - 1 dftr = j - 1 dferror = (n-1)(j-1) -> different to between because error is not interaction of participant x treatment

Identify the formulae for all degrees of freedom in 3-way factorial ANOVA

dftotal = N - 1 dfinteraction = product of df for factors involved (dfjkl = (j - 1) (k - 1) (l - 1) dferror = N - jkl (n-1) x (# cells)

Indicate the formulae for degrees of freedom for: o total effects o all between-participants effects o all within-participants effects

dftotal = N - 1 (number of observations - 1) dfBP = (g)(n)-1 (# of groups) (number of rats in each group) - 1 dfWP = dftotal - dfparticipants

Indicate the general and specific notations for a factorial design with Factor A (with 2 groups/levels), Factor B (with 4 groups/levels), and Factor C (with 3 groups/levels)

e.g. 2 (age) x 3 (alcohol) x 2 (gender) between-subjects design

Partial correlation (pr²)

examines the association of the IV and the DV after controlling for other factors and removing them from both (removing other variables before considering the impact). - the proportion of residual variance in the criterion uniquely accounted for by predictor 1. Advantage: If IV 2 is a confound then we have purified our test of IV 1. But if the shared variability is of interest of us, this wouldn't make sense.

Semi-partial correlation (sr²)

examines the unique contribution of the IV divided by the total variability in the DV. - the proportion of total variance in the criterion uniquely accounted for by predictor 1. Advantage: effect size measure to identify the unique contribution.

decomposing an interaction

examining simple slopes and plotting on a grph

What are confidence intervals?

how significant the effect is (it is significant when the 95% confidence interval does not include zero)

What happens to β if IV's are correlated with each other?

if IVs are NOT correlated with each other, then the β is equal to the correlation When they are correlated, the β changes

a (in regression equation)

intercept (value of Y when X = 0)

Collinearities

intercorrelations among predictors →highlight associations among the DVs

Path b.c (mediation)

mediator is related to the DV when the effect of the IV is controlled for

The sphericity assumption is violated when epsilon = ___

more than 1 ( >1 )

ANOVA vs STANDARD MULTIPLE REGRESSION

no test of overall model vs tests model automatically tests main effects vs tests unique effects of each IV (covarying residual DV scores with IV once all other IV effects are controlled. tests interactions automatically vs does not test for interactions report Fs and effect sizes plus follow ups vs report R² with F test plus each IV's β's with ps plus follow ups

Explain the difference between an ordinal interaction and a disordinal interaction

ordinal: Same effect, just bigger in other places (the direction generalises, but the effect changes shape). disordinal: effect disappears of reverses (failure of generalisability)

Yhat (in regression equation)

predicted value of Y (DV)

2-way ANOVA. Identify the degrees of freedom formulae for: the effect of the interaction

product of df for factors in the interaction dfBA = (b - 1) x (a - 1)

The difference between r and r adjusted

r is a sample statistic (biased to the sample) r adj is the population correlation coefficient. r adj is more conservative than r. The difference between the two becomes smaller as sample size increases.

List the various terms used to indicate a bivariate correlation

r, Pearson correlation, zero-order correlation.

Interaction in Multiple Regression

relationship between a criterion and a predictor varies as a function of a second predictor. (the second predictor is called a 'moderator')

Validities

relationship between each predictor and the criterion

Which is overestimated (inflated/overly liberal) in regression?

r² (with small samples)

Define the coefficient of determination in the context of bivariate correlation (in words), and identify the letter used to indicate it

r² → the proportion of variance in one variable that is explained by the variance in another.

b (in regression equation)

slope of regression line

What variables should be mean centred in MMR

subtract the mean of each predictor from each observation's score on that predictor

X (in regression equation)

value of predictor (IV)

What if there is an interaction between the covariate and the focal IV in ANCOVA?

violates the assumptions of ANCOVA

When do we test simple slopes?

when interaction is significant

Singularity

when your predictors are so highly correlated that you can't compute a solution