Week 5: Two-Way ANOVA (independent measures)
two-way ANOVAs address what three questions?
1. What happens to the DV as factor A (IV1) changes in levels? a. Is there a statistically significant main effect on factor A on the DV? 2. What happens to the DV as factor B (IV2) changes in levels? a. Is there a statistically significant main effect on factor B on the DV? 3. How does specific combinations of factor A and factor B affect the DV? a. Is there a statistically significant interaction between factor A and factor B upon the DV?
post hoc in SPSS
Click on Post Hoc If one variable has only two levels we do not need to select post hoc tests for that variable. If the variable as more than two levels we can conduct post hoc tests. Select the variable and transfer it to the box labelled Post Hoc Tests for.
interaction effect
Interaction effects examines two or more factors at the same time, which may not be predictable based on the effects of either factor on their own.
main effect
Main effect is the effect of one factor (IV) on its own
total sums of squares, SSt
calculate in the same way as one-way
the residual sum of squares, SSr
calculated the same was as in one-way anova SS1 + SS2 + SS3 etc it represents individual differences in performance or the variance that can't be explained by factors that were systematically manipulated
how many levels?
can use numerical code to designate the number of levels of each factor 2 X 3 means that there are 2 levels of factor A and 3 levels of factor B- because there are two numbers, and therefore, two factors, it is a two-way design 2 X 3 X 3 means there are 2 levels of factor A, 3 levels of factor B, and 4 levels of factor C-because there are three numbers, therefore, three factors, it is a three-way design
the f-ratios
each effect in a two way ANOVA has its own f-ratio to calculate these we have to first calculate the MS for each effect by taking the SS and dividing by the respective degrees of freedom
why factorial design?
enhances external validity: by not limiting the study to examining the influence of only one IV on the DV you can improve the generalizability of the study's findings more efficient: we can use fewer participants than compared to if we did separate studies on gender/depression scores and diagnosis/depression scores. Whilst factorial designs require more participants and resources than a single factor study, compared to conducting several single factor studies to thoroughly examine a particular DV, conducting one factorial design will be more efficient and provide more information. test for interactions between factors: Factorial designs cannot only examine the effects of each IV on its own, but also the combined effects of the IVS- the interaction between factors. used to control systematic and non-systematic variance: Including participant characteristics such as age or gender as IVs and reduce the error variance of the analysis- which will make the statistical analysis more powerful Including additional variables also enables you to test whether these variables are acting as confounding variables
Multi-subscript notation Xijk
in two-way ANOVA< three subscripts are needed the first, i, identifies the level of factor a (row) the second subscript, j, identifies the level of factor B- column together (i,j) identify the cell in the data table subscript K identifies the individual score within a score n = number of scores in each cell (in a balanced design) nij = number of scores in a specific cell (i,j) a = number of levels of factor A b = number of levels of factor B N= total number of scores
ANOVA notation
we use subscript notation to represent and locate individual scores in an experiment single subject notation Xi the ith score in a set
heterogeneity and sample size
• Unequal sample sizes and heterogeneity do not mix • Gamst et. Al (2008) provide further information about this situation: • Heterogeneity can become a serious issue with unequal sample sizes. For example, large group variances associated with small sample sizes tend to produce a liberal F statistic whose nominal alpha level such as .05 is actually operating at a less stringent level such as .10. • Conversely, when the situation arises where large sample sizes and large group variances are associated, we produce a conservative F statistic, where the nominal alpha level such as .05 is actually more stringent, for example, .01 (Stevens, 2002). (p. 58)
how do we test the interaction?
An interaction tests whether the effect of one factor is the same across all levels of another factor. If there is no interaction the difference between means should be the same. Therefore, if differences are not the sample then there is an interaction — we can reject the null hypothesis because they are not the same. H 0 = (A1B1 - A1B2) - (A2B1 - A2B2) = 0
when is a liberal F statistic produced?
Large group variances associated with small sample sizes tend to produce liberal F statistic - more likely to find a significant difference when there isn't one
when is a conservative F statistic produced?
Large groups variances associated with large sample tend to produce conservative F statistic - more likely not to find a significant difference when one does exist.
why can't we use graph interactions as an indication of significance?
We can use graphs to give an indication of whether there is a main effect or interaction but you CANNOT use graphs to tell you if the main effects or interactions are statistically significant. This is because differences between means can be due to sampling error. Therefore, you must ALWAYS use statistical tests to determine significance.
factorial ANOVA using SPSS
analyze > general linear model > univariate IV placed into fixed factors and DV placed in dependent variable to graph interactions we click on plots and place factor 1 on horizontal axis and factor 2 on separate lines (doesn't matter which way round the variables are plotted)
variance
assume the variance is roughly equal between groups/levels of factor if it is violated we can use a more strict p value, typically p <0.025 this helps guard against type 1 error transform data- may reduce heterogeneity
model sum of squares SSm
broken into three components: SSA, SSB, and SSAB we can calculate SSAB by using subtraction SSAB = SSM - SSA- SSB
what does a factorial ANOVA do?
examines the effect of each factor IV on its own, as well as the combined effect of the factors main effect is the effect of one factor (IV) on its own interaction effect examines two or more factors at the same time- that is the combined effect, which may not be predictable based on the effects of either factor on their own thus, factorial ANOVA produce multiple F ratios- one for each main effect and interaction term knowing about one f-ratio tells you nothing about the others- each term sits by itself
what kinds of factorial designs are there?
independent measures repeated measures or mixed design- a combination of independent and repeated measure factors (mixed models)
between subjects factorial designs
independent measures there are a number of questions we can ask: is there an effect of being in condition A1-3? is there an effect of being in condition B1-2? then there is the extra question of : is there an interaction between A and B e.g. o IV: psychiatric diagnosis AND gender o DV: depression scores o Is there an effect of diagnosis (factor 1) on depression score- ignoring gender? o Is there an effect of gender (factor 2) on depression score- ignoring diagnosis? o Then we can ask the more complicated question: does the effect of diagnosis depend on your gender (interaction)?
main effect with interaction
main effect with interaction: nonparallel lines
main effect with no interaction
main effect with no interaction: parallel lines, there is no crossing over
factorial designs
more often than not, behaviour. affect and cognitions are influenced by more than one factor therefore, we often design studies which involve examining the impact of two or more IVs on a particular dependent variable factorial designs allow us to investigate complicated relationships
double notation, Xij
needed when we have multiple variables for the sample participant the first subject, i, refers to the row that the particular value is in the second subscript, j, refers to the column what is X2,3? the second person (second row) and the third value (third column)
effect size for two-way ANOVA
omega squared- need to first compute a variance component for each of the effects and the error and then use these to calculate the effect sizes for each In these equations, a is the number of levels of the first independent variable, b is the number of levels of the second independent variable and n is the number of people per condition. we also need to estimate the total variability: this involves the sum of all variance component for each of the effects and error plus the residual MS the effect size is then calculated by the effect/total variance
planned contrasts- weights
set up contracts- assign positive/negative, magenta and weight to each chunk set up two-way independent ANOVA as usual but press paste instead of OK to copy the syntax into the syntax editor window Add a line like: /CONTRAST (ALCOHOL)SPECIAL (2 -1 -1) Add at top: attractiveness BY Gender Alcohol
dealing with normality violations
transformation remove outliers use a nonparametric method- Friedman's test
factorial
two or more IVs
assumptions of two-way ANOVA
we have the same overall assumptions we had with one-way ANOVA o Interval/ratio level data for DV o Independence o Normality o Homogeneity of variance • Overall, ANOVA is a relatively robust procedure to violations of assumptions: o This is especially true for violations of normality, especially when sample sizes are large o Normality is not a crucial assumption and can be violated without great consequence
underlying theory of a two way ANOVA
we still find the SSt and break this variance down into variance that can be explained by the experiment (SSM) and variance that cannot be explained (SSR) in two way ANOVA, the variance explained by the experiment is made up of not one experimental manipulation but two therefore we break the model sum of squares down into variance explained by the first IV (SSA), second IV (SSB) and variance explained by the interaction of these two variables (SSA X SSB)
interpreting interactions graphs
whilst you can use graphs to give you an indication of whether there is a main effect or interaction, you cannot use graphs to tell you if main effects or interactions are statistically significant interaction- non parallel lines no interaction- parallel lines/no crossing over
planned contrasts
you can do planned contrasts for a two-way design but only for the main effects you can run contrasts involving one IV at a time if you have predictors to test.
normality
• ANOVA is considered to be quite resilient or robust to departures from normality. As Winer et al. (1991) suggest "A reasonable statement is that the analysis of variance F statistic is robust with regard to moderate departures from normality when sample sizes are reasonably large and equal..." (p. 101) • Confidence in the "robustness" of the normality assumption come in part from an important statistical principle known as the central limit theorem, which indicates that as we increase the sample size (n), the sample mean will increasingly approximate a normal distribution. • Hence, Keppel and Wickens (2004) argue persuasively that "once the samples become as large as a dozen or so, we need not worry much about the assumption of normality" (p. 145).
contrasts in SPSS
• For one-way ANOVA, SPSS has a procedure for entering codes that define the contrasts we want to do. However, for two-way ANOVA no such facility exists and instead we are restricted to doing one of several standard contrasts. • We can use standard contrasts • In reality, most of the time we want contrasts for our interaction term, and they can be obtained only through syntax. • To get contrasts for the main effect of factor 1, click on contrast in the main dialog box.
using syntax to do simple effects analysis
• Setup the analysis as normal, but when you're ready to run it, instead of pressing OK, press paste- which will paste the analysis info into a new syntax file • Type an extra command in your syntax file e.g. /EMMEANS=TABLES (GENDER*ALCOHOL) COMPARE (ALCOHOL) ADJ (BONFERRONI) • Compare(alcohol) tells SPS to examine the interaction by comparing the different alcohol levels one gender at a time • ADJ(bonferroni) this will conduct post-hoc tests for the differences between alcohol levels by gender.
simple effects analysis
• When you find a significant interaction between two or more variables, simple effects analysis (i.e. post hoc tests) can be used to examine that interaction. • If there is no significant interaction you cannot do simple effects analysis. • Simple effects analysis looks at the effects of one independent variable at one level of the other independent variable, then repeats the process for all other levels • For example, we could use simple effects analysis to look at the effects of alcohol on attractiveness for males, then at the effects of alcohol on attractiveness for females. • For males we would do a normal ANOVA with the different levels of alcohol and then separate we do a one-way ANOVA for females. • We have to use syntax to do this