Stats Exam 2 Review
An ANOVA produces SStotal = 80 and SSwithin = 30. What is the SSbetween?
50 SStotal = SSwithin + SSbetween so 80 = 30 + SSbetween SSbetween = 80 - 30 SSbetween = 50
Testwise Alpha Level
the risk of a Type I error for an individual statistical test
Levels
the specific changes to the independent variable that the researcher chose to use in an experiment Also referred to as "conditions"
Experimentwise Alpha Level
the total probability of a Type I error that accumulates from the series of statistical tests within the experiment Formula: αew = 1 - (1 - αpc)^c αew: experimentwise error rate αpc: testwise error rate c: number of comparison groups
What is a Two-Way Factorial ANOVA?
A Two-Way Factorial ANOVA is a statistical test that allows us to analyze a research situation with TWO independent variables where we can examine the effect of each independent variable separately (i.e. main effect) and the interaction between two independent variables (i.e. interaction effect)
What is a Type I Error?
A Type I error occurs when the null hypothesis is rejected when it is true, a false positive. Ex. Your Covid test comes back positive but you don't actually have Covid
What is a biased estimate?
A biased estimate is an estimate of a population parameter that is likely systematically to OVERestimated or UNDERestimated the true value of the population parameter
Cell Mean
A cell mean is the mean of a particular combination of levels of the variables that divide the groups in a factorial ANOVA and are used to explore possible interaction effects
Two-Way ANOVA: Interpreting Graphs
A large distance between both lines indicates a main effect of Factor B A steep slope of both lines indicates a main effect of Factor A The lines intersecting indicate an interaction effect of Factors A and B Remember, when marginal means of Factor A and/or Factor B are the same, there is no main effects
A smaller error variance means...
A larger F-ratio
What is a One-Way ANOVA?
A statistical test that allows us to analyze a research situation with ONE Independent Variable with THREE or more levels F = MSbetween / MSwithin
What are Planned Comparisons?
A test that is conducted when there are multiple groups of scores, but specific comparisons have been specified prior to data collection These comparisons are based on theoretical findings from literature reviews and previous research Also called Priori Comparisons
F-Ratio
ANOVAs analyze the variance of scores both between groups and within groups in an attempt to determine if the treatment conditions affect scores differently
One-Way ANOVA: Mean Square Between (MSbetween)
An estimate of the variance between groups MSbetween = SSbetween / dfbetween
One-Way ANOVA: Mean Square Within (MSwithin)
An estimate of the variance within groups MSwithin = SSwithin / dfwithin
What is another name for a Two-Way Factorial ANOVA?
Factorial Design ANOVA
What is the Tukey's HSD formula?
HSDα = q (k, dfwithin) √MSwithin / n q = studentized q k = number of treatments (groups) in the overall experiment n = number of scores in each group
What are the Small, Medium, and Large Effect Sizes for Eta-Squared?
Small Effect Size: η^2 = .02 Medium Effect Size: η^2 = .15 Large Effect Size: η^2 = .26
One-Way ANOVA: Between-Groups Sums of Squares (SSbetween)
The sum of the squared deviations of each group's mean from the grand mean, multiplied by the number of subjects in each group. The variation attributed to, or between, the groups. Formula: SSbetween = ∑ [ (ΣXg )^2 / ng ] - (ΣX)^2 / N ∑ = Sum of SSbetween for all of the groups in the study Xg = the scores within each group or condition ng = the number of participants in each group
One-Way ANOVA: Different F-Ratio Formula Descriptions
There are several different ways the F-Ratio is stated: F = MS-between / MS-within F = Variance between treatments / Variance within treatments (**aka Error term) F = systematic treatment effects + random, unsystematic differences / random, unsystematic differences F = 0 + random, unsystematic differences / random, unsystematic differences
What does an ANOVA that is "2 x 3 x 2" tell you about the experiment?
There is 3 independent variables The first IV has 2 levels, the second IV has 3 levels, and the third IV has 2 levels There are 12 (2x3x2) conditions for this factorial design
What is Tukey's HSD?
Tukey's Honestly Significant Difference (HSD) A test that provides a single value to determine the minimum difference between treatment means that is necessary to claim statistical significance- a difference large enough that p < alpha (experimentwise) Probability of making at least one Type I error given all Null Hypotheses (H0s) are true Controls for experimentwise error rate Best for doing all pair-wise comparison post-hoc Not appropriate for heterogeneous variance or unequal sample sizes
What are some of the commonly used Post-Hoc Tests?
Tukey's Honestly Significant Difference (HSD) test Bonferroni test Scheffé test Dunnett's test Fisher's protected t-test Neumann-Keuls Duncan's
When is a One-Way Repeated ANOVA used?
Used when limited number of subjects are available Used to minimize the level of error variance Used in research topics on learning or change in performance over time or phases in an experiment
Within-Subjects Factorial ANOVA
When all independent variables are within-subjects
Complex/Mixed Factorial ANOVA
When the independent variables are a mix of between-subjects and within-subject measures
When is Tukey's HSD not appropriate to use?
When there is unequal sample sizes or heterogeneous variance
What is the final goal for an ANOVA?
a F-Ratio F= Variance between treatments / Variance within treatments
One-Way ANOVA: What Degrees of Freedom are used to find the F-critical Value?
df-between (numerator) and df-within/error (denominator)
One-Way Repeated Measures ANOVA: What Degrees of Freedom are used to find the F-critical Value?
df-between and df-error
Two-Way Factorial ANOVA: What Degrees of Freedom are used to find the F-critical Value?
df-factor(a, b, axb) and df=error
Two-Way ANOVA: What is the formula to find degrees of freedom for Factor A?
dfa = A - 1
Two-Way ANOVA: What is the formula to find degrees of freedom for A x B?
dfaxb = (A-1)(B-1)
Repeated Measures ANOVA: What is the formula to find degrees of freedom for Subject?
dfsubject = n -1 n = number of participants within each group
Two-Way ANOVA: What is the formula to find degrees of freedom for Total?
dftotal = N - 1
Repeated Measures ANOVA: What is the formula to find degrees of freedom for Total?
dftotal = N - 1 N = total number of participants in the entire study
Two-Way ANOVA: What is the formula to find MSerror?
MSerror = SSerror / dferror
What are we looking at when interpreting Post-Hoc scores?
Magnitude of the Difference from the Tukey HSD Value **subtract so that difference is positive or take the absolute value of the difference scores
What is MSwithin also known as?
Mean Square Error (MSerror)
What is the "MSE"?
Mean Squared Error aka MS-within (denominator of F-ratio)
What is the difference between "N" and "n"?
"N" is the total number of participants in the STUDY while "n" is the number of participants in each GROUP
What are changes in between-groups variance caused by?
1. Systematic differences caused by treatments 2. Random, unsystematic differences i.e. Individual differences and/or experimental (measurement) error
In terms of Variance, what are we wanting?
A big treatment effect with only a little error
Why do we perform a Post-Hoc Test when our F-ratio value has already determined significance?
A significant F-ratio indicates that at least one difference in means is statistically significant but does not indicate which means differ significantly from each other, therefore, we would use a Post-Hoc test to see specifically which mean differences are significant.
What is the APA formatted statistic for Repeated-Measures ANOVA?
F (df-between, df-error) = F-obt, MSE, p-value, eta-squared value
Two-Way ANOVA: What is the formula to find F-value for Factor A?
F = MSa / MSerror
True or False: A disadvantage of combining 2 factors in an experiment is that you cannot determine how either factor would affect participants' scores if it were examined in an experiment by itself
False A Two-Way Factor ANOVA allows us to determine the effect of one variable controlling for the effect of the other (i.e. main effects)
True or False: Two separate One-Way ANOVAs provide exactly the same information that is obtained from a Two-Factor ANOVA
False Main Effects (i.e. the specific effect of one Factor on the DV) in a Two-Factor ANOVA are identical to results of two One-Way ANOVAs BUT Two-Factor ANOVAs also provide Interaction Effect results that One-Way ANOVAs do not
True or False: Post tests are needed if the decision from an ANOVA is "fail to reject the null hypothesis" (i.e. the Null Hypothesis (H0) is true)
False Post-Hoc tests are needed ONLY if you reject the Null Hypothesis (H0)- indicating at least one mean difference is significant Remember: If the Null Hypothesis (H0) is true that means that mu1 = mu2 = mu3 which indicates there is no difference between means and therefore no statistical significance
Factorial Design Matrix: Determining Interaction
If the patterns of the rows in a FDM are different then there is an interaction
What are sources of experimental error?
Individual differences (subject variables) -motivation -ability Testing environment -Time of day -Noise Measurement error Variability of testing -any given manipulation cannot be administered to participants exactly the same way
Scheffe Test
Less power for pairwise comparisons, but better for complex comparisons ex. Comparing the combined effect of 2 drug doses to a single control group (placebo) Uses weight means to achieve desired comparison
What type of ANOVA is used for Between-Subjects Design?
One-Way Randomized ANOVA
What two components are analyzed for total variability?
SS-total and df-total SS-total = SS-between + SS-within df-total = df-between + df-within
Repeated Measures ANOVA: What is the formula to find MSbetween?
SSbetween / dfbetween
What is the alternative name for SSwithin?
SSerror
Repeated Measures ANOVA: What is the formula to find MSerror?
SSerror / dferror
Repeated Measures ANOVA: What is the formula to find MSsubject?
SSsubject / dfsubject
What does the F-ratio for an One-Way Repeated Measures ANOVA not account for?
Subject
What is a Main Effect?
The effect of a single independent variable on the dependent variable. Specifically, a main effect is examining the mean differences among levels of one factor or independent variable and differences are tested for statistical significance Each factor (or independent variable) is evaluated independently of the other factor(s) in the study There is as many main effects as there are independent variables in a study
What is the most important thing about Groups for an One-Way Repeated Measures ANOVA?
They must be equal
What is the Sum of Squares?
The sum of squared deviations scores
What does a F-ratio close to 1 tell you?
There are no systematic treatment effects
True or False: ANOVAs are always two-tailed?
True
Two-Way Factorial Design: True or False: The larger the Degrees of Freedom, the larger the Variabilities?
True
Between-Subjects Factorial ANOVA
When all independent variables are between-subjects
One-Way ANOVA: What is the formula for finding the degrees of freedom for Within-Groups?
df = N - k N = total number of participants k = number of groups *Remember: "N" and "n" are two different things
One-Way ANOVA: What is the formula for finding the degrees of freedom for the Total?
df = N -1 N = number of participants
One-Way ANOVA: What is the formula for finding the degrees of freedom for Between-Groups?
df = k -1 k = number of groups
One-Way ANOVA: What are the two parameters for finding "q"?
k and df-within k = number of groups df-within = N - k
Factorial Design
studies that combine two or more factors (i.e. independent variables)
What is the difference between t-tests and ANOVAs?
t-tests directly examine the difference between means while ANOVAs examine the difference in variances to determine which means are different *Remember: variances is a measure of variability that indicates how far each data point in a group is from the mean
Repeated Measures ANOVA: What is the formula to find F-value?
MSbetween / MSerror
What are the assumptions of an ANOVA test?
1. Data is on interval-ratio scale 2. Underlying population distribution is normally distributed 3. Assumption of homogeneity of variance: σ^2 1 = σ^2 2 = σ^2 3 4. Distributions are equal except for the means 5. Observations are independent of each other
A researcher obtains an F-ratio with df = 2, 12 in a Repeated-Measures ANOVA. How many subjects participated in each study?
7 We are given: dfbetween = 2 and dferror = 12 dfbetween = k-1 2 = k-1 k = 3 dferror = (k-1)(n-1) 12 = (3-1)(n-1) 12 = (2)(n-1) 6 = n-1 n = 7
How can a One-Way ANOVA have a Between-Subjects Design?
A One-Way ANOVA can have a between-subjects design by having each level of the independent variable have different participants. Each participant experiences only one level of the independent variable F= MSbetween / MSwithin Ex. It is hypothesized that exam scores will differ depending on the type of music students listen to while studying. To test this, students are assigned to listen to either Rock, Classical, OR Rap music while they study for an exam. Student exam scores are then measured and compared.
What is a One-Way Repeated Measures ANOVA?
A One-Way Repeated Measures ANOVA is a statistical test that allows us to analyze a research situation with ONE independent variable that has THREE or more levels It is a Within-Subjects Design where each participant experiences ALL levels of the independent variable. I.e. participants are measured repeatedly F = MSbetween treatments / MSerror
What is a Factorial Design Matrix?
A factorial design matrix organizes data such that cell means from conditions and marginal means from the levels of each independent variables separately can be easily determined examining a factorial design matrix is a preliminary assessment of the effects before statistical analyses are conducted information provided in the matrix is used to create a line graph
Marginal Mean
A marginal mean is a mean score for all the participants at a particular level of one of the independent variables and are used to explore possible main effects Computed by finding the average of their respective interaction means
What is an Interaction Effect?
An interaction effect ( or Dependence of Factors) exists when difference in the effect of one factor (or IV) depend on the level of another factor (IV)
Which combination of factors is most likely to produce a large value for the F-ratio? A. large mean differences and large sample variances B. large mean differences and small sample variances C. small mean differences and large sample variances D. small mean differences and small sample variances
B. large mean differences and small sample variances
What variance do we want more of?
Between-Groups Variance If Within-Groups Variance is larger that means greater error rate and therefore smaller F-ratio, remember Within is also called Error
What are the two types of variances in ANOVAs?
Between-groups variance and within-groups variance
What is used to counteract the increase in alpha error due to multiple hypothesis test? (i.e. if you were to use multiple t-tests)
Bonferroni Adjustment
Dunnett's Test
Comparing all treatments with a control group Most powerful to hold experiment-wise error rates at or below alpha when comparisons are made to control group
If a Two-Factor ANOVA produces a statistically significant interaction, then you can conclude that A. Either the main effect for factor A or the main effect for factor B is also significant B. Neither the main effect for factor A nor the main effect for factor B is significant C. Both the main effect for factor A and the main effect for factor B are significant D. The significance of the main effects is not related to the significance of the interaction
D. The significance of the main effects is not related to the significance of the interaction Remember: When results provide a statistically significant INTERACTION that is saying that Factor A x Factor B together effected the dependent variable that is being tested. A significant Main Effect is when one factor (A OR B) can be said to be statistically significant in relation to the dependent variable. You can have a significant main effect for Factor A or B and no significant interaction effect for Factor AxB, and vice versa. The calculations are independent of one another.
Factorial Design Matrix Layout
Means found on the INSIDE of the matrix are the interaction means (cell means), i.e. interaction of AxB, and these means are what are plotted in a figure Means for each main effect are referred to as marginal means and are found OUTSIDE the matrix, i.e. main effect of factor a, main effect of factor b
What is Mean Square (MS)?
Estimates of variance across groups To calculate MS we divide the Sum of Squares by their respective dfs in order to get the average deviation from the mean, this lets us compare ratios to determine whether there is a significant difference due to treatment-the larger the ratio, the more the treatments affect the outcome By dividing by df we are able to create an unbiased estimate
What is the APA formatted statistic for One-Way ANOVA?
F (df-between-groups, df-within-groups) = F-obt, MSE, p-value, eta-squared value
Two-Way ANOVA: What is the formula to find F-value for AxB?
F = MSaxb / MSerror
Two-Way ANOVA: What is the formula to find F-value for Factor B?
F = MSb / MSerror
Two-Way Factorial ANOVA F-Ratio Description
F = Variance (mean differences) between sample means / Variance (mean differences expected with no treatment effect)
What is the F-Distribution formula?
F = Variance between sample means / Variance expected with no treatment effect F = MSbetween treatments / MSerror (also called MSwithin)
What are two other names for Interaction Effect?
Non-Additive Effects-because the main effects do not "add" together AxB Interaction where A represents one IV and B represents the other IV
One-Way ANOVA: Within-Groups Sums of Squares (SSwithin)
The sum of the squared deviations of each score from its group mean. The sum of squares of the residual error is the variation attributed to the error. Formula: SSwithin = ∑ [ ΣXg^2 - (ΣXg )^2 / ng ] ∑ = Sum of SSwithin for all of the groups in the study Xg = the scores within each group or condition ng = the number of participants in each group
One-Way ANOVA: Total Sums of Squares (SStotal)
The sum of the squared deviations of each score from the grand mean and helps express the total variation that can be attributed to various factors. The Sum of Squares considers both the sum of squares from the factors (i.e. what you're testing) and from randomness or error Formula: SStotal = ΣX^2 - (ΣX)^2 / N N = total number of participants in study
True or False: A report shows ANOVA results: F(2,27) = 5.36, p < .05. You can conclude that the study used a total of 30 participants.
True dftotal = N-1 AND dftotal = dfbetween + dfwithin so dfs given are (2, 27) dfbetween = 2 dfwithin = 27 therefore dftotal = 27 + 2 = 29 AND dftotal = N-1 so 29 = N-1 N = 29 + 1 N = 30 Remember "N" is the total number of participants in the entire study
Are Main Effects and Interaction Effects independent or dependent on each other?
Independent Remember: A Main Effect is looking at whether the effect of one specific factor (or IV) is significantly different than the dependent variable while Interaction Effect is looking to see if there is a significant interaction that is dependent on the level or value of the other factor Main Effect: IV on DV Interaction Effect: (IVa x IVb) on DV
Two-Way ANOVA: What is the formula to find MS Factor A?
MSa = SSa / dfa
How are ANOVAs and t-tests related?
F = t^2 For independent samples, either t or F can be used, it will always result in the same decision
F-Distribution
F-distribution is a distribution of all possible values when the null hypothesis (H0) is true Every F distribution has separate degrees of freedom for the numerator and denominator A Fcv value cannot be negative because it is a ratio of two variances and variances must be positive F-distributions are always positive and positively skewed
One-Way Repeated ANOVA: F-Ratio Formula Descriptions
F= variance (differences) between sample means (without individual differences) / variance (differences) expected with no treatment effect (individual differences removed mathematically)
What is the APA formatted statistic for Two-Way Factorial ANOVA?
Factor A: F (df-factor a, df-error) = F-obt, MSE, p-value, eta-squared value Factor B: F (df-factor b, df-error) = F-obt, MSE, p-value, eta-squared value Factor AxB: F (df-factor axb, df-error) = F-obt, MSE, p-value, eta-squared value
How can you use MS scores to look at error?
If MS-within is really large in comparison to MS-between there is more error in the study than treatment effect, which results in a small F-ratio
When using Tukey's HSD, when can we claim significance between two groups?
If the difference of two sample means (larger minus smaller) is greater than the HSDalpha then the difference between the two groups is significant at that alpha level
Factorial Design Matrix: Determining Main Effect
If the mean for Level 1a is different from the mean of Level 2a, there is a main effect of the independent variable A, but significance is still unknown Same means for Level 1a and Level 2a, there is no main effect of IV A
Two-Way ANOVA: What is the formula to find MS AxB?
MSaxb = SSaxb / dfaxb
Two-Way ANOVA: What is the formula to find MS Factor B?
MSb = SSb / dfb
What does "k" represent?
Number of treatments or groups in experiment *Remember: This is not the same as number of participants within a group. This is equivalent to the number of levels of the given independent variable
What type of ANOVA is more sensitive to small differences between groups?
One-Way Repeated Measures ANOVA
What type of ANOVA is used for Within-Subjects Design?
One-Way Repeated Measures ANOVA
What are errors from individual differences?
Participant characteristics may vary considerably from one person to another The differences in participant characteristics can influence measurements, especially in the dependent variable Repeated measures design controls for the effects of participant characteristics Due to the research design, individual differences are removed from the numerator of the F-ratio. However, individual differences must be statistically removed from the denominator
What are Post-Hoc Tests?
Post-Hoc Tests are follow-up tests done to determine exactly which mean differences are significant, and which are not, by comparing two individual means at a time (pairwise comparison) and each comparison includes risk of a Type I error How you decide which PAIRS of groups are significantly different aka the between-group differences *post-hoc is Latin for "after that"
What do Post-Hoc tests do in relation to error rate?
Post-Hoc tests are intended to keep the experiment-wise error rate to acceptable levels
Between- vs. Within-Subject ANOVAs
Random assignment of subjects in a between-subject design, observations are both change effect and treatment effect Chance effect due to random assignment minimized in within-subject design -Lowers error variance -Estimated by subjects' average performance across treatments
What is another name for Main Effect?
Simple Effects-because it is only looking at ONE independent variable
What does it mean when the Alternative Hypothesis (HA) is true?
Size of treatment effect is more than 0 F is noticeable larger than 1.00
What does it mean when the Null Hypothesis (H0) is true?
Size of treatment effect is near 0 F is near 1.00 Chance factors and error are the driving forces behind any differences No or little treatment effects Both between and within estimates should be the same
How is each variance for a F-Ratio computed?
Sum of Squares / Degrees of Freedom SS/df Variance between treatment: SS-between/df-between Variance within treatment: SS-within / df-within
Why is the Sum of Squares important?
Sum of Squares allows us to look at each score and see how far away the score is from the mean The bigger the sum of squares, the farther away your scores are from your mean, which means there is more variability in your data
What do we use to calculate variance in a Two-Way Factorial ANOVA? And why?
Sum of Squares because there are several sources of variance, therefore, several sums of squares must be calculated
For a Two-Way Factorial ANOVA, what do you know about independent variables?
That you have at least TWO independent variables
What is the alternative hypothesis (HA) for an ANOVA?
The alternative hypothesis for an ANOVA states that there is at least one mean difference among the populations HA: μ1 ≠ μ2 ≠ μ3 OR HA: μ1 = μ3 but μ2 is different OR HA: μ1 > μ2 < μ3 *Remember: the alternative hypothesis is rarely directional in an ANOVA and it should be based off previous research and theory
What is the biggest adjustment between Between-Subjects ANOVAs and Within-Subjects ANOVAs?
The biggest adjustment is the addition of a process to mathematically remove the individual differences variance component from the denominator of the F-ratio. F = variance (differences) between sample means (without individual differences) / variance (differences) expected with no treatment effect (individual differences removed mathematically)
What are random, unsystematic factors?
The error within the study which must be accounted for in both between- and within-treatment variance
One-Way ANOVA: Effect Size (Eta-Squared)
The measure of the proportion of variability in the scores that is accounted for by the differences between treatments Eta-Squared does NOT care about the percentage of variance accounted for by error Eta-Squared shows how much the effect of treatment is, with the understanding that significance was obtained by having little error in the study η^2 = SSbetween / SStotal
What is the null hypothesis for an ANOVA?
The null hypothesis for an ANOVA states that all THREE levels of the IV are the same H0: μ1 = μ2 = μ3
Within-Groups Variance
The variability within each sample Individual scores are not the same within each sample *Within-Groups Variance would be like when you are given data from treatments and are looking at the individual scores WITHIN the treatments Treatment 1: 5, 8, 9, 2 Treatment 2: 4. 7, 9, 8 Treatment 3: 1, 8, 7, 3
What is the difference between Tukey's HSD and the Scheffe Test?
Tukey's HSD lets you know how much each pair of groups must differ and the Scheffe Test just does an F-ratio with only the two groups you want to compare
Two-Way ANOVA: Graphing
Type of Graph: Line Graph X-Axis: More important Independent Variable, i.e. IV1-Level 1 and IV2-Level 2 will be graphed Y-Axis: Dependent Variable *Other Independent Variable goes in the Legend For Legend IV to be significant you need a large distance between both lines of IV that is graphed For X-Axis IV to be significant you need a line with a steep slope If the lines intersect or would intersect if they continued in either direction, then there is an interaction Without error bars we can only make educated guesses regarding the interaction Marginal means are the midpoints between cell means, the dots (not on the ends of the lines) represent the marginal means for the IVs
Two-Way ANOVA: What is the formula to find degrees of freedom for Factor B?
dfb = B - 1
Repeated Measures ANOVA: What is the formula to find degrees of freedom for Between?
dfbetween = k - 1 k = number of groups in the study
Repeated Measures ANOVA: What is the formula to find degrees of freedom for Error?
dferror = (k-1)(n-1) k = number of groups in study n = number of participants within each group
Two-Way ANOVA: What is the formula to find degrees of freedom for Error?
dferror = AB(N-1)
Between-Group Variance
the variability that results from general differences between the treatment conditions Variance between treatments measures DIFFERENCES among sample means *Between-Group Variance would be like when you are given data from treatments and are looking at the "M" (mean) value Treatment 1: M = 3 Treatment 2: M = 8 Treatment 3: M = 5