Stats Exam 2 Review

An ANOVA produces SStotal = 80 and SSwithin = 30. What is the SSbetween?

50 SStotal = SSwithin + SSbetween so 80 = 30 + SSbetween SSbetween = 80 - 30 SSbetween = 50

Testwise Alpha Level

the risk of a Type I error for an individual statistical test

Levels

the specific changes to the independent variable that the researcher chose to use in an experiment Also referred to as "conditions"

Experimentwise Alpha Level

the total probability of a Type I error that accumulates from the series of statistical tests within the experiment Formula: αew = 1 - (1 - αpc)^c αew: experimentwise error rate αpc: testwise error rate c: number of comparison groups

What is a Two-Way Factorial ANOVA?

A Two-Way Factorial ANOVA is a statistical test that allows us to analyze a research situation with TWO independent variables where we can examine the effect of each independent variable separately (i.e. main effect) and the interaction between two independent variables (i.e. interaction effect)

What is a Type I Error?

A Type I error occurs when the null hypothesis is rejected when it is true, a false positive. Ex. Your Covid test comes back positive but you don't actually have Covid

What is a biased estimate?

A biased estimate is an estimate of a population parameter that is likely systematically to OVERestimated or UNDERestimated the true value of the population parameter

Cell Mean

A cell mean is the mean of a particular combination of levels of the variables that divide the groups in a factorial ANOVA and are used to explore possible interaction effects

Two-Way ANOVA: Interpreting Graphs

A large distance between both lines indicates a main effect of Factor B A steep slope of both lines indicates a main effect of Factor A The lines intersecting indicate an interaction effect of Factors A and B Remember, when marginal means of Factor A and/or Factor B are the same, there is no main effects

A smaller error variance means...

A larger F-ratio

What is a One-Way ANOVA?

A statistical test that allows us to analyze a research situation with ONE Independent Variable with THREE or more levels F = MSbetween / MSwithin

What are Planned Comparisons?

A test that is conducted when there are multiple groups of scores, but specific comparisons have been specified prior to data collection These comparisons are based on theoretical findings from literature reviews and previous research Also called Priori Comparisons

F-Ratio

ANOVAs analyze the variance of scores both between groups and within groups in an attempt to determine if the treatment conditions affect scores differently

One-Way ANOVA: Mean Square Between (MSbetween)

An estimate of the variance between groups MSbetween = SSbetween / dfbetween

One-Way ANOVA: Mean Square Within (MSwithin)

An estimate of the variance within groups MSwithin = SSwithin / dfwithin

What is another name for a Two-Way Factorial ANOVA?

Factorial Design ANOVA

What is the Tukey's HSD formula?

HSDα = q (k, dfwithin) √MSwithin / n q = studentized q k = number of treatments (groups) in the overall experiment n = number of scores in each group

What are the Small, Medium, and Large Effect Sizes for Eta-Squared?

Small Effect Size: η^2 = .02 Medium Effect Size: η^2 = .15 Large Effect Size: η^2 = .26

One-Way ANOVA: Between-Groups Sums of Squares (SSbetween)

The sum of the squared deviations of each group's mean from the grand mean, multiplied by the number of subjects in each group. The variation attributed to, or between, the groups. Formula: SSbetween = ∑ [ (ΣXg )^2 / ng ] - (ΣX)^2 / N ∑ = Sum of SSbetween for all of the groups in the study Xg = the scores within each group or condition ng = the number of participants in each group

One-Way ANOVA: Different F-Ratio Formula Descriptions

There are several different ways the F-Ratio is stated: F = MS-between / MS-within F = Variance between treatments / Variance within treatments (**aka Error term) F = systematic treatment effects + random, unsystematic differences / random, unsystematic differences F = 0 + random, unsystematic differences / random, unsystematic differences

What does an ANOVA that is "2 x 3 x 2" tell you about the experiment?

There is 3 independent variables The first IV has 2 levels, the second IV has 3 levels, and the third IV has 2 levels There are 12 (2x3x2) conditions for this factorial design

What is Tukey's HSD?

Tukey's Honestly Significant Difference (HSD) A test that provides a single value to determine the minimum difference between treatment means that is necessary to claim statistical significance- a difference large enough that p < alpha (experimentwise) Probability of making at least one Type I error given all Null Hypotheses (H0s) are true Controls for experimentwise error rate Best for doing all pair-wise comparison post-hoc Not appropriate for heterogeneous variance or unequal sample sizes

What are some of the commonly used Post-Hoc Tests?

Tukey's Honestly Significant Difference (HSD) test Bonferroni test Scheffé test Dunnett's test Fisher's protected t-test Neumann-Keuls Duncan's

When is a One-Way Repeated ANOVA used?

Used when limited number of subjects are available Used to minimize the level of error variance Used in research topics on learning or change in performance over time or phases in an experiment

Within-Subjects Factorial ANOVA

When all independent variables are within-subjects

Complex/Mixed Factorial ANOVA

When the independent variables are a mix of between-subjects and within-subject measures

When is Tukey's HSD not appropriate to use?

When there is unequal sample sizes or heterogeneous variance

What is the final goal for an ANOVA?

a F-Ratio F= Variance between treatments / Variance within treatments

One-Way ANOVA: What Degrees of Freedom are used to find the F-critical Value?

df-between (numerator) and df-within/error (denominator)

One-Way Repeated Measures ANOVA: What Degrees of Freedom are used to find the F-critical Value?

df-between and df-error

Two-Way Factorial ANOVA: What Degrees of Freedom are used to find the F-critical Value?

df-factor(a, b, axb) and df=error

Two-Way ANOVA: What is the formula to find degrees of freedom for Factor A?

dfa = A - 1

Two-Way ANOVA: What is the formula to find degrees of freedom for A x B?

dfaxb = (A-1)(B-1)

Repeated Measures ANOVA: What is the formula to find degrees of freedom for Subject?

dfsubject = n -1 n = number of participants within each group

Two-Way ANOVA: What is the formula to find degrees of freedom for Total?

dftotal = N - 1

Repeated Measures ANOVA: What is the formula to find degrees of freedom for Total?

dftotal = N - 1 N = total number of participants in the entire study

Two-Way ANOVA: What is the formula to find MSerror?

MSerror = SSerror / dferror

What are we looking at when interpreting Post-Hoc scores?

Magnitude of the Difference from the Tukey HSD Value **subtract so that difference is positive or take the absolute value of the difference scores

What is MSwithin also known as?

Mean Square Error (MSerror)

What is the "MSE"?

Mean Squared Error aka MS-within (denominator of F-ratio)

What is the difference between "N" and "n"?

"N" is the total number of participants in the STUDY while "n" is the number of participants in each GROUP

What are changes in between-groups variance caused by?

1. Systematic differences caused by treatments 2. Random, unsystematic differences i.e. Individual differences and/or experimental (measurement) error

In terms of Variance, what are we wanting?

A big treatment effect with only a little error

Why do we perform a Post-Hoc Test when our F-ratio value has already determined significance?

A significant F-ratio indicates that at least one difference in means is statistically significant but does not indicate which means differ significantly from each other, therefore, we would use a Post-Hoc test to see specifically which mean differences are significant.

What is the APA formatted statistic for Repeated-Measures ANOVA?

F (df-between, df-error) = F-obt, MSE, p-value, eta-squared value

Two-Way ANOVA: What is the formula to find F-value for Factor A?

F = MSa / MSerror

True or False: A disadvantage of combining 2 factors in an experiment is that you cannot determine how either factor would affect participants' scores if it were examined in an experiment by itself

False A Two-Way Factor ANOVA allows us to determine the effect of one variable controlling for the effect of the other (i.e. main effects)

True or False: Two separate One-Way ANOVAs provide exactly the same information that is obtained from a Two-Factor ANOVA

False Main Effects (i.e. the specific effect of one Factor on the DV) in a Two-Factor ANOVA are identical to results of two One-Way ANOVAs BUT Two-Factor ANOVAs also provide Interaction Effect results that One-Way ANOVAs do not

True or False: Post tests are needed if the decision from an ANOVA is "fail to reject the null hypothesis" (i.e. the Null Hypothesis (H0) is true)

False Post-Hoc tests are needed ONLY if you reject the Null Hypothesis (H0)- indicating at least one mean difference is significant Remember: If the Null Hypothesis (H0) is true that means that mu1 = mu2 = mu3 which indicates there is no difference between means and therefore no statistical significance

Factorial Design Matrix: Determining Interaction

If the patterns of the rows in a FDM are different then there is an interaction

What are sources of experimental error?

Individual differences (subject variables) -motivation -ability Testing environment -Time of day -Noise Measurement error Variability of testing -any given manipulation cannot be administered to participants exactly the same way

Scheffe Test

Less power for pairwise comparisons, but better for complex comparisons ex. Comparing the combined effect of 2 drug doses to a single control group (placebo) Uses weight means to achieve desired comparison

What type of ANOVA is used for Between-Subjects Design?

One-Way Randomized ANOVA

What two components are analyzed for total variability?

SS-total and df-total SS-total = SS-between + SS-within df-total = df-between + df-within

Repeated Measures ANOVA: What is the formula to find MSbetween?

SSbetween / dfbetween

What is the alternative name for SSwithin?

SSerror

Repeated Measures ANOVA: What is the formula to find MSerror?

SSerror / dferror

Repeated Measures ANOVA: What is the formula to find MSsubject?

SSsubject / dfsubject

What does the F-ratio for an One-Way Repeated Measures ANOVA not account for?

Subject

What is a Main Effect?

The effect of a single independent variable on the dependent variable. Specifically, a main effect is examining the mean differences among levels of one factor or independent variable and differences are tested for statistical significance Each factor (or independent variable) is evaluated independently of the other factor(s) in the study There is as many main effects as there are independent variables in a study

What is the most important thing about Groups for an One-Way Repeated Measures ANOVA?

They must be equal

What is the Sum of Squares?

The sum of squared deviations scores

What does a F-ratio close to 1 tell you?

There are no systematic treatment effects

True or False: ANOVAs are always two-tailed?

True

Two-Way Factorial Design: True or False: The larger the Degrees of Freedom, the larger the Variabilities?

True

Between-Subjects Factorial ANOVA

When all independent variables are between-subjects

One-Way ANOVA: What is the formula for finding the degrees of freedom for Within-Groups?

df = N - k N = total number of participants k = number of groups *Remember: "N" and "n" are two different things

One-Way ANOVA: What is the formula for finding the degrees of freedom for the Total?

df = N -1 N = number of participants

One-Way ANOVA: What is the formula for finding the degrees of freedom for Between-Groups?

df = k -1 k = number of groups

One-Way ANOVA: What are the two parameters for finding "q"?

k and df-within k = number of groups df-within = N - k

Factorial Design

studies that combine two or more factors (i.e. independent variables)

What is the difference between t-tests and ANOVAs?

t-tests directly examine the difference between means while ANOVAs examine the difference in variances to determine which means are different *Remember: variances is a measure of variability that indicates how far each data point in a group is from the mean

Repeated Measures ANOVA: What is the formula to find F-value?

MSbetween / MSerror

What are the assumptions of an ANOVA test?

1. Data is on interval-ratio scale 2. Underlying population distribution is normally distributed 3. Assumption of homogeneity of variance: σ^2 1 = σ^2 2 = σ^2 3 4. Distributions are equal except for the means 5. Observations are independent of each other

A researcher obtains an F-ratio with df = 2, 12 in a Repeated-Measures ANOVA. How many subjects participated in each study?

7 We are given: dfbetween = 2 and dferror = 12 dfbetween = k-1 2 = k-1 k = 3 dferror = (k-1)(n-1) 12 = (3-1)(n-1) 12 = (2)(n-1) 6 = n-1 n = 7

How can a One-Way ANOVA have a Between-Subjects Design?

A One-Way ANOVA can have a between-subjects design by having each level of the independent variable have different participants. Each participant experiences only one level of the independent variable F= MSbetween / MSwithin Ex. It is hypothesized that exam scores will differ depending on the type of music students listen to while studying. To test this, students are assigned to listen to either Rock, Classical, OR Rap music while they study for an exam. Student exam scores are then measured and compared.

What is a One-Way Repeated Measures ANOVA?

A One-Way Repeated Measures ANOVA is a statistical test that allows us to analyze a research situation with ONE independent variable that has THREE or more levels It is a Within-Subjects Design where each participant experiences ALL levels of the independent variable. I.e. participants are measured repeatedly F = MSbetween treatments / MSerror

What is a Factorial Design Matrix?

A factorial design matrix organizes data such that cell means from conditions and marginal means from the levels of each independent variables separately can be easily determined examining a factorial design matrix is a preliminary assessment of the effects before statistical analyses are conducted information provided in the matrix is used to create a line graph

Marginal Mean

A marginal mean is a mean score for all the participants at a particular level of one of the independent variables and are used to explore possible main effects Computed by finding the average of their respective interaction means

What is an Interaction Effect?

An interaction effect ( or Dependence of Factors) exists when difference in the effect of one factor (or IV) depend on the level of another factor (IV)

Which combination of factors is most likely to produce a large value for the F-ratio? A. large mean differences and large sample variances B. large mean differences and small sample variances C. small mean differences and large sample variances D. small mean differences and small sample variances

B. large mean differences and small sample variances

What variance do we want more of?

Between-Groups Variance If Within-Groups Variance is larger that means greater error rate and therefore smaller F-ratio, remember Within is also called Error

What are the two types of variances in ANOVAs?

Between-groups variance and within-groups variance

What is used to counteract the increase in alpha error due to multiple hypothesis test? (i.e. if you were to use multiple t-tests)

Bonferroni Adjustment

Dunnett's Test

Comparing all treatments with a control group Most powerful to hold experiment-wise error rates at or below alpha when comparisons are made to control group

If a Two-Factor ANOVA produces a statistically significant interaction, then you can conclude that A. Either the main effect for factor A or the main effect for factor B is also significant B. Neither the main effect for factor A nor the main effect for factor B is significant C. Both the main effect for factor A and the main effect for factor B are significant D. The significance of the main effects is not related to the significance of the interaction

D. The significance of the main effects is not related to the significance of the interaction Remember: When results provide a statistically significant INTERACTION that is saying that Factor A x Factor B together effected the dependent variable that is being tested. A significant Main Effect is when one factor (A OR B) can be said to be statistically significant in relation to the dependent variable. You can have a significant main effect for Factor A or B and no significant interaction effect for Factor AxB, and vice versa. The calculations are independent of one another.

Factorial Design Matrix Layout

Means found on the INSIDE of the matrix are the interaction means (cell means), i.e. interaction of AxB, and these means are what are plotted in a figure Means for each main effect are referred to as marginal means and are found OUTSIDE the matrix, i.e. main effect of factor a, main effect of factor b

What is Mean Square (MS)?

Estimates of variance across groups To calculate MS we divide the Sum of Squares by their respective dfs in order to get the average deviation from the mean, this lets us compare ratios to determine whether there is a significant difference due to treatment-the larger the ratio, the more the treatments affect the outcome By dividing by df we are able to create an unbiased estimate

What is the APA formatted statistic for One-Way ANOVA?

F (df-between-groups, df-within-groups) = F-obt, MSE, p-value, eta-squared value

Two-Way ANOVA: What is the formula to find F-value for AxB?

F = MSaxb / MSerror

Two-Way ANOVA: What is the formula to find F-value for Factor B?

F = MSb / MSerror

Two-Way Factorial ANOVA F-Ratio Description

F = Variance (mean differences) between sample means / Variance (mean differences expected with no treatment effect)

What is the F-Distribution formula?

F = Variance between sample means / Variance expected with no treatment effect F = MSbetween treatments / MSerror (also called MSwithin)

What are two other names for Interaction Effect?

Non-Additive Effects-because the main effects do not "add" together AxB Interaction where A represents one IV and B represents the other IV

One-Way ANOVA: Within-Groups Sums of Squares (SSwithin)

The sum of the squared deviations of each score from its group mean. The sum of squares of the residual error is the variation attributed to the error. Formula: SSwithin = ∑ [ ΣXg^2 - (ΣXg )^2 / ng ] ∑ = Sum of SSwithin for all of the groups in the study Xg = the scores within each group or condition ng = the number of participants in each group

One-Way ANOVA: Total Sums of Squares (SStotal)

The sum of the squared deviations of each score from the grand mean and helps express the total variation that can be attributed to various factors. The Sum of Squares considers both the sum of squares from the factors (i.e. what you're testing) and from randomness or error Formula: SStotal = ΣX^2 - (ΣX)^2 / N N = total number of participants in study

True or False: A report shows ANOVA results: F(2,27) = 5.36, p < .05. You can conclude that the study used a total of 30 participants.

True dftotal = N-1 AND dftotal = dfbetween + dfwithin so dfs given are (2, 27) dfbetween = 2 dfwithin = 27 therefore dftotal = 27 + 2 = 29 AND dftotal = N-1 so 29 = N-1 N = 29 + 1 N = 30 Remember "N" is the total number of participants in the entire study

Are Main Effects and Interaction Effects independent or dependent on each other?

Independent Remember: A Main Effect is looking at whether the effect of one specific factor (or IV) is significantly different than the dependent variable while Interaction Effect is looking to see if there is a significant interaction that is dependent on the level or value of the other factor Main Effect: IV on DV Interaction Effect: (IVa x IVb) on DV

Two-Way ANOVA: What is the formula to find MS Factor A?

MSa = SSa / dfa

How are ANOVAs and t-tests related?

F = t^2 For independent samples, either t or F can be used, it will always result in the same decision

F-Distribution

F-distribution is a distribution of all possible values when the null hypothesis (H0) is true Every F distribution has separate degrees of freedom for the numerator and denominator A Fcv value cannot be negative because it is a ratio of two variances and variances must be positive F-distributions are always positive and positively skewed

One-Way Repeated ANOVA: F-Ratio Formula Descriptions

F= variance (differences) between sample means (without individual differences) / variance (differences) expected with no treatment effect (individual differences removed mathematically)

What is the APA formatted statistic for Two-Way Factorial ANOVA?

Factor A: F (df-factor a, df-error) = F-obt, MSE, p-value, eta-squared value Factor B: F (df-factor b, df-error) = F-obt, MSE, p-value, eta-squared value Factor AxB: F (df-factor axb, df-error) = F-obt, MSE, p-value, eta-squared value

How can you use MS scores to look at error?

If MS-within is really large in comparison to MS-between there is more error in the study than treatment effect, which results in a small F-ratio

When using Tukey's HSD, when can we claim significance between two groups?

If the difference of two sample means (larger minus smaller) is greater than the HSDalpha then the difference between the two groups is significant at that alpha level

Factorial Design Matrix: Determining Main Effect

If the mean for Level 1a is different from the mean of Level 2a, there is a main effect of the independent variable A, but significance is still unknown Same means for Level 1a and Level 2a, there is no main effect of IV A

Two-Way ANOVA: What is the formula to find MS AxB?

MSaxb = SSaxb / dfaxb

Two-Way ANOVA: What is the formula to find MS Factor B?

MSb = SSb / dfb

What does "k" represent?

Number of treatments or groups in experiment *Remember: This is not the same as number of participants within a group. This is equivalent to the number of levels of the given independent variable

What type of ANOVA is more sensitive to small differences between groups?

One-Way Repeated Measures ANOVA

What type of ANOVA is used for Within-Subjects Design?

One-Way Repeated Measures ANOVA

What are errors from individual differences?

Participant characteristics may vary considerably from one person to another The differences in participant characteristics can influence measurements, especially in the dependent variable Repeated measures design controls for the effects of participant characteristics Due to the research design, individual differences are removed from the numerator of the F-ratio. However, individual differences must be statistically removed from the denominator

What are Post-Hoc Tests?

Post-Hoc Tests are follow-up tests done to determine exactly which mean differences are significant, and which are not, by comparing two individual means at a time (pairwise comparison) and each comparison includes risk of a Type I error How you decide which PAIRS of groups are significantly different aka the between-group differences *post-hoc is Latin for "after that"

What do Post-Hoc tests do in relation to error rate?

Post-Hoc tests are intended to keep the experiment-wise error rate to acceptable levels

Between- vs. Within-Subject ANOVAs

Random assignment of subjects in a between-subject design, observations are both change effect and treatment effect Chance effect due to random assignment minimized in within-subject design -Lowers error variance -Estimated by subjects' average performance across treatments

What is another name for Main Effect?

Simple Effects-because it is only looking at ONE independent variable

What does it mean when the Alternative Hypothesis (HA) is true?

Size of treatment effect is more than 0 F is noticeable larger than 1.00

What does it mean when the Null Hypothesis (H0) is true?

Size of treatment effect is near 0 F is near 1.00 Chance factors and error are the driving forces behind any differences No or little treatment effects Both between and within estimates should be the same

How is each variance for a F-Ratio computed?

Sum of Squares / Degrees of Freedom SS/df Variance between treatment: SS-between/df-between Variance within treatment: SS-within / df-within

Why is the Sum of Squares important?

Sum of Squares allows us to look at each score and see how far away the score is from the mean The bigger the sum of squares, the farther away your scores are from your mean, which means there is more variability in your data

What do we use to calculate variance in a Two-Way Factorial ANOVA? And why?

Sum of Squares because there are several sources of variance, therefore, several sums of squares must be calculated

For a Two-Way Factorial ANOVA, what do you know about independent variables?

That you have at least TWO independent variables

What is the alternative hypothesis (HA) for an ANOVA?

The alternative hypothesis for an ANOVA states that there is at least one mean difference among the populations HA: μ1 ≠ μ2 ≠ μ3 OR HA: μ1 = μ3 but μ2 is different OR HA: μ1 > μ2 < μ3 *Remember: the alternative hypothesis is rarely directional in an ANOVA and it should be based off previous research and theory

What is the biggest adjustment between Between-Subjects ANOVAs and Within-Subjects ANOVAs?

The biggest adjustment is the addition of a process to mathematically remove the individual differences variance component from the denominator of the F-ratio. F = variance (differences) between sample means (without individual differences) / variance (differences) expected with no treatment effect (individual differences removed mathematically)

What are random, unsystematic factors?

The error within the study which must be accounted for in both between- and within-treatment variance

One-Way ANOVA: Effect Size (Eta-Squared)

The measure of the proportion of variability in the scores that is accounted for by the differences between treatments Eta-Squared does NOT care about the percentage of variance accounted for by error Eta-Squared shows how much the effect of treatment is, with the understanding that significance was obtained by having little error in the study η^2 = SSbetween / SStotal

What is the null hypothesis for an ANOVA?

The null hypothesis for an ANOVA states that all THREE levels of the IV are the same H0: μ1 = μ2 = μ3

Within-Groups Variance

The variability within each sample Individual scores are not the same within each sample *Within-Groups Variance would be like when you are given data from treatments and are looking at the individual scores WITHIN the treatments Treatment 1: 5, 8, 9, 2 Treatment 2: 4. 7, 9, 8 Treatment 3: 1, 8, 7, 3

What is the difference between Tukey's HSD and the Scheffe Test?

Tukey's HSD lets you know how much each pair of groups must differ and the Scheffe Test just does an F-ratio with only the two groups you want to compare

Two-Way ANOVA: Graphing

Type of Graph: Line Graph X-Axis: More important Independent Variable, i.e. IV1-Level 1 and IV2-Level 2 will be graphed Y-Axis: Dependent Variable *Other Independent Variable goes in the Legend For Legend IV to be significant you need a large distance between both lines of IV that is graphed For X-Axis IV to be significant you need a line with a steep slope If the lines intersect or would intersect if they continued in either direction, then there is an interaction Without error bars we can only make educated guesses regarding the interaction Marginal means are the midpoints between cell means, the dots (not on the ends of the lines) represent the marginal means for the IVs

Two-Way ANOVA: What is the formula to find degrees of freedom for Factor B?

dfb = B - 1

Repeated Measures ANOVA: What is the formula to find degrees of freedom for Between?

dfbetween = k - 1 k = number of groups in the study

Repeated Measures ANOVA: What is the formula to find degrees of freedom for Error?

dferror = (k-1)(n-1) k = number of groups in study n = number of participants within each group

Two-Way ANOVA: What is the formula to find degrees of freedom for Error?

dferror = AB(N-1)

Between-Group Variance

the variability that results from general differences between the treatment conditions Variance between treatments measures DIFFERENCES among sample means *Between-Group Variance would be like when you are given data from treatments and are looking at the "M" (mean) value Treatment 1: M = 3 Treatment 2: M = 8 Treatment 3: M = 5