Psy 201 Test 3
G is the sum of all scores in the experiment
(G stands for grand total) -You can find it by adding up all N scores -Or you can add up the treatment totals: G = ΣT
T is the sum of scores (ΣX) in a treatment condition
(T stands for treatment total) -T1 is the sum of scores in Group 1, T2 is the sum of scores in Group 2, etc.
Formula for Tukey's HSD
(Will be provided on exam) -𝐻𝑆𝐷=𝑞*sqrt(𝑀𝑆𝑤𝑖𝑡ℎ𝑖𝑛/𝑛) -MSwithin is the within-treatment variance from ANOVA -n is the sample size (number of scores) in each treatment condition -This test requires that sample sizes be the same for all treatments -q is a critical value—the Studentized range statistic
k is the number of treatment conditions
(the number of levels of the factor) For a between-subjects design, it is also the number of separate groups or samples Study time example: We considered four session times (15, 30, 60, and 120 min), so k = 4
n is the number of scores in each treatment condition
(the sample size in each group) -n1 is the sample size in Group 1, n2 is the sample size in Group 2, etc.
N is the total number of scores in the experiment
(the sum of all the sample sizes across groups) -N = kn if all groups have equal sample sizes -Example: If there are three groups (k = 3) that each have five scores (n = 5), then N = kn = (3)(5) = 15
Variance in a Two-Factor ANOV
**pic on Word
Determining Significance continued:
--If your calculated F ratio is less than the critical F value, then it is nonsignificant and has a p value greater than alpha --In other words, it is relatively likely that you would have obtained that F ratio, by chance alone, if H0 were true --Consequently, you do not reject H0 and you cannot draw any conclusion—you need to maintain the possibility that all means might be equal **Remember that we either reject or do not reject H0 (we never "accept" it)
Tukey's HSD Test
-A commonly used post hoc test -You calculate a single value—the "honestly significant difference" or HSD—that is the minimum difference between treatment means that is significant -If the absolute difference between two means is greater than the HSD, then you can conclude that the difference is significant -If the absolute difference between two means is less than the HSD, then you cannot conclude the difference is significant (but you also cannot conclude that the means are equal)
Hypotheses for Correlations
-A correlation can be calculated as part of a hypothesis test to determine whether a relationship between variables is statistically significant -When we did hypothesis testing with the t test or ANOVA, our hypotheses were about the values of population means (μ) -With correlations, the hypotheses are about the value of the correlation coefficient in the population (ρ)
More about repeated measure of ANOVA:
-A repeated-measures ANOVA is similar to a between-subjects ANOVA, but the calculations differ a bit --Variability due to individual differences is removed or eliminated from the calculated variance because the same subjects experience all conditions ---Similar to what happens in a paired-samples t test --By removing individual differences, a repeated-measures ANOVA has more statistical power than a between-subjects ANOVA ---It is easier to detect an effect if one truly exists (We will not be covering this type of ANOVA in detail)
The ability to examine interactions is an advantage of two-factor ANOVA compared with single-factor ANOVA....
-Besides looking at the effect of each factor by itself, you can see whether the factors influence each other -This is similar in concept to "drug interactions" that doctors and pharmacists get concerned about -They want to know whether the effect of one drug is altered by taking a second drug; if it is, then the two drugs interact, which might be bad for the patient
Decisions in a Two-Factor ANOVA
-Each F ratio is compared with its own critical F value to determine whether to reject the null hypothesis (H0) --This is done separately for each main effect and the interaction, resulting in three decisions: ---Reject or do not reject H0 for the main effect of A ---Reject or do not reject H0 for the main effect of B ---Reject or do not reject H0 for the interaction of A and B
Decisions in a Two-Factor ANOVA continued:
-Each decision reflects an independent test of either a main effect or an interaction effect -The tests are independent because the outcome of any one test is unrelated to the outcomes of others -In other words, each effect (main effect of A, main effect of B, and the interaction of A and B) can be either significant or nonsignificant -Knowing the significance of one effect does not tell you anything about the significance of the others
Example: Investigating the effects of study method and course material on exam scores..
-Factor A: Study method --Levels: A1 = Re-reading; A2 = Self-testing -Factor B: Course material --Levels: B1 = Statistics; B2 = Biology -This is a 2 x 2 ANOVA design: 2 levels of Factor A x 2 levels of Factor B -There are 2 x 2 = 4 distinct combinations of the factor levels
Each factor in a two-factor ANOVA can be manipulated either between- or within-subjects
-If both factors are between-subjects, then it is a between-subjects design -If both factors are within-subjects, then it is a within-subjects or repeated-measures design -If one factor is between-subjects and the other is within-subjects, then it is a mixed design -The same logic applies in each case, but the calculations are slightly different -Its value—the correlation coefficient—is represented by r for sample data and ρ (the Greek letter rho) for population data
Testwise alpha level:
Probability of a Type I error for an individual hypothesis test. -We usually set alpha = .05, meaning there is only a 5% chance the test will result in a Type I error
We can use the basic formula for sample variance to determine each of those variances:
Sample variance formula on notebook
Limitation of t-tests:
T-tests compare 2 means between groups. If you were to do a series of tests comparing more than 2 means for a wider variety of conditions, it would inflate the experiment-wise alpha level. -For example: --Test-wise alpha level: If you do six t tests to compare four means, then you could set it equal to .05 --Experiment-wise alpha level: Reflects the probability that any of the six t tests will result in a Type I error ---Prob. of not making a Type I error (1 - α) = .95 ---Joint prob. of not making a Type I error in a set of six t tests = .956 ---Experiment-wise alpha = 1 - .956 = .265 ** this means that there is a 5% chance of a Type I error for each individual t test but a 26.5% chance of a Type I error for the experiment as a whole (all six t tests together). We want a 5% chance, not 26.5%
F Ratio
The F ratio compares the variances in exactly this way: 𝐹= variance between treatments/variance within treatments Or 𝐹= systematic treatment effects + random, unsystematic effects/random, unsystematic effects -If there are no treatment effects (H0 is true), then F = 1 -If there are treatment effects (H0 is false), then F > 1
Factor
The independent variable that designates the groups being compared (e.g., "session time")
Level
The individual conditions that make up a factor (e.g., 15, 30, 60, and 120 min are four levels of the factor "session time")
Strength
The numerical value of r indicates the strength or the consistency of the linear relationship -The value of r can only be between -1 and +1 -A value near ± 1 indicates a strong relationship -A value near 0 indicates a weak relationship (a value of 0 indicates no relationship) -A straight line that best fits the data on a scatter plot can be used to highlight the relationship (later we will cover how to determine the best-fitting line)
The F Distribution:
To determine whether a calculated F ratio is extreme enough to reject H0, we compare it with a critical F value from an F distribution for the specific numerator and denominator df. The asymmetry of the distribution is positively skewed because F cannot be negative **pic on Word
Experiment-wise alpha level:
Total probability of a Type I error that is accumulated from all the individual hypothesis tests in an experiment. -It is usually higher than the test-wise alpha level
How do we solve such a problem?
You do a single test that compares all means simultaneously for a specified alpha level (e.g., .05) -Analysis of variance allows you to do such a test
Two-Factor ANOVA
a two-factor (or two-way) ANOVA in which two independent variables are manipulated
Post hoc tests
additional hypothesis tests done after an ANOVA to determine which means are significantly different from each other
For The F Distribution Table,
alpha = .05 uses numbers in regular font alpha = .01 uses numbers in bold font
The simplest ANOVA design...
single-factor (or one-way), between-subjects ANOVA
If the null hypothesis is false, then....
there are differences between the means and the distributions shift apart
2. For the main effect of factor B:
𝐹= variance between the means for factor B/variance expected if there is no effect of B -A main effect of factor B (i.e., a systematic treatment effect) will increase the numerator
The F ratio is then the ratio of those variances (MS)
𝐹= variance between treatments/variance within treatments= 𝑀𝑆 𝑏𝑒𝑡𝑤𝑒𝑒𝑛/𝑀𝑆 𝑤𝑖𝑡ℎ𝑖𝑛
1. For the main effect of factor A:
𝐹=_variance between the means for factor A/variance expected if there is no effect of A -A main effect of factor A (i.e., a systematic treatment effect) will increase the numerator
Hypotheses for Correlations
-If we do a two-tailed hypothesis test, then we want to know whether our sample correlation (r) is different from the hypothesized population correlation (ρ) of 0 -If r is far from 0 (close to either -1 or +1), then we reject H0, and we conclude there is a significant relationship between the variables -If r is close to 0, then we do not reject H0, and we do not have sufficient evidence to make a conclusion -We determine whether r is far enough away from 0 by comparing it with a criterion: a critical r value (rcrit)
Determining Significance:
-If your calculated F ratio is greater than or equal to the critical F value, then it is statistically significant with a p value less than alpha --In other words, it is very unlikely that you would have obtained such an extreme F ratio, by chance alone, if H0 were true --Consequently, you reject H0 and conclude that at least one mean differs from the others **However, you need to do post hoc tests to determine which means are significantly different
4. Make a Decision
-Is the calculated F ratio more extreme than the critical F value determined earlier? -What does the relevant F distribution look like? -What is our decision about the null hypothesis (H0)? -How would we report the outcome of the ANOVA? **notes
Get Sample Data and Calculate Stats
-Let's rearrange the data into a table with 15 rows (one for each subject's score) -Table in notes
A two-factor ANOVA determines whether the main effects and interaction are significant by testing hypotheses about them
-Main effect of study method (Factor A): --H0: μA1 = μA2 (no difference between means for re-reading and self-testing) --H1: μA1 ≠ μA2 (some difference between means for re-reading and self-testing) -Main effect of course material (Factor B): --H0: μB1 = μB2 (no difference between means for statistics and biology) --H1: μB1 ≠ μB2 (some difference between means for statistics and biology) -Interaction between study method and course material --H0: There is no interaction between factors (the effect of study method does not depend on the course material) --H1: There is an interaction between factors (the effect of study method depends on the course material)
How do you identify main effects and interactions in graphs?
-Main effects: Differences between means for each factor (averaging over levels of the other factor) -Interaction: Existence of nonparallel lines that cross or converge
2. Interaction:
-Mean differences among the levels of one factor that differ based on the levels of another factor -Interaction between study method and course material: Does the difference between means for re-reading and self-testing differ for statistics versus biology? -An interaction is a difference of differences that cannot be explained by the overall main effects
Null and alternative hypotheses:
-Null hypothesis: There are no differences between the population means (all means are equal) H0: μ1 = μ2 = μ3 (no means are different) -Alternative hypothesis: There is at least one difference between the population means --H1: at least one μ is different from the others **Notice that H1 is vague—it does not specify which population means might be different -With three means, there are multiple ways in which you could have at least one difference between means -ANOVA determines whether any of those possibilities exists, but it does not tell you which one --Later, you will learn about post hoc tests that allow you to do that
How can this be?
-Remember that a t statistic compares distances—the distance (or difference) between two sample means -The F ratio compares variances, but remember that a variance is a mean squared distance (or deviation) -Therefore, when comparing two means from separate groups: F = t^2 -This relationship does not hold for more than two means because you cannot calculate a t statistic
If we are interested in differences between means, why is it called analysis of variance?
-Remember, if an IV has an effect, we assume it changes the mean—not the variance—of the distribution of sample means (i.e., the variability within a condition does not change) -However, when two (or more) distributions are shifted apart, the variance between their means changes (i.e., the variability between conditions changes)
Example of Repeated-Measures ANOVA:
-Solving math problems that vary in difficulty --Factor: Problem type --Levels: Easy, medium, and hard --Each subject does a mix of problem types, making this a one-way, repeated-measures ANOVA design
You can also have a single-factor, within-subjects ANOVA design....
-The same subjects participate in all conditions, experiencing every level of the factor -Because each subject is measured repeatedly, this is sometimes called a repeated-measures design
Direction
-The sign of r (+ or -) indicates whether the relationship is positive or negative -A positive correlation (+r) is when two variables change in the same direction: as one variable goes up, so does the other, and vice versa -A negative correlation (-r) is when two variables change in opposite directions: as one variable goes up, the other goes down, and vice versa
Effect size for ANOVA
-The simplest way to measure effect size is to calculate the proportion of variance accounted for by the treatment condition: Prop. of variance accounted for= 𝑆𝑆𝑏𝑒𝑡𝑤𝑒𝑒𝑛/𝑆𝑆𝑡𝑜𝑡𝑎𝑙 This effect size measure is called η2 ("eta squared") (Is a percentage)
More about the F ratio:
-The value of the F ratio will allow us to make a decision about H0: --If F is close to 1, then we cannot reject H0 --If F is much greater than 1, then we can reject H0 -How much greater? That will be determined by comparing the F ratio to a critical F value, as explained later
there are two sources of variability:
-The variability within each condition—random, unsystematic effects due to sampling error -The variability between conditions—systematic treatment effects that shift the distributions apart -When the distributions are combined into one, the overall distribution will have higher variance than the individual group distributions
Recall that in an ANOVA design:
-There can be one or more factors -Each factor has two or more levels -Each factor can be either between- or within-subjects
A researcher is studying simulated driving performance under three conditions: no phone, hands-free phone, and hand-held phone
-There is a single factor (type of phone) with three levels (none, hands-free, and hand-held) -Separate groups of 5 people are assigned to each condition, making it a between-subjects design; Thus, we will be doing a single-factor, between-subjects ANOVA
we need to calculate nine values when doing an ANOVA:
-Three values for SS (SSbetween, SSwithin, and SStotal) -Three values for df (dfbetween, dfwithin, and dftotal) -Two values for variance (between and within treatments) -One value for the F ratio
2. Set the Criterion for a Decision
-To determine the criterion, we need to know the df -There are 3 conditions (groups), so k = 3 -There are 5 subjects per condition, so n = 5 -Total sample size N = kn = (3)(5) = 15 -df total= 14 (N-1) -df within= 12 (N-k) -df between= 2 (k-1) -Fcrit=3.88 (alpha 0.05)
Example of a single-factor (or one-way), between-subjects ANOVA design
-To introduce key concepts, let's assume a single factor with three levels, resulting in three different groups of subjects
The formulas are easier to understand if you remember what the labels (subscripts) refer to:
-Total: The entire set of scores, so SS is calculated for all N scores and df = N - 1 -Within: Within each treatment condition, so SS and df are calculated inside each treatment, then summed across treatments -Between: Between treatment conditions, so SS is calculated using treatment means and df = k - 1 -Finally, for both SS and df: Total = Between + Within
1. State the Hypothesis
-We can classify the no-phone, hands-free phone, and hand-held phone conditions as groups 1, 2, and 3 -Null hypothesis: There are no differences between the population means (all means are equal) H0: μ1 = μ2 = μ3 -Alternative hypothesis: There is at least one difference between the population means H1: at least one μ is different from the others
Factorial design
-When a study involves more than one factor -All possible combinations of factor levels are represented in a factorial design --Each factor can be labeled with a letter (A, B, C...) -A two-factor ANOVA has an A x B design, where A and B indicate the number of levels of the factors --Each level of a factor can be labeled with a number (1, 2, 3...)
Hypotheses for ANOVA:
-With three levels of a single factor, there are three hypothesized population means, one for each group: --μ1 = population mean for Group 1 --μ2 = population mean for Group 2 --μ3 = population mean for Group 3 -ANOVA allows us to determine whether there are any differences between the population means
ANOVA compares which two types of variances?
-Within & between treatment variance
Each F ratio is associated with two types of df:
-dfbetween (called the "numerator" df because it is used to calculate MS between in the numerator of the F ratio) -dfwithin (called the "denominator" df because it is used to calculate MSwithin in the denominator of the F ratio) -Usually written as F(dfbetween, dfwithin)
Each SS has a certain df associated with it:
-dftotal is associated with SStotal -dfwithin is associated with SSwithin -dfbetween is associated with SSbetween -Total df (dftotal) --The total number of scores (N) minus 1: d𝑓𝑡𝑜𝑡𝑎𝑙=𝑁−1
3. For the interaction of factors A and B:
-𝐹= variance between means not explained by main effects/variance expected if there is no interaction effect -An interaction between factors A and B ("a difference of differences") will increase the numerator
You need three pieces of information to find a critical F value in the table:
1. Alpha level 2. Numerator df (a.k.a. dfbetween) 3. Denominator df (a.k.a. dfwithin)
You need three pieces of information to find a critical r value in the table:
1. Alpha level 2. One- or two-tailed test 3. Degrees of freedom (df) 4. For a correlation, df = n - 2, where n equals the number of pairs of scores (number of X-Y pairs)
With this type of ANOVA design, we can examine three separate sets of mean differences:
1. Is there a difference between means based on study method? (comparing MA1 and MA2) **use pic on Word to help visualize this better 2. Is there a difference between means based on course material? (comparing MB1 and MB2) 3. 3. Does the difference for one factor depend on the other factor? (e.g., comparing re-reading and self-testing for each type of course material)
Correlations have many different applications:
1. Prediction 2. Validity 3. Reliability 4. Theory verification
The different variances underlying the F ratios are determined in two stages:
1. Separate total variance into between-treatments and within-treatments variance -This is what we did in the between-subjects ANOVA when we separated total SS and df into between and within SS and df 2. Separate the between-treatments variance into components associated with factor A, factor B, and the interaction -We did not have to do this in the single-factor ANOVA because all treatment effects in the between-treatments variance reflected just one factor
To find the appropriate q value in the table, you need:
1.) Alpha level (.05 or .01) 2.) Number of treatments (k) 3.) Degrees of freedom within treatments (dfwithin); shown as df for error term on table **alpha=0.01 is in boldface
Two important characteristics of the F ratio:
1.) Because F ratios are computed from variances (which are always positive numbers), F values are always positive 2.) Remember what the underlying variances represent: 𝐹= systematic treatment effects + random, unsystematic effects/random, unsystematic effects --Thus, if there are no treatment effects (H0 is true), then F = 1. If there are treatment effects (H0 is false), then F > 1
In an ANOVA experiment, we will follow the four steps of our hypothesis testing procedure:
1.) State the hypotheses 2.) Set the criterion for a decision 3.) Get sample data and calculate statistics 4.) Make a decision
A between-subjects ANOVA depends on the same assumptions as the independent-samples t test:
1.) The observations within each sample must be independent 2.) The populations from which the samples are selected must have normal distributions 3.) The populations from which the samples are selected must have equal variances -Also called homogeneity of variance -Reflects the idea that treatments change the means (not the variability) of distributions
Three other points to consider about the relationship:
1.) When comparing only two means, the hypotheses for ANOVA and t test are identical: --H0: μ1 = μ2 (all means are equal) --H1: μ1 ≠ μ2 (there is at least one difference between means—in this case, there can be only one) 2.) The degrees of freedom (df) for the t statistic and the denominator df for the F ratio (dfwithin) are identical when there are only two means: -Degrees of freedom for an independent-samples t statistic: df = n1 + n2 - 2 -Denominator degrees of freedom for the F ratio: dfwithin = N - k --Remember that N = n1 + n2 and k = 2, so: --dfwithin = N - k = n1 + n2 - 2
You only do a post hoc test when:
1.) You reject H0 (indicating there is at least one difference between means) 2.) There are three or more treatments (k ≥ 3) -If there are only two treatments, then it is obvious that those two treatment means are different **statisticians have come up with post hoc tests that control for Type I errors
Scatter plot
A graph in which the values of two variables are plotted against each other -Scatter plots are useful for visualizing the relationship between two variables
Analysis of variance (ANOVA) (definition)
A statistical procedure used to evaluate differences between means for two or more treatments or conditions
which approach should you use—ANOVA or t test—if you want to compare only two means?
Answer: It doesn't matter—they produce the same outcome
Treatment effect
Between-treatments variance should be greater than within-treatment variance (their ratio should be greater than 1)
No treatment effect
Between-treatments variance should equal within-treatment variance (their ratio should equal 1)
Calculating the Correlation
Conceptually, the formula for calculating r for two variables, X and Y, is: 𝑟=_covariability of X and Y/variability of X and Y separately_
For a two-tailed hypothesis test:
H0: ρ = 0 (there is no correlation in the population; i.e., there is no relationship between the variables) H1: ρ ≠ 0 (there is a non-zero correlation in the population; i.e., there is a relationship between the variables)
For a one-tailed hypothesis test for a positive relationship
H0: ρ ≤ 0 (there is not a positive correlation) H1: ρ > 0 (there is a positive correlation)
Between-treatments variance:
How much the scores are spread out between conditions Two components: Random, unsystematic effects due to sampling error Systematic treatment effects due to manipulation of the IV -If the null hypothesis is true, then there are no differences between the means, so the distributions for the three conditions should completely overlap The only type of variability is within each condition—random, unsystematic effects due to sampling error
Within-treatment variance:
How much the scores are spread out within a condition One component: Random, unsystematic effects due to sampling error
1. Prediction
If two variables are related, then one variable can be used to make predictions about the other variable -Example: There is a moderately positive correlation between SAT scores and first-year college GPA -Students with higher SAT scores tend to have higher GPAs in college -College admissions officials often ask for SAT scores because the scores predict (to some extent) whether students will be successful in college
Interpretation of Tukey's HSD Test
If you get a value of 2.38, it means that 2 means should differ by 2.38 in order to be significant
1.) Main effect:
Mean differences among the levels of one factor -Main effect of study method: Is there any difference between means for re-reading and self-testing? -Main effect of course material: Is there any difference between means for statistics and biology?
ANOVA is (more/less) flexible than the t test?
More
1. State the Hypotheses
Null hypothesis: There is no relationship (zero correlation) between performance and self-reward H0: ρ = 0 Alternative hypothesis: There is a relationship (non-zero correlation) between performance and self-reward H1: ρ ≠ 0
