Chapter 13, "Comparing Three or More Means,"

*ANOVA* assumes that the samples used are independent random samples . If you calculate the total *degrees of freedom* for 3 groups with 6 observations per group, you get 17. If you calculate the *grand mean* when there are a total of 20 observations and the sum of all individual observations is 140, you get 7.

...

- Using a t-test with three or more samples will increase the likelihood of committing a Type I error. A Type I error occurs if a null hypothesis is rejected that is actually true. Instead, use ANOVA hypothesis testing to find the F ratio. - ANOVA deals with the ratio of the variance between groups to the variance within groups. This ratio is the F ratio. More variation between groups than within groups results in a larger F ratio, which increases the probability that you will reject the null hypothesis. - The null hypothesis for ANOVA assumes that there are no mean differences between the groups. The research hypothesis assumes that at least one mean is different. H subscript 0 space equals space mu subscript 1 space equals space mu subscript 2 space equals space mu subscript 3... space equals space mu subscript k - If Fobt ≥ Fcrit, you reject the null hypothesis; the difference in the means is significant. - If Fobt < Fcrit, you fail to reject the null hypothesis; there are no mean differences.

...

1 Draw the ANOVA Summary Table. 2 Write the null hypothesis. 3 Calculate the Grand Mean. 4 Calculate the degrees of freedom. 5 Calculate the sum of squares between, and record it in the table. 6 Calculate the within groups sum of squares, and record it in the table. 7 Add the within groups sum of squares to the table, and calculate the SS<sub class="subTerm">total. 8 Calculate the between groups mean square. 9 Calculate the within groups mean square. 10 Calculate the F ratio. 11 Look up the critical value for F using both degrees of freedom values.

...

More variation between than within three or more groups will result in a larger F ratio.

...

Using *ANOVA* with three or more groups analyzes the existing between group and within group variation. Using t-tests for hypothesis testing to compare several mean values increases the chance of committing a(n) *Type I error*. If you look for significant results in hypothesis testing with three groups, you actually look for more *variation* between the groups than within the groups.

...

Final Step in Solving for the F-Ratio

ANOVA is a widely used statistical procedure. There are, however, a few items that need mentioned. First, the ANOVA we learned to do in this lesson is a one-way analysis of variance. A one-way ANOVA is used to deal with the relationship between two variables. In the example we walked through, we tested the relationship between the amount of time spent in therapy and the number of intrusive thoughts. If we had more than two variables, we would have to use a different form of ANOVA, perhaps a two-way analysis of variance. Another item that needs mentioning is the fact that ANOVA assumes that the samples we use are independent random samples. A modified ANOVA procedure is required if the groups are matched or related samples. Finally, as we mentioned earlier, Tukey's HSD is only one possible post hoc test that could have been used. There are many post hoc tests we could have chosen to use.

Review

ANOVA is an analysis of variance that exists between groups and within groups. The ANOVA Summary Table helps you organize the sum of squares, the degrees of freedom, and the mean square for between groups and within groups. You then calculate the F ratio by dividing the between-groups mean square by the within-groups mean square. To determine if F is significant, use the degrees of freedom values to find the critical value for F in the F-table. If Fobt ≥ Fcrit, you reject the null hypothesis; the difference in the means is significant. If Fobt < Fcrit, you fail to reject the null hypothesis; there are no mean differences. With three or more groups, follow-up testing, or post hoc testing, is needed to determine where the significant mean differences are. One such post hoc test is Tukey's HSD, which provides a pair-by-pair comparison of the sample means. Using Tukey's HSD, a Q statistic is calculated for each pair of means and compared to a critical value for Q. If Qobt ≥ Qcrit, the difference in the means is significant. If Qobt < Qcrit, the difference in the means is not significant.

Summary

An analysis of variance, or ANOVA, is used to determine if the mean differences among groups are significant when you have three or more samples. If the null hypothesis is rejected, follow-up testing is needed to determine which pairs are significantly different. Tukey's HSD is one of many post hoc tests that you can use to compare sample means.

Data for four groups

Assume that we have four groups of trauma patients: -One group receives therapy for 6 months -One group receives therapy for 4 months -One group for two months -One group doesn't receive therapy at all At the end of each therapy period, the patients are asked to record the number of intrusive thoughts they have in a one day period. The data are listed below.

Data for Four Groups

Assume that we have four groups of trauma patients: One group receives therapy for 6 months One group receives therapy for 4 months One group for two months One group doesn't receive therapy at all At the end of each therapy period, the patients are asked to record the number of intrusive thoughts they have in a one day period. The data are listed below.

A column in the ANOVA Table:

Degrees of Freedom (df)

Causes us to reject the null hypothesis

Fobt ≥ Fcrit

ANOVA Table;Mean Squares

From here on out, calculating F is pretty simple. The next column is the mean square, and we again have a between groups mean square and a within groups mean square. Note that there is no mean square total. Calculating both mean square values is pretty simple. We pull information from the table to plug in to the two formulas. M S subscript B space equals space fraction numerator S S subscript B over denominator d f subscript B end fraction The between groups mean square is found by dividing the between groups sum of squares by the between groups degrees of freedom. Both of those values are already in our summary table, which makes this easy to calculate. M S subscript w space equals space fraction numerator S S subscript w over denominator d f subscript w end fraction The within groups mean square is found by dividing the within groups sum of squares by the within groups degrees of freedom. Both of these values are already in the summary table, so this is easy to calculate.

Calculated by adding together all of the individual observations or scores across all groups and dividing by the total number of observations or scores:

Grand Mean

ANOVA Examples

Here we explore two examples in which we would use a one-way ANOVA for hypothesis testing. In both of these examples, we would want to see more variation between the groups than within the groups. More between group variation than within group variation will result in a larger F ratio, which increases the chances of rejecting the null hypothesis: Comparing test scores from three courses A teacher wants to know if there's a significant difference between her students' test scores depending on the type of course they take from her. One group takes her statistics course in a traditional face-to-face way; one group takes the course face-to-face half the time and online the rest of the time; and the third group takes the course completely online. She administers the same final exam in exactly the same way to all of her students. Comparing three approaches to marketing A business would like to know which marketing approach results in more new customers. They compare mailed brochures, emailed brochures, and telemarketing calls to determine if any approach generated significantly more new business than the other approaches.

Overview

Hypothesis testing for three or more groups is more involved than t-tests for two sample groups. You use an analysis of variance test to determine if the mean differences among sample groups are significant.

Correction Procedures

If our group sizes are not the same across all groups, then we have to use a correction procedure when calculating Q. Q space equals space fraction numerator open vertical bar X with bar on top subscript 1 space minus space X with bar on top subscript 2 close vertical bar over denominator square root of begin display style fraction numerator M S subscript w over denominator n with tilde on top end fraction end style end root end fraction space space space space space space space space space space n with tilde on top space equals space fraction numerator K over denominator begin display style 1 over n subscript 1 end style space plus space 1 over n subscript 2 space plus space fraction numerator 1 over denominator n 3 end fraction space plus... space plus space 1 over n subscript k end fraction The numerator is always the absolute value (meaning, ignore those negative signs!) of the difference between the two group means we are comparing. The denominator only needs to be calculated once, as the same denominator is used for all mean pairs Once we have calculated Q for each pair of means, we compare the calculated Q to the critical value for Q. Just as before, if Q subscript o b t end subscript space greater or equal than space Q subscript c r i t end subscript, we have a significant mean difference; If however, Qobt < Qcrit, then those two means are not significantly different from one another.

ANOVA Table - Source

Look first at the column headed Source: Remember that ANOVA is an analysis of the variance that exists between the groups and within the groups. Source in this table refers to where the variation is coming from: Between Groups or Within Groups. The Total row at the bottom of the table is just that, a sum of the cells above it.

Intro 3

Many steps are involved in completing an ANOVA Summary Table. However, by completing the table, you will have calculated and organized all of the information needed to determine the F ratio. Hypothesis testing does not end once you find the F ratio. You also need to perform post hoc testing to determine which pairs of means are significantly different from one another. Once these steps in the process are completed, you can draw conclusions about the data.

Is F Significant?

Now that we've completed the calculations for each cell of the ANOVA Summary Table, we need to determine if F is significant. We use our degrees of freedom values to find the critical value for F in the table. Step 1 Using the F-Table, we move down the far left table to the value of our Degrees of Freedom for the Denominator (this is our within groups degrees of freedom, dfw). Remember to get as close as possible without going over. Step 2 We then scroll across to the column with our Degrees of Freedom for the Numerator (the between groups degrees of freedom, dfB). The value listed there at the intersection is the critical value for F (Fcrit). Step 3 If F subscript o b t end subscript space greater or equal than space F subscript c r i t end subscript we reject the null hypothesis, indicating that we have mean differences. Step 4 If, however, F subscript o b t end subscript space less than space F subscript c r i t end subscript, then we fail to reject the null hypothesis and no mean differences exist.

ANOVA Table—F

The final cell we need to complete is F. The formula for F was mentioned earlier in this lesson. F is calculated by dividing the between groups mean square by the within groups mean square. Again, both values are already in the table, so the calculation is very simple. F space equals space fraction numerator M S subscript B over denominator M S subscript w end fraction

Steps to Solving for Tukey's HSD

The final steps teach us how to determine which pairs of means are significantly different from one another. Step 1: Perform a Tukey's HSD post hoc test to determine which pairs of means are significantly different from one another. This involves multiple steps that we will work through one by one. Step 2: Determine all possible pairs of means. It is easier if we create a table to keep us organized. The first column is all possible pair combinations of means. The second column is where we will record the calculated Q for each pair of means. The third column is the critical value for Q that we will get from the table The critical value for Q is the same for each pair of means. The final column is where we will answer the question, is Q subscript o b t end subscript space greater or equal than space Q subscript c r i t end subscript Then we record yes or no in the last column, depending on the answer to this question.. Step 3: Calculate the denominator of the formula for Q. Since our groups each contain seven participants, we can use the simpler Q formula. Step 4: Calculate Q for each pair of means and add them to the table. Step 5: Find the critical value for Q using the Q-Table foralpha = .05. The columns across the top represent the number of groups we have (Number of Means Being Compared) and the column on the far left represents our Degrees of Freedom for MSError. This is our dfw from the ANOVA Summary Table. The point at which those two values intersect is our critical value for Q. Move down the far left column until you reach the dfw (24), and move across the columns until you reach the number of groups we have (4). The critical value for Q is 3.90. Step 6: Determine for each pair of means if the calculated Q is greater than or equal to the critical value for Q. If yes, record YES in the cell; if no, record NO in the cell. Those pairs of means with YES in the last column are the means that are significantly different from one another. Step 7: The final step, once you have solved for Tukey's HSD, is to determine the results and state the conclusions. The F ratio is significant, and the significant mean differences are between groups 1 and 3 and groups 1 and 4. Therefore, it appears that therapy is helpful to reducing the number of intrusive thoughts in trauma patients, but only after 4 months. There were no differences between the group receiving no therapy and those receiving therapy for two months, indicating that the therapy period is too short to reduce intrusive thoughts. There were no differences between the group receiving 4 months of therapy and the group receiving 6 months of therapy either, indicating that the extra two months of therapy is not effective at further reducing intrusive thoughts in trauma patients.

Steps to Solving for the F-Ratio

The following steps illustrate how to complete the ANOVA table and determine whether we reject or fail to reject the null hypothesis. Step 1: Draw the ANOVA Summary Table. This will keep us organized, and give order to the many steps involved in solving for the F ratio. Step 2: Write the null hypothesis. Step 3: Calculate the Grand Mean (X with bar on top subscript G r a n d end subscript ). Sum together all of the individual observations, and divide by 28, the total number of observations there are across all groups. Step 4: Calculate the degrees of freedom. While this doesn't go in order of the table, it's an easy first step. Once the calculations are complete, record the information in the ANOVA Summary Table. Step 5: Calculate the sum of squares between, and record it in the table. Step 6: Calculate the within groups sum of squares, and record it in the table. This step is much easier to accomplish if we create a table to keep us organized. Step 7: Now that the table is completed, simply add the open parentheses X space minus space X with bar on top close parentheses squared column sums together to get the within groups sum of squares. Add this value to the table, and calculate the SStotal. Step 8: Calculate the between groups mean square. Using the table, locate the between groups sum of squares, and divide that value by the between groups degrees of freedom. Step 9: Calculate the within groups mean square by dividing the within groups sum of squares by the within groups degrees of freedom. Step 10: Calculate the F ratio by dividing the between groups mean square by the within groups mean square. Step 11: Look up the critical value for F using both degrees of freedom values. Use that value to determine if the F ratio is significant. The critical value for F is 3.01. Do we reject the null hypothesis? Remember to ask the question: Is F subscript o b t end subscript space greater or equal than space F subscript c r i t end subscript Specifically, is 7.21 space greater or equal than space 3.01? Yes! Therefore, we reject the null hypothesis. We have significant mean differences somewhere among our groups. But where? Which group means are significantly different from the others? We need to perform a Tukey's HSD to answer that question.

The Formula to Calculate F

The formula to calculate F looks pretty simple: F space equals space fraction numerator M S subscript B over denominator M S subscript W end fraction But what on earth is MS? The simplest way to begin thinking about calculating F is to start with an examination of the ANOVA Summary table. Each cell requires a value, and there's a unique formula to use for each cell. Let's start by defining the table.

ANOVA Table;Degrees of Freedom

The next column of the summary table is degrees of freedom. We again have two separate formulas for calculating the degrees of freedom for within the groups and for between the groups. dfB = k - 1, where k equals the total number of groups dfw= m- k, where m equals the total number of participants across all groups and k equals the total number of groups. dftotai = dfB + dfw Once we calculate our degrees of freedom, we place those values in the appropriate cells.

ANOVA Table;Sum of Squares

The next column, Sum of Squares, requires that we find the deviations from the mean and square them. We will calculate a sum of squares for between groups (SS8) and a sum of squares for with in groups (SSw). The two sum of squares formulas are different, so lets take a look at each of them now. (Insert Sbb equation) The formula for the between groups sum of squares uses the squared deviations between the mean for the group and the grand mean, and multiplies that squared difference by the number of observations in the group. The grand mean is calculated by adding together all of the individual observations across all groups, and dividing by the total number of observations there are. Once this has been completed for each group, those values are summed to find the between groups sum of squares. Once we find the SSB and the SSw, we record the values in the ANOVA Summary Table in the appropriate cells. We add the two values together to find the SStotal.

F-Ratio

The null hypothesis in the ANOVA situation assumes that there are no mean differences between the groups. We note the hypothesis this way, where k equals the number of groups we have: H subscript 0 colon space mu subscript 1 equals space mu subscript 2 space equals space mu subscript 3 space equals space times times times space equals space mu subscript k In other words, we assume that the mean from each group will be the same, statistically speaking, as the other means. And we have as many means in our null hypothesis as we have groups, hence the mu symbol at the end. The research hypothesis for ANOVA assumes that at least one mean is different from the others. As with the other hypothesis testing techniques that we learn about in this statistics course, we will calculate the F ratio and compare it to a significant value for F. If F subscript o b t end subscript space greater or equal than space F subscript c r i t end subscript, we reject the null hypothesis, indicating that we have mean differences. If, however, F subscript o b t end subscript space less than space F subscript c r i t end subscript, then we fail to reject the null hypothesis and no mean differences exist.

ANOVA Table; Within Groups Sum of Squares

The within groups sum of squares formula uses the sum of the squared deviations between each observation and the group mean. Once that is calculated for each group, those values are summed to find the within groups sum of squares. (Insert SSw equation)

The Q-Table

To find the critical value for Q we use the Q-Table. The columns across the top represent the number of groups we have (Number of Means Being Compared) and the column on the far left represents our Degrees of Freedom for MSError. This is our dfw from the ANOVA Summary Table. The point at which those two values intersect is our critical value for Q.

Gives us a pair-by-pair comparison of the sample means

Tukey's HSD

Tukey's HSD

Tukey's HSD will enable us to find where the significant differences between means are. It gives us a pair-by-pair comparison of the sample means. With the use of Tukey's HSD, a Q statistic is calculated for each pair of means, and the calculated Q is compared to a critical value for Q to determine if that pair of means is significantly different from one another. To calculate Q, we use information from the ANOVA Summary Table, specifically the MSW. We also use the number of observations in each group. The formula we use to calculate Q if the number of observations is the same for each group is: Q space equals space fraction numerator open vertical bar X with bar on top subscript 1 space minus space X with bar on top subscript 2 close vertical bar over denominator square root of begin display style fraction numerator M S subscript w over denominator n end fraction end style end root end fraction

Mean Differences Among Three or More Groups

When we have three or more group means to compare, we cannot use t-tests for hypothesis testing. Even if we divided our means into all possible pairs (mean 1 and mean 2, mean 1 and mean 3, and mean 2 and mean 3 for example), we still couldn't use t-tests. Not only would this process be a lot of work - imagine running multiple t-tests for each group of data - it would also greatly increase the likelihood that we would commit a Type I error. A Type I error occurs if the null hypothesis is rejected when it is actually true. In other words, we make a Type I error if we indicate that significant differences exist between our group means when such differences do not actually exist. If we commit a Type I error, we would say that significant differences exist when they really do not. We rejected the null hypothesis when we really should have failed to reject it. In order to get around both problems, we use an analysis of variance (ANOVA) to test for significant mean differences among three or more groups.

Follow-Up Testing

When we performed hypothesis testing with t-tests, we were finished at this point. We either rejected or failed to reject the null hypothesis and there was nothing left to do. If we found mean differences, it was pretty obvious what two means were significantly different from one another: we only had two means!! With ANOVA, however, interpreting significant results is not that simple. Since we have three or more groups when we use ANOVA, we have to do some more investigating to determine where the significant mean differences are. Further investigation is referred to as post hoc testing or follow-up testing. There are several follow-up tests that we could use to determine where the significant mean differences are. In this lesson, we will use Tukey's Honestly Significant Difference (Tukey's HSD).

Intro 2

When working with three or more samples, use ANOVA, an analysis of variance that exists between groups and within groups. Filling in an ANOVA Summary Table will help you organize statistics calculated between groups and within groups. Then all the information you need to calculate the F ratio will be readily available in the summary table.

Review 2

When you have three or more samples to compare, use an analysis of variance, ANOVA, to test for significant mean differences. Things to remember about ANOVA: -A one-way ANOVA is used when the relationship is between two variables. -With more than two variables, use a different form of ANOVA, such as a two-way ANOVA. -ANOVA assumes that the samples are independent random samples. A modified ANOVA procedure is required if the groups are matched or related samples. -Many post hoc tests are available, including Tukey's HSD.

Intro

You cannot use t-tests for hypothesis testing when you have three or more samples to compare. Instead, you would use an analysis of variance (ANOVA) to test for significant mean differences. ANOVA is a comparison of variance between groups and within groups.

ANOVA

is an inferential method used to test the equality of three or more population means. -If we have three groups, we may have three means that are similar to one another. But, there may be great variability between the group means. Also, as with other samples, these samples may have substantial variation in the observations within each group. Analysis of variance is a comparison of the estimates of variance between groups and within groups. So, how does ANOVA measure the amount of variation between groups and within groups? ANOVA is a calculation of the ratio of the variance between groups to the variance within groups. This ratio is the F ratio, and is named for its developer Sir Ronald Fisher. If we seek significant results in hypothesis testing with three groups, what we are actually looking for is more variation between the groups than is seen within the groups. More variation between than within will result in a larger F ratio, which will increase the probability that we will reject the null hypothesis. Remember that when we reject the null hypothesis, we have evidence for significant mean differences.