T-Test and ANOVA

Effect Size - Cohen's d

Standardised difference between two means. Possible values for Cohen's d range from 0 to infinity. Interpretation: Small = .20 Medium = .50 Large = .80

Family-wise Error Rate

By conducting multiple comparisons, the chance of obtaining a Type I error in any one test increases. With six comparisons, the chances of committing a Type I error is 26%. When conducting confirmatory research: Only collect data on variables that you will analyse. Only analyse variables where you hypotheses. Avoid running analyses between variables "because you can". Running analyses between all available variables and only selecting results that are significant is known as p-hacking, and is BAD.

Welch's t-test

A variation on the independent-samples t-test: More robust when variances between groups are unequal. Degrees of freedom is estimated in a slightly different way. There is no disadvantage to using Welch's t-test over a standard independent-samples t-test. Comparison: Standard independent-samples t-test: t(67) = 1.97, p = 0.053 t(38.76) = 1.92, p = 0.062

Post-Hoc Comparisons

ANOVA is an omnibus test: It tells you whether there is a difference between groups, but not which groups are different. In order to know where the difference is between groups, we must conduct post-hoc comparisons. -> should only be run if the ANOVA is significant. Pairwise Comparisons: Run a t-test between all group combinations.

Matching Participants

Between-subjects designs can be influenced by individual differences between the groups. Therefore, you may want to pair participants to reduce this variation. Can match participants on variables, such as age, sex. Best match for any participant is themselves. i.e., a within-subjects research design.

Statistical tests

Independent-Samples t-test •Categorical IV with two levels. •Between-subjects. Paired-Samples t-test •Categorical IV with two levels. •Within-subjects. One-Way ANOVA (Analysis of Variance) •Categorical IV with more than two levels. •Between-subjects.

One-way ANOVA

One-Way ANOVA (Analysis Of Variance): When the DV is continuous. The IV is categorical and has more than two levels. Between-subjects Design When an ANOVA is applied to an IV with only two levels, results are identical to an independent-samples t-test. ANOVA is an omnibus test. -> It tells you whether there is a difference between groups, but not which groups are different. Test Statistic: F-statistic. Comparison between variance explained by the model and variance unexplained by the model. Comparison between variance between group means compared to variance within groups. A higher ratio indicates groups may have different mean values. Degrees of Freedom: Number of groups - 1

Bonferroni Correction

The Bonferroni correction is a method for controlling for family-wise error rate. Divide the alpha (significance level) by the number of comparisons. For example: For six comparisons: .05/6 = .008. Bonferroni Correction = .05/Number of comparisons

Independent samples t-test

Used when you have a categorical IV (2 levels) and a continuous DV. Determines whether there is a statistically significant difference between the means of two unrelated groups (i.e., between-subjects design). Also called: two-sample t-test, student's t-test. Example: Research question: Are cat people more likely to be introverts compared to dog people? Using the class data, we group people as cat-people or dog-people based on self-report. Removing participants who report being "both" or "neither" • Measure introversion on a continuous scale. Null Hypothesis: The means between the two levels of the IV are equal. Example: The means on introversion between cat-people and dog-people are equal. Alternative Hypothesis: The means between the two levels of the IV are not equal. Example: The means on introversion between cat-people and dog-people are not equal. Test statistic: t statistic Degrees of Freedom: n-2

Paired samples t-test

When to use a paired-samples t-test: The IV is within-subjects categorical variable (2 levels). Examples: Before/after a treatment. Comparison between two measures collected from same participant. When not to use a paired-samples t-test: When groups are separate participants, even when participants have been age and sex matched. -> Conduct an independent-samples t-test in this case. Degrees of Freedom: n-1