Chapter 13

Distinguish between a Type I and a Type II error. Why is your significance level the probability of making a Type I error?

A Type I error is made when the null hypothesis is rejected but it is actually true. Our decision was that the IV had an effect on the DV when it the truth is that it did not. The significance level is chosen before running the test. Researchers decide what level of probability of making a Type I error they are willing to tolerate during the planning stage, and that becomes the alpha or significance level. A Type II error is made when the null hypothesis is not rejected when the research hypothesis is actually true. We conclude that the IV does not impact the DV when it the truth is that it does.

What is a confidence interval?

An interval of values that defines the most likely range of the of actual population values. For example, with a 95% confidence interval around a sample mean, there is a 95% probability that the population mean lies within that range.

Define statistical significance

Infers whether the results will hold up if the experiment is repeated several times, each time with a new sample of research participants.

Distinguish between the null hypothesis and the research hypothesis. Which one is tested?

Null Hypothesis (H0): No effect of the IV on the DV or no difference between groups. Research Hypothesis (H1): There is an effect of the IV on the DV or a difference between groups. Example (quasi-experimental design): 3rd grade students are provided with an experimental reading program geared towards increasing reading ability. Another class is given the traditional reading program. H0 = There is no difference in reading scores between classes. H1 = There is a difference in reading score between classes; the experimental class will score higher on reading ability compared to the traditional class. The null hypothesis (H0) is tested.

What influences the probability of making a Type II error?

Significance (alpha) level: Setting significance level too low. Sample size: True differences are more likely to be detected with a large sample size. Effect size: A small effect size might not be significant with a small sample.

What is the difference between statistical significance and practical significance?

Statistical significance tells us the likelihood that the results we saw were due to random error. If the probability is low, the test is statistically significant. However, with large samples, very small effects can be significant. That doesn't mean that difference is meaningful (i.e., it may not make any difference in practical terms).

Describe which factors are most important in determining whether obtained results will be significant. Why is sample size important?

The most important factors are sample size, effect size, and alpha level. Sample size is important because larger samples offer more precise estimates of the true population value.

Discuss the reasons that a researcher might obtain non-significant results

The null hypothesis may be true. Small sample size: The effect size may be too small to detect with the size of the available sample, especially if the data is "noisy" (i.e., lots of within-group variability). The significance level may be too stringent (too low).

What does it mean to reject the null hypothesis?

We support the research hypothesis. We are 95% confident there is a differnce between the two groups.