Chapter 20

Effect size

The difference between the null hypothesis value and the true value of a model parameter is called the effect size.

Type II error

The error of failing to reject a null hypothesis when it is in fact false (also called a false negative). The probability of a type II error is often denoted beta, and depends on the effect size.

Type I error

The error of rejecting the null hypothesis when it is in fact true (also called a false positive), The probability of a type I error is the alpha level.

Statistically significant

When the P-value falls below the alpha level, we say that the test is "statistically significant" at that alpha level.

Significance level

The alpha level is also called the significance level, most often in a phrase that such as a conclusion that a particular test is "significant at the 5% significance level".

Power

The probability that a hypothesis test will correctly reject a false null hypothesis is the power of a test. To find power, we must specify a particular alternative parameter value as the "true" value. For any specific value in the alternative, the power is (1-Beta).

Alpha level

The threshold P-value that determines when we reject a null hypothesis. If we observe a statistic whose P-value based on the null hypothesis is less than the alpha level, we reject that null hypothesis.