06.02 Errors, Power, and Significance

Some ways to increase the power of the test and reduce the probability of committing a Type II error are to...

increase the sample size, decrease the variability of the data, or increase the significance level If you increase the significance level, α, you make it easier to reject the null and are more likely to reject it regardless of whether it is true—meaning, you are now more likely to make a Type I error.

Practice 2 A Type II error occurs in which of the following situations?

H0 is not rejected when it should be rejected

NOTICE

The only way to reduce both types of errors is to collect more data and increase the sample size.

Probability of making errors

The probability of committing a Type I error is equal to α, the significance level. The probability of committing a Type II error is equal to β.

Type I errors

rejecting the null when the null is true --When a statistic calls for the rejection of the null hypothesis, which is actually true --When H0 is rejected when it is true --False positive --Error probability denoted by the Greek letter α (alpha)

Example 5 An experimenter conducts a significance test of H0: p = 0.40 and Ha: p < 0.40. He isn't aware of it, but the actual p-value is equal to 0.35. With which sample size and significance level will the test have the greatest power?

α = 0.05, n = 500

Example 1 Ella is accused of shoplifting and is arrested. The judge says she is innocent until proven guilty. What would be a Type I error and Type II error in this situation?

H0: Ella is innocent (because the Judge said she is innocent until proven guilty). Ha: Ella is guilty (because this requires proof to reject the null). With a Type I error, we reject a true null hypothesis. In the context of the problem, this means Ella is innocent, but she is found guilty and she goes to jail for a crime she did not commit. With a Type II error, we fail to reject a false null hypothesis. In the context of the problem, this means Ella is guilty but is found innocent even though she committed the crime. The example with Ella-the-shoplifter is an easy way to remember the difference between Type I and Type II errors. A Type I error sends an innocent person to jail. A Type II error lets a guilty person go free. In this situation, neither one is correct; they both result in a miscarriage of justice.

Practice 3 I. A researcher conducts a significance test using data from a well-designed study. Which of the following is/are TRUE? II. If you increase the significance level, you increase the power of the test. II. A larger p-value is strong evidence against the null hypothesis. Set a higher standard of proof by choosing α= 0.05 instead of 0.01.

I only

Example 2 Prenatal genetic testing can be a very long and expensive process. A new home test called the RapidGene In-home Genetic Test provides fast test results in less than 20 minutes. A patient who tests positive for possible genetic defects that could affect a pregnancy may want more follow-up testing and medical diagnosis.

1. What would a Type I error mean? For this example, the null hypothesis is the RapidGene test results indicate the patient does not have any genetic defects. The alternative hypothesis is the RapidGene test results indicate the patient has genetic defects. A Type I error means the RapidGene test results indicate the patient has genetic defects present when she does not. 2. What would a Type II error mean? A Type II error means the RapidGene test results indicate the patient has no genetic defects when she actually does have them. 3. Which error would be considered worse—a Type I error or Type II error? Explain. In this situation, a Type II error is more serious because the patient who has genetic defects is told she does not have them, which means she does not receive the correct medical diagnosis and information.

Example 4 A large department store finds only 40% of all its customers make a purchase. Management is concerned employee customer service is responsible for this low purchase rate. In an effort to increase the percentage of sales, management decides to offer incentives to employees whose customers demonstrate higher purchasing rates. If the incentive is successful, the company will continue with the incentive program.

1. Write the company's null and alternative hypotheses. Let p = the percentage of sales. H0: p = 0.40 following the incentive program (or, the percentage of sales is equal to 40%) Ha: p > 0.40 following the incentive program (or, the percentage of sales is greater than 40%) 2. Describe a Type I error in the context of the problem and explain its consequence. A Type I error is to determine the percentage of sales has increased when it has not. The consequence of this type of error is that the company will spend money on an incentive program that does not increase the percentage of sales. 3. Describe a Type I error in the context of the problem and explain its consequence. A Type II error is to determine the percentage of sales has not increased when it has. The consequence of this type of error is that the company misses the opportunity to affect a successful incentive program to increase its percentage of sales.

Example 3 The Boy Who Cried Wolf is a classic children's fable. It warns readers about the danger of raising a false alarm. In the story, the boy claims repeatedly that a wolf is approaching the village, despite no such real danger. He does this so frequently that when a wolf actually approaches, no one believes him and the village is attacked.

1. Write the null and alternative hypotheses for the story. H0: No wolf is present when the boy cries wolf. Ha: A wolf is present when the boy cries wolf. 2. Describe a Type I error in the context of the problem and explain its consequence. A Type I error rejects the null when the null is true. In this case, if the townspeople believe the boy and assume a wolf is present when there is no wolf, a Type I error is made. The consequence is they react when there is no real threat. 3. Describe a Type II error in the contest of the problem and explain its consequence. A Type II error fails to reject the null when it is false. In this story, if the townspeople believe there is no wolf when there is, in fact, a wolf, a Type II error is made. The consequence is they do not respond to the very real threat.

Practice 1 The National Highway Traffic Safety Administration is testing new road signs with the hope of decreasing the number of drivers who text while driving. In this context, what is a Type I error?

A Type I error is to claim the new road signs decrease the number of drivers who text while driving, when it doesn't decrease the number.

Example of practical significance

A pharmaceutical company finds a statistically significant difference (p = 0.04) between the percentage of people who get the flu and take a new vitamin daily compared with those who get the flu and do not take the vitamin daily. Although the result is statistically significant, it is not practically significant because each bottle of 60 vitamins costs $1,000 whereas a flu vaccination costs $50 and flu medication costs $25.

How to not make Type I errors

You are able to decrease the probability of committing a Type I error by changing the significance level, α. Because a Type I error is made when you decide incorrectly to reject the null hypothesis, you can require the evidence to be more convincing before deciding to reject it. To make the evidence more convincing, you reduce the significance level. If the null hypothesis is rejected when p < 0.05, you can, instead, lower the significance level to 0.01, which reduces the chances of the null hypothesis being rejected. In theory, you can even decrease the significance level to such a small value (such as α = 0.001) that it would almost completely avoid a Type I error. However, in doing so, you almost never have enough evidence to reject the null hypothesis, even when it is false, which increases the probability of committing a Type II error. Notice in the graph that when the shaded area (Type I error area) is made smaller, the orange area (Type II area) becomes much larger. Adjusting the significance level is not without consequences.

Type II errors

failing to reject the null when the alternative is true --When a statistic does not give enough evidence to reject the null hypothesis, which is actually false --When H0 is not rejected when Ha is true --False negative --Error probability denoted by the Greek letter β (beta)

Power of the test

the probability a test will reject the null hypothesis at a chosen significance level α when the specified alternative value of the parameter is true ( is = to 1 - β)

Practical significance

the real-life implications of the statistical findings