Lesson 30: Probabilities and Power

T/F It is not always the case that one type of error is worse than the other.

True

Which is harder? a. Detecting a difference when data are noisy (highly variable) b. Detecting a difference when data do not vary much.

[I think b]

α is the probability or risk of How to deal with this?

accepting Type I Error (rejecting Ho when it is true) Set a low enough significance level (alpha)

When we choose a level of significance (α), we are choosing a ____ for _______ the null hypothesis.

benchmark / rejecting

Which type of error is more serious, Type I or Type II?

Depends on the situation.

The probability of a type II error is much more complicated to calculate, but it is

inversely related to the probability of making a type I error. Thus, reducing the chance of making a type II error increases the likelihood of a type I error.

2-sided test requires a ______ sample size than a one-sided test.

larger

Higher power requires a _____ sample.

larger

Smaller effect size requires a ____ sample.

larger

smaller alpha requires a _____ sample to achieve desired power.

larger

Once the sample size is set, larger values of α will _______ the probability of a type II error while ______ the probability of a type I error.

decrease increasing

the probability of a type I error is equal to the ______ level ____.

significance α

Regarding Type I and Type II Errors, what is ART?

Alpha Reject Ho when it is True

ART

Alpha Rejects the Ho when it is True

General guidelines for choosing a level of significance (3):

--If the consequences of a type I error are more serious, choose a small level of significance (α). --If the consequences of a type II error are more serious, choose a larger level of significance (α). But remember that the level of significance is the probability of committing a type I error. --In general, we choose the largest level of significance that we can tolerate as the chance of making a type I error.

You can only have both low risk of false positive and high power if:

--make the decision in the planning stage of the study AND --have lots of money (very expensive)

a. "Power" = symbol(s) b. "power" is the chance you _____ Ho when Ho is _____

1 - Beta (1 minus beta) = the chance you reject Ho when Ho is false. (You correctly conclude there is a treatment effect when there really IS a treatment effect.)

What is the symbol for "power?"

1 - β

If β denotes the probability of making a Type II error (failing to reject a false null hypothesis), then power = (symbol):

1- β.

When designing a study, there is a tradeoff between (4):

1. power 2. alpha level 3. sample size 4. minimum detectable difference (specific Ha) --industry standard -- 80% power, alpha = .05 http://wise.cgu.edu/power_applet/power.asp

If the significance level is α = .05, then _____ percent of the time we will reject the null hypothesis even if it is true.

5% Of course we will not know whether the null hypothesis is true. But if it is, the natural variability that we expect in random samples will produce "rare" results 5 percent of the time.

BFF

Beta Fails to reject the False Ho

Regarding Type I and Type II Errors, what is BFF?

Beta Fail to reject Ho when it is a False Ho

The probability (risk) of Type II error can only be set _______ by ______ of (3) . . .

Can only be set INDIRECTLY by CHOICE OF 1. Sample size 2. Alpha 3. Effect size (smallest difference of practical importance)

Type II Error

FAIL TO REJECT Ho -- when Ha is true You BELIEVE A FALSE Ho. (Trial: Accused is pronounced "not guilty" when he's guilty)

Regarding Type I and Type II error: a good test procedure has:

Has a small probability of rejecting the null hypothesis when it is true Has a high probability of rejecting the null hypothesis when it is false

The effect of "effect size" in a [probability hypothesis] is... a, more difficult to detect if there is _____________. b. easier to detect if there is __________________.

It is more difficult to detect a very small difference (crocodile/alligator) It is easier to detect a larger (obvious) difference. (crocodile/chicken -- from same family)

The sampling distribution assuming Ho is false when it is, is . . .

Power (or the power of the test) Usually the darkly shaded curve

Type I Error

REJECT A Ho -- when it is true (Trial: Accused is pronounced guilty when is innocent)

Factors that affect power (3):

Significance level α Sample size n Effect size (the difference between the actual value of the parameter and the hypothesized null value)

The power of the test (or just power)

The probability of correctly rejecting the null hyp. when it is false. This is the probability that the test successfully detects a significant difference and is called the POWER OF THE TEST or just POWER.

The sampling distribution assuming Ho is true

Usually the more lightly shaded curve

Tests with _____power are highly sensitive to deviations from the null hypothesis. a. high b. low

a. high

level of significance measured by

alpha

We want the power of the test (power) [probability] to be _____ and many researchers like to have a statistical power of at least _____.

high 80%.

The best way to reduce the probability of a type II error is to

increase the sample size.

For a fixed alpha, when n increases, power ____

increases

Rreducing the chance of making a type II error________ the likelihood of a type I error.

increases

As "effect size" increases, the power ____.

increases.

The chance that you reject Ho when Ho is false is

power

A low risk of a false negative is _____

powerful. (you are told you don't have strep, but you DO)

A low risk of a false positive is ______

safe. (says you have strep, but you don't)

The smaller α is, the ______ the probability of a type I error there will be.

smaller

The probability that the test successfully detects a significant difference and is called . . .

the POWER OF THE TEST or just POWER.

In general, the probability of a type I error is shown by the symbol ____.

α

If the null hypothesis is true, then the probability that we will reject a true null hypothesis is

α.