Lesson 26: Overview of Hypothesis Testing

What do the levels of alpha mean?

Alpha levels are used in hypothesis tests. The significance level α is the probability of making the wrong decision when the null hypothesis is true. .01 .05 (the one generally used) 1.0 VERY infrequent

If alpha is unknown, what alpha-level is used?

.05 alpha

p-values range between

0 and 1

Elements of Tests of Significance (4)

1. Draw a conclusion about parameter (ex.: mu or p) based on the data 2. Claim 1 (the Null Hypothesis): H-not always represents the parameter. Null Hyp. always has =. Ex: p = .37 3. Claim 2 (alternate hypothesis H(sub A). Is always >, < or NOT =. 4. Outcome is represented by a standardized statistic called a TEST STATISTIC (which summarizes all evidence w/a test statistic) (Later 5. Related measure of uncertainty is alpha which represents the probability of falsely rejecting the null hyp.; alpha defines what is a rare or unlikely outcome.)

If an alpha value is LOW, we can conclude (2):

1. The null hyp. is NOT true 2. Claim 2 (alt. hyp.) IS true

Meaning of p-value

1. The probability of getting a test statistic as extreme or more extreme than observed if the null hyp. were true. ( the probability of observing data like those observed assuming that Ho is true. ) (By "extreme" we mean extreme in the direction of the alternative hypothesis.) --probability on statistic --computed assuming the null hyp. is true 2. A value between 0 and 1: 0< or = p < or = 1 3. A measure of the strength of agreement between the observed test statistic and the null hyp. --measures evidence against the null hyp. --the p-value tells us how likely the test statistic is to be true.

Choosing alpha-level . . . (3)

1. always subjective (but imp.) choice 2. alpha = risk of false positive --risk should generally be small (.10) --if consequences of false positive serious, alpha must be VERY small (.01 or so) 3. another issue: how skeptical is audience? --if highly skeptical, alpha must be very small to win them over

alpha (definition)

A. pre-specified cutoff for p-value 1. artificial (but imp. sharp boundary between rejection and non-rejection regions for p-value) 2. If p-value < or = alpha --no difference is statistically significant --reject the null hyp. and conclude it's false 3. alpha = risk of false positive

when do you choose an alpha-level?

BEFORE the study is done

Comment on p-value (ebook)

Comment By "extreme" we mean extreme in the direction of the alternative hypothesis. Specifically, for the z-test for the population proportion: If the alternative hypothesis is Ha:p<p0 (less than), then "extreme" means small, and the p-value is: The probability of observing a test statistic as small as that observed or smaller if the null hypothesis is true. If the alternative hypothesis is Ha:p>p0 (greater than), then "extreme" means large, and the p-value is: The probability of observing a test statistic as large as that observed or larger if the null hypothesis is true. if the alternative is Ha:p≠p0 (different from), then "extreme" means extreme in either direction either small or large (i.e., large in magnitude), and the p-value therefore is: The probability of observing a test statistic as large in magnitude as that observed or larger if the null hypothesis is true.

Don't reject the null hyp. means

DIFFERENCES between claimed parameter value and calculated statistic could be due to CHANCE.

Rejecting the null hyp. means

DIFFERENCES between claimed parameter value and calculated statistic is REAL

Interpret this data: H-not: p = .20 H-a: p-hat < 20 n = 400 Claim 2: p-hat = .16 z score = -2.0

The p-value in this case is: * The probability of observing a test statistic as small as -2 or smaller, assuming that Ho is true. OR (recalling what the test statistic actually means in this case), * The probability of observing a sample proportion that is 2 standard deviations or more below p0=.20, assuming that p0 is the true population proportion. OR, more specifically, * The probability of observing a sample proportion of .16 or lower in a random sample of size 400, when the true population proportion is p0=.20.

To determine when we have enough evidence to say the differences between Claim 1 (null hyp.) and Claim 2 (alt.) are enough, we use . . .

alpha

If p-value > or = to alpha . . . .

fail to reject the null hyp. (NOT ACCEPT) Why? Study might be weak. Can's say: I can't prove it false, so it must be true.

If a standardized test statistic that would rarely happened (determined by alpha) is true, it is good evidence that the null hypothesis is _______________. Hence we can reject ___________ and believe ___________ is true.

null hyp. is NOT true reject the NULL HYPOTHESIS and believe CLAIM 2 (alternate claim) is TRUE.

You never "accept" a . . .

null hypothesis --fail to reject the null hyp, but NOT accept

Which of the following p-hat values will give the smallest p-value? (p=.6) p-hat = .4 (10/25) p-hat = .5 p-hat = .58

p-hat = .4 For the alternative hypothesis H-not: p < 0.60, we are asking, "what is the probability of observing a test statistic smaller than that given by the data?" Of the options given, the probability is the smallest for p-hat = 0.40, since it is the furthest from p = 0.60.

If p-value is < or = alpha, then you . . .

reject the null hypothesis the difference is statistically significant --diff. likely real, not due to chance --real diff. not necessarily important diff. --statistical tests do not address issue of importance (or practical signif.) --other info. (economic?) required to address importance