Chapter 9

meaning of statistically significant

"Significant" in the statistical sense does not necessarily mean "important." It means simply "not likely to happen just by chance."

the two circumstances to determine how small a P-value is convincing evidence against the null hypothesis

1) How plausible is Ho?: If Ho represents an assumption that the people you must convince have believed for years, strong evidence (small P-value) will be needed to persuade them. 2) What are the consequences of rejecting Ho?: If rejecting Ho in favor of Ha means making an expensive change of some kind, you need strong evidence that the change will be beneficial.

influences on how large a sample is needed

1) If you insist on a smaller significance level (such as 1% rather than 5%), you have to take a larger sample. A smaller significance level requires stronger evidence to reject the null hypothesis. 2) If you insist on higher power (such as 99% rather than 90%), you will need a larger sample. Higher power gives a better chance of detecting a difference when it is really there. 3) At any significance level and desired power, detecting a small difference requires a larger sample than detecting a large difference.

factors that increase the POWER of a test

1) Increasing the sample size (reduces the probability of a Type II error) 2) Increasing the significance level (increases the risk of a Type I error)

questions we must answer to decide how many observations we need

1) Significance level: How much protection do we want against a Type I error--getting a significant result from our sample when Ho is actually true? By using alpha = 0.05, there is a 5% chance of making a Type I error. 2) Practical importance: How large a difference between the hypothesized parameter and the actual parameter value is important in practice? 3) Power: How confident do we want to be that our study will select a difference of the size we think is important?

one-sample z test for a proportion

Choose an SRS of size n from a large population that contains an unknown proportion p of successes. To test the hypothesis Ho: p = po, compute the z statistic. Find the P-value by calculating the probability of getting a z statistic this large or larger in the direction specified by the alternative hypothesis Ha.

one-sample t test

Choose an SRS of size n from a large population with unknown mean. To test the hypothesis Ho: μ = μo, compute the one-sample t statistic. Find the P-value by calculating the probability of getting a t statistic this large or larger in the direction specified by the alternative hypothesis Ha in a t distribution with df = n - 1. Use this test only when (1) the population distribution is Normal or the sample is large ( n >/= 30), and (2) the population is at least 10 times as large as the sample.

statistically significant

If the P-value is smaller than alpha, we say that the data are statistically significant at level alpha. In that case, we reject the null hypothesis Ho and conclude that there is convincing evidence in favor of the alternative hypothesis Ha.

test statistic

Measures how far a sample statistic diverges from what we would expect if the null hypothesis Ho were true, in standardized units. That is, test statistic = (statistic - parameter) / standard deviation of statistic

For significance tests, "robust" means that the _______ is pretty accurate.

P-value

conditions for one-sample t test

Random: The data were produced by random sampling or a randomized experiment. Normal: The population distribution is Normal (based on graph) OR the sample size is large (n >/= 30). Independent: Individual observations are independent. When sampling without replacement, check that the population is at least 10 times as large as the sample.

significance tests: a four-step process

State: What hypotheses do you want to test, and at what significance level? Define any parameters. Plan: Choose the appropriate inference method. Check conditions. Do: If the conditions are met, perform calculations. Compute the test statistic. Find the P-value. Conclude: Interpret the results of your test in the context of the problem.

beware of multiple analysis

Statistical significance ought to mean that you have found a difference that you were looking for. In other settings, significance may have little meaning.

null hypothesis (Ho)

The claim we seek evidence AGAINST that is tested by a statistical test. The test is designed to assess the strength of the evidence against this. Often this is a statement of "no difference."

power

The power of a test against a specific alternative is the probability that the test will reject Ho at a chosen significance level alpha when the specified alternative value of the parameter is true.

power and Type II error

The power of a test against any alternative is 1 minus the probability of a Type II error for that alternative; that is, power = 1 - beta

P-value

The probability, computed assuming Ho is true, that the statistic (such as p-hat or x-bar) would take a value as extreme as or more extreme than the one actually observed. The smaller the P-value, the stronger the evidence against Ho provided by the data.

paired data

The result of study designs that involve making two observations on the same individual, or one observation on each of two similar individuals. This can be analyzed by first taking the difference within each pair to produce a single sample. Then use one-sample t procedures.

significance and Type I error

The significance level alpha of any fixed level test is the probability of a Type I error. That is, alpha is the probability that the test will reject the null hypothesis Ho when Ho is in fact true. Consider the consequences of a Type I error before choosing a significance level.

statistical significance and practical importance

When a null hypothesis ("no effect" or "no difference") can be rejected at the usual levels (α = 0.05 or α = 0.01), there is goof evidence of a difference. But that difference may be very small. When large samples are available, even tiny deviations from the null hypothesis will be significant.

significance test

a formal procedure for comparing observed data with a claim (called a hypothesis) whose truth we want to assess; the claim is a statement about a parameter

Statistical tests ask if sample data give good evidence (for/against) a claim.

against

The smaller the P-value, the great the chance the null hypothesis will (be rejected/fail to be rejected).

be rejected

The hypotheses should express the hopes or suspicions we have (before/after) we see the data. It is cheating to look at the data first and then frame hypotheses to fit what the data show.

before

Failing to find evidence against Ho means only that the data are _______ with Ho, NOT that we have clear evidence that Ho is true.

consistent

When planning a study, verify that the test you plan to use has a (low/high) probability (power) of detecting a difference of the size you hope to find.

high

one-sided alternative hypotheses

if the alternative hypothesis states that a parameter is larger than the null hypothesis or if it states that the parameter is smaller than the null value

two-sided

if the alternative hypothesis states that the parameter is different from the null hypothesis value (it could be either larger or smaller)

Type II error

if you fail to reject the null hypothesis Ho when the null hypothesis Ho is false

Type I error

if you reject the null hypothesis Ho when the null hypothesis Ho is true

Increasing the significance level (increases/decreases) the risk of a Type I error.

increases

Statistical significance (is/is not) the same thing as practical importance.

is NOT

If the df you need isn't provided in Table B, use the next (lower/higher) df that is available.

lower

You should (always/never) "accept Ho" or use language implying that you believe Ho is true.

never

Statistical tests deal with claims about a _______.

population

Always state Ho and Ha in terms of (sample statistics/population parameters).

population parameters

Hypotheses always refer to a (sample/population), NOT to a (sample/population).

population, sample

A test is significant if we (reject/fail to reject) the null hypothesis Ho.

reject

The power of a test to detect a specific alternative is the probability of reaching the (right/wrong) conclusion when that alternative is true.

right

The basic idea of a _______ is this: an outcome that would rarely happen if a claim were true is good evidence that the claim is not true

statistical test

Be sure to report the degrees of freedom with any (z/t) procedure, even if technology doesn't.

t

alternative hypothesis (Ha)

the claim about the population that we are trying to find evidence FOR

The significance level of a test is the probability of reaching the (right/wrong) conclusion when the null hypothesis is true.

wrong