Chapter 7: Hypothesis Testing

factors that can influence statistical power

- levels of significance - sample size - effect size - type of statistical test

one-tailed test

A test of significance that looks for an effect in a particular direction

two-tailed test

A test of significance that looks for an effect without concern as to the direction of the effect

significance level

Alpha

Statistical vs. Clinical Significance

Clinical significance is usually measured in effect size, usually used to determine the impact of medical interventions. Statistical significance usually happens when it is very unlikely to have occurred given the null hypothesis. Statistical significance can be present without clinical significance being present.

Alternative Hypothesis (Ha)

Hypothesis denoted by H1 or Ha, states an effect

Null Hypothesis (H0)

Hypothesis denoted by Ho, no effect, based on no change

A researcher sets the probability at 0.05, and determines that the analysis is statistically significant, what would the researcher do next?

Next the researcher should propose an appropriate statistical test. The researcher should do the appropriate test for the research question. To then keep going and eventually figure out if he will reject or not reject the null hypothesis.`

Type I Error (False Positive)

Occurs when the null hypothesis is rejected when it shouldn't be

generalizability

The accuracy with which results from a sample can be extrapolated to encompass the population as a whole

effect size

The measure of the magnitude of the relationship or difference between groups; often used as a measure of efficacy

p value

The probability level which forms basis for deciding if results are statistically significant (not due to chance).

statistical power

The probability of correctly rejecting the false null hypothesis.

A null hypothesis was rejected at the 0.01 level and the results were determined to be statistically significant, what does this mean?

This means that the level of significance was 10% (0.01) and if it was rejected and statistically significant than this means that rejecting the null hypothesis in this case is statistically correct and seems to have been the correct thing to do.

clinical significance

Usually measured as effect size, may be used to determine the magnitude of impact of an intervention; useful for evaluating clinical practice

a two-tailed hypothesis statement is only applicable to the

alternative hypothesis

if p value is larger than α

do not reject the null hypothesis

what will happen to the statistical power if the sample size increases?

if sample size increases, the z value increases, which means it's more likely to reject the null hypothesis, so the statistical power increases.

If P-value is smaller than α

reject the null hypothesis in favor of the alternative hypothesis

if the sample statistic and the hypothesized population parameter are very different, the null hypothesis is

rejected

type II error

when the null hypothesis is not rejected when it should be

Steps in Hypothesis Testing

1. State the null and alternative hypothesis 2. Choose the level of significance (alpha) 3. Propose an appropriate statistical test 4. Check assumptions of the chosen test 5. Compute the test statistics (find p value) 6. Find the critical value (use alpha) 7. Compare the test statistics and the critical value to make conclusions

Suppose you rejected a null hypothesis at the 0.05 level of significance. Does this mean that you would reject the same null hypothesis at the 0.01 level? Explain.

No. Rejecting and not rejecting the null hypothesis goes off of p value and Alpha. In this case the Alpha is 0.05, so if we reject the null hypothesis at 0.05 then the p value is smaller than 0.05, but if the p value is larger than 0.01 then we wouldn't reject the null hypothesis.

Suppose you rejected a null hypothesis at 0.05 level of statistical significance. If you were to repeat the experiment over multiple times, would you get the same result? Explain.

No. You wouldn't necessarily get the same results every time. By choosing 0.05 level of statistical significance you are still giving yourself a 5% chance that you will incorrectly reject or not reject the null hypothesis. Also, there is always room for error when doing tests and/or studies.

the researcher will make a decision whether to reject or fail to reject the null hypothesis by looking at the

P-value & Alpha