Research Methods Ch. 6

What is usually the easiest way to increase the power of a planned study?

Increase the sample size

Type 1 Error

Rejecting the null hypothesis when in fact it is true; getting a statistically significant result when in fact the research hypothesis is not true.

Power Table

Table for a hypothesis testing procedure showing the statistical power of a study for various effect sizes and sample sizes.

How and why does sample size affect power?

The larger a sample size is, the more power there is, because a larger sample size makes the distributions of means narrower and thus have less overlap; sot he area in the predicted distribution that is more extreme than the cutoff in the known distribution is greater.

If the result is NOT statistically significant, what is the conclusion if the sample size is (a) small, or (b) large?

(a) Small - Inconclusive (b) Research hypothesis probably false

If the result is statistically significant, what is the conclusion if the sample size is (a) small, or (b) large?

(a) Small - important result (b) Large - Might or might not have practical importance

How do you calculate power?

1) Determine raw-score cutoff point on comparison distribution 2) Find the Z score for this point on the distribution of means for the experimental population 3) Use the normal curve table to find probability of finding a more extreme Z score 4) Percentage in tail region above Z score cutoff is your power level.

What does effect size add to just knowing whether a result is significant?

A significant result can be just barely big enough to be significant or much bigger than necessary to be significant; so the effect size tells you how big that effect is.

Why do researchers use a standardized effect size?

A standardized effect size makes results of studies using different measures comparable

How and why does using a one-tailed vs two tailed test affect power?

A study with a one-tailed test has more power because with a one-tailed test, the cutoff in the predicted direction in the known distribution is less extreme, so the corresponding area that is more extreme than this cutoff is larger. there is an added cutoff in the opposite side with a two-tailed test, but this is so far out on the distribution that it has little effect on power.

What is the probability of getting a significant result if the research hypothesis is false?

Alpha; probability of making a Type 1 error, which is your significance level (ex: 1%, 5%)

Relationship between beta and power

Beta + Power = 100%

What two factors determine effect size?

Difference between known and predicted population means and the population standard deviation?

What is the equation for standardized effect size?

Difference between the population means, divided by the population's standard deviation

Type 2 Error

Failing to reject the null hypothesis when in fact it is false; failing to get a statistically significant result when in fact the research hypothesis is true

If you set an extreme alpha level (say .001), what is the effect on the probability of Type 1 error or Type 2 Error?

For a Type 1 Error, the probability is low. For a Type 2 Error, the probability is high.

Decision Errors

Incorrect conclusions in hypothesis testing in relation to the real but unknown situation, such as deciding the null hypothesis is false when it is actually true.

What are the two basic ways of increasing the effect size of a planned study?

Increase the predicted difference between population means, and reduce the population standard deviation

Beta

Probability of making a Type 2 Error

Statistical Power

Probability that the study will give a significant result if the research hypothesis is true

Alpha

The probability of making a Type 1 error

When a result is significant, what can you conclude about effect size if the study had a very large sample size?

The research hypothesis is probably not true

What is the probability of making a Type 1 Error

The significance level (such as .05)

What factors determine the power of a study?

1) Effect Size 2) Significance level (alpha) 3) One vs. two tailed test 4) Type of hypothesis-testing procedure

Why is statistical power important?

It can help you determine how many participants are needed for a study you are planning, and understanding power can help you make sense of results that are not significant or results that are statistically but not practically significant.

A study with a larger or smaller effect size is more likely to come out statistically significant?

Larger effect size

What is the role of effect size in a meta-analysis?

Meta-analysis use an average effect size across studies to compare different subgroups of studies

Effect Size Conventions

Small: d = 0.2; overlap of 85% Medium: d = 0.5; overlap of 67% Large: d = 0.8; overlap of only 53%

Effect Size

Standardized measure of difference between populations; increases with greater differences between means

Meta-Analysis

Statistical method for combining effect sizes from different studies

How is statistical power different from just the probability of getting a significant result?

Statistical power is the probability IF THE RESEARCH HYPOTHESIS IS TRUE, not just the probability of getting a significant result.

Why is statistical significance not the same as practical importance?

Statistical significance means you can be confident the effect would be unlikely to happen if the null hypothesis were true; however, doesn't mean that it is a large or substantial effect

How and why does the significance level used affect power?

The more lenient the significance level is, the more power there is. This is because it makes the cutoff in the known distribution less extreme; so the corresponding area that is more extreme than this cutoff in the predicted distribution of means is larger.