Statistics Chap 6
type II error
failing to reject the null hypothesis when in fact it is false; failing to get a statistically significant result when in fact the research hypothesis is true
alpha
probability of making a type I error; same as significance level
beta
probability of making a type II error
effect size conventions
standard rules about what to consider a small, medium, large, effect size, based on what is typical in psychology research; also known as Cohen's conventions
statistical power
probability that the study will give a significant result if the research hypothesis is true
List situations in which it is useful to consider power, indicating what the use is for each.
-After a study is done that had nonsignificant results, to evaluate whether the result should be attributed to the null hypothesis being true (in the high-power situation) or to inadequate power so that it is still reasonable to think that future research might have a chance of being significant. -When planning an experiment, to permit changes of various kinds (or even abandon the project) if power is too low (or possibly to make the study less costly, for example by reducing the number of participants, if power is higher than reasonably needed). - After a study is done that had significant results with a large sample. In this case, if power is very high, this suggests that a low effect size is possible, so that although the result is significant, it may not be very important.
The power of a study represents the likelihood of getting a statistically significant result in your study, if in fact the research hypothesis is true. Effect size can be thought of as the degree to which populations do not overlap. The larger the effect size is, the greater the power will be. Given this study's high level of power for detecting large and medium-sized effects, the researchers were able to conclude that the most likely reason for the nonsignificant study results is that the research hypothesis is false. Thus, it is unlikely that the results of the study would be nonsignificant if there were a medium or large effect in the population.
A study placed strangers in pairs and asked them to talk together following a series of instructions designed to help them become close. At the end of 45 minutes, individuals privately answered some questions about how close they now felt to their partners. (The researchers combined the answers into a "closeness composite.") One key question was whether closeness would be affected by either (a) matching strangers based on their attitude agreement or (b) leading participants to believe that they had been put together with someone who would like them. The result for both agreement and expecting to be liked was that "there was no significant differences on the closeness composite." The researchers went on to argue that the results suggested that there was little true effect of these variables on closeness. An excerpt has been linked below. Explain this result to a person who understands hypothesis testing but has never learned about power or effect size.
significance
Significance is based on the difference between the mean of the sample and the known population mean, divided by the standard deviation of the distribution of means. If the sample size is very large, then the standard deviation of the distribution of means is very small. Thus, even a small difference between the means when divided by a very small denominator can give a large overall result.
A group of researchers analyzed results from a large-scale longitudinal study of a sample of children born in a specific city. As one part of their study, the researchers compared the 94 in their sample who were, at age 21, alcohol dependent (clearly alcoholic) versus the 863 who were not alcohol dependent. The researchers compared these two groups in terms of personality test scores from when they were 18 years old. After noting that all results were significant, they reported the results linked below. Explain these results, including why it was especially important for the researchers in this study to give effect sizes, to a person who understands hypothesis testing but has never learned about effect size or power.
The power of a study represents the likelihood of getting a statistically significant result in your study, if in fact the research hypothesis is true. Effect size can be thought of as the degree to which populations do not overlap. The larger the effect size is, the greater the power will be. It is important to give the effect sizes because the sample sizes for each of the two samples are different. It is possible to get a statistically significant result with a large sample size, even if the effect size is very small. So getting significant results for both samples does not indicate how practically significant the results actually are.
power tale
a table for hypothesis testing procedure showing the statistical power of a power of a study for various effect sizes and same sizes
decision errors
incorrect conclusions in hypothesis testing in relation to the real (but unknown) situation, such as deciding the null hypothesis is false when it is really true
type I error
rejecting the null hypothesis when in fact it is true; getting a statistically significant result when in fact the research hypothesis is not true
effect size
standarized measure of difference (lack of overlap) between populations. Effect size increases with greater differences between means
meta-analysis
statistical method for combining effect sizes from different studies
how does the following affect the power of a planned study?
(a) A larger predicted difference between the means of the populations increases power. (b) A larger population standard deviation decreases power. (c) A larger sample size increases power. (d) Using a more extreme significance level decreases power. (e) Using a two-tailed test instead of a one-tailed test decreases power.
