Statistics Chap 6

type II error

failing to reject the null hypothesis when in fact it is false; failing to get a statistically significant result when in fact the research hypothesis is true

alpha

probability of making a type I error; same as significance level

beta

probability of making a type II error

effect size conventions

standard rules about what to consider a small, medium, large, effect size, based on what is typical in psychology research; also known as Cohen's conventions

statistical power

probability that the study will give a significant result if the research hypothesis is true

List situations in which it is useful to consider​ power, indicating what the use is for each.

-After a study is done that had nonsignificant​ results, to evaluate whether the result should be attributed to the null hypothesis being true​ (in the​ high-power situation) or to inadequate power so that it is still reasonable to think that future research might have a chance of being significant. -When planning an​ experiment, to permit changes of various kinds​ (or even abandon the​ project) if power is too low​ (or possibly to make the study less​ costly, for example by reducing the number of​ participants, if power is higher than reasonably​ needed). - After a study is done that had significant results with a large sample. In this​ case, if power is very​ high, this suggests that a low effect size is​ possible, so that although the result is​ significant, it may not be very important.

The power of a study represents the likelihood of getting a statistically significant result in your​ study, if in fact the research hypothesis is true. Effect size can be thought of as the degree to which populations do not overlap. The larger the effect size​ is, the greater the power will be. Given this​ study's high level of power for detecting large and​ medium-sized effects, the researchers were able to conclude that the most likely reason for the nonsignificant study results is that the research hypothesis is false. ​Thus, it is unlikely that the results of the study would be nonsignificant if there were a medium or large effect in the population.

A study placed strangers in pairs and asked them to talk together following a series of instructions designed to help them become close. At the end of 45​ minutes, individuals privately answered some questions about how close they now felt to their partners.​ (The researchers combined the answers into a​ "closeness composite.") One key question was whether closeness would be affected by either​ (a) matching strangers based on their attitude agreement or​ (b) leading participants to believe that they had been put together with someone who would like them. The result for both agreement and expecting to be liked was that​ "there was no significant differences on the closeness​ composite." The researchers went on to argue that the results suggested that there was little true effect of these variables on closeness. An excerpt has been linked below. Explain this result to a person who understands hypothesis testing but has never learned about power or effect size.

significance

Significance is based on the difference between the mean of the sample and the known population​ mean, divided by the standard deviation of the distribution of means. If the sample size is very​ large, then the standard deviation of the distribution of means is very small.​ Thus, even a small difference between the means when divided by a very small denominator can give a large overall result.

A group of researchers analyzed results from a​ large-scale longitudinal study of a sample of children born in a specific city. As one part of their​ study, the researchers compared the 94 in their sample who​ were, at age​ 21, alcohol dependent​ (clearly alcoholic) versus the 863 who were not alcohol dependent. The researchers compared these two groups in terms of personality test scores from when they were 18 years old. After noting that all results were​ significant, they reported the results linked below. Explain these​ results, including why it was especially important for the researchers in this study to give effect​ sizes, to a person who understands hypothesis testing but has never learned about effect size or power.

The power of a study represents the likelihood of getting a statistically significant result in your​ study, if in fact the research hypothesis is true. Effect size can be thought of as the degree to which populations do not overlap. The larger the effect size​ is, the greater the power will be. It is important to give the effect sizes because the sample sizes for each of the two samples are different. It is possible to get a statistically significant result with a large sample​ size, even if the effect size is very small. So getting significant results for both samples does not indicate how practically significant the results actually are.

power tale

a table for hypothesis testing procedure showing the statistical power of a power of a study for various effect sizes and same sizes

decision errors

incorrect conclusions in hypothesis testing in relation to the real (but unknown) situation, such as deciding the null hypothesis is false when it is really true

type I error

rejecting the null hypothesis when in fact it is true; getting a statistically significant result when in fact the research hypothesis is not true

effect size

standarized measure of difference (lack of overlap) between populations. Effect size increases with greater differences between means

meta-analysis

statistical method for combining effect sizes from different studies

how does the following affect the power of a planned study?

​(a) A larger predicted difference between the means of the populations increases power. ​(b) A larger population standard deviation decreases power. ​(c) A larger sample size increases power. ​(d) Using a more extreme significance level decreases power. ​(e) Using a​ two-tailed test instead of a​ one-tailed test decreases power.