Chapter 6: Making Sense of Statistical Significance

Can you know if you have made a Type I or a Type II error?

No. But researchers can try to carry out studies so as to reduce the chance of making one: 1) lenience significance level.

What is the difference between significance and effect size?

Significance tells us that the results of the experiment should convince us that there is an effect (i.e. that it is "not due to chance"). But significance does not tell us how big this non-chance effect is. Put another way, the effect size is a measure of the difference between population means.

What is this Chapter about?

Statistical significance is about more than p < .05. This chapter helps us become sophisticated about making sense of significance. This required learning about 3 interrelated issues: Decision Errors, Effect Size, and Statistical Power.

What is the significance level? What is its symbol?

The chance of making a type I error, aka Alpha. it's symobl is the greek letter that looks like infinity.

Define Effect Size

the standardized measure of difference (lack of overlap) between populations. Effect size increases with greater differences between means.

What are 2 main factors that statistical power depends on?

1) How big an effect size the research hypothesis predicts. Effect size is influenced by: ** Differences between means ** The population standard deviation 2) How many participants are in the study (sample size)

When calculating effect size, what standard deviation do you use? Why?

1) Pick one standard deviation and use it. 2) Because in hypothesis testing you usually assume that both populations have the same standard deviation.

What other factors influence power?

1) Significance level: Less extreme significance (p<.20) level result in more power. 2) Using a one tailed test instead of a two tailed test 3) Type of hypothesis testing procedure employed.

What is the raw score effect size?

An effect size given in terms of the raw score on the measure

Why does sample size effect statistical power

Because the larger the sample size the smaller the standard deviation. The smaller the standard deviation, the greater the power

What are effect size conventions used for? What are the associated number for effect size?

Cohen's effect size provides a standard for deciding on the importance of the effect of a study in relation to what is typical in psychology. Determining whether an effect is considered strong or weak.

Effect size indicates the extent to which _______?

Two populations do NOT overlap.

What is the relationship between Type I and Type II errors?

When it comes to setting significance levels, protecting against one kind of decision increasing the risk of making the other. i.e. lowering your alpha reduces probability of making a Type II error.

What is a Type II error?

When the research hypothesis is true but the result does not come out extreme enough to reject the null hypothesis. In other words: you make this kind of decision error when the hypothesis testing procedure leads you to decide that the results of the study are inconclusive when in reality the research hypothesis is true.

Decision errors concern what?

Wrong conclusions that you make about your data. Despite doing all your figuring correctly, your conclusions regarding the hypothesis are incorrect --> "how the right procedures can lead to wrong conclusions."

Decision errors are possible in hypothesis testing because ____?

You are making decisions about populations based on information in samples. For example, rejecting a null hypothesis because a sample's mean has a very small probability that of being true if the null hypothesis is true. That said, it is there is still a possibility that the null hypothesis is true.

Type I Error

You reject the null hypothesis, when in fact the null hypothesis is true. in other words, you conclude that the study supports the research hypothesis when in reality the research hypothesis is false. MEMORY AID: "Good I-nitiative, bad judgement"

What is the standardized effect size?

You use a standardized effect size when you want to compare the effect size of one study with the effect size of a similar study that uses different measures of a variable. The standardized effect size is the difference between the population means divided by the population's standard deviation