Methods 301 Final

you reject the null hypothesis, you do not...

1. prove the null hypothesis 2. they're simply not statistically significant 3. the study is inconclusive

not knowing the population variance presents what two problems?

1. We have to estimate population variance 2. When you estimate, a second problem arises, the shape of the distribution is not quite normal→ we have to use a t distribution

• What is the probability of getting a type 1 error?

The same as your significant level (p-value). (e.g. 5%)

with t-test you don't...

compare to a "known" population

Statistical Power:

probability that the study will give a significant result if the research hypothesis is true

Hypothesis Testing:

procedure for deciding whether the outcome of a study supports a particular hypothesis or theory

rejecting or failing to reject the null hypothesis does not...

prove anything

There's a cost of setting the alpha too high-->

making a type 1 error

What is meant by p < .01?

the probability is less than 1%

Five-Steps of the Hypothesis-Testing Process:

• 1) Restate the question as a research hypothesis and a null hypothesis about the population • 2) Determine the characteristics of the comparison distribution • 3) Determine the cutoff sample score on the comparison distribution at which the null hypothesis should be rejected • 4) Determine you sample's score on the comparison distribution Decide whether to reject the null hypothesis

Reduce type 2 errors →

Maximize statistical power

Type II Error:

-fail to reject the null hypothesis (and thus failing to support the research hypothesis) when the null hypothesis is actually false (and the research hypothesis is actually true -or: You conclude the study does not support the research hypothesis when in reality the research hypothesis is true

in a t-test you might compare your sample by...

1) Comparing scores between two or more groups (experimental vs. control group; 5 year olds to 8 year olds) 2) Compare scores for the same individual at two different time points (before and after treatment) 3) Compare a sample to a population with a known mean, but an unknown variance

(a) What is a comparison distribution? (b) What role does it play in hypothesis testing?

7a. A comparison distribution is a distribution to which you compare the results of your study. 7b. In hypothesis testing, the comparison distribution is the distribution for when the null hypothesis is true. To decide whether to reject the null hypothesis, check how extreme the score from your sample is on this comparison distribution. In other words, how likely it would be to get a sample with a score this extreme (in the tails) if your sample came from this comparison distribution.

What can you conclude when (a) a result is so extreme that you reject the null hypothesis and (b) a result is not very extreme so that you cannot reject the null hypothesis?

9a. The research hypothesis is supported when a result is so extreme that you reject the null hypothesis; the result is statistically significant. 9b. The result is not statistically significant when a result is not very extreme; the result is inconclusive.

probability:

Probability is the expected relative frequency of a particular outcome.

Independent samples t-test:

Sample exp. Grp. Mean compared to sample control group mean • Don't need distribution of means (compare mean to mean)

4) Determine you sample's score on the comparison distribution

carry out your study and get the results

Single-sample t test:

compare your sample to a population with a known mean, but an unknown variance

Protecting against one decision error ...

increases the chance of making the other decision error

There is a cost of setting the alpha level too low→

making a type 2 error.

Reduce type 1 errors →

o Can set a more stringent alpha (e.g. 1%) o Replicate

why is a type 2 error possible?

o The null hypothesis was false, but the particular sample you had was not extreme enough to get into that rejection region

With null hypothesis testing:

o We reject the idea that there is no effect (double negative) o What's the probability of getting our results if there is no difference between our sample and the population? o We decide if there is an effect by seeing if its unlikely to get our result if there were no effect

1) Restate the question as a research hypothesis and a null hypothesis about the population

o population 1: babies who took special vitamin o population 2: babis in general→ baseline comparison group o our prediction: babies who take the special vitamin (pop 1) will on averagw walk earlier than the general population of babies (pop 2) • → Research Hypothesis (H1) • statement in hypothesis testing about the predicted relationship between populations (often a prediction of difference in population means) H1: M, < M2 o Opposite of our prediction: the populations (1 and 2) are not different in the way predicted • → Null hypothesis (H0) • statement about the relationship between populations that is the opposite of the research hypothesis there is no difference or difference opposite to predicted • contrived statement to examine whether it can be rejected as part of hypothesis testing H0: M, = M2

distribution of means is only...

previous studies' means

Type I Error (false positive):

reject the null hypothesis (and support the research hypothesis) when the null hypothesis is actually true (and research hypothesis is actually false)

Compromise →

set alpha at 5% (p< .05)

Effect Size:

tells us the size of an effect when we have a significant result

statistical significance:

when a sample score is so extreme that researcher rejects the null hypothesis.

Decision Errors:

when the right procedures lead to the wrong decision... our conclusion from hypothesis testing does not match reality

Interpreting Effect Size:

• .20= small • .50= medium • .80= large • total overlap= no effect • meta analysis need effect size

Basic Logic of Hypothesis Testing:

• Consider our known population of babies o The chance of a baby walking 8 months is unlikely (~2%) • If the special vitamin had no effect, we'd expect our baby to look like babies in general • If our baby did start walking at 8 months: o We reject the idea that the special vitamin had no effect → accepting that the special vitamin has an effect

What influences power?

• Effect size: larger the effect, the more power you have • Sample size: the larger the sample, the more power you have • Easy to see difference with big effect or big sample (lots of power) • Hard to see difference with small effect or small sample (low power)

Dependent-samples t-test:

• Have same people in both groups o Ex: pre and post test • Compare means directly

Single-sample t test using Steps in Hypothesis Testing Process:

• Restate the problem as hypotheses about populations. • Determine the characteristics of the comparison distribution. • Estimating the Population Variance • Biased estimate

Z tests vs. T tests

• Z Test • Random sample • Normal distribution • Comparison copulation a "known population" o We know the mean and standard deviation • T Test: • Random sample • Normal distribution (approximates) • Don't compare to a "know population" • Instead we might... o Compare scores between two or more groups (experimental vs. control group; 5 year olds to 8 year olds) o Compare scores for the same individual at two different time points (before and after treatment) o Compare a sample to a population with a known mean, but an unknown variance

5) Decide whether to reject the null hypothesis

• compare actual sample with cutoff score • if the sample score is more extreme than the cutoff score (a.k.a. critical value) you reject the null hypothesis • if sample score is not more extreme than critical value, you fail to reject the null hypothesis

3) Determine the cutoff sample score on the comparison distribution at which the null hypothesis should be rejected

• cutoff sample score/ critical value: the cutoff point for how extreme a sample score needs to be for it to be too unlikely to occur if the null hypothesis were true (that is that there's no difference) • Conventional levels of significance: o In general, Most common convention is 5% for psychologists→ researcher will reject the null hypothesis if the probability of getting a sample score this extreme is less than or equal to 5% • p<.05 • statistical significance: when a sample score is so extreme that researcher rejects the null hypothesis.

2) Determine the characteristics of the comparison distribution

• population (if we know the population) • control group

why is a type I error possible?

• → you reject the null when a sample's result is so extreme that is unlikely to have obtained the result of the null hypothesis was true • it is possible to get an extreme result with the null hypothesis being true. o If this happens, you would reject the null but that decision would be wrong

Alpha:

The probability of making a type 1 error

Conventional levels of significance:

In general, Most common convention is 5% for psychologists→ researcher will reject the null hypothesis if the probability of getting a sample score this extreme is less than or equal to 5%

Beta (B) :

The probability of making a type 2 error

The higher the alpha...

the lower the chance of making a type 2 error

The lower the alpha...

the lower the chances of making a type 1 error

the study is inconclusive →

the study didn't detect a difference