PSYC 241

What does a large z-score indicate?

A large value indicates a large discrepancy between the sample data and null hypothesis. Also that the sample data are very unlikely to have occurred by chance alone so we conclude that it must've been caused by treatment effect/we should reject the null hypothesis.

What does increasing the size of the sample do for the reject or fail to reject of null?

A larger sample produces a smaller standard error which leads to a larger z score. Therefore, larger n --> increased likelihood of rejecting the null (falling in the critical region).

Critical region

Composed of the extreme sample values that are very unlikely (as defined by the alpha level) to be obtained if the null hypothesis is true. The boundaries for the critical region are determined by the alpha level. If sample data fall in the critical region, the null hypothesis is rejected.

How would you express null and alternative directional hypotheses in symbols based on supplements making people score higher on tests?

H1: μ > 80 (with the supplement, the average score is greater than 80). H0: μ ≤ 80 (with the supplement, the average score is not greater than 80).

What is the distribution of sample means?

The bell curve that you always see. It is divided into 2 sections: 1) Samples means that are likely to be obtained if H0 is true; that is, sample means that are close to the null hypothesis (this is the majority of the curve, the middle portion). 2) Sample means that are very unlikely to be obtained if H0 is true; that is, sample means that are very different from the null hypothesis (these are the two tails sections on the curve).

What is p?

The possibility of type I error (wrongly rejecting the null). It is the researcher's report that the treatment had an effect but admits that it could be a false report by however much p value is.

What does it mean if your Cohen's d is 0.5?

This means that your treatment changed the mean by .5 of a standard deviation. Similarly, if d=1.00, the size of the treatment effect is equal to one whole standard deviation.

Estimation

We are estimating an unknown population mean. We go from sample to population mean.

What does increasing the standard deviation do for the reject or fail to reject of null?

A larger standard deviation produces a larger standard error, which leads to a smaller z score. Therefore, larger standard deviation--> less likely to reject null (more likely to fall into confidence interval).

Effect size

A measure of effect size is intended to provide a measurement of the absolute magnitude of a treatment effect, independent of the size of the samples being used.

How would you state the null and alternative hypothesis for a directional test based on supplements making people score higher on tests?

H0: Test scores are not increased. (The treatment does not work). H1: Test scores are increased. (The treatment works as predicted.)

Why are the 3 popular alpha levels so good?

.05, .01, .001 are considered reasonable good values because they provide a low risk of error without placing excessive demands on research results (w/o making it impossible to reject null).

What are the four steps of hypothesis testing?

1) State hypothesis. 2) Decision criteria/predict a population mean. 3) Collect data from random sample, calculate statistics. 4) Make decisions.

Alpha level/level of significance

A probability value that is used to define the concept of "very unlikely" in a hypothesis test. Make up the critical region. Most commonly used alpha levels are .05 (5%), .01 (1%), and .001 (.1%). E.g. By having an alpha of .05, we are separating the most unlikely 5% of the samples means (the extreme values) from the most likely 95% of the samples means (the central values.

Statistical significance

A result is said to be statistically significant if it is very unlikely to occur when the null hypothesis is true. That is, the result is sufficient to reject the null hypothesis. Thus, a treatment has a significant effect if the decision from the hypothesis test is to reject H0. If your z score point that you find falls into the critical regions (shaded), it is significant.

Test statistic

A term that indicates that the sample data are converted into a single, specific statistic that is used to test the hypotheses. The z-score statistic that we use in the hypothesis test is an example of a test statistic.

Define standard deviation. What does a low SD tell us? High SD?

A value that shows how much variation there is from the average (mean or expected value). Low standard deviation means that the data points are very close to the mean but high standard deviation indicates that the data points are spread out over a large range of values.

What does a small z-score indicate?

A z-score near zero indicates that the data support the null hypothesis/we should fail to reject the null hypothesis.

As the power of a test increases, what happens to the probability of Type II error?

As power increases, the probability of a Type II error decreases.

Alternative hypothesis

Assumes that there will be a change, difference, or relationship for the general population. In an experiment, it predicts that the IV does have an effect on the DV. It is written as H1: μ with supplement≠ 80 (with supplement, mean test score will be different from 80).

What does the z score tell us?

Describes exactly where the sample mean is located relative to the hypothesized population mean from H0.

Degrees of freedom

Describes the number of scores in a sample that are independent and free to vary. Because the sample mean places a restriction on the value of one score in the sample, there are n-1 degrees of freedom for the sample.

Confidence interval

Estimate of an unknown population mean. Confidence interval is an interval estimate. It's the center section of a distribution that is surrounded by the tails.

Why would you fail to reject the null hypothesis?

If sample data are not in the critical region, the data would be reasonably close to the null hypothesis (in center of distribution). Because the data would not provide strong evidence that the null is wrong, our conclusion would be to fail to reject the null. This means that the treatment does not appear to have an effect.

Difference between z test and t test

If you have a sigma, use a z test. If you don't have one, use a t test.

What is a directional "one-tailed" hypothesis test?

In this test, the statistical hypotheses (H0 and H1) specify either an increase or a decrease in the population mean. That is, they make a statement about the direction of the effect.

How does increasing sample size influence the outcome of a hypothesis test?

Increasing sample size increases the likelihood of rejecting the null hypothesis.

How does increasing sample size influence the power of a hypothesis test?

Increasing sample size increases the power of a test.

How would a scientific report state that the 1. "null hypothesis was rejected"? 2. And "failed to reject null hypothesis"?

It would say that 1. "the effect of the treatment was statistically significant". And 2. "the treatment effect was not statistically significant"/"there was no evidence for a treatment effect".

Point estimate

It's one number/one value. In terms of population mean, the point estimate is a sample mean (M) (a sample mean representing the population mean). We know it's not going to be the exact same, it's a little above or below the population mean.

Flynn Effect

Noticed average IQ score was going up over time and it's not supposed to do that. So IQ scores tend to go up over time. Again, the average result is set to 100. However, when the new test subjects take the older tests, in almost every case their average scores are significantly above 100.

Null hypothesis

Null= "nothing". In a general population, it states that there is no change, difference, or relationship. In an experiment, it predicts that the IV (treatment) will have no effect on the DV (scores) of population. It is written as H0: μ with supplement= 80 (even with supplement, mean test score will still be 80).

Type II error

Occurs when a researcher fails to reject a null hypothesis that is really false. In a typical research situation, a Type II error means that the hypothesis test has failed to detect a real treatment effect. Probability of type II error is β.

Type I error

Occurs when a researcher rejects a null hypothesis that is actually true. In a typical research situation, a Type I error means the researcher concludes that a treatment does have an effect when it actually doesn't have an effect.

Cohen's d

One of the simplest and most direct methods for measuring effect size. Recommended that effect size can be standardized by measuring the mean difference in terms of the standard deviation.

Examples of situations using Directional "one-tailed" hypothesis tests

Researcher beings an experiment with specific prediction about direction of treatment effect. E.g. Alcohol consumption is expected to slow reaction times.

What are 2 things that make power fluctuate?

Sample size- Larger sample produces greater power for hypothesis test. Alpha level- Reducing alpha level for a hypothesis test reduces the power of the test.

Example of Type I error

Suppose researcher selected sample of n=25 people who were already above average smart. When they are tested 6 months after being on supplement, they will score higher than average even though supplement may have had no effect. In this case, the researcher would conclude that the treatment has had an effect, when really, it didn't. This is a type I error.

Interval estimate

Taking that point estimate and building an interval around it. You have the sample mean and you construct a point above it and below it.

What happens as the alpha level is lowered?

The boundaries for the critical region move farther out and become more difficult to reach. The boundaries for the critical region determine how much distance between the sample mean and population mean is needed to reject the null. As the alpha level gets smaller, this distance gets larger and it becomes more hard/impossible to ever reject the null hypothesis.

What does a hypothesis test consist of?

The concepts of z-scores, probability, and the distribution of sample means. A hypothesis test is a statistical method that uses sample data to evaluate a hypothesis about a population.

What things can affect the z statistic?

The difference between sample mean (for treated group) and population mean for population of untreated group. If the difference between these 2 are small, z score will be small. If they are the same, z score will be 0. As standard deviation gets larger, standard error gets larger and so z score gets smaller. Standard error also gets smaller as sample size gets larger.

Power

The power of a statistical test is the probability that the test will correctly reject a false null hypothesis. That is, power is the probability that the test will identify a treatment effect if one really exists.

What is the p value?

The probability that the result would occur if null hypothesis were true (without any treatment effect), which is also the probability of Type I error. The probability needs to be small.

What is the alpha level for a hypothesis test?

The probability that the test will lead to a Type I error. That is, the alpha level determines the probability of obtaining sample data in the critical region even though the null hypothesis is true.

What does p<.01 indicate in a hypothesis test?

The test will probably reject the null hypothesis. If the probability is less than .01, it means that it is very unlikely that the result occurred without any treatment effect (so the data are in the critical region, and H0 is rejected).

Why would you reject your null hypothesis?

This decision is made whenever the sample data fall in the critical region. A sample value in the critical region indicates that the sample is not consistent with the population define by the null hypothesis. This sample is very unlikely to occur if the null is true so we reject it.

Significance vs. importance

When a statistic is significant, it simply means that you are very sure that the statistic is reliable. It doesn't mean the finding is important.

Example of Type II error

Whenever you fail to reject a null hypothesis, there is a risk of Type II error which is the failure to reject a false null hypothesis. Type II error means that the research data do not show the results that the researcher had hoped to obtain. The researcher can accept this outcome and conclude that the treatment either has no effect or has only a small effect that is not worth pursuing.

Point estimate

You use a single number as your estimate of an unknown quantity.

What is a small/medium/large effect interms of Cohen's d?

d=0.2 is a small effect. d=0.5 is a medium effect. d=0.8 is a large effect.