Unit 7: Introduction to Hypothesis Testing

Alpha (two-tailed) 0.05 (5%), What it is it's critical Z-score?

+/- 1.96

Alpha (two-tailed) 0.01 (1%), What it is it's critical Z-score?

+/- 2.58

Alpha (two-tailed) 0.001 (0.1%), What it is it's critical Z-score?

+/- 3.30

What are the 4 assumptions that the hypothesis test makes?

-Random sampling -Independent observations -Standard Deviations is unchanged by the treatment -Normal sampling distribution

How do you determine whether you should reject the null hypothesis?

-Sample means falls within the critical region -big discrepancy between sample and H0 -Unlikely to occur if H0 is true -Demonstrates the treatment effect

How do you determine whether you should fail to reject the null hypothesis?

-sample mean does not fall within critical region -Data is reasonably close to H0 -Likely to occur if H0 is true -Treatment effect is not demonstrated

4 steps of hypothesis testing

1. State the HYPOTHESIS about the population. 2. Set the CRITERIA for making a DECISION about accepting or rejecting your hypothesis 3. Collect data and compute sample statistics 4. Make a decision -compare the obtained sample data to the prediction made in step 2; evaluate the hypothesis according to these criteria.

Reject H0

2.5% of values

Extreme

5%

Hypothesis Test

A statistical method that uses sample data (stats) to test a claim about a property of a population (hypothesis)

Based on the value of μ for the untreated population, what do score do you expect M to be close to if the null is true?

Around 10 or Larger

What is β called?

Beta

A researcher is conducting a one-tailed test with a α = .01 to determine whether a treatment produces a significant increase in scores. What Z-score value(s) would define the critical region for this test?

Beyond +2.58

What is a type 2 error?

Failing to reject the null hypothesis when it is actually false -treatment effect exists but is not detected -consequences: not serious; can accept outcomes or repeat experiment. -probability of type 2 error (β)-depends on a variety of factors; impossible to determine single, exact probability value

As the alpha level increases, the boundaries for the critical region move farther out into the tails of the distribution.

False

Changing α from .05 to .01 increases the risk of a Type I error.

False

If a researcher is predicting that a treatment will produce an increase in scores, then the critical region for a directional test would be located entirely in the left-hand tail of the distribution

False

If a sample mean falls into the critical region, then your decision should be to "fail to reject H 0 ."

False

If the alpha level is increase from 0.01 to 0.05, then the boundaries for the critical region move farther away from the center of the distribution (i.e., the critical region gets smaller).

False

Rejecting the null hypothesis with α = .05 means that you have more confidence in your decision than if you had rejected the null hypothesis with α = .01.

False

What is the symbol for the null hypotheses?

H0

What is they symbol for the alternative hypotheses?

H1

In general, increasing the sample size (for example, from n=4 to n=50) will___________ the risk of a type 1 error (assume that α is held constant at .05)

Have no influence on

What type of treatment effect is likely to affect in a type 2 error?

Having a small (but real) treatment effect. It is dependent in part of the size of the treatment effect.

What are the critical regions in hypothesis testing?

It is composed of extreme sample values that are very unlikely to be obtained if the null hypothesis is true. Boundaries are determined by the alpha level.

If given a problem, be able to calculate the standard error and the z-score by using the formulas

Ok

If given a z-score, be able to determine the alpha levels that support or reject the null hypothesis

Ok

Alternative Hypothesis

Predicts that the ID (treatment) WILL HAVE AN EFFECT on the DV for the population

Null hypothesis

Predicts that the independent variable (treatment) will have NO EFFECT on the dependent variable for the population.

A researcher conducts a hypothesis test to evaluate the effect of a treatment. The hypothesis test produces a z-score of z = -2.40. Assuming that the researcher is using a two-tailed test, what is the correct statistical decision?

Reject the null hypothesis with α = .05 but not with α = .01.

What is a type 1 error?

Rejecting the null hypothesis when it is actually true -treatment effect found when effect does not exist -Consequences: false reports of effects in the scientific literature -Probability of type 1 error= alpha level (α)

If a treatment has a very small effect, then a hypothesis test is likely to

Result in Type 2 error

Which regions or non-regions in hypothesis testing have a high probability?

The 95%

The school district is considering increasing the class size in elementary schools. The school board is concerned this could have a negative effect on student learning. What would the null hypothesis say about the effect of class size on student learning?

The mean of student learning with large class size would be equal to what it is currently

A Type I error occurs when you conclude that a treatment effect exists, but the treatment has no effect.

True

A type 1 error occurs when you conclude that a treatment effect exists, but the treatment has no effect.

True

Finding a significant treatment effect, means that you have rejected the null hypothesis.

True

In Hypothesis testing we assume that the SHAPE of the population and the SD will remain the SAME after treatment.

True

It is impossible to PROVE that the null is true or false with hypothesis testing. We can only provide evidence that the results found are highly likely or unlikely if the null is true (based on whether they fall in the critical or non-critical region). In science, we never prove or disprove anything, we only provide supporting or opposing evidence.

True

Whenever your decision is to reject the null hypothesis, there is a risk of a Type I error.

True

What does β represent?

Type 2 error

When is null true?

When there is no change (it is assumed true until proven otherwise).

Based on the value of μ for the untreated population, what do score do you expect M to be close to if the null is false?

X

Based on your z-score, how do you find the critical region, and how do you determine whether you should reject or accept the null hypothesis? Remember, for a two tailed test, you need to have a ± in front of the Z-score for your critical region, because it is positive AND negative.

X

Can the probability of type 1 and /or type 2 be determined specifically? How?

X

If given a hypothesis, how can you tell where the critical and noncritical regions will be located on a distribution? (Remember these are based off of α and whether your test is one-tailed (directional hypothesis) or two-tailed (non-directional hypothesis)).

X

In hypothesis testing, what does a low-probability (extreme) value for the z-score indicate?

X

Remember, there can be a one or two-tailed hypothesis. Know how to correctly look up the critical region for either in the unit normal table. With a two-tailed hypothesis, if your alpha level is .05, then you will be looking at a critical region of .025 split between the two tails. With a one-tailed test, you would have a critical region of .05 all in one-

X

What does each level of significance mean in research study?

X

What is α called?

alpha level (also called level of significance)

A researcher is conducting a one-tailed test with α = .01 to determine whether a treatment produces a significant increase in scores. What z-score value(s) would define the critical region for this test?

beyond +2.33

Effect Size is Measured by Cohen's d, how is that calculated?

mean difference/standard deviation

If the sample data produce a test statistic (z-score) that is in the critical region, then which of the following is the appropriate conclusion for the test?

reject H 0

A type 1 error is defined as

rejecting a true null hypothesis

In a hypothesis test, the critical region consists of

sample values that are very unlikely to occur if the null hypothesis is true

As the alpha level decreases, the size of the critical region gets ________ and the risk of a Type I error gets ________.

smaller, smaller

In a typical hypothesis testing situation, the null hypothesis makes a statement about

the population after treatment

What does α represent?

the probability value that is used to define the very unlikely sample outcomes if H0 is true

What are the non-critical regions in hypothesis testing?

the range of values of the test statistic that indicates that the difference was probably due to chance and the null hypothesis should not be rejected.

A hypothesis test is used to evaluate the effect of a treatment. If the test decision is to reject the null hypothesis with α = .05, then

there is a 5% chance that the treatment really has no effect.

If score lies within our 95% middle distribution, that means the probability of H0 being true is

very high

Will raising α increase the risk making a type 1 error?

yes, it is determined by the alpha level and is under the experimenter's control.

For a regular two-tailed test with α = .01, the boundaries for the critical region would be defined by z-scores of

z = ±2.58

Which alpha level provides the smallest chance of committing a Type I error?

α = .01

A researcher is conducting an experiment to evaluate a treatment that is expected to INCREASE the scores for individuals in a population which is known to have a mean of μ=80. The result will be examined using a one-tailed hypothesis test. Which of the following is the correct statement of the null hypothesis?

μ ≤ 80

A researcher is conducting an experiment to evaluate a treatment that is expected to increase the scores for individuals in a population which is known to have a mean of μ = 80. The results will be examined using a one-tailed hypothesis test. Which of the following is the correct statement of the null hypothesis?

μ ≤ 80