Unit 7: Significance Tests
Pooled Proportion (p̂) Equation (In Words)
Pooled Proportion (p̂) = Number of Successes in Both Samples Combined / Number of Individuals in Both Samples Combined
Power =
Power = 1 - 𝑃(Type II Error)
The Four Step Process for Significance Tests with Proportions: State
State the hypothesis you want to test and the significance level. Also, be sure to define any parameters you use.
Type 1 Error Probability
The probability of making a Type I Error in a significance test is equal to the significance level, α.
The Conclusion of a Significance Test: The Type II Error
This is if a test fails to reject the null hypothesis when the alternative hypothesis is true. That is, the test does not find convincing evidence that the alternative hypothesis is true when it actually is not.
Alternative Hypothesis: The One-Sided Alternative Hypothesis
This states that a parameter is greater than the null value, or this states that the parameter is less than the null value.
Alternative Hypothesis: The Two-Sided Alternative Hypothesis
This states that the parameter is different from the null value. It could be either greater than the null value or less than the null value.
How many major types of hypotheses are there for Significance Tests?
Two types of major hypotheses here: The Null Hypothesis and the Alternative Hypothesis
How many types of errors are there for the conclusion of a significance test?
Two: the Type I Error and the Type II Error
Degrees of Freedom (df) Formula
df = 𝑛 - 1
𝑃(Type II Error) =
𝑃(Type II Error) = 1 - Power
Mean Equation for the Sampling Distribution of p̂₁ - p̂₂
𝜇 subscript p̂₁ - p̂₂ = 𝑝₁ - 𝑝₂
The Three Ways to Increase the Power of a Significance Test
1. The sample size, 𝑛, is larger. 2. The significance level, ɑ, is larger. 3. The null and alternative parameter values are farther apart.
Significance Tests about 𝑝 Conditions: The Large Counts Condition
Both 𝑛𝑝₀ ≥ 10 and 𝑛(1-𝑝₀) ≥ 10 must be true.
What boundary should you consider using when determining whether or not a result is statistically significant?
Consider using a boundary of 5% when determining whether or not a result is statistically significant (i.e. view a P-value less than 0.05 as small).
The Conditions for Performing a Significance Test About a Difference Between Two Population Means: The Normal/Large Sample Condition
For each sample, (only) one of the following must be true: 1. The corresponding population distribution (or the true distribution of response to the treatment) is Normal, 2. The sample size is large (𝑛 ≥ 30), or 3. For each sample, if the population (treatment) distribution has an unknown shape, and 𝑛 < 30, a graph of the sample data must be drawn and show that there is no strong skewness nor outliers in the sampling distribution
For the 𝑃-Value df (degree of freedom) number in significance tests with two population means, use the ___ df value.
For the 𝑃-Value df number in significance tests with two population means, use the smaller df value.
Significance Tests with a Mean: Plan
Identify the appropriate interference method (a one sample 𝒕 test for 𝜇 in this case), and check conditions (the Random Condition, the 10% Condition, and the Normal/Large Sample Condition).
The Four Step Process for Significance Tests with Proportions: Plan
Identify the appropriate interference method, and check conditions (the Random Condition, the 10% Condition, and the Large Counts Condition).
The Four Step Process for Significance Tests with Proportions: Do
If all three conditions (the Random Condition, the 10% Condition, and the Large Counts Condition) are met, do all three of the following calculations: 1. Give the sample statistic(s). 2. Calculate the standardized test statistic. 3. Find the 𝑃-value.
Significance Tests with a Mean: Do
If the conditions are met, perform the calculations (do all three of the following): 1. State the sample statistic(s), 2. Calculate the standardized test statistic (a one sample 𝒕 test for 𝜇), and 3. Find the 𝑃-value.
If the 𝑃-value is less than α, we say that the result is [finish the statement].
If the 𝑃-value is less than α, we say that the result is statically significant at the α = ___ level.
The Four Step Process for Significance Tests with Proportions: Conclude
Make a conclusion about the hypotheses in the context of the problem.
Significance Tests with a Mean: Conclude
Make a conclusion about the hypotheses in the context of the problem. In the two sentence conclusion, be sure to define what Ha means in context.
Never Do This When Talking About the Null Hypothesis
Never "accept the null hypothesis" or conclude that the null hypothesis is true.
Writing the Conclusion to a Significance Test (The Structure)
The first sentence should give a decision about the null hypothesis (reject the null hypothesis or fail to reject the null hypothesis) based on an explicit comparison of the 𝑃-value to a stated significance level. The second sentence should provide a statement about whether or not there is convincing evidence for the alternative hypothesis in the context of the problem.
Significance Tests for 𝜇 Conditions: The Normal/Large Sample Condition
The population has a Normal distribution, the sample size is large (𝑛 ≥ 30), //or// if the population has an unknown shape and 𝑛 < 30, use a graph of the sample data to asses the Normality of the population. Do not use 𝒕 procedures, if the graph shows strong skewness or outliers.
The sampling distribution of p̂₁ - p̂₂ is ___ ___ if the Large Counts Condition is met for both samples.
The sampling distribution of p̂₁ - p̂₂ is approximately Normal if the Large Counts Condition is met for both samples.
What are the three most common significance levels?
The three most common significance levels are α = 0.01, α = 0.05, and α = 0.10.
𝑃-value
This is a test of the probability of getting evidence for the alternative hypothesis, Ha, as strong or stronger than the observed evidence when the null hypothesis, H₀, is true.
The Conclusion of a Significance Test: The Type I Error
This is if a test rejects the null hypothesis when the null hypothesis is true. That is, the test finds convincing evidence that the alternative hypothesis is when when it actually is not.
Power
This is the probability that the test will find convincing evidence for Ha when a specific alternative value of the parameter is true. This is very related to a Type II Error, and it is about you rejecting H₀ and caught the issue/were right to reject H₀.
Significance Level: α
This is the value that one can use as a boundary for deciding whether an observed value is unlikely to happen by chance alone when the null hypothesis is true. This should be stated before the data is produce.
A Standardized Test Statistic (AKA: A Z-Score)
This measures how far a sample statistic is from what we would expect if the null hypothesis, H₀, were true, in standard deviations.
Standard Deviation Equation for the Sampling Distribution of p̂₁ - p̂₂
σsubscript p̂₁ - p̂₂ = √((𝑝₁(1-𝑝₁) / 𝑛₁) + 𝑝₂(1-𝑝₂) / 𝑛₂) (as long as the 10% Condition is met for both samples: 𝑛₁ < 0.10𝑁₁ and 𝑛₂ < 0.10𝑁₂
Two Sample 𝒕 Test for the Difference Between Two Proportions
𝒕 = ((x̄₁ - x̄₂)-0) / √((𝑠₁)² / 𝑛₁) + (𝑠₂)² / 𝑛₂)
Standardized Test Scores: The 𝒕 Score Formula
𝒕 = (x̄-𝜇₀) / ((𝑠 sub 𝑥) / √𝑛)
Two Sample 𝒛 Test for the Difference Between Two Population Proportions
𝒛 = ((p̂₁ - p̂₂)-0) / √((Pooled p̂(1-Pooled p̂) / 𝑛₁) + ((Pooled p̂(1-Pooled p̂) / 𝑛₂)
One Sample 𝒛 Test for a Proportion Equation
𝒛 = (p̂-𝑝₀)/(√𝑝₀(1-𝑝₀)/𝑛)
Significance Test (AKA: A Hypothesis Test)
A formal procedure for using observed data to decide between two competing claims (called hypotheses), the claims are usually statements about a parameter. Parameter examples include population proportions, 𝑝, or population means, 𝜇. This is also another way (besides confidence intervals) to make statistical inferences, and this is used to weigh the evidence in favor of or against a particular claim.
As your Power increases, your probability of getting a Type II Error ___.
As your Power increases, your probability of getting a Type II Error decreases.
One can ___ the probability of making a Type I Error in a significance test by using a ___ significance level.
One can decrease the probability of making a Type I Error in a significance test by using a smaller significance level.
Pooled Proportion (p̂) Equation (In Symbols)
Pooled Proportion (p̂) = (𝑥₁ + 𝑥₂) / (𝑛₁ + 𝑛₂)
The Four Step Process for Significance Tests: Conclude Script
Sentence One: Since 𝑝 is [< or >] ɑ, we [reject or fail to reject] H₀. Sentence Two: There [is or is not] convincing evidence that the true [mean or proportion] is [Ha].
A Standardized Test Statistic (AKA: A Z-Score) Equation
Standardized Test Statistic = (Statistic - Parameter) / (The Standard Deviation of the Statistic)
Significance Tests with a Mean: State
State the hypotheses you want to test and the significance level. Also, define any parameters you use.
The Four Step Process for Significance Tests
State, Plan, Do, and Conclude
Significance Tests about 𝑝: Conditions
The Random Condition, the 10% Condition, and the Large Counts Condition
The Conditions for Performing a Significance Test About a Difference Between Two Population Proportions
The Random Condition, the 10% Condition, and the Large Counts Condition
Significance Tests for 𝜇: Conditions
The Random Condition, the 10% Condition, and the Normal/Large Sample Condition
The Conditions for Performing a Significance Test About a Difference Between Two Population Means
The Random Condition, the 10% Condition, and the Normal/Large Sample Condition
The best significance level to use depends on whether a Type I Error or a Type II Error is ___ ___.
The best significance level to use depends on whether a Type I Error or a Type II Error is more serious.
Alternative Hypothesis (Ha)
The claim that one is trying to find evidence for in a significance test
Null Hypothesis (H₀)
The claim that one weighs evidence against in a significance test
The Conditions for Performing a Significance Test About a Difference Between Two Population Proportions: The Large Counts Conditions
The counts of "successes" and "failures" in each sample group: 𝑛₁𝑝₁ ≥ 10, 𝑛₁(1-𝑝₁) ≥ 10, 𝑛₂𝑝₂≥ 10, and 𝑛₂(1-𝑝₂) ≥ 10.
Significance Tests about 𝑝 Conditions: The Random Condition
The data must come from a random sample from the population of interest.
Significance Tests for 𝜇 Conditions: The Random Condition
The data must come from a random sample from the population of interest.
The Conditions for Performing a Significance Test About a Difference Between Two Population Means: The Random Condition
The data must come from two independent random samples or from two groups in a randomized experiment.
The Conditions for Performing a Significance Test About a Difference Between Two Population Proportions: The Random Condition
The data must come from two independent random samples or from two groups in a randomized experiment.
Significance Tests about 𝑝 Conditions: The 10% Condition
When sampling without replacement, 𝑛 < 0.10𝑁.
Significance Tests for 𝜇 Conditions: The 10% Condition
When sampling without replacement, 𝑛 < 0.10𝑁.
The Conditions for Performing a Significance Test About a Difference Between Two Population Means: The 10% Condition
When sampling without replacement, 𝑛₁ < 0.10𝑁₁ and 𝑛₂ < 0.10𝑁₂.
The Conditions for Performing a Significance Test About a Difference Between Two Population Proportions: The 10% Condition
When sampling without replacement, 𝑛₁ < 0.10𝑁₁ and 𝑛₂ < 0.10𝑁₂.
Making a Conclusion in a Significance Test: When the 𝑃-value is Not Small
When the 𝑃-value is not small, fail to reject the null hypothesis (H₀) and conclude that there is not convincing evidence for the alternative hypothesis (Ha) [in context].
Making a Conclusion in a Significance Test: When the 𝑃-value is Small
When the 𝑃-value is small, reject the null hypothesis (H₀) and conclude that there is convincing evidence for the alternative hypothesis (Ha) [in context].
With P(Type I Error) and P(Type II Error), when one increases, the other ___.
With P(Type I Error) and P(Type II Error), when one increases, the other decreases. If we make it more difficult to reject the null hypothesis, H₀, by decreasing α, we increase the probability that we will not find convincing evidence for the alternative hypothesis, Ha, when it is true.