Module 4.1: Hypothesis Testing and Null Hypothesis Significance Testing
1. State the Null Hypothesis
(1) State the hypotheses and select an α level. The null hypothesis, H0, always states that the treatment has no effect (no change, no difference). (2) According to the null hypothesis, the population mean after treatment is the same is it was before treatment. · Always in null form
Test statistic
- Quantity based on sample data and used to test between null and alternative hypotheses § Statistic is used to test between the null and alternative hypotheses · Used as an evidence to decide and conclude o Rejecting the null o Retaining the null
Ho. Null hypothesis
- Statement regarding the value(s) of unknown parameter(s). Typically will imply no difference or relationship (implies equality - Null state) § What is being evaluated using the sample data § Statement regarding the unknown population parameter in null form § Refers that there is no difference/equal/there is no effect § Have to state in a null form § What is assumed to be true
Rejection region
- Values of the test statistic for which we reject the null in favor of the alternative hypothesis § How strong(threshold) should be the evidence be for us to decide to reject the null hypothesis
Two-tailed test
- the critical region is located at both sides
Inferential testing
- these are statistical techniques or procedures that we use to make inferences to the population based from the sample data
2 ways to make inferences of the Population:
1. Hypothesis Testing-· Also known as Statistical significance testing, Null Hypothesis Significance testing and etc. 2. Estimation- 1. Using the sample mean(statistic) as an estimate of the population parameter (possible values).
STEPS IN STATISTICAL TESTING:
1. State the Null Hypothesis 2. Determine Significance Level (Alpha)/ Locate Critical Region -Alpha level -Critical Region 3. Compute Test Statistic (And P-Value) 4.Decision/Conclusion and Interpretation
The purpose of the hypothesis test is to decide between two explanations:
1. The difference between the sample and the population can be explained by sampling error (does not appear to be a treatment effect) 2. The difference between the sample and the population is too large to be explained by sampling error (does appear to be a treatment effect).
Decision/Conclusion and Interpretation
A large value for the test statistic shows that the obtained mean difference is more than would be expected if there is no difference. If it is large enough to be in the critical region we conclude that the difference is significant or that the treatment has a significant effect. In this case we reject the null hypothesis. If the mean difference is relatively small, then the test statistic will have a low value. In this case, we conclude that the evidence from the sample is not sufficient, and the decision is fail to reject the null hypothesis. · We are not sure that the observed difference is not due to sampling error · The evidence is insufficient
Non-Parametric tests
Assumed Distribution: Any Typical Data: Nominal or Ordinal Data Usual Central measures: Median Benefits: Simplicity less affected by outliers
Parametric Tests
Assumed Distribution: Normal Typical Data: Ratio or interval Usual Central Measures: Mean Benefits: Can draw many conclusions assume the distribution is normally distributed while the non-parametric tests are not assuming any distribution (distribution free tests) desired because we can draw many conclusions and gives us more information
Inferences made on the population using sample data
Because we cannot study the whole population that's why we use sample data to make generalizations
Research Hypothesis
Claims about the relationships between variables. Happening in a population/general level in reality Ex: Stress decreases academic performance · Claiming the effect between stress and the decreased academic performance
Statistical Hypothesis
Claims about the unknown population parameter and different with the research hypothesis · Claims about the state of the parameter Ex: Students with Higher mean Stress levels will have lower mean grades
Compute Test Statistic (And P-Value)
Compute the test statistic. The test statistic forms a ratio comparing the obtained difference between the sample mean and the hypothesized population mean versus the amount of difference we would expect without any effect. · P-value < α = .05. · Reject the null · P-value > α = .05 · Retain the null
P values
Currently popular because it gives us the exact probabilities Represents Probability of getting a test statistic at least as extreme as the one calculated if the null is true Basically, the value of the probability of the statistic We Compare the Alpha value and the _______ to decide if we retain or reject the null
Elements of a hypothesis test:
Ho. Null hypothesis Ha. Alternative hypothesis Test statistic Rejection region
Statistical Significance
If we decide to reject the null hypothesis that means that the statistic is significant A statement in the research literature that the statistical test was significant indicates that the value of the calculated statistic warranted rejection of the null hypothesis o For a difference question, this suggests a real difference and not one due to sampling error o Whatever we observed are not likely due to sampling error o It means that the result (Statistic) is reliable
Uses for Inferential Statistics
Inferences made on the population using sample data Test claims about the population using evidence from sample data · Meaning with the sampling error the statistic can be different with the parameter . Statistics for determining differences between experimental and control groups in experimental research Statistics used in descriptive research when comparisons are made between different groups
Errors in Hypothesis Tests
Just because the the sample mean (following treatment) is different from the original population mean does not necessarily indicate that the treatment has caused a change. You should recall that there usually is some discrepancy between a sample mean and the population mean simply as a result of sampling error. Because the hypothesis test relies on sample data, and because sample data are not completely reliable, there is always the risk that misleading data will cause the hypothesis test to reach a wrong conclusion. Two types of error are possible. -Type 1 Error -Type 2 Error
The lower the alpha value the stricter the test is
Let's say that the alpha value is .05 or 5% is enough for us to say that it is improbable. Then we compute the statistic—its critical region would now be greater than 1.96 SD. Because these are the outcomes that occurs 5% of the time or less
Test claims about the population using evidence from sample data
Primary reason why we use inferential statistics Ex: Population level: Males and female students have difference in grades · However, we don't have the resources to gather the data and instead we move on to getting the sample from the population Sample Level: Getting samples of Male and Female students and Concluding that their scores are different · But we are not certain with the validity of our results because they are associated with the sampling error
Ha. Alternative hypothesis
Statement contradictory to the null hypothesis (will always contain an inequality) § Statement opposite to the null hypothesis § Refers that there is a difference, there is a relationship, there is an effect and they are not the same § Stated to mean that there is inequality § Usually the research hypothesis · However, the null hypothesis is what we evaluate in research
Hypothesis Testing
Stating a claim about the population parameter that claim is called as Hypothesis and we test it Ex: Population level: Males and female have difference in grades · Claim about the unknown population parameter
Primary reason why we use inferential statistics:
We cannot study the whole population that's why we use a sample. The population parameter is mostly unknown that's why we use the sample data to make generalizations about the population.
Directional Tests
When a research study predicts a specific direction for the treatment effect (increase or decrease), it is possible to incorporate the directional prediction into the hypothesis test. The result is called a directional test (2 tailed) or a one-tailed test. A directional test includes the directional prediction in the statement of the hypotheses and in the location of the critical region. Determines the specific direction for the difference in the statistic
The differences are also normally distributed
and because of that we know the probability of us obtaining a particular value o 95%= we can obtain a particular value different from the mean within 2SD § High probability values
Rationale behind Statistical tests:
basing on known distributions like normal distributions. o Known because We know the Properties, Areas and probabilities associated the Normal Distribution o Making inferences based on the normal distribution If We take different samples from the population and calculate their means, it will approximate in a normal distribution o Instead of raw data, we have sample means as scores in the distribution
The α level establishes a criterion, or "cut-off", for making a decision about the null hypothesis. The alpha level also determines the risk of a Type I error.
i. α level also known as the significance level ii. Establishes a criterion for making a decision about the null hypothesis iii. Determines our threshold or risk tolerance—how much error you are willing to accept when you decide to reject the null hypothesis iv. Set by the researcher depending on the purpose (relaxed or strict)
a. Locate the critical region. The critical region consists of outcomes that are very unlikely to occur if the null hypothesis is true.
is defined by sample means that are almost impossible to obtain if the treatment has no effect. The phrase "almost impossible" means that these samples have a probability that is less than the alpha level. i. The tails of the distribution ii. Almost impossible (P < α) iii. Values within 2SD=High Probability values(Values that are within the Ho) iv. We have to decide what is the α value
Meaning with the sampling error the statistic can be different with the parameter
o If in the population level: there are no difference in grades between men and women students o Sample level: there is a difference § Different because of the sampling error Ex: Pre-Surveys in election: Who will you vote for this coming election? (came from a sample) · Candidate A=60% But this can be different with the results of the population because of the sampling error We use inferential statistic to make inferences to the population while accounting for sampling error
Retain H0 when P > α
o If the P value is larger than the alpha value, the difference can be due to the sampling error, therefore retaining the null o The evidence is not strong to reject the null hypothesis
Reject H0 when P ≤ α
o If the P value is within the threshold or equal, we can say that the difference is not due to sampling error and reject the null o smaller and smaller P-values provide stronger and stronger evidence against H0
The purpose of the statistical test is to evaluate the null hypothesis (H0) at a specified level of significance (e.g., p < .05)
o In other words, do the means (among others) differ significantly so that these differences would be attributable to chance occurrence less than 5 times in 100? · Ex: In a claim that means are different in a population o Do the means differ significantly o Are they reliable and not likely due to sampling error?
One tailed test
o One Mean is less than the other o Depending to the hypothesized mean where the critical region lies (either (+) or (-))
Type I error
occurs when the sample data appear to show a difference (effect) when, in fact, there is none. In this case the researcher will reject the null hypothesis and falsely conclude that there is a difference (effect). The hypothesis test is structured so that Type I errors are very unlikely; specifically, the probability of a Type I error is equal to the alpha level. · The higher you set your alpha level, the higher the risk you are willing to take and vice versa · Type 1 Error = α
Type II error
occurs when the sample does not appear to have a difference (effect), in fact, there is a difference (effect). In this case, the researcher will fail to reject the null hypothesis and falsely conclude that there is no difference (effect). · Failing to reject the null hypothesis Type 2 Error = β
Type II errors are commonly the result ______________
of a very small treatment effect. Although the treatment does have an effect, it is not large enough to show up in the research study. · Small differences are not likely to be detected by the tests
· Note that the statistic and parameter have naturally occurring difference between them called as the ________
sampling error · Something that is always present
Type I errors are caused by _____________
unusual, unrepresentative samples. Just by chance the researcher selects an extreme sample with the result that the sample falls in the critical region even though the treatment has no effect.
The general goal of a hypothesis test is to rule out chance (sampling error) as a plausible explanation for the results from a research study.
· The difference can be due to the sampling error and the goal is to eliminate that through testing · We are cautious in trusting the statistic without testing it.
In experiments, hypothesis testing is a technique to help determine whether a specific treatment has an effect on the individuals in a population.
· What we are making on are samples: · Ex: Findings o Experimental: Improved o Control group: Did not improved Are the findings due to the interaction of variables(IV affected the DV) or due to the sampling error? Goal: Make statement(s) regarding unknown population parameter values based on sample data
Significance - Conventions
· α = .05 , .01, <.001(lowest significant level that we use) o Most common alpha value in Psychology: 0.5 or 5% o .001(lowest significant level that we use) · P> α = not significant evidence · P< α = significant evidence Example · Set α = .05. Find P = 0.27 Þ retain H0 · Example: Set α = .01. Find P = .001 Þ reject H0
Critical Region values:
α = .05(beyond 1.96SD) α = .01(beyond 2.58SD) α = .001(beyond 3.30 SD)