Module 4.1: Hypothesis Testing and Null Hypothesis Significance Testing

1. State the Null Hypothesis

(1) State the hypotheses and select an α level. The null hypothesis, H0, always states that the treatment has no effect (no change, no difference). (2) According to the null hypothesis, the population mean after treatment is the same is it was before treatment. · Always in null form

Test statistic

- Quantity based on sample data and used to test between null and alternative hypotheses § Statistic is used to test between the null and alternative hypotheses · Used as an evidence to decide and conclude o Rejecting the null o Retaining the null

Ho. Null hypothesis

- Statement regarding the value(s) of unknown parameter(s). Typically will imply no difference or relationship (implies equality - Null state) § What is being evaluated using the sample data § Statement regarding the unknown population parameter in null form § Refers that there is no difference/equal/there is no effect § Have to state in a null form § What is assumed to be true

Rejection region

- Values of the test statistic for which we reject the null in favor of the alternative hypothesis § How strong(threshold) should be the evidence be for us to decide to reject the null hypothesis

Two-tailed test

- the critical region is located at both sides

Inferential testing

- these are statistical techniques or procedures that we use to make inferences to the population based from the sample data

2 ways to make inferences of the Population:

1. Hypothesis Testing-· Also known as Statistical significance testing, Null Hypothesis Significance testing and etc. 2. Estimation- 1. Using the sample mean(statistic) as an estimate of the population parameter (possible values).

STEPS IN STATISTICAL TESTING:

1. State the Null Hypothesis 2. Determine Significance Level (Alpha)/ Locate Critical Region -Alpha level -Critical Region 3. Compute Test Statistic (And P-Value) 4.Decision/Conclusion and Interpretation

The purpose of the hypothesis test is to decide between two explanations:

1. The difference between the sample and the population can be explained by sampling error (does not appear to be a treatment effect) 2. The difference between the sample and the population is too large to be explained by sampling error (does appear to be a treatment effect).

Decision/Conclusion and Interpretation

A large value for the test statistic shows that the obtained mean difference is more than would be expected if there is no difference. If it is large enough to be in the critical region we conclude that the difference is significant or that the treatment has a significant effect. In this case we reject the null hypothesis. If the mean difference is relatively small, then the test statistic will have a low value. In this case, we conclude that the evidence from the sample is not sufficient, and the decision is fail to reject the null hypothesis. · We are not sure that the observed difference is not due to sampling error · The evidence is insufficient

Non-Parametric tests

Assumed Distribution: Any Typical Data: Nominal or Ordinal Data Usual Central measures: Median Benefits: Simplicity less affected by outliers

Parametric Tests

Assumed Distribution: Normal Typical Data: Ratio or interval Usual Central Measures: Mean Benefits: Can draw many conclusions assume the distribution is normally distributed while the non-parametric tests are not assuming any distribution (distribution free tests) desired because we can draw many conclusions and gives us more information

Inferences made on the population using sample data

Because we cannot study the whole population that's why we use sample data to make generalizations

Research Hypothesis

Claims about the relationships between variables. Happening in a population/general level in reality Ex: Stress decreases academic performance · Claiming the effect between stress and the decreased academic performance

Statistical Hypothesis

Claims about the unknown population parameter and different with the research hypothesis · Claims about the state of the parameter Ex: Students with Higher mean Stress levels will have lower mean grades

Compute Test Statistic (And P-Value)

Compute the test statistic. The test statistic forms a ratio comparing the obtained difference between the sample mean and the hypothesized population mean versus the amount of difference we would expect without any effect. · P-value < α = .05. · Reject the null · P-value > α = .05 · Retain the null

P values

Currently popular because it gives us the exact probabilities Represents Probability of getting a test statistic at least as extreme as the one calculated if the null is true Basically, the value of the probability of the statistic We Compare the Alpha value and the _______ to decide if we retain or reject the null

Elements of a hypothesis test:

Ho. Null hypothesis Ha. Alternative hypothesis Test statistic Rejection region

Statistical Significance

If we decide to reject the null hypothesis that means that the statistic is significant A statement in the research literature that the statistical test was significant indicates that the value of the calculated statistic warranted rejection of the null hypothesis o For a difference question, this suggests a real difference and not one due to sampling error o Whatever we observed are not likely due to sampling error o It means that the result (Statistic) is reliable

Uses for Inferential Statistics

Inferences made on the population using sample data Test claims about the population using evidence from sample data · Meaning with the sampling error the statistic can be different with the parameter . Statistics for determining differences between experimental and control groups in experimental research Statistics used in descriptive research when comparisons are made between different groups

Errors in Hypothesis Tests

Just because the the sample mean (following treatment) is different from the original population mean does not necessarily indicate that the treatment has caused a change. You should recall that there usually is some discrepancy between a sample mean and the population mean simply as a result of sampling error. Because the hypothesis test relies on sample data, and because sample data are not completely reliable, there is always the risk that misleading data will cause the hypothesis test to reach a wrong conclusion. Two types of error are possible. -Type 1 Error -Type 2 Error

The lower the alpha value the stricter the test is

Let's say that the alpha value is .05 or 5% is enough for us to say that it is improbable. Then we compute the statistic—its critical region would now be greater than 1.96 SD. Because these are the outcomes that occurs 5% of the time or less

Test claims about the population using evidence from sample data

Primary reason why we use inferential statistics Ex: Population level: Males and female students have difference in grades · However, we don't have the resources to gather the data and instead we move on to getting the sample from the population Sample Level: Getting samples of Male and Female students and Concluding that their scores are different · But we are not certain with the validity of our results because they are associated with the sampling error

Ha. Alternative hypothesis

Statement contradictory to the null hypothesis (will always contain an inequality) § Statement opposite to the null hypothesis § Refers that there is a difference, there is a relationship, there is an effect and they are not the same § Stated to mean that there is inequality § Usually the research hypothesis · However, the null hypothesis is what we evaluate in research

Hypothesis Testing

Stating a claim about the population parameter that claim is called as Hypothesis and we test it Ex: Population level: Males and female have difference in grades · Claim about the unknown population parameter

Primary reason why we use inferential statistics:

We cannot study the whole population that's why we use a sample. The population parameter is mostly unknown that's why we use the sample data to make generalizations about the population.

Directional Tests

When a research study predicts a specific direction for the treatment effect (increase or decrease), it is possible to incorporate the directional prediction into the hypothesis test. The result is called a directional test (2 tailed) or a one-tailed test. A directional test includes the directional prediction in the statement of the hypotheses and in the location of the critical region. Determines the specific direction for the difference in the statistic

The differences are also normally distributed

and because of that we know the probability of us obtaining a particular value o 95%= we can obtain a particular value different from the mean within 2SD § High probability values

Rationale behind Statistical tests:

basing on known distributions like normal distributions. o Known because We know the Properties, Areas and probabilities associated the Normal Distribution o Making inferences based on the normal distribution If We take different samples from the population and calculate their means, it will approximate in a normal distribution o Instead of raw data, we have sample means as scores in the distribution

The α level establishes a criterion, or "cut-off", for making a decision about the null hypothesis. The alpha level also determines the risk of a Type I error.

i. α level also known as the significance level ii. Establishes a criterion for making a decision about the null hypothesis iii. Determines our threshold or risk tolerance—how much error you are willing to accept when you decide to reject the null hypothesis iv. Set by the researcher depending on the purpose (relaxed or strict)

a. Locate the critical region. The critical region consists of outcomes that are very unlikely to occur if the null hypothesis is true.

is defined by sample means that are almost impossible to obtain if the treatment has no effect. The phrase "almost impossible" means that these samples have a probability that is less than the alpha level. i. The tails of the distribution ii. Almost impossible (P < α) iii. Values within 2SD=High Probability values(Values that are within the Ho) iv. We have to decide what is the α value

Meaning with the sampling error the statistic can be different with the parameter

o If in the population level: there are no difference in grades between men and women students o Sample level: there is a difference § Different because of the sampling error Ex: Pre-Surveys in election: Who will you vote for this coming election? (came from a sample) · Candidate A=60% But this can be different with the results of the population because of the sampling error We use inferential statistic to make inferences to the population while accounting for sampling error

Retain H0 when P > α

o If the P value is larger than the alpha value, the difference can be due to the sampling error, therefore retaining the null o The evidence is not strong to reject the null hypothesis

Reject H0 when P ≤ α

o If the P value is within the threshold or equal, we can say that the difference is not due to sampling error and reject the null o smaller and smaller P-values provide stronger and stronger evidence against H0

The purpose of the statistical test is to evaluate the null hypothesis (H0) at a specified level of significance (e.g., p < .05)

o In other words, do the means (among others) differ significantly so that these differences would be attributable to chance occurrence less than 5 times in 100? · Ex: In a claim that means are different in a population o Do the means differ significantly o Are they reliable and not likely due to sampling error?

One tailed test

o One Mean is less than the other o Depending to the hypothesized mean where the critical region lies (either (+) or (-))

Type I error

occurs when the sample data appear to show a difference (effect) when, in fact, there is none. In this case the researcher will reject the null hypothesis and falsely conclude that there is a difference (effect). The hypothesis test is structured so that Type I errors are very unlikely; specifically, the probability of a Type I error is equal to the alpha level. · The higher you set your alpha level, the higher the risk you are willing to take and vice versa · Type 1 Error = α

Type II error

occurs when the sample does not appear to have a difference (effect), in fact, there is a difference (effect). In this case, the researcher will fail to reject the null hypothesis and falsely conclude that there is no difference (effect). · Failing to reject the null hypothesis Type 2 Error = β

Type II errors are commonly the result ______________

of a very small treatment effect. Although the treatment does have an effect, it is not large enough to show up in the research study. · Small differences are not likely to be detected by the tests

· Note that the statistic and parameter have naturally occurring difference between them called as the ________

sampling error · Something that is always present

Type I errors are caused by _____________

unusual, unrepresentative samples. Just by chance the researcher selects an extreme sample with the result that the sample falls in the critical region even though the treatment has no effect.

The general goal of a hypothesis test is to rule out chance (sampling error) as a plausible explanation for the results from a research study.

· The difference can be due to the sampling error and the goal is to eliminate that through testing · We are cautious in trusting the statistic without testing it.

In experiments, hypothesis testing is a technique to help determine whether a specific treatment has an effect on the individuals in a population.

· What we are making on are samples: · Ex: Findings o Experimental: Improved o Control group: Did not improved Are the findings due to the interaction of variables(IV affected the DV) or due to the sampling error? Goal: Make statement(s) regarding unknown population parameter values based on sample data

Significance - Conventions

· α = .05 , .01, <.001(lowest significant level that we use) o Most common alpha value in Psychology: 0.5 or 5% o .001(lowest significant level that we use) · P> α = not significant evidence · P< α = significant evidence Example · Set α = .05. Find P = 0.27 Þ retain H0 · Example: Set α = .01. Find P = .001 Þ reject H0

Critical Region values:

α = .05(beyond 1.96SD) α = .01(beyond 2.58SD) α = .001(beyond 3.30 SD)