Hypothesis Testing

Research hypothesis (H1)

A substantive hypothesis expressed in terms of population parameters. • We use our sample statistics to make statements about population parameters. • H1 states that there is a difference between the two groups. (American workers differ in the number of hours they work compared to the general population (workers in industrialized nations).

Testing the Null Hypothesis

Rather than testing H1 directly we test the null hypothesis H0 (there is no real difference in the number of hours worked) because we hope to reject the null hypothesis in order to provide support for the research hypothesis.This is why H1 is also referred to as the "alternative hypothesis". If we reject the H0 , we conclude that there is a real difference

Statistical Significance

Remember, there are two ways to assess statistical significance in hypothesis testing. •With our Z obtained, we determine its probability of occurrence in a normal distribution and compare P and alpha values: •If P ≤ α, then we reject the null hypothesis and conclude that the difference is statistically significant. •Or we compare our Z obtained with a Z critical: •If the absolute value of the Z (obtained) ≥ critical value, then we reject the null hypothesis and conclude that the difference is statistically significant.

P value

•The probability of getting a sample mean of 43 hours if H0 is true is less than 0.0001. •If there is no difference in the sample of American workers (43 hours) from the population mean (40 hours), then this sample mean would be extremely unlikely (less than 1 out of 1000 sample means).

Testing the Significance of the Difference Between Two Sample Proportions

•The procedure to test for significant differences between two samples for estimating population proportions (or percentages) is identical. •Use when the dependent variable is measured at a nominal or ordinal level (dichotomous variables). •π1 and π2 are the proportions for population 1 and population 2 •p1 and p2 are the proportions for sample 1 and sample 2

Testing the Significance of the Difference Between Two Sample Proportions

•The procedure to test for significant differences between two samples for estimating population proportions (or percentages) is identical. •Use when the dependent variable is measured at a nominal or ordinal level (dichotomous variables). •π1 and π2 are the proportions for population 1 and population 2 •p1 and p2 are the proportions for sample 1 and sample 2 Hypotheses •Hypotheses are stated in terms of population notation. •H0: π1 = π2 •H1: π1 ≠ π2 •H1: π1 > π2 •H1: π1 < π2

2. State the hypotheses and select alpha

•The research question only asks if the sample proportion is different from the population proportion. •Two-tailed test •H0: πUALR female students = 0.38 •H1: πUALR female students ≠ 0.38 •Set alpha = 0.10

4. Compute the Test Statistic

•This is the information for our two samples: Middle-class N1 = 412 p1 = 0.50 Working class N2= 360 p2 = 0.46 •Compute the standard error 4. Compute the Test Statistic 1080.10361.004.00361.046.050.02121ppSppZ What is the P value for a Z obtained of 1.11? Based on our α (0.10) and the P value, what would you conclude? Can we reject H0? Is there any other way to determine the statistical significance of our Z-test?

Statistical Significance

•Typically, researchers hope to find differences that are statistically significant. •This may confirm our hypotheses about differences across groups in society, such as gender, race/ethnicity, age, political affiliation, etc. •In substantive terms, how important are statistically significant differences? •As a researcher, you must determine if the magnitude or the size of the difference has a theoretical or practical importance.

1. Make assumptions

•We have a random sample. •Determine level of measurement of dependent variable. •To test proportions, we must have ordinal or nominal-level dependent variable (dichotomous). •In this example, the dependent variable is measured at an ordinal level. •Sampling distribution •We have a large sample size and based on the CLT, we can assume that the sampling distribution of proportions is normally distributed.

1. Making assumptions

•We have a random sample. •The level of measurement of the variable is interval ratio (SAT score), which is appropriate for testing the difference in means. •Sampling distribution is normal. •Even if we didn't have a normally distributed score, our sample size is large enough to apply the Central Limit Theorem.

Hypothesis test for one-sample proportion

•We have been estimating a population mean using one sample and comparing it to a known population mean. •We can also use a sample proportion and compare it to a population proportion. •We will follow a 5-step procedure.

Magnitude of the Difference

•We must decide if this small difference is important and has practical implications. •Example 1: A researcher finds that making a slight change in one's diet creates a very small decrease in the side effects from chemotherapy. •Is this small difference of practical importance? •Yes! Those experiencing side effects would probably find any degree of relief to be very important, particularly since it involves a very modest change in diet.

No Statistically Significant Results May Also be Important

•We usually expect to find differences between groups, but sometimes there is no significant difference. •This finding can also be important •Example: We hypothesize that SAT scores for poor African Americans in Chicago are lower than SAT scores for poor Whites (perhaps due to lack of access to quality schools, racial discrimination, etc.) •We find no significant difference in the SAT test scores across racial groups. •This may be an important finding! This may help us to understand that poverty is a problem for both groups, not differences in educational opportunities associated with their race.

Ethnicity and SAT Scores

•We want to study the relationship between ethnicity and SAT scores among high school graduates. •In particular, we are interested in comparing writing scores of Hispanics to writing scores of all H.S. students. •We have data for ALL high school students in Arkansas and know that the average SAT writing score in 2009 was 550, with SD = 100. •We have a sample of N=768 Hispanic students, and we calculate their mean SAT writing score to be 561.

4. Computing the test statistic

•What are the chances that we would have randomly selected a sample of Hispanics such that the sample's mean SAT score (561) was different than the average SAT score for all high school students in Arkansas (550)? •We determine the probability that the sample was different due to chance error. •We know the population (all high school students) average SAT score, and the population standard deviation: •μ = 550 •σ = 100 •First, we need to estimate the standard error •SE = 3.61 •Then, we convert our sample mean into a Z score •By converting the sample mean to a Z score, we obtained the Z statistic . •Our Z statistic for the Hispanics sample mean is 3.05. •Now we determine the probability of observing a Z statistic of 3.05 assuming that the null hypothesis is true. •What's the probability of getting a sample mean that is 3.05 standard errors away from the true population mean? •In the standard normal table, we find the area to the right of a Z = 3.05. •This probability is known as the P value •P = 0.0011

3. Select the Sampling Distribution and the Test Statistic

•Which test statistic? •To estimate differences of proportions (or percentages) between two samples, we use the Z-statistic. •Normal distribution

Null hypothesis (H0)

Contradicts the research hypothesis and usually states that there is no difference between the population mean and some specified value. H0 : μy = some value (In our example: There is no difference between the number of hours worked per week by American workers compared to the general population.)

One-Tailed Tests

H1 is directional, specifying that the population mean is greater than or less than some specified value. • If we state that the population mean is greater than some specified value, it is a right-tailed test. => H1 : μy > some value • If we state that the population mean is less than some specified value, it is a left-tailed test. => H1 : μy < some value

Rejection Level and Decision-Rules

The decision to reject the null hypothesis is based on the alpha level. Alpha is usually set at the 0.05, 0.01, or 0.001 level. Alpha of 0.05 means that there is a 5 in 100 probability of observing a test statistic of that size due to chance. We are confident that 95% of the time we can reject the null hypothesis without making an error. If the obtained probability (P) ≤ alpha , reject HO (null hypothesis) Our differences are statistically significant. If the obtained probability (P) > alpha , fail to reject HO Our differences are not statistically significant. The relationship between the p-value and alpha allow us to make statements regarding significance of the results. American works and the general population. -The difference is statistically significant. It's not just due to random chance alone. •If there is no difference in the sample of American workers (43 hours) from the population mean (40 hours), then this sample mean would be extremely unlikely (less than 1 out of 1000 sample means). -Thus, we can reject this explanation. -There really is a difference in the mean number of hours American works and the general population. -The difference is statistically significant. It's not just due to random chance alone.

Errors in Hypothesis Testing

Two kinds of errors - Type I and Type II Type I error: Probability associated with rejecting a null hypothesis when it is true. We conclude that our sample mean is different of the population mean when in reality it is not. H0 should not have been rejected. The probability of a Type I error = alpha () Type II error: Probability of failing to reject a null hypothesis when it is false. We conclude that there is no difference when in reality there is one. H0 should have been rejected.

Directional versus Non-Directional Research Hypotheses

Two-Tailed Tests One-Tailed Tests

Errors in Hypothesis Testing

Type I error: Probability associated with rejecting a null hypothesis when it is true. We conclude that our sample mean is different of the population mean when in reality it is not. H0 should not have been rejected. The probability of a Type I error = alpha () Type II error: Probability of failing to reject a null hypothesis when it is false. We conclude that there is no difference when in reality there is one. H0 should have been rejected.

Hypothesis Testing

We evaluate hypotheses by using sample statistics about population parameters and all statistical tests assume "random sampling."

For tests of hypotheses about means

We need: • A variable measured at an interval-ratio level • A variable that is normally distributed, or • A sample size larger than 50 to apply the central limit theorem

Two-Tailed Tests

When the researcher is interested in testing if the population mean is different from a specified value. H1 does not designate direction. • We have no theoretical basis to specify a direction, but we do believe there is some difference between the population mean and a specified value. • H1 : μy ≠ some value • H0 (The null hypothesis) is no difference. • H0 : μy = some value

The Process of Hypothesis Testing

• Ask a research question • Formulate a hypothesis = tentative answer based on theory or informed reasoning ("Tentative" because the hypothesis can only be verified after empirical testing)

Forms of a Research Hypothesis

• H1 usually specifies that the population parameter is: • Not equal to some specified value: μY ≠ some value • Greater than some specified value: μY > some value • Less than some specified value: μY < some value

Directional hypotheses (one-tailed)

• Null hypothesis • American workers work the same number of hours per week than workers in industrialized countries. => H0 : μy = 40 hrs • Research hypotheses • American workers work more hours per week than workers in industrialized countries (right-tailed test) => H : μ > 40 hrs • American workers work less hours per week than workers in industrialized countries (left-tailed test) => H1 : μy < 40 hrs **When in doubt, use a non-directional hypothesis. **

Non-directional hypotheses (two-tailed)

• Null hypothesis • American workers work the same number of hours per week than workers in industrialized countries. => H0 : μy = 40 hrs • Research hypothesis • American workers do not work the same number of hours per week than workers in industrialized countries. => H : μ ≠ 40 hrs

we formulate two hypotheses

• Research hypothesis (H1) • Null hypothesis (H0)

Z statistic

• The obtained Z is the number of standard errors that our sample is from Y , assuming the null hypothesis is true. • What is the probability of finding this Z statistic? • The probability corresponds to the area beyond Z. In this example, the area is on the right tail of the curve. • p < 0.0001

One-Sample Proportion

•According to the Arkansas Higher Education Board, 37.6% of college students graduate within a six year period after entering college. •A sample of 303 female students in UALR revealed that 40.5% finished college after six years. •Is the proportion of UALR female students who are finishing college different from the proportion of Arkansas college students?

5. Making a decision about the null hypothesis and interpreting our results

•Because the P value is < alpha (P = 0.0022), we reject the null hypothesis and conclude that the mean SAT score of Hispanics is different from the average SAT score for all high school students, and this difference is statistically significant.

3. Select the sampling distribution and the test statistic

•Because we are estimating a one-sample proportion, we use the Z statistic and normal sampling distribution.

Example: How important is religion in our life?

•Do people in different social classes perceive the importance of religion in the same way? •Using the World Values Survey 2006, we find that 50.1% of people in the middle class and 45.8% of working-class people reported religion as being very important in their life. •We will test if the difference in the percentage of people reporting religion as very important in their life is statistically significant.

Magnitude of the Difference

•Example 2: A researcher finds that a very expensive regimen of drugs taken daily reduces the chances of developing prostate cancer in young men by half. •Is this difference of practical importance? •Perhaps not. Say the chances of developing prostate cancer as a young man are 2 in 1,000,000. If you take the very expensive drugs, your chances drop to 1 in 1,000,000. With such a small chance to begin with, is it worth the time and money to try and decrease it?

2. State the Hypotheses and Select Alpha

•H0: There is no difference in the importance of religion between people in the middle-class and working-class people. •H0: πmiddle-class = πworking-class •H1: The importance of religion is different between people from middle- and working classes. •H1: πmiddle-class ≠ πworking-class •Two-tailed test because H1 is non-directional. •We set alpha at 0.10

Testing the Null Hypothesis with a Two-Tailed Test

•In a two-tailed test, outcomes may be located at both the right and left tails of the sampling distribution. •The null hypothesis will be rejected if our mean falls either at the left or right tail. •A .05 alpha means that H0 will be rejected if our mean falls among either the lowest or the highest 5% of the sampling distribution. •The obtained Z is calculated using the same formula. •To find P for a two-tailed test, look up the area in Column C of Appendix B that corresponds to the obtained Z, and multiply the P value by 2. •Because this is a two-tailed test, multiply the P value by 2: •P = 2 * 0.0011 = 0.0022 •The P value shows the probability of getting a sample mean that is as extreme as the one we got for Hispanics •P value < alpha •Is 0.0022 < 0.05?

Testing the Null Hypothesis

•In our example, the sample mean falls in the shaded area of the sampling distribution (left tail). •If there is no difference in the sample of American workers (43 hours) from the population mean (40 hours), then this sample mean would be extremely unlikely (less than 1 out of 1000 sample means). -Thus, we can reject this explanation. -There really is a diTesting the Null Hypothesis •We test the null hypothesis H0 -H0 states that there is no real difference between our sample statistic and the population parameter. •We hope to reject the null hypothesis in order to provide support for the research hypothesis. •If we find that the probability that the null hypothesis is true is very small (< 5%, for example). -We conclude that H0 is probably not true and we reject it. -Hence, there is a real difference between our sample statistic and the population parameter. -And the difference is statistically significant.fference in the mean number of hours

5. Make a decision about the null hypothesis and interpret the results

•Is 0.2802 < 0.10? •No, P > α, we fail to reject the null hypothesis. •We conclude that there is not a significant difference between the proportion of UALR female students finishing college in Arkansas and the state as a whole.

2. Stating the hypotheses and selecting alpha

•Is this a one- or two-tailed test? •Two-tailed, because we don't specify a direction. •We are just saying that Hispanics have different SAT scores than all high school students in Arkansas.

Steps in Hypothesis Testing

•Making assumptions •Stating the research and null hypotheses, and selecting alpha •Selecting the sampling distribution and specifying the test statistic •Computing the test statistic •Making a decision and interpreting the results

Type I and Type II Errors

•Minimize the Type I error (chance of rejecting the null hypothesis when it is true) by limiting the alpha () level. • = 0.01 means that you are willing to limit the probability of rejecting null hypothesis when it is true (type I error) to 1 out of 100 times. •In non-statistical terms, an alpha level of .01 means that we have a 1% chance of being wrong. •Type I and Type II errors are inversely related. If we reduce alpha and lower the risk of making Type I error, we increase the risk of making a Type II error.

Alpha-Level in Hypothesis Testing

•Minimize the Type I error (chance of rejecting the null hypothesis when it is true) by limiting the alpha () level. • alpha= 0.01 means that you are willing to limit the probability of rejecting null hypothesis when it is true (type I error) to 1 out of 100 times. •In non-statistical terms, an alpha level of .01 means that we have a 1% chance of being wrong.

α and P value

•P value is the probability associated with the obtained test statistic. •It is a measure of how unusual or rare our obtained statistic is. •Since we want to reject H0, we need small p values. •We also need to define a cut-off point. If p falls below this cut-off point, we reject the null hypothesis. • α is our cut-off point, or level of significance at which the null hypothesis is rejected. •α represents the level of risk we are willing to take in rejecting H0 if H0 is true. •α is set in advance by the researcher.

Sampling Distribution and Test Statistic for Two-Sample Proportions

•Population distributions of proportions are normal. •We use the normal distribution and the Z statistic. •The standard error has to include both proportions.

1. Make assumptions

•Random sampling and independent samples •Sample size larger than 50 •Normal sampling distribution •Original variable (importance of religion in your life) is measured at an ordinal level (very important, fairly important, not very important, no opinion) •We create a dichotomous nominal variable and estimate the percentage that say "religion is very important".

Two ways to determine significance

•So far, we: •Decide on an alpha level. •Calculate the Z statistic (obtained) and find the P value associated with it. •Compare the P value to the alpha level to determine if the result is statistically significant. •We could take a shortcut too •Decide on an alpha level. •Calculate the Z statistic (obtained). •Compare the obtained Z statistic with the Z statistic that corresponds to our alpha. •Ex: For alpha = 0.05, the Z statistic is 1.96 for a 2-tailed test. •If we calculate a Z ≥ 1.96, then we know it is statistically significant. •This Z statistic is called the critical value. •If P ≤ α, then we reject the null and the difference is statistically significant. •If the absolute value of the Z we estimated is ≥ critical value, then the difference is statistically significant.

4. Compute the test statistic

•The Z statistic for proportions: •Where is the standard error of the population proportion: •Look for 1.08 in the Standard Normal Table (Column C, Appendix B) •This corresponds to a P value = 0.1401 •Because this is a two-tailed test, multiply this P value by 2 •P = 2 * 0.1401 = 0.2802

2. Stating the hypotheses and selecting alpha

•The null hypothesis says that there is no difference between SAT scores of Hispanics and SAT scores of all high school students. H0: μHispanics = 550 •The alternative hypothesis says that SAT scores of Hispanics are different from scores for the general population. H1: μHispanics ≠ 550 •Alpha = 0.05

3. Selecting the sampling distribution and the test statistic

•The population SD is known and we have a large sample size, so we use the normal distribution and the Z statistic.