SIP 7

Two Caveats of the t test for Dependent Means

1. In order to compress two scores into one score per person, a difference score (also known as a change score) must be generated by subtracting one score from the other, this will give you the amount of change or difference between the first score and the second score Change Score = [Time 2/Second/After] Score - [Time 1/First/Before] Score Difference scores are NOT the same as deviation scores! They are the difference between the first condition and the second! 2. The population mean of the difference scores is 0: if mean of the population is 0, this is essentially saying there is no difference between the two sets of scores μMDifference = μ2 = 0

8 people watch the Super Bowl game. Their scores on an excitement scale are 14, 8, 6, 13, 10, 10, 5, 6 (1= not at all to 20=extremely excited). People in general have a mean of 6.00. Using a .05 significance level, does watching the Super Bowl game make a difference in excitement?

1. P1: People who watch a Super Bowl game. P2: People in general. H1: μ1 ≠ μ2 H0: μ1 = μ2 2. μM = μ = 6.00 S2 = SS/N - 1 = 78.00/ (8 - 1) = 11.14 S2M = S2/N = 11.14/8 = 1.39 SM = √ S2M = √1.39 = 1.18 Shape = t distribution with df = 7 (8 - 1) 3. t cutoff for two-tailed with df of 7 = ±2.37 4. t = (m - μM)/SM = (9.00 - 6.00)/1.18 = 2.54 5. Our t score of 2.54 exceeds the cutoff t score of 2.37. We can reject the null hypothesis. Our research hypothesis is supported. Watching a Super Bowl game does have an effect on excitement. People who watch a Super Bowl game are more excited than people in general. t(df) = t-score, p < .05 if significant, OR t(df) = t-score, n.s. if non-significant Our results in APA format: t(7) = 2.54, p < .05

Conducting a t test for a single sample

1. Restate the question as null and research hypotheses: Population 1, Population 2, Research hypothesis and Null hypothesis 2. Determine the characteristics of the comparison distribution: a. μ (mean of the distribution of means) b. SM (standard deviation of the distribution of means) c. M (sample mean) d. S2 (estimated population variance), S2M (variance of the distribution of means) e. Shape, df (degrees of freedom), 3. Determine the sample score on the comparison distribution for which the null hypothesis should be rejected (Get the cutoff t scores!) 4. Determine your sample's score on the comparison distribution: t = M - μM / SM 5. Decide whether to accept or reject the null

The Five Steps of a t test for Dependent Means

1. Restate the question as the research and null hypothesis Population 1: people who receive experimental treatment Population 2: people who show no change (no difference) Research Hypothesis: μ1 ≠ μ2 Null Hypothesis: μ1 = μ2 2. Determine the characteristics of the comparison distribution Assume the mean of the distribution of means of difference scores is 0 (μDifference = 0) Figure the difference score for each participant. After you find this score forget about your before and after scores. Only focus on the different scores. From here its nearly the same as the t tests that we have been doing Figure the mean of the difference scores (MDifference) Figure the deviation scores and find SS (sum of squaresDifference) Find the SD of the distribution of means of difference scores using the formulas below: a. S2Difference = Σ (X - M)2 / N - 1 OR SS / N - 1 Here, "X" is the difference score for each person b. S2MDifference = S2/N c. SMDifference = √S2M 3. Determine the cutoff scores on the comparison distribution for which the null hypothesis should be rejected Use t table on Moodle or in the back of your textbook: find your degrees of freedom (df = N-1), select one or two-tailed test, select appropriate significance level (p =.10, .05, .01) - Use 5% for our PSY2510 class! 4. Determine your sample's score on the comparison distribution: t = M - μM / SM μm is always zero for a t test for dependent means. 5. Decide whether to accept or reject the null hypothesis/ Same as previous t test conclusions

When we know both μ and σ, the shape of our comparison distribution is approximately normal if:

1. The comparison distribution follows a normal curve, or 2. There are more than 30 participants

t test for Dependent means (paired sample t test)

A t test for dependent means (SPSS: "paired samples t test") is used when you do not have the population mean or standard deviation, but you have two sets of scores from a group of people It's called "t test for dependent means" because the means for each set of scores are dependent on each other We call this type of research design a within-subjects design or a repeated measures design A t test for dependent means doesn't always have to come from the same person (for before and after testing, time 1/time 2 testing etc.), it could be from researcher matched pairs (husband/wife, twins) Then you get the difference scores from the 2 sets of scores. After you get the set of difference scores (After - Before), you run a t test for single sample. One difference between t test of single sample and t test for dependent means in the 2nd step: In t test of single sample: µm = µ In t test for dependent means: µm = 0 Not "People in general" anymore, instead it's "People who show no change"

t test for single sample = One Sample t test

A t test is a hypothesis testing procedure For all the hypothesis testing we've done so far, we've had to have three things given to us: the population mean, standard deviation/variance, and shape In a t test, we don't have the population variance, thus we need to estimate it from the sample scores (the variance of the scores in the sample should be similar to the variance of the scores in the population): a. This is a biased estimate of the population variance (dividing by N) because the sample's variance is consistently slightly smaller than the population's variance, so we have to identify an unbiased estimate of the population variance. b. Population estimated variance: S2 = Σ (X - M)2/(N - 1) OR SS/(N - 1) (estimated population variance unbiased) c. Population SD: S = √S2 We can use the estimated population variance to estimate the variance and standard deviation of the distribution of means (comparison distribution) using the results of the formulas above: a. Distribution of Means Variance: S2M = S2/N b. Distribution of Means SD: SM = √S2M

Advantages and Disadvantages of Repeated-Measures Design

Advantages: A repeated-measures design often has larger effect sizes and provides more power than studies using other designs with the same number of participants It's usually really easy to get significant results The standard deviation of difference scores is usually very low

Hypothesis Testing: Hypothesis testing (Exam 1) Z test t test for single sample t test for dependent means

Comparison Distribution: Distribution of scores Distribution of means Distribution of means Distribution of difference scores

Step 3:

Cutoff = ±2.78

Advantages and Disadvantages of Repeated-Measures Design

Disadvantages: However, variation among participants on each testing's scores is not part of the variation involved in the analysis (so in some ways the test is blind) However, testing a group of people "before and after" an experimental procedure, without using a control group (no comparison group), is a weak research design This is because you have no way of knowing if the potential change from before to after is because of your manipulation or not! Other events may have happened in between Initial test itself may have caused changes Participants may drop out There may be effects of practice or fatigue from taking the same measure twice Participants may have changed or matured between the tests

Dividing by N - 1 makes the estimate of the population variance slightly larger making it (more/less) bias.

Dividing by N - 1 makes the estimate of the population variance slightly larger making it less bias.

SPSS for t Test for Dependent Means

Enter the scores from your distribution in two columns of data window Analyze -> Compare means -> Paired Sample t test -> Select variables for t test -> OK Output gives 3 "Paired Sample" tables: First table has descriptive statistics, second table has the correlation between scores (don't need to know right now), and the third table has your statistics (df, t score, exact significance level)

What if a researcher gets the same t-score of -2.00 with the sample of 100 participants, how would she/her report the results in APA format?

First, we must look to see what our degrees of freedom will be with 100 participants; df=N-1; df = 100-1= 99. If we look at the table, the 5% cutoff for 99 degrees of freedom is not there, so use the degrees of freedom for 95, which is ±1.984 (±1.98). Our sample's t score of -2.00 exceeds the cutoff. We can reject the null hypothesis! t(99) = -2.00, p < .05 14. Yes, you would run a single sample t-test because we are provided with one sample of student's scores. We also are given the mean for the students in general. However, we are not given the population variance, so we would not run a z-test.

If a researcher gets the t-score = -2.00 in their two-tailed test, and there are 21 participants, how would she/he report the results in APA format?

First, we must look to see what our degrees of freedom will be with 21 participants; df=N-1; df = 21-1=20. If we look at the t-table, the 5% cutoff for 20 degrees of freedom is ±2.086 (±2.09). Sample's t score of -2.00 does not exceed the cutoff. This means results are inconclusive! We can report this in APA format as, t(20) = -2.00, n.s.

SPSS for t test for single sample (example)

In variable view, name the variable: Anxiety In data view, enter scores: 8, 6, 4, 9, 3 Click analyze -> compare means -> click one sample t test -> enter test value (µ = 4.00) -> click ok Output should give 2 tables (sig. (two-tailed) has to be .049 or lower)

Why is power important—what is its purpose in designing studies?

Low power suggests that a study has less chance to reach statistical significance. We want to maximize power so that we stack the odds in our favor for reaching significance. Power is also used when proposing research and obtaining funding.

Given this story problem, would you run a single sample t test? Why or why not? A researcher wants to see if students who drink 1 cup of coffee every morning for a semester perform differently in their calculus class than students in general whose mean score is known to be 150.00 out of total 600 possible points. Their sample was 10 students, who received scores of 200.00, 345.00, 127.00, 88.00, 398.00, 56.00, 300.00, 450.00, 490.00, and 210.00 in their course. Run SPSS to find out if students who drink coffee perform better or worse than students in general.

Our t value exceeds the cutoff. We can reject the null hypothesis. Research hypothesis is supported. Students who took 1 cup of coffee every morning throughout the semester did better than students in general, t(9) = 2.40, p < .05.

Step 1:

P1: People who become vegan. (μ1) (mean of the difference scores in people who become vegan) P2: People who show no change. (μ2) (mean of the difference scores in people who show no change) H1: μ1 ≠ μ2 H0: μ1 = μ2 (=0)

Hypothesis tests

Population 1: N = 1, x Population 2: μ, σ (people in general) Comparison distribution: a. mean μ b. variance & SD σ2 σ c. shape Normal curve

t test

Population 1: N >= 2 (30), mean Population 2: μ (people in general) Comparison distribution: a. mean μM b. variance & SD S2M = S2/N SM = √S2/N c. shape t -distribution for df = N-1

Z tests

Population 1: N >= 2 (30), mean Population 2: μ, σ (people in general) Comparison distribution: a. mean μM b. variance & SD σ2M = σ2/N σM = √σ2/N (SE) c. shape Approx. normal curve

Is power typically used before or after collecting data?

Power is used when planning research, therefore it is typically used before collecting data.

What does "robust" mean? Is the t test a robust statistical test?

Robustness indicates that the statistical test in question will still perform well despite violations of its assumptions. Therefore, yes, the t test is robust to moderate violations of the assumption of a normally distributed population.

estimated population

S

sample

SD

What is the difference between SM and S (in formula and conceptually)?

SM is divided by N, whereas S is divided by N-1. S is the standard deviation estimate of a population, whereas SM is the standard deviation for a distribution of means for a t-test. S2 is found first, then SM is found using S2.

We are using our sample to estimate the population variance in a single sample t-test, so we can estimate the comparison distribution:

Sample variance = Σ(X - M)2/N or SS/N (biased population variance) S = √S2 estimated population standard deviation S2 = SS/(N - 1) (unbiased estimated population variance) S2m = S2/N (distribution of means variance) Sm = √S2/N Homogeneity of variance dictates that the variances of pop 1 & pop 2 should be the same The samples variance is consistently smaller than the population variance

Comparison Distribution for t Test: The t Distribution

Since we are using an estimated population variance, we have less true information and more room for error, so that's why we have a t distribution (it's not a normal curve) The comparison distribution is now a t distribution, which has heavier tails compared to a normal curve, and varies in shape based on degrees of freedom: a. standard normal b. t-distribution with df=5 c. t-distribution with df=2 This difference in shape affects how extreme a score you need to reject the null hypothesis (i.e., cutoff). You want LARGE degrees of freedom. (the t shape will be closer to a normal curve) t score: number of standard deviations from the mean on a t distribution t = M - μM / SM How to use the t table: the t table can be found in the appendix of the textbook (Aron, Aron, & Coups) and on Moodle: a. Orient yourself by finding the degrees of freedom for your sample 1a. When selecting the degrees of freedom (and your exact number doesn't exist), ALWAYS select the degrees of freedom less than your actual degrees of freedom 2a. Example: You have a sample of 74 people, 73 df: t table only has cutoff scores for 75 df and 70 df. Use the cutoff score for 70 df because it is a more conservative approach and makes it harder to reject the null, and thus less likely that you are inappropriately rejecting the null. b. Decide if you are using a one or two tailed test c. Decide your significance level, and follow across to the cutoff t score d. The t table doesn't have every cutoff value (unlike the normal curve table), instead it has the most commonly used significance values (p =.10, .05, .01) and associated cutoff scores (it's more efficient this way) therefore use the cutoff value that is lower than your N-1 if it is not on the t-table

Which steps of the 5 steps for conducting a t test change based on conducting a t test for dependent means? How do those steps change?

Step 1 changes slightly because population 2 for a t test for dependent means is people who show no change (it is no longer people in general). Step 2 changes because you need to create a difference score (and use the mean of the difference scores for further calculations). Steps 3-5 are the same as we've been doing!

Example of t test for dependent means: A researcher tested 4 individuals before and after going to Applebees on their hunger. A researcher tests the hypothesis that there is a difference in the scores, using the .05 significance level. m = 6.00 S2 = SS/(N - 1) = 14/3 = 4.67 You only need A & B scores to determine the difference scores. You run the t test for single sample with your difference scores Sign matters for difference scores!

Step 1: Population 1: People who go to Applebee's (µ of the difference scores for the population 1 = µ1) Population 2: People who show no change (µ of the difference scores for the population 2 = µ2) H1: µ1 ≠ µ2 H0: µ1 = µ2 (= 0) Step 2: 1st get difference scores, the mean of difference scores µm = 0 S2m = S2/N = 4.67/4 = 1.17 Sm = 1.08 (Standard Error) Shape: t distribution for df = 3 Step 3: Cutoff t scores: ± 3.18 Step 4: t = (m - µm)/Sm t = (6.00 - 0)/1.08 = 5.56 Step 5: The sample's t score of 5.56 exceeds the t cutoff of ±3.18 . We can reject the null hypothesis. Our research hypothesis is supported. Going to Applebees does make a difference. It increases satisfaction. APA format: t(3) = 5.56, p <.05 However, if our sample's t score did not exceed the cutoff, the results would look like this: The t score of 2.56, does not exceed the 5% cutoff. We cannot reject the null hypothesis. Our research is inconclusive. APA format: t(3) = 2.56, n.s.

A professor wants to test if providing students with study guides helps students perform significantly better or worse on the final exam than students in general, which is out of 100 total possible points. Her sample was 10 students, who received scores of 91.00, 85.00, 72.00, 88.00, 95.00, 95.00, 90.00, 74.00, 86.00, and 89.00 on the final exam. Students in general received a mean score of 80.00. Perform a one sample t test at the .05 significance level to determine if providing study guides makes difference.

Step 1: Population 1: people who receive study guide for final exam Population 2: people in general Research Hypothesis: μ1 ≠ μ2 Null Hypothesis: μ1 = μ2 Step 2: μM = μ = 80 S2 = Σ (X - M)2 / N - 1 = 554.50/9 = 61.61 S = 7.85 S2M = S2/N = 61.61/10 = 6.16 SM = √S2M = 2.48 Shape: t distribution with 9 degrees of freedom df = (10-1) = 9 Step 3: For 9 degrees of freedom, Two Tailed t Test, at .05 significance (use t Table on Moodle): t Cutoff score = -2.26 or +2.26 Step 4: t = M - μ / SM = (86.50-80.00)/2.48 = 2.62 Step 5: Our sample t score of 2.62 is more extreme than the cutoff t score of 2.26, therefore, we can reject the null hypothesis, and our research hypothesis should be supported. Providing a study guide to the students helps them perform better on the final exam than students in general.

t test for single sample:

Step 1: The same as our step 1 from previous tests Step 2: Find μm = μ, S2m, Sm and shape Step 3: Find the t scores cutoff(s). Step 4: Get sample's t score t = (m - μm)/ Sm Step 5: Conclusion Does it exceed the cutoff? Reject or cannot reject the null. Research is supported or inconclusive.

Is this distribution bimodal? Are there multiple modes?

The distribution is not bimodal, but there are multiple modes. It is not bimodal because there is no gap between the multiple modes.

Assumptions of t test for Single Sample and Dependent Means

The population distribution is a normal curve However, even when it is not, the t test is robust and accurate (especially if sample is larger than 50) Data collected (i.e. dependent variable) is equal-interval data Cannot use nominal, categorical, rank order variables The t test is robust over moderate violations (such as non-normal distribution) Robust means the test performs well even though the assumptions are violated EXCEPTION: when using a one-tailed test, and the population is highly skewed, a t test is likely to give a distorted result -> this is why it's good to use a two-tailed test!

What value goes in the "test value" box of SPSS? The sample mean? The population mean? Or the mean of the distribution of means?

The population mean (the mean that you must know, make up, or be given), goes in the test value box. SPSS has no problem calculating the mean of your sample by itself (we have done this multiple times), and technically, the mean of the distribution of means and the population mean are the same thing (so it's a bit of a trick question)!

Type 5:

The sample's t score of 3.38 exceeds the cutoff of 2.78. We can reject the null hypothesis. The research hypothesis is supported. Becoming vegan increases happiness. APA format: t(4) = 3.38, p < .05 if your t score did not exceed the cutoff you would write APA format like this: t(4) = 2.38, n.s.

When you know only μ (mu), then the shape is a t distribution with df = N - 1

The t distribution varies depending on the amount of information we have (sample size matters)

A research article reported a statistic of t(32) = 4.33, p < .05 (two tailed). How many participants were in the study? What was the t cutoff? Were the results significant?

There were 33 participants in the sample (N-1; 33-1 = 32). The t cutoff was +/- 2.043 and was found by referring to the t distribution chart. The results were significant because the sample t score of 4.33 surpassed the cutoff. The results were reported with a p value of less than .05.

Reporting Single Sample t test and t test for Dependent Means

To report a t test in your research, the proper format includes: degrees of freedom, t score, and significance level t(18) = 2.54, p < .05 (significant result) OR t(18) = 1.54, n.s. (non-significant result) To write out your results: The researcher used a [type of test] test with [insert value] degrees of freedom, found a t score of [insert value], and the result was significant at the [insert significance] level. If you used a one or two tailed test, you can mention this in your write-up, however, if nothing is indicated, it is assumed you used a two-tailed test Reminder: You must italicize the t, p, and n.s for APA format!

When selecting the degrees of freedom cutoff for a sample of 53 people, should you use the degrees of freedom cut off score for 50 or 55?

When selecting the degrees of freedom cut off for a sample of 53 participants, you should select the degrees of freedom cut off of 50 because it is the more conservative approach and makes it harder to reject the null

Degrees of Freedom

Why does the above formula use N - 1? a. Because it has degrees of freedom! Degrees of freedom: the number of scores free to vary when estimating a population df = N - 1* For a one-sample t test Why do we need degrees of freedom? a. Because the variance of a sample is generally smaller than the variance of the population, the variance of a sample is BIASED. This means the variance of the sample consistently underestimates the actual population variance. b. Dividing by N - 1 makes the estimate of the population variance slightly larger, therefore helping to make it UNBIASED. The more people in your sample, the larger the degrees of freedom, and the better the estimate of the actual population a. The t distribution differs most from normal curve when degrees of freedom are low, so we want a sample to be as large as possible, only if we want to make a better estimation! b. When the sample size is ∞, the t distribution is normally curved. On t table- when looking at df cut off move down to the row for the specific df..if it is in between numbers shown on the table (ex. Collect from 35 people so df=34) use the df that is lower than yours (ex. Use df 30 when df of your study is 34) because you don't want to accidently reject the null if it is not supposed to be.

How do you increase power? What does increasing power do to a study?

You increase power by increasing sample size, choosing a less strict significance level (higher than .5) *watch for type 1 error*, use a one-tailed test instead of a two-tailed test, increase effect size *watch for generalizability*. Increasing power makes your study more likely to get significant results (rejecting the null hypothesis).

Number of scores for each participant

Z test: 1 t test for a single sample: 1 t test for dependent means: 2

Population variance is known

Z test: Yes- σ2 t test for a single sample: No t test for dependent means: No

Population mean is known

Z test: Yes-µ t test for a single sample: Yes-µ t test for dependent means: 0

Formula

Z test: Z = (M - μm)/σm t test for a single sample: t = (M-μm)/Sm t test for dependent means: t = (M-μm)/Sm

True or False: you can use nominal, categorical, rank order variables for t tests (single sample and dependent means)

false

What will give you a better estimate of the population variance? A (larger/smaller) sample is better because (greater/lesser) degrees of freedom create a more accurate estimate of the population.

larger greater

A t test for dependent means tend to have (larger/smaller) effect sizes, and (more/less) power.

larger more

EX: 5 participants after a job interview on the scale of anxiety. Their scores are 8, 6, 4, 9, and 3 (range: 1 - 10). Mean of the people in general is 4.00. 5% significance level.

m = 6.00 Σ(x-m)=0 Estimated population variance: SS=26.00 S2 = SS/N - 1 S2 = 26.00/(5- 1) = 6.50 Step 1: Population 1: People who have a job interview (µ1) Population 2: People in general (µ2) H1: µ1 ≠ µ2 H0: µ1 = µ2 Step 2: µm = µ= 4.00 S2m = S2/N = 6.5/5 = 1.30 Sm (SE) = 1.14 Shape: t distribution for df = 4.00 Step 3: Cutoff t scores = ± 2.78 Step 4: t = (m - µm)/Sm t = (6.00 - 4.00)/1.14 = 1.75 Step 5: The sample's t cutoff does not exceed the cutoff. The null hypothesis cannot be rejected. The research is inconclusive. APA format: t(4) = 1.75, n.s. If the results were significant, then say, t(4) = 5.75, p < .05

The (more/less) degrees of freedom a sample has will make the t distribution look more like a normal distribution.

more

A researcher test five people on their level of happiness before and after becoming vegan. The researcher hypothesizes that becoming a vegan has an impact on happiness. Test if this hypothesis is true.

step one through 5 below

Step 4:

t = (M - μm)/Sm = (5.00 - 0)/1.48 = 3.38

When you collect data from a single group twice, which statistical test would you run? What's the comparison distribution?

t test for dependent means with a t distribution for degrees of freedom = N - 1

Why is the t test for dependent means a weak research design? Does it have any strengths?

t tests for dependent means (also called dependent samples t tests) do not have a control condition. This means that we cannot say for sure if our treatment is really what caused or affected our dependent variable. But it does have some strengths! When we use t test for dependent means, because we only use the difference scores, we ignore a lot of individual differences. If we ignore individual differences, standard deviation decreases. If standard deviation decreases, then we increase effect size and power!

In APA format, express the following information about a t test for dependent means. t=1.03, with a sample size of 20, a 5% significance level was used.

t(19)=1.03, n.s.

In APA format, express the following information about a t test for single sample. t=4.01, with a sample size of 20, a 5% significance level was used.

t(19)=4.01, p < .05

For a one-sample t test, population 2 is _______________________ For a t test for dependent means, population 2 is_____________________

the general population (people in general) people who show no change

Comparison Distribution

z test: Distribution of means σ2M=σ2/N σM= √ σ2/N Shape: Approximately normal t test for a single sample: μm=μ S2m=S2/N Sm=√S2/N Shape: t-distribution for df=?

P2

z test: People in general μ,σ, σ2 t test for a single sample: People in general μ, S, S2

P1

z test: People who do something N>= 2 Mean t test for a single sample: People who do something N >= 2 Mean

Step 2:

μm = μ2 = 0 S2 = SS/df = 44.00/(5 - 1) = 44.00/4 = 11.00 Sm2 = S2/N = 11.00/5 = 2.20 (narrower than population variance) Sm = √Sm2 = √2.20 = 1.48 (also known as the SE) Shape: t distribution with df = 5 - 1 = 4

population

σ

distribution

σM