Activity 5-1

The point estimate for the CI for the mean difference is -4.06. That means that our best guess for the population mean difference is -4.06. The LB of our 95% confidence interval is -6.52 and the UB is -1.60. All values within that confidence interval are not equally likely to be the population mean difference value. Based on our data, which of the following values is more likely to be the population mean difference value?

-4.0, because it is the closest to the point estimate

Compute a single sample t hypothesis test using statistical software. The data file is available on the textbook website. For all programs, use 0 as the test value. What is the observed difference between the means?

-4.058

What was the average number of minutes that students underestimated the time required (i.e., the observed mean difference)?

-4.058

What are the LB and UB for the 95% confidence CI around the mean difference? These values are generated by the statistical software.

-6.52, -1.60

What is the two-tailed p value for this study? This is the area under the t curve to the right of +3.271 and to the left of -3.271.

.001

Suppose that two researchers each obtained a mean difference of 1 with a standard deviation of 9. Researcher 1 used a sample size of 20 while Researcher 2 used a sample size of 150. What would the effect size be for both researchers?

.11

How much sampling error would be expected with a sample size of 100 people, assuming the SD is 10?

1

Sampling error is measured by the SEM. Suppose that a researcher knows that the SD for a sample is 10. How much sampling error would be expected with a sample size of 25 people?

2

The summary above provides all of the relevant statistics, but you still need to interpret it. In your interpretation, you should discuss five things: 1) the p value (i.e., evidence against the null hypothesis), 2) the size of the effect (i.e., what is the direction of the effect, do the results appear to be large enough to be important in this context), 3) the CIs, 4a) the methodological rigor, and 4b) the scientific literature. Below we have provided a complete scientific conclusion which includes all five of the above required components. Indicate which of the five components is addressed in each sentence by writing the numbers 1-4b in each space. The students underestimated the time it would take to complete a test by 4.06 minutes, which is a small-medium effect (d). When you consider that most of us perform multiple tasks every day, habitually underestimating required time by 4 minutes could lead to problems with scheduling and planning. The small p value provides strong evidence against the null hypothesis. The range of plausible effect sizes suggests that the actual difference is probably somewhere between small and medium. The range of plausible effect sizes suggests that the actual difference is probably somewhere between small and medium. Overall, these results are consistent with research found in the literature suggesting students do exhibit an optimism bias, specifically a planning fallacy. One possible methodological concern with the study is that people may have spent very different amounts of time on the test and this may have influenced the participants' responses differently.

D. 2 (the size of the effect) C. 1 (the p value) E. 3 (confidence intervals) A. 4b (scientific literature) B. 4a (methodological rigor)

Fill in the blanks with the appropriate numbers from the statistical analyses you ran. We computed a single sample t and d to compare the average difference between the students' predicted time for task completion and their actual completion time (M = [a], SD = [b]) to zero, t ([c]) =[d], p = [e] (two-tailed), 95% CI ([f],[g]) , d = [h], ([i]).

Fill in the blanks with the appropriate numbers from the statistical analyses you ran. We computed a single sample t and d to compare the average difference between the students' predicted time for task completion and their actual completion time (M = -4.06, SD = 12.59) to zero, t (102) =-3.27, p = .001 (two-tailed), 95% CI (-6.52,-1.60) , d = -.32, (-.52,-.12).

The t statistic is computed as:

The observed mean difference divided by expected sampling error

Based on the 95% CI around d, to what degree does the range of plausible d's for the population suggest different scientific conclusions? (i.e., does the range span dramatically different d values and therefore dramatically different conclusions?)

The scientific conclusions are dramatically different

People tend to be overly optimistic about many things including income, debt, grades, illness, and marriage (e.g., Shepperd, Waters, Weinstein, & Klein, 2015; Weinstein, 1985). People also exhibit unrealistic optimism when estimating how long tasks will take to complete, called the planning fallacy (e.g., Buehller, Griffin, & Ross, 1994; Kahneman & Tversky, 1979). When students habitually commit the planning fallacy it can cause them to overcommit, leaving insufficient time for important tasks. The planning fallacy may explain why students do not perform optimally in school and/or at work. Do statistics students exhibit the planning fallacy? Can they accurately predict the time required to complete a practice test for a Statistics course? You recruit a representative sample of 103 students in a statistics courses at your university and ask them to estimate how many minutes it will take them to complete the test. This is the third practice test of the semester so the students have experience completing this sort of task. After making their predictions, students complete the practice test online and each student's completion time is recorded to the nearest minute. You compute the difference between each student's predicted time and the actual completion time (predicted-actual). A negative difference indicates a student underestimated, a positive difference indicates overestimation, and a zero would be a perfect prediction. Based on previous research on unrealistic optimism, you expect students will underestimate the task time. However, you recognize that both underestimation and overestimation have important implications for statistics students. You need to consider this when you determine if a one- or two-tailed hypothesis test is appropriate. The data file is on the textbook website. Why is a single sample t test the correct statistic for this research scenario?

You are comparing one sample mean to a known test value (0).

The closer the p value is to 0, the _______ (Choose one: weaker/stronger) the evidence is against the null hypothesis (assuming there are no methodological flaws in the study)

stronger

Use software to construct a graph of the sample data. Consider what you can conclude based on the graph your software provides. JASP created the graph in Figure 5.1. Based on this graph, does the sample mean suggest overestimation or underestimation? Explain your answer. Remember, the DV is Estimated time - actual time = time difference.

underestimation (the prediction time was less than actual time)

Choose the correct statistical null hypothesis .

µ1 (time difference) = 0

Sketch a graph of a t distribution with the obtained t value labeled. This is a two-tailed test, so you should place the positive and negative obtained t value on the t distribution you drew. You placed your obtained t value at ______.

+3.27 and -3.27

The mean difference is one way of thinking about the size of the effect. You can also compute a standardized effect size (d). If the statistical software does not compute this for you, you can compute it by hand by dividing the mean difference by the SD.

-.32

Use your software or the program on the textbook website to fill in the blanks. The confidence interval around d has a LB of _____ and an UB of _____.

-.52, -.12

What is the t value for this study?

-3.271

What is the standard error of the mean (SEM)?

1.241

In this research situation is a one- or two-tailed hypothesis test appropriate?

A two-tailed test is appropriate, both under estimation and overestimation have important implications for statistics students.

Assuming there are no methodological flaws in the study, which of the following is the correct interpretation of the p value? (Choose 2)

The probability of obtaining a t value as large or larger than the one computed if the null hypothesis is true An index in which a smaller value is stronger evidence against the null hypothesis

All the necessary assumptions are met by this study. Match each of the statistical assumptions to the fact that implies the assumption was met. Homogeneity of variance Appropriate measurement of the IV and DV Normality Independence

D. It is unlikely that the sample SD and population standard deviation will be dramatically different. B. The DV is measured on an interval ratio scale. C. The population of scores has a normal shape. A. The scores from each participant were collected so one participant's scores did not affect anyone else's scores.

True or False. A study with a lower p value (e.g., .001) will always have a larger effect size than a study with a higher p value (e.g., .05).

False

Interpret the width of the CIs. Do they provide compelling evidence that the sample data provide sufficiently precise estimates of the population? Explain your answer.

No, the CIs are useful but imprecise.

Which of the following is a methodological concern with this study?

People may have spent very different amounts of time on the test and it is possible that biases are influenced by the length of time doing the task.

Which researcher would have the smaller SEM and smaller p value? Researcher 1 or Researcher 2?

Researcher 2

Based on the 95% CIs, it appears that the effect size in the population is:

Somewhere between small and medium

What does the SEM measure? (select two)

The typical distance between all possible sample means of a given sample size and the population mean. Expected sampling error

Are these results consistent with research found in the literature on the optimism bias? Do students exhibit an optimism bias, specifically a planning fallacy?

Yes

Do you think that the effect is sufficiently large to have practical importance? Why or why not? Consider both of the effect size measures as you make your determination (i.e., the observed mean difference in minutes and the d).

Yes, given the number of tasks we perform each day even smaller mis-estimations of time can add up across a full day or week. The d is small-medium. In this context this observed effect size probably has practical importance.

Based on the p value, do you find the evidence against the null hypothesis compelling?

Yes, the p value is close to zero.

The sample size can have a very dramatic effect on a study's p value. Generally speaking, larger sample sizes are better, up to a point. The sample size that is required for good statistical evidence depends on the size of the effect that exists in the population. A general guide to follow is if the effect in the population is large a reasonable sample size would be ______, if it is medium it would be ______, and if it is small it would be ______.

about 25; about 65; about 400

Assuming everything else is held constant, as the absolute value of a t statistic increases, the p value ________.

decreases

If the observed difference is unlikely to be due to sampling error, you would expect the observed difference to be ______________ (smaller/larger) than the SEM.

larger

The data file has the difference scores for each participant computed as (predicted time - actual time = time difference). Therefore, if students are overly optimistic in their time estimates, the time difference would be

negative.

Describe the size of the effect as small, small-medium, medium, medium-large, or large.

small-medium

In general, a larger sample size results in a ________ denominator of the t statistic (SEM), which will ________the value of the obtained t.

smaller; increase