Ch. 5 RQs
When reporting results in APA style you should include information about:
ALL OF THE ABOVE: -the hypothesis test (i.e., p value) -practical importance (i.e., effect size) -plausible parameters (i.e., LB to UB range of CI)
When interpreting 95% CIs you should determine if...
BOTH OF THE ABOVE: (a) the lower boundary value surpasses your idea of practical importance and (b) the lower and upper boundary values suggest similar or different interpretations
Assuming there are no methodological flaws in the study, which of the following is the correct interpretation of the p value?
(1) the probability of obtaining a t value as large or larger than the one computed if the null hypothesis is true (2) an index in which a smaller value is stronger evidence against the null hypothesis
What does the SEM measure?
(1) the typical distance between all possible sample means of a given sample and the population mean (2) expected sampling error
The higher the t value (i.e. the farther the t is away from zero), the _________ the p will be to zero.
closer
The number in the parentheses in the character string "t (24) = 2.73, p = .01, 95% CI [.76, 5.50], d = .55, [.12, .96]." is the
degrees of freedom
In general, the smaller the effect you are trying to detect, the _________ your sample size.
larger
If the observed difference is unlikely to be due to sampling error, you would expect the observed difference to be _________________ than the SEM
larger (because large t's, which are associated with small p values, have a large observed mean difference (in numerator) compared to its SEM (in denominator))
Do small p values (ie large t values) have small sampling error?
-not necessarily because you can't make a definitive conclusion about sampling error from a p value (you have to also interpret and take into account the effect size (d), as well as the methodological rigor and preexisting scientific literature) -you are *more likely* to find less sampling error if you have a smaller p value -it is difficult to find a low p value if you have a lot of sampling error
The statistical techniques used in this book most closely follow which approach to statistical reasoning? Fisher or Neyman Pearson
Fisher
Which of the following is the best summary of the differences in the graphs generated by computer programs?
The graphs generated by statistical programs do not look identical, but convey similar information.
Which of the following is a correct interpretation of a p value of 0.06?
The probability of the obtained t OR A MORE EXTREME t is 0.06 IF THE NULL HYPOTHESIS IS TRUE
You can use a single sample t test to compare
a sample mean to a known value of interest (e.g. a population mean)
If the effect in the population is large, a reasonable sample size would be __________; if it is medium it would be ____________; and if it is small it would be ____________.
about 25, about 65, about 400
T/F: sample size matters when computing and interpreting sampling error. effect sizes are not influenced by sample size.
both true statements
The p values from a hypothesis test provides information about:
the likelihood that an outcome occurred due to sampling error.
A p value is the probability of getting the sample mean or a more extreme sample mean assuming that:
the null hypothesis is true
The t statistic is computed as:
the observed mean difference divided by expected sampling error
Which of the following information is provided by CIs and not other statistics?
the precision of population estimates (i.e., how precisely the sample estimates the population parameters)
In order for CIs to give precise estimates of the population parameters, both estimates have to lead to
the same scientific conclusion (i.e. both UB and LB of the 95% CI around the "Cohen's d" have to either be both small, both medium, or both large)
if the null is not true, the t value will be close to
the tail(s)
When interpreting p values using three-valued logic, you can reach one of three conclusions. Which of the following is NOT one of these conclusions?
there is no difference The conclusions you can reach: (1) the difference is probably negative (2) the difference is probably positive (3) suspend judgement about the difference (as result of p value being too high to support concl. 1 and 2)
two-tailed null hypothesis is:
μ1=μ0 (the test value, the value if there is no effect)
The traditional threshold approach to p values is flawed because: (Choose all that apply)
-nearly identical p values can lead to opposite scientific conclusions -p values vary along a continuous range from p=1 to p=0 and dichotomizing/separating/classifying these values into "significant" and "not significant" categories can frequently mischaracterize the statistical evidence.
CI values accomplish which of the following?
-they describe the range of plausible values for a population parameter based on the expected sampling error. -they help with interpreting results by making researchers explicitly aware of the uncertainty involved in generalizing sample results to populations
How to tell if a p value is considered compelling evidence:
-when p value< 0.05, it is considered significant -when p value>0.05, it is considered not significant *keep in mind that p values close to 0.05 (like 0.05,0.06/0.07 which are close to being considered "significant") could or could not be significant depending on the problem, so you must be cautious/careful in your interpretation* -the smaller the p value, the more compelling the evidence is against the null (the more compelling the evidence is that the dependent variable is the reason for the deviation from actual average/mean and not sampling error)
For this example, the observed difference between the sample mean and the population mean was ______________; the difference expected due to sampling error was____________
3.13;1.146
Your written scientific conclusion should include all the statistical evidence (i.e., exact p value, observed effect size, the CIs, the mean of the sample, the df, the obtained t value) and then it should also provide an interpretation that considers:
BOTH OF THE ABOVE: -the scientific literature -the methodological rigor of the study (these two concepts make up the fourth pillar of reasoning)
Which of the following is a correct interpretation of a p value of 0.02?
IF THE NULL HYPOTHESIS IS TRUE the probability of the obtained t value OR A MORE EXTREME t value is 0.02.
purpose of pillar one (finding p value) is to ask the question:
Is the observed difference (b/w sample mean and test value) likely or unlikely to be due to sampling error? (depends on if it's a large or small p value respectively) -Assuming the null hypothesis is true and that there are no methodological flaws in the study, how likely is it that the observed difference is due to sampling error? ----Answer: probability of obtaining a t value of # or a more extreme t assuming null is true is (p value, % form); it is p-value (% form) likely that my observed difference is due to sampling error
The number [.12] in the character string "t (24) = 2.73, p = .01, 95% CI [.76, 5.50], d = .55, [.12, .96]." is the
LB of the confidence interval for d
Assuming everything else is held constant, as the absolute value of a t statistic increases the p value ________
decreases (because larger t's are farther from population mean (t=0) and so the values farther from the mean are more rare)
T/F: A study with a lower p value (ex. 0.001) will always have a larger effect size than a study with a higher p value (ex. 0.05).
false (because larger effect sizes more likely to result from high p-values, not lower p-values)
low t value corresponds to
high p value
When performing a single sample t test, the _____________ assumption is the most difficult to assess because you do not know the population standard deviation.
homogeneity of variance
One of the reasons confidence intervals are so valuable to scientific reasoning is that they encourage researches to:
interpret their results cautiously by making them consider the range of possibilities that might reasonably exist in the population.
The statistical hypothesis is often referred to as the _________ hypothesis.
null
Evidenc against the null hypthesis is considered compelling when the.....
p value is close to zero
Researchers sometimes report a p value of 0.041 as p<0.05 rather than p=0.041. Which is the better, more informative way to present p values?
p=0.042 (the exact p value provides more precise evidence which is more desirable)
Values closest to the ________ _______________ are more likely to be the population mean difference value
point estimate
When computing an effect size, you are getting info on the...
practical importance (i.e., magnitude of the effect)
define p value
probability of getting a certain t value or one further from zero (one more extreme) if null hypothesis is true and assuming there are NO methodological flaws
Your _____________ hypothesis is your expectation based on your understanding of the relevant scientific literature.
scientific
Finding evidence in support of your ______________ hypothesis is a two-stage process. First you find statistical evidence that the data are incompatible with the _____________ hypothesis then you attempt to construct an inferential argument to support your ______________ hypothesis.
scientific, null, scientific
One-tailed p values are ___________ two-tailed p values for the same t statistic.
smaller than
In general, a larger sample size results in a ______________ denominator of the t statistic (SEM), which will ___________ the value of the obtained t.
smaller, increase
The _____________ hypothesis is usually stating that the mean difference between the population mean and the test value is zero.
statistical null
what does it mean when you say the null hypothesis is true?
that means that any deviation from the test value of interest (could be population mean, could be any value that represents the actual average/mean of something) is due to sampling error; says that the sample mean will remain close to the expected score close to the test value (i.e. within decimal points of it)= more likely to be due to sampling error when sample mean way above that test value (as in at least 1.0 sample mean higher)=shows more support for the dependent variable u are measuring to be the cause of the deviation, not sampling error
T/F: Values closer to the observed mean difference (i.e., the point estimate) are more plausible as estimates of the population mean difference than values close to the LB and UB.
true
Difference between each participant's predicted time and actual completion time=predicted-actual. A negative difference indicates a student _________________, a positive difference indicates ____________________, and a __________ would be a perfect prediction.
underestimated; overestimation; zero
How do you obtain the p values for a single sample t test?
use statistical software to compute them
Which of the following is NOT a pillar of scientific reasoning?
your own personal beliefs and political ideology 4 pillars: -hypothesis testing -effect sizes -confidence intervals -contextualizing the results by considering the methodological rigor and the scientific literature
A smaller p value reflects stronger evidence _______________ the null hypothesis. (against/for)
against
One-tailed hypothesis tests are appropriate if:
an outcome in the opposite direction as you predicted does not have any implications for the population
Basing your scientific conclusion on all four pillars of scientific reasoning is a good idea because it encourages you to make:
appropriately cautious scientific conclusions (NOT scientific conclusions with absolute certainty)
If the null is true, you can expect to get a t value close to
zero
purpose of pillar 2 is to ask the question:
is the effect large enough to have a meaningful effect/practical importance on the population?
The major advantage of d as an effect size measure is that:
it is easily compared to other ds from other studies w/ diff scales
Wider confidence intervals provide _______ precise estimates of the population parameters.
less
High t value corresponds to
low p value
There are 3 kinds of confidence intervals (CI for a single mean, CI around the mean difference (when have two samples), CI around the standardized effect size (d)), but all three estimate the range of values one might expect in the population based on a:
margin of sampling error (MOE) around a point estimate from the sample.
The SEMs estimates the amount of...
mean difference one should expect in a study due to sampling error given the sample size
The closer the *p value* is to 0, the _____________ the evidence is against the null hypothesis (assuming there are no methodological flaws in the study).
stronger
The mean and the mean difference will only be the same if the test value is ______
zero! (always look at problem to see what the given test value is, not stated explicitly so you have to know what's going on in the problem)
