Test 2
What are one of the most common misuses of statistical methods?
Examine data and use it to decide what to test
What is Type 2 error? Why does it not lead to lawsuits?
Failing to reject Ho when Ha is true; conclusion is weaker.
What do you do when question asks, "Find the 95% CI for the average activation time. Does result agree with the 2 tailed test from part B?" When Sample mean is 27, and Ha:M=/25.
We wanna know if the true average activation time, 25, is in teh range of plausible values for the mean. Go to explore output stats and 95% CI to get "24.5,31.8). Yes 25 is in the range.
Practical significance?
When the difference is big enough to matter in the real world context of interest
Is the sample t test possible for skewed large pops?
Yes but meaningless because mean doesn't describe pops typical value
Fail to reject Ho?
You want to reject that there's no effect but really there is no effect. You have to accept the null saying the one without the effect is true.
CI and 1 tailed hyp tests can lead to different ______ because.. Is it statistically invalid to use both the CI and hypotheses test?
conclusions; 2 tailed p value is diff from 1 tailed p values Yes
How to find the effect size?
sample mean - test value
In other words of 5% level of significance, the researcher must be 95% confidant...
that they are not making a Type I error before willing to reject Ho, concluding Ha is "proven".
If hte sample average is far enough away from the test value....
the P value will be low enough to let us reject the test value with low risk of type 1 error
Negative values of T support..
the left tailed alternative
The P value does not tell us how big the ..
Effect size is because it is also a function of population variability and sample size.
T tests for skewed large samples: A) Not valid but useless B) Not valid but not useless C) Valid but useless D) Valid but not useless
Valid but useless
What does valid and useful mean?
Valid: Low chance of making Type 1 error. Useful: Is the mean a reasonable way to descreibe the center of the population? (looking at a histogram if the sample is large to determine this)
What does it mean when the P Value>α, and we cannot conclude Ha is "proven"?
We are NOT saying Ho is true, just saying there is not enough evidence to prove Ha beyond a reasonable doubt. (fail to reject Ho).
In an example, if the 1 tailed P value is .028, what does this mean regarding rejecting or failing to reject?
We can reject Ho at the 5% level of significance. Since the chance of being wrong is 2%.
What do you do when you're asked "What is the smallest level of significance that allows us to conclude Ha:M=/25?" When the P value is 8.5%
.085
Ha:M>Test Value Ha:M<Test Value Ha: M=/ Test Value Most are _____ Which do you use for scale for accuracy?
1 Tailed 1 Tailed 2 tailed **1 tailed, researchers are trying to prove one way or the other **2 tailed
In a 2 tailed test, what is the general rule for adjusting 2 tailed P values?
1 tailed P value is half the 2 tailed p value. Divide the P value by 2!
What are the other commonly used significance levels?
1% (.01) 10% (.10)
What is the general relationship between the 1 and 2 tailed P values summarized with 2 rules?
1) 2 tailed p value divided by 2 is the P value for 1 tailed Ha that is supported by the data. We know it is supported because we looked at the sample average. 2) The two 1 tailed P values add up to 1. They are opposites of each other in the probability sense.
Cautions about Hyp Testing? ALMR
1) Assumptions of random sampling. Biased data is usually wrong. 2) Looking at the observed sample average is the proper way to measure the effect size, NOT the p value. (Ha:M>100, and P value is .0001, implies that the mean is bigger than 100 by some amount whether it is huge or small) 3) Most violated! Ha should be stated by looking at the data. Scientist thinks something is true and then tests. Cannot decide what to test based on the data. You can't test data you already know is statistically significant. 4) Repeated use of Test/CI. If you do 100 Confidence Intervals, and 95% sure each one is right, 5 out of 100 would be wrong/5 type 1 errors occur. Eventually mistakes will come because of multiplicity effect. If you use the same test with α=.05, on every person and so not surprising after all since could expect to make 25 type 1 errors (if population is 500 people).
Example of using the T table to find the exact P value: Experiment produces 100 measurements with sample average 17.23 and std dev 1.3 Can we conclude the average measurement exceeds 17 using a 5% level of significance?
1) Ha:M>17 (17.23-17)/(1.3/sqrt(100))= 1.769 The Degrees of freedom= 100-1=99. Since we don't have 99, go to the 100 on the T table and find what 1.769 is between 2) It's between 1.660 and 1.984, which is .05 and .025 .025<P value<.05 (smaller than 5% and bigger than 2.5%) 3) Or the SPSS way: 1-cdf.t(1.769,99)= .0400 4) Conclude that it is less than 5% so we can reject Ho and conclude that the mean is higher than 17 for the whole population
What are the assumptions for Sample T tesT?
1) Random Sample 2) Population is mound shaped or sample is over 30
What if trying to prove Ha:M<30? We find the sample mean is 27. At the 5%? 10%? 1%?*******
1) We can support the data because it is less than 30. (we change the test value from 27 to 30) 2) To conclude how strong, see the P value is .207. We divide by 2 (since data is supported). We get .1035 (10%) 5%= .10 is greater than .05, so we must fail to reject Ho 10%=Fail to reject Ho because .1035 is greater than .10 1%= Cannot fail to reject Ho (support Ha), because .10 is smaller than .01
If claim is not supported, how do you find the P Value?
1-(2 tail/2)
Values of T diff from zero postive or neg support...
2 tailed alternative
Reject Ho?
A very strong conclusion stating that Ha is true but needs a lot of strong evidence for this.
What is the philosophy of statistical hypothesis testing?
Assume Ho is true unless the observed data proves that Ha is true
The standard terminology of hyp testing has been based on the view that A) Ha is more important B) Ho is more important
B) Ho is more important; making it more confusing yet more widely used in journal articles
When the question is asking if the 1 sample T test can be used despite being a small sample size what does that mean?
Because it is bound shaped, you look at the SW, and if it is above .05, then you can assume mound shaped, and can use the One Sample T Test.
If the 1 tailed test is used we can or cannot use the CI? Because 2 tailed and CI both ...
CANNOT. Because the real confidence that both are correct is not the same as individual confidence levels. Look at 2 directions.
Which is better choice , CI or 2 tailed? Because 2 tailed only shows.. CI shows..
CI because information is easier to understand and more detailed. 2 tailed only shows how sure we are that the mean is different than the test value. How much the mean may differ from the test value
Statistically significant?
Can reject Ho
Ha
Claim that the researcher is trying to prove/prove beyond a reasonable doubt
What does 5% level of significance mean?
Comparing the P Value to .05
Effect size?
Diff between the true value of M and the test value; how much average actually changes (Effect size is a statistical concept that measures the strength of the relationship between two variables on a numeric scale. For instance, if we have data on the height of men and women and we notice that, on average, men are taller than women, the difference between the height of men and the height of women is known as the effect size. The greater the effect size, the greater the height difference between men and women will be. Statistic effect size helps us in determining if the difference is real or if it is due to a change of factor)
If activation time was more than 25 seconds? Does this agree with the CI?
Do 1 tailed test for Ha:M>25. Get P value .085, and data supports, so only divide by 2 to get .0425. So yes the evidence is strong enough to conclude that M>25. Must look at the range, and there are values less than 25, so no it does not agree.
Which tells us how much average changes: P value or effect size?
Effect size
True or False: In journal articles, when the evidence is not strong enough to conclude Ha is true it is called "failing to reject Ho". This is NOT the same as accepting or concluding Ho.
False It is the same
How do we find the area for the two tailed tests?
Find the area in one tail and double it
List 3 hypotheses that can be tested using this output and give correct P value for each. Test value: 8 Sample mean: 7.9 Sig 2 tailed: .046
Ha: M≠ 8 has p-value 0.046 Ha: M < 8 has p-value 0.046/2 = 0.023 Ha: M > 8 has p-value 1 - (0.046/2) = 0.977
If in a 1 tailed Test, Ha:M>.08, what would the null be?
Ho:M<.08
When can statistical significance be obtained for small effect sizes?
If a large sample is used
What does the observed sample average tell us? What does it NOT tell us?
If the data supports Ha. (SUPPORT) How strongly. (CONCLUDE)
Why do 2 tailed hyp tests and CI always agree?
If the error rates match. CI=90% α=.10 CI=95% α=.05 CI=99% α=.01 Hypothesis testing relates to a single conclusion of statistical significance vs. no statistical significance. Confidence intervals provide a range of plausible values for your population. You can use either P values or confidence intervals to determine whether your results are statistically significant. If a hypothesis test produces both, these results will agree. The confidence level is equivalent to 1 - the alpha level. So, if your significance level is 0.05, the corresponding confidence level is 95%. For our example, the P value (0.031) is less than the significance level (0.05), which indicates that our results are statistically significant. Similarly, our 95% confidence interval [267 394] does not include the null hypothesis mean of 260 and we draw the same conclusion.
All in all, CI and Hyp testing..
In statistical analyses, there tends to be a greater focus on P values and simply detecting a significant effect or difference. However, a statistically significant effect is not necessarily meaningful in the real world. For instance, the effect might be too small to be of any practical value. It's important to pay attention to the both the magnitude and the precision of the estimated effect. That's why I'm rather fond of confidence intervals. They allow you to assess these important characteristics along with the statistical significance. You'd like to see a narrow confidence interval where the entire range represents an effect that is meaningful in the real world.
What does it mean when the P Value is close to zero?
It is evidence that Ha is true because you can reject Ho with a small chance of making a Type 1 error.
What does it mean when the P Value is is close to 1?
It is evidence that Ho is true because you have to take a large chance of making a Type 1 error to reject Ho.
If the value is 25, and we find the sample mean, and it is 27, what can we say?
Knowing that it is NOT 25, we can support the data, saying the mean is NOT 25. Next we want to know if it is strong enough conclusion and look at P value. If it is, for example, 8%, then that is too high of a risk to reject Ho, so we cannot conclude it is proven. If we went ahead and concluded it, we would have an 8.5% chance of making a Type 1 error.
What do you do when a question asks "is the evidence strong enough to conclude Ha:M=/25 at the 10% level of sig?" when P value is 8.5%
Look at P value, if it is 8.5% , then it is under alpha, meaning a low risk of type 1 error. So yes, can reject Ho because evidence is strong enough.
If claim is not supported after looking at the sample average, there is no need to...
Look at the P Value
What does it mean to be more statistically significant?
Lower p value
What is the main purpose of the P Value?
Measure how strong the support is and tells us if it should be considered proof beyond a reasonable doubt which allows us to conclude Ha is true.
Valid tests are possible with small samples as long as.. Why are the results meaningful?
Mound shaped; The mean describes the typical value of a mound pop but difficult to prove hypothesis with small samples
What does it mean when it asks is the evidence strong enough?
Must use P value to see
Does data support the 1 tailed Ha:M<25, if the sample average is 27 and the P value is .085? Although there is no point in making a P Value calculation since doesn't support, but what do you do if you need to find it for this situation?
No, because it is bigger than 25, the data does NOT support. P value: 1-.085/2= .9575...96% of making a type 1 error. AKA, having to be willing to take a 96% chance of being wrong.
Ho
Null; No effect. The research has no effect.
When the sample is small, we will see results that show _____ even though the result is not statistically sig
Practical significance (will we see a big impact in the real world?) *Statistical sig- less than 5%
When the sample is large, we will see results that are statistically significant but not _____
Practical significant
Right tailed area?
Proving the average is bigger than Mo (the P value is in the area to the right of T)
Left tailed area?
Proving the average is smaller than Mo
Confidence Interval?
Random intervals from the data you collected. If you run enough experiments, the percentage that the intervals will contain the actual mean of pop is 95%
Type 1 error?
Rejecting Ho when Ho is actually true. Too confident that Ha is correct. Yet not enough evidence to prove the causal when really Ha is not correct.
Multiplicity effect?
Repeated use of a test on diff samples or subjects means we have to expect some decisions will be wrong. If 100 tests are done using 5% level of significance we must expect 5% of 100 =5 type 1 errors to be made.
Standard error of the mean?
S/sqrt(n) ...an estimate of the standard dev of the sample average
Suppose testing Ha:M=/40 vs Ho:M=40 for sample A and B. Sample A: Sample mean=40.3, S=2, n=200 Sample B: Sample mean=100, S=250, n=30 Calculate effect size and P value for each sample Which has bigger effect size? Which has lowest P value? What do the effect sizes and P values show us about hyp testing?
Sample A: Effect size: .3 T= (40.3-40)/2/sqrt(200))=2.121 P value: .01(*2)<p value<.025(*2) .02<P value< .05 OR SPSS ...2*(1-cdf.t(2.121,199)=.032 Sample B: Effect size: 60 T= (100-40)/250/sqrt(30))=1.315 P value: .05(*2)<p value<.10 (*2) .10<p value<.20 OR SPSS ....2*(1-cdf.t(1.315,29))=.199 Sample A is statistically more sig We cannot base conclusion just on the sample average
Failing to reject Ho is another way of saying:
Saying Ho is right
P value?
Sig. in SPSS. If P value is less than .05, the % risk is low so you can safely reject Ho. If it is above .05, then we cannot safely reject Ho, we must fail to reject Ho.
When answering "Does a 95% Confidence Interval for the mean weight loss give the same conclusion?" What do you do? (if M>10) and range is (13.56,23.74)
Since entire range for plausible values for the mean is higher than 10, the conclusion is the same.
If Ha:M>25 and P Value is .085... "Does the data support the Ha:M>25 strongly enough to conclude (prove beyond a reasonable doubt) it is true at the 1% level of significance? 5% level of significance?"
Since it is 1 tailed we divide to get .0425 (4.3% chance of making Type 1 error if conclude Ha). 1%= No, .0425 is greater than .01, so because the greatest risk would be no bigger than 1%, this is a 4.3% risk and so fail to reject Ho. 5%= Yes, .0425 is less than .05, and so can safely take the chance. It is convincing evidence. Can reject Ho. It is a statistically significant result.
Prove there is a dif= Matters in the real world=
Statistical significance Practical significance
One sample test statistic?
T=sample mean -one sample T test/std dev/sqrt of the pop Counts the number of standard errors that the sample average is away from the test value.
What does the One Sample T Test allow us to do?
Test hypotheses that compare a population mean to a fixed value.
How are the CI and 2 tailed test the same thing mathematically?
Test rejects values outside of CI range, and fails to reject test values that are in the CI.
Two tailed area?
The P value is in the sum of the areas in the two tails
The one tailed P values are probability opposites, meaning..
They sum up to 1.
What is the One Sample T test using the summary data?
To calculate the P values when you only have (n, sample mean, and st dev) Ha:M is either >,<,=/ to the test value Mo
When should you look at the estimated effect size?
To see if the difference is big enough to really matter in the real world context of interest.
True or False: In journal articles, concluding Ha is true is called "rejecting Ho"
True
Which error can lead to lawsuits?
Type 1 because you falsely conclude Ha to be "proven"
Positive values of test statistic can happen...
if sample average is bigger than the test value so the pos values of T support the right tailed alternative
Values of T that are farther away from zero in the appropriate direction...
lead to smaller P values
Large sample size can...
make a little diff be statistically significant
In Paired Samples T Procedures, the pairs are from... THE SUBJECTS ARE ...
the same person; SAME PERSON
If the p-value is less than alpha (i.e., it is significant), then the confidence interval ... Looking at the Minitab output above, the 95% confidence interval of 365.58 - 396.75 does not include $400. Thus, we know that the P value .... If the p-value is greater than alpha (i.e., it is not significant), then the confidence interval ...
will NOT contain the hypothesized mean. the p-value will be less than 0.05. will include the hypothesized mean.
What is the maximum probability of making a Type 1 error?
α (5% level of sig) . Typically researcher is only 5% willing to take a risk of making Type 1 error.
