Behavioral Statistics, Test 4
Type I error
A decision to reject the null hypothesis when the null hypothesis is true
Type II error
A decision to retain the null hypothesis when the null hypothesis is false
Two-sample t test, independent groups - alternative hypothesis
A directional alternative hypothesis would predict that the samples are random samples from populations where the mean of the first group is larger than the mean of the second group, or the mean of the second group is smaller than the mean of the first group, depending on the direction. A non-directional alternative hypothesis would predict that the means of the two groups are not equal.
Why is the F test used instead of using the t test for independent groups multiple times?
The F test is used because the probability of a Type I error is smaller. When calculating multiple groups of t comparisons, the probability of getting t values equal to or greater than t critical goes up, which increases the probability of a Type I error.
F test
The F test uses the variance of the data for hypothesis testing. F obtained is equal to the average of two independent variance estimates from the same population.
Which test is more powerful: correlated groups or independent groups?
The correlated groups design
Two-sample t test, independent groups design - How are the subjects selected and assigned?
Subjects are randomly selected from the subject population and then randomly assigned
ANOVA
The ANOVA is a statistical technique used to analyze multigroup experiments. Using the F test allows us to make one overall comparison that tells whether there is a significant difference between the means of the groups.
F distribution - degrees of freedom
The F distribution has two values for the degrees of freedom, one for the numerator and one for the denominator. Df1 = n1-1 (numerator) Df2 = n2-1 (denominator)
When is the F test appropriate?
The F test is appropriate in any experiment in which the scores can be used to form two independent estimates of the population variance. One frequent situation the F test is appropriate occurs when analyzing the data from experiments that use more than two groups or conditions. Using several groups allows a more clearer interpretations of the findings. Using several levels increases the possibility of a positive result occurring from the experiment.
Robust
A test is said to be robust if it is relatively insensitive to violations of its underlying mathematic assumptions.
What happens to the t distribution as the degrees of freedom increase?
As degrees of freedom approaches infinity, the t distribution approaches the normal curve. As the degrees of freedom increases, sample size increases and the estimated variance gets closer to the actual variance.
What are the differences between the t and z distributions?
At any degrees of freedom other than infinity, the t distribution has more extreme t values than the z distribution since there is more variability in t (since we do not know the standard deviation). The tails of the t distribution are elevated relative to the z distribution. For a given alpha level, the critical values of ti s higher than for z, making the t test less powerful than the z test. This means that t obtained must be higher than z obtained to reject the null hypothesis. As degrees of freedom increases, the critical value of t approaches that of z.
Alternative hypothesis
Claims that the difference in results between conditions is due to the independent variable
Null hypothesis
Claims that the independent variable does not have an effect on the alternative hypothesis and that any differences are due to chance
Two-sample t test, dependent/correlated groups - Design
Each subject gets two or more treatments. A difference score is calculated for each subject, and the resulting difference scores are analyzed. The simplest experiment of this type uses two conditions, often called control & experimental or before & after.
Two-sample t test, independent groups - null hypothesis
If the null hypothesis states that the independent variable has no effect. The samples would be random samples from a population where the two group means are equal and chance alone would account for the differences between the sample means.
What is the shape of the sampling distribution of t?
If the null-hypothesis population is normally shaped, or if N is greater than or equal to 30, the t distribution looks like the z distribution (normal distribution, standard bell curve) except that there is a family of t curves that vary with sample size.
Decision rule
If the probability is less than or equal to alpha, reject the null hypothesis and retain the alternative hypothesis. If the obtained probability is greater than alpha, fail to reject and retain the null hypothesis.
Two-sample t test, dependent/correlated groups - Matching
Instead of the same subject being used in both conditions, pairs of subjects that are matched on one or more characteristics serve in the two conditions. The difference scores between the matched pairs are then analyzed in the same manner as when the same subject serves in both conditions
How is the sampling distribution of t derived empirically?
It can be derived empirically by taking a specific population of raw scores, drawing all possible different samples of a fixed size N, and then calculation the t value for each sample. By varying N and the population scores, one can derive sampling distributions for various populations and sample size.
How is the critical region of rejection determined?
It is determined by the alpha level.
Why is the independent groups design more efficient than the correlated groups design?
It is more efficient from a degrees of freedom per measurement analysis. The degrees of freedom are important because the higher the degrees of freedom, the lower t critical is. The independent groups design is also beneficial because many experiments preclude using the same subject in both conditions. When the same subject cannot be used, matching can be used; however, matching is time-consuming, costly, and the experimenter often does not know which are the important variables for matching in order to produce a higher correlation.
What does it mean if two samples show larger differences in their variances?
It may indicate that the independent variable is not having an equal effect on all the subjects within a condition. This can be an important finding in its own right, leading through further experimentation into how the independent variable varies in its effects on different types of subjects.
How many degrees of freedom are associated with the mean?
N
How many degrees of freedom are associated with the standard deviation?
N-1
What are the degrees of freedom for the t test?
N-1
Two-sample t test, independent groups - degrees of freedom
One degree of freedom is lost each time a standard deviation is calculated. Since we calculate the standard deviation twice, we lost two degrees of freedom. We enter table D with N-2 degrees of freedom, but it varies uniquely only with degrees of freedom.
Differences between correlated groups vs. independent groups design
The correlated groups experiment analyzes difference scores instead of raw scores. Data collected from the two designs can be the same, but sometimes the conclusion can be different. This can happen because the correlated groups design allows us to use the subjects as their own control. This maximizes the possibility that there will be a high correlation between the scores in the two conditions. When the correlation is high, the difference scores will be much less variable than the original scores. This causes the correlated groups design to be potentially more powerful than the independent groups design.
What is the critical region of rejection?
The critical region for rejection of the null hypothesis is the area under the curve that contains all the values of the statistic that allow rejection of the null hypothesis.
Two-sample t test - Design
The essential feature of this design is that there are paired scores between conditions. The difference scores from each score pair are analyzed to determine whether chance alone can reasonably explain them.
Two-sample t test, independent groups design- Design
The independent groups design involves experiments using two or more conditions. Each condition uses a different level of the independent variable. The most basic experiment only has two conditions - an experimental condition and a control condition.
Degrees of Freedom
The number of scores that are free to vary
What is the two-sample t test also known as?
The repeated or replicated measures design
What is a major limitation of the single sample experiment?
The requirement that at least one population parameter must be specified
When is the t test appropriate?
The t test is appropriate when the experiment only has one sample, the mean is specified, the standard deviation is unknown, and the mean of the sample is used as the basic statistic. The sampling distribution of the mean must be normal. To be normal, N must be greater than or equal to 30 or the population of raw scores must be normal.
How is the sampling distribution of F generated empirically?
The sampling distribution of F can be generated empirically by taking all possible samples of size n1 and n2 from the same population, then estimating the population variance from each of the samples using s1 and s2. Then, calculate f obtained fro all possible combinations of s1 squared and s2 squared. Finally, calculate the probability of F for each different value of F obtained.
Sampling distribution of F - Characteristics
The sampling distribution of F gives all the possible F values along with the probability of F for each value, assuming sampling is random from the population. Since F is a ratio of variance estimates, it never has a negative value. The F distribution is positively skewed. The median F value is approximately equal to 1. There is a family of curves.There is a different curve for each combination of df1 and df2. The critical values for F can be found under Table F in Appendix D.
What is the sampling distribution of t?
The sampling distribution of t is a probability distribution of the t values that would occur if all possible different samples of a fixed size N were drawn from the null-hypothesis population. It gives all the possible different t values for samples of size N and the probability of getting each value if sampling is random from the null-hypothesis population.
Assumptions underlying the t test
The sampling distribution of the mean of the first group minus the mean of the second group is normally distribution. The populations from which the samples were taken should be normally distributed. There is homogeneity of variance.
What does the t distribution look like?
The t distribution is symmetrical about zero and becomes closer to the normally distributed z distribution with increasing degrees of freedom. As degrees of freedom approaches infinity, the t distribution approaches the normal curve.
Homogeneity of variance - t test
The t test assumes that the variances of the two populations are equal. If the variances of the samples in the experiment are very different, the two samples probably are not random samples from populations where the standard deviations are equal. If this is true, the homogeneity of variance assumption is violated (the standard deviations are not equal).
Two-sample t test, dependent/correlated groups - Difference scores
The t test for correlated groups allows utilization of both the magnitude and direction of the difference scores. It treats the difference scores as though they were raw scores and tests the assumption that the difference scores are a random sample from a population of difference scores having a mean of zero.
Violations of the assumptions of the t test
The t test is a robust test. The t test is relatively insensitive to violations of normality and homogeneity of variance, depending on sample size and the type and magnitude of the violation. If the two samples are equal in size and if the size of each sample is equal to or greater than 30, the t test for independent groups may be used without appreciable error despite moderate violation of the normality and/or the homogeneity of variance assumptions. If there are extreme violations of these assumptions, then an alternative test such as the Mann-Whitney U test should be used.
What are the advantages of two-sample experiments?
The two-treatment experiment does not need to measure population parameters. This is a benefit because investigators should not use previously acquired population data because we do not know how accurate the information is.
Which test is more powerful: z or t?
The z test is more powerful.
Two-sample t test, independent groups design - Pairing
There is no basis for pairing of subjects and each subject is only tested once. When analyzing the data, since subjects are randomly assigned to conditions, there is no basis for pairing scores between conditions. Rather, a statistic is computed for each group separately and the two group statistics are compared to determine whether chance alone is a reasonable explanation of the differences between the group statistics. The statistic that is computed on each group for the t test is the mean.
What two characteristics must the alternative and null hypothesis have?
They must be mutually exclusive and exhaustive.
Why are there different sampling distributions of t for different sample sizes?
This is because we are estimating the standard deviation and because the size of the sample influences the accuracy of the estimate. The t distribution varies uniquely with the degrees of freedom associated with t.
Limitation of two-group study
Two groups are often not sufficient enough to allow a clear interpretation of the findings.
When do we reject the null hypothesis?
We reject the null hypothesis if the obtained probability is equal to or less than the critical probability level.
