Stats Exam Unit 3 Thought Questions
What are the two biggest drawbacks to using nonparametric tests?
1. Confidence intervals and effect size measures are not typically available for nominal or ordinal data 2. Nonparametric tests tend to have less statistical power than parametric tests, which increases the risk for making a Type II error (False Negative-fail to reject the null) -nonparametric tests are often seen as the fall back method and are not usually the go-to method for statistical test
What is a cell and what does it show researchers when they are analyzing data?
A cell is a box in a table that depicts one unique combination of levels of the IV in a factorial (two-way between groups ANOVA); when cells contain numbers, they are usually the mean scores of all the participants who were assigned to that combination of levels
What is the effect size measure of chi-square tests and how is it calculated?
Cramer's V is the measure of effect size for chi-square tests and it is the quotient of the chi-square value divided by the number of participants multiplied by the smallest degree of freedom of either a) the row or b) the columns
Why is the within-groups SS smaller for the within-groups ANOVA as compared to the between-groups ANOVA?
For the between groups ANOVA, we only have to subtract the SSbetween from the SStotal to get the SSwithin; but for a within groups ANOVA, we must take into account the fact that we have the same number of participants so we must remove that effect. This is why we subtract SStotal-SSbetween-SSsubjects in order to get our SSwithin -by subtracting the SSsubjects, we are removing the variability caused by participant differences from the estimate of variability across conditions
Heteroscedastic populations
Populations that have different variances, will be greater than two when you divide the largest sample variance by the smallest sample variance
Homoscedastic populations
Populations which come from the same variance, will be less than two when you divide the largest sample variance by the smallest sample variance
Give examples where linear regression would typically be useful or appropriate.
Predicting student success at a university based on GPA and SAT/ACT score Insurance companies input demographic data into an equation to predict the likelihood of a class of people (such as young male drivers) to submit a claim
How is the total variance partitioned in a one-way between-groups ANOVA? What contributes to, or is represented by, each of the components?
Total variance (SStotal) is composed of the product of the between groups variance and within groups variance SS total=total deviations; for each score, subtract the grand mean from every score and square the deviations, then sum all the squared deviations
reliability
a reliable measure is one that is consistent -a score of shyness will reveal nearly the same score every time
correlation coefficient
a statistic that quantifies a relation between two variables -can either be positive or negative -always falls between +/- 1 -the strength (magnitude) or the coefficient, not the sign, indicates how large it is
multiple regression
a statistical technique that includes two or more predictor variables in a prediction equation
correlation
a systematic association (or relation) between two variables -correlations cannot tell us which explanation is right, but it can force us to think about possible explanations -we can assess the direction and size of a correlation -Pearson's correlation coefficient is the most common form of correlation
coefficient alpha
commonly used to estimate a test or measure's reliability and is calculated by taking the average of all possible split-half correlations -you should always aim to have an alpha value of 0.9 or 0.95; you want high reliability when you are using a test to make decisions, especially when those decisions are related to people (ex. surgery odds, etc.)
adjusted standard residual
the difference between the observed frequency and the expected frequency for a cell in a chi-square research design, divided by the standard error; it is the calculation of a statistic for each cell based on its residual -a cell's residual is the difference between the expected frequency and the observed frequency for that cell -the ASR is a measure of the number of standard errors that an observed frequency falls from its associated expected frequency (similar to the z stat for each cell) -a larger ASR indicates that an observed frequency is farther from its expected frequency than a smaller adjusted standardized residual frequency
latent variables
the ideas that we want to research but cannot directly measure
regression to the mean
the tendency of scores that are particularly high or low to drift towards the mean over time
Each of the three F statistics has its own between groups sum of squares (SS), degrees of freedom (df), mean square (MS), and critical value, but they all share a ___________________
within-groups mean square (MSwithin)
How many degrees of freedom must you find for a one-way within-groups design ANOVA, how do we calculate them, and what do they each mean?
4- between groups, subjects, within groups, and total 1. Calculate the between groups and subjects degrees of freedom first because multiplying them together will get us our within groups degrees of freedom dfbetween=Ngroups-1 dfsubjects=n-1 dfwithin=(dfbetween)(dfsubjects) dftotal=Ntotal-1
source table
A source table presents important calculations and final results of an ANOVA in an easy-to-read format -In the Source column, the population variance comes from the spread between the means and the second source comes from the spread within each sample
What are the three possible causal explanations for a correlation?
A---->B B----->A ------>A C ------>B
Quantitative interaction
An interaction in which the effect of one variable is strengthened or weakened at one or more levels of the other IV, but the direction of the initial effect does not change -the effect of one IV is modified in the presence of another IV
Qualitative interaction
An interaction of two (or more) IVs in which one IV reverses its effect depending on the level of the other IV -the effect of one variable doesn't just become stronger or weaker, it actually reverses direction in the presence of another variable
Two Ways to Estimate Population Variance
Between-groups variance and within-groups variance estimate two different kinds of variance in the population. If there two estimates are the same, then the F stat will be 1.0 (different from z test or t test where 0 would mean no difference) -As the sample means get farther apart, the between groups variance (numerator) increases and the F stat also increases
What are expected frequencies and observed frequencies?
Expected frequencies are the frequencies that we expect to see about a given group of participants based on the null hypothesis Observed frequencies are the frequencies that we actually see about a given group of participants -When we state the null and research hypotheses, we discuss them in terms of frequencies and not means -The chi-square stat is calculated using the observed and expected frequencies
Reporting the F stat and looking it up in the F stat table is done by this format: For a within-groups design, we are only interested in the between-groups F stat because this tells us that there is a significant difference between groups or not
F (between groups df, within groups df)
How are ANOVA results typically presented in APA journals?
F( df between, df within)= F stat, p<>0.05 <=reject >=fail to reject
True or False: Correlation determines causation
FALSE! Correlations only provide clues to causation; they do not demonstrate or test for causality -correlations only quantify the strength and direction or the relation between variables
What is the main difference between a one-way ANOVA and a two-way ANOVA in terms of variability?
In a two-way ANOVA, we have three ideas being tested and each idea is a separate source of variability 1. Main effect of the first independent variable 2. Main effect of the second independent variable 3. The interactions between the first main effect and the second main effect -with a two-way ANOVA, we also have a fourth source of variance from the within-groups variance
How does the Tukey HSD attempt to prevent an inflated chance of a Type I error?
It allows you to take a second look at whether or not you have enough of a difference to reject the null hypothesis
The importance of scatterplots and correlation analysis
It is important to construct a scatterplot for correlations because you can see if there is a general trend with your data and you can also see if there are any obvious outliers that may be skewing the data
Effect Size, R^2, for Two-Way ANOVAs
R^2 is a measure of effect size and is the same as the effect size for a one-way ANOVA -Since we have two main effects and an interaction effect, we must isolate the effect size for a single effect at a time; we do this by subtracting the sums of squares for both of the other effects from the total sum of squares ex. R^2 (row)= SSrow/ (SStotal-SScolumn-SSinteraction)
Conceptually, what is meant by SSbetween, SSwithin, and SStotal (think of the deviation scores that are the basis for each of these SS terms)?
SS between= deviations between groups; for each score, subtract the GM from that score's group mean and square the deviation. Sum all the squared deviations - this is the step that estimates how much each group, not each individual participant, deviates from the overall group mean SS within= deviations within groups; for each score, subtract the group mean from that score and square the deviation SS total=total deviations; for each score, subtract the grand mean from every score and square the deviations, then sum all the squared deviations
What is the difference between SS and MS? What is another name for MS? Which is additive?
SS= sum of squares, represent between groups variability, within groups variability, and total variability (1st two add up to the total). Remember that deviations is another term used to describe variability (variability for a between groups, SSb, is the same as the deviations for a between group) MS= mean squares, it is the SS divided by the degrees of freedom for both the between groups and the within groups (not total). Both MS between and MS within values will be used later to find the F stat -SS is additive and MS is not
What are the four sources of variability in a two-way ANOVA?
SStotal SSbetween (row) SSbetween (column) SSwithin SSbetween (interaction)= SStotal-(SSbetween(row)+SSbetween(column)+SSwithin)
What are the standards for a small, medium, and large correlation coefficient?
Small= 0.10 Medium= 0.30 Large= 0.50
What is a factorial ANOVA?
This is another name for a two-way between groups ANOVA, which is a hypothesis test that includes two nominal IV, regardless of their numbers of levels and a scale DV -a factorial ANOVA must have at least two IVs to be called a factorial ANOVA (also called factors)
What is one problem associated with using stepwise multiple regression?
Two samples with very similar data can, and sometimes do, overlap and can lead to drastically different conclusions
Type I and Type II errors
Type I- False Positive, rejecting the null when the null is true Type II- False Negative, failing to reject the null when the null is false
What does it mean to assess the proportion of variance explained by each ANOVA component?
We must calculate the effect size, R^2, which allows us to determine the variability explained, but only by the between-groups differences -the numerator of the effect size equation is the measure of the variability that takes into account just the differences among means, SSbetween -the denominator of the effect size equation is the SStotal-SSsubjects, and this takes into account the total variability but removes the variability caused y differences among participants
What is the difference between a quantitative interaction and a qualitative interaction?
While both of these interactions show one of the variables strengthening or weakening, a qualitative interaction will show that one of the levels will reverse its effect depending on the level of the other IV -a quantitative interaction will show that one variable is strengthened or weakened at one or more levels of the other IV but the direction of the initial effect does not change
When would you perform a post-hoc test in a two-way between-design ANOVA?
You only need to run a post-hoc test in a two-way study when you have a significant main effect with more than three levels of this particular significant main effect; you do this in order to determine exactly where differences lie -you would not need to run a post-hoc if you have under three levels
After you find the predicted z score on Y (Y hat), what do you do next?
You then find the raw score for Y using the equation: Y (hat)= Zy (SDy)+ My Y (hat)= (-0.573)(15.040)+76=67.38
psychometrics
a branch of statistics that is used in development of tests and measures
covariate
a scale variable that we suspect associates, or covaries, with the IV of interest
simple linear regression
a statistical tool that lets us predict a person's score on a DV from his or her score on one IV -also allows us to calculate the equation for a straight line that describes the data -a regression also provides specific quantitative information that predicts the relations between variables, which goes beyond the information that a correlation can provide
Be able to explain the logic behind each df term for a one-way independent groups ANOVA.
dfbetween= Ngroups-1 This is the number of groups in the study minus 1, because the sample between-groups variance estimates the population variance through the differences among the means of the samples dfsample=n1-1 or n2-1 This is the degrees of freedom for each sample where n represents the number of participants in each group dfwithin=df1+df2+df3+....+dfn This is the total degrees of freedom from each of the samples in the study; it is the summation of the degrees of freedom for each of the samples in the study dftotal=dfbetween+dfwithin or Ntotal-1 This is the degrees of freedom for the total number of participants in the study ex. If there are 60 participants in three groups (20 in each), it can be 60-1=59 or 2+57=59
conditional proportions
divide the number of given outcomes by the total number of people in a certain group; this proportion is called the conditional proportion because you haven't calculated the proportions out of ALL people in the study, just out of a certain condition or category
N'
harmonic mean -when sample sizes are different, you have to calculate a weighted sample size; must be calculated before calculating the standard error
The strength of the correlation is determined by....
how close to "perfect" (+/- 1) the data points are; the closer the data is to the imaginary perfect line, the stronger the relation between the two variables -if your data points are far from this imaginary line, the farther the correlation is from perfect, the weaker relationship between the two variables
slope
is the amount that Y is predicted to increase for an increase of 1 in X
validity
measures what is intended to be measured
statistical interaction
occurs in a factorial design when two or more IV have an effect on the DV in combination that neither IV has on its own; both variables work together to create the interaction; this can be calculated through the F stat of the interaction effect
Be able to compare r, r^2, sy,sx, and sy-hat for various data sets. What does each reveal and how do they relate to each other?
r= correlation coefficent r^2= proportionate reduction in error sy
marginal mean
the mean of a row or column in a table that shows the cells of a study with a two-way ANOVA design
Cramer's V (effect size)
the standard effect size used with the chi-square test for independence (also called Cramer's phi) -the denominator represents the sample size, N, and either the degrees of freedom for the rows or the degrees of freedom for the columns, whichever is smaller. We take the square root in order to get the quotient for Cramer's V
path
the term that statisticians use to describe the connection between two variables in a statistical model
manifest variables
the variables in a study that we can observe and that are measured
mixed-design ANOVA
used to analyze the data from a study with at least two IVs; at least one of the IVs must be within-groups and at least one variable must be between-groups -includes both within-groups variables and between-groups variables
Assumptions for a Chi-Square
1. The variable is nominal (one nom value for goodness of fit, two nom values for test for independence) 2. Each observation is independent, no single participant can be in more than one category 3. The participants were randomly selected (unlikely) 4. There is a minimum number of expected participants in every category (usually 20)
Two main problems with matched groups:
1. We might not be aware of all of the variables of interest 2. If one of the people in the matched pair leaves the study or backs out, both data points must be discarded
What is meant by regression to the mean? When is it most severe? What impact can it have on experimental design and interpretation of experimental results?
Regression to the mean occurs when extreme scores, such as very high and very low scores, start to move towards the mean, or regress, to become less extreme; usually extreme scores can show the most regression towards the mean, such as stocks; midrange values usually hover around the mean ex. Very tall parents do tend to have tall children but they won't be as tall as they are; likewise, very short parents will likely have short children, but they are usually not as short as they are Previous performance is not necessarily indicative of future performance ex. For investment decisions, it is wise to ride out a decrease in a mutual fund rather than panic and sell before the likely drift back toward the mean; it might also help you to avoid buying into the fund that has been on top for several years, knowing that it stands a chance of sliding back towards the mean
Effect Size for Within-groups ANOVA
SSbetween/(SStotal-SSsubjects)
What are the two types of research designs for a one-way ANOVA?
Within groups ANOVA and between groups ANOVA
multivariate analysis of covariance (MANCOVA)
an ANOVA with multiple dependent variables and covariates
What are the six steps of hypothesis testing?
1. Identify the populations, distribution, and assumptions (ANOVAs will follow the F distribution) 2. State the null and research hypothesis 3. Determine the characteristics of the comparison distribution 4. Determine the critical value or cutoff 5. Calculate the test statistic 6. Make a decision (if the F stat falls beyond the critical F value, reject; if the F stat falls within the critical F value, fail to reject)
Using the standard regression equation:
1. If the Zx is 0, use the direction of the correlation, Rxy, in the equation to find the predicted z score Zy=(-0.85)(0) 2. If Zx is 1, using the direction of the correlation, Rxy, in the equation to find the predicted z score Zy=(-0.85)(1) 3. If you are not given the Zx score, you need to find the z score given the raw score X=5 Mx=3.4 SDx=2.375 Zx=(5-3.4)/2.375=0.674 Zy=(-0.85)(0.674)=-0.573
Matched Groups
A matched groups design has different people in each group but they are similar (matched) on characteristics important to the study; these groups have more statistical power because it allows us to analyze the data as if the same people were in each group -this type of design is especially useful when participants can't be in two groups at the same time -allows us to use within-groups design even if different participant experiences each level of the IV; rather than using the same participants, we match different participants on possible confounding variables ex. Are psych majors or history majors more interested in current events? (you couldn't have a psych major be a history major too...plus you can't randomly assign people to a major)
How does the calculation of the effect size, R^2 or n^2, differ for the one-way within-groups ANOVA and the one-way between-groups ANOVA? How does the calculation of the Tukey HSD differ for these two types of ANOVAs?
For the within groups ANOVA, we subtract SSsubjects from SS total in the denominator The Tukey HSD test is the same for both one-way between and one-way within ANOVAs, in that you compare each pair of means and divide by the standard error (you would not need to use a corrected standard error because you have the same # of people in each group because they are the same people) -any value that falls beyond the critical q table cutoff value (determined by the within-groups degrees of freedom and p level of 0.05) will mean that that main effect had a statistically significant difference
orthogonal variables
IVs that make separate and distinct contributions in the prediction of a DV, as compared with the contributions of other variables -they do not overlap each other
If the null hypothesis is true (i.e. no difference between means), what is expected regarding the relationship between MSbetween and MSwithin? What size of F would be anticipated?
MS between and MS within would be the same value and therefore the resulting F stat would be 1 -It is the same when a z test or t test reveals a value of 0, in which the sample and the population show no difference
If someone has scores for both variables above the mean (two positive deviations) or scores for both variables below the mean (two negative deviations), then the two variables are likely to be _________ correlated. If someone has a score above the mean for one variable (positive deviation) and below the mean for the other variable (negative deviation), then the two variables are likely to be ______ correlated.
Positively Negatively
For a two way ANOVA, how many sets of hypotheses do we need?
We need three sets of hypotheses, one for each main effect (2) and one for the interaction effect -you will have a null and research hypothesis for all three
What happens when you don't have any significant main effects and you are unable to find a significant interaction effect?
You would conclude that there is insufficient evidence for the study to support the research hypothesis
standard regression equation
Zy=(Rxy)(Zx) The subscripts in the formula indicate that the first z score is for the DV, Y, and that the second Z score is for the IV, X The ^ above the Y is called a "hat" by statisticians and it refers to the fact that this variable is predicted, there for Zy (hat) is the z score for the predicted DV, not the actual score -z scores tell us how far a participant falls from the mean in terms of standard deviations
multivariate ANOVA (MANOVA)
a form of ANOVA in which there is more than one DV; multivariate refers to the number of DVs, not the number of IVs (a normal ANOVA can handle multiple IVs, but you need a MANOVA is you have multiple DVs)
statistical (theoretical) model
a hypothesized network of relations, often portrayed graphically, among multiple variables
linear relations
a linear relation is shown when a correlation coefficient is calculated; a linear relation is when you can draw a straight line through a set of data points in order to show a general direction or pattern
What is the goal of multiple regression? What advantages are present for using multiple regression?
a regression equation using more than one IV is likely to be an even better predictor that using the mean -the goal of multiple regression is to include two or more predictor variables in a prediction equation to create a simple linear regression ex. Behavior tends to be influenced by many factors, so a multiple regression would allow you to use these variables together in your prediction and give an even better outcome
intercept
is the predicted value for Y when X is equal to 0, which is the point at which the line crosses, or intercepts, the y-axis
Identify similarities and differences between linear regression and correlation.
1. Correlation quantifies the degree to which two variables are related. Correlation does not fit a line through the data points. You simply are computing a correlation coefficient (r) that tells you how much one variable tends to change when the other one does. When r is 0.0, there is no relationship. When r is positive, there is a trend that one variable goes up as the other one goes up. When r is negative, there is a trend that one variable goes up as the other one goes down. Linear regression finds the best line that predicts Y from X. 2. Correlation is almost always used when you measure both variables. It rarely is appropriate when one variable is something you experimentally manipulate. Linear regression is usually used when X is a variable you manipulate (time, concentration, etc.) 3. With correlation, you don't have to think about cause and effect. It doesn't matter which of the two variables you call "X" and which you call "Y". You'll get the same correlation coefficient if you swap the two. The decision of which variable you call "X" and which you call "Y" matters in regression, as you'll get a different best-fit line if you swap the two. The line that best predicts Y from X is not the same as the line that predicts X from Y (however both those lines have the same value for R2) 4. Correlation computes the value of the Pearson correlation coefficient, r. Its value ranges from -1 to +1. Linear regression quantifies goodness of fit with r2, sometimes shown in uppercase as R2. If you put the same data into correlation (which is rarely appropriate; see above), the square of r from correlation will equal r2 from regression.
Hypothesis Test for Chi-Square Test for Independence
1. Identify the populations, comparison distribution (chi-square), and assumptions (variable is nominal (2), each observation is independent, the participants were randomly selected, and there is a minimum number of expected participants in every category 2. State the null and research hypotheses in words 3. Determine the characteristics of the comparison distribution -degrees of freedom for each variable are calculated and then multiplied together to get the overall degrees of freedom dfrow=krow-1 dfcolumn=kcolumn-1 dfx^2=(dfrow)(dfcolumn) 4. Determine the critical values or cutoffs, based on calculated degrees of freedom and a p value of 0.05 5. Calculate the test statistic; determine the expected frequencies; errors are often made in this step and if wrong expected frequencies are used, the chi-square derived from them will also be wrong Expected Frequency=f(e)=(Totalcolumn/N)(Totalrow) 6. Make a decision
What are some limitations with using regressions?
1. It is extremely rare that the data analyzed in a regression equation are form a true experiment (one that used randomization to assign participants to conditions); this is especially true when IVs are scale variables instead of nominal variables, in which we cannot randomly assign participants to conditions 2. When drawing conclusions from regression, we must consider confounding variables (such as an outside C variable like we saw in correlation) that may impact the data; this limits our confidence in a correlation 3. Beware of regression to the mean
What are the assumptions that represent optimal conditions for valid data in an ANOVA?
1. Random selection is necessary if we want to generalize beyond the sample: if this assumption is not met, generalize to the population with caution 2. A normally distributed population allows us to examine the distributions of the samples to get a sense of what the underlying population distribution might look like; this assumption becomes less important as sample size increases 3. Homoscedasticity (homogeneity of variance) assumes that the samples all come from the same population with the same variances
What assumptions are made for between-groups ANOVA? How robust is the test regarding these assumptions?
1. Random selection is necessary if you want to generalize beyond the sample to the remainder of the population. If you do not have random sampling (which we never do...) then you would use caution when generalizing 2. A normally distributed population allows for us to examine the distribution of the samples to get a sense of what the underlying population distribution might look like 3. Homoscedasticity (homogeneity of variance) assumes that the samples all come from the same population with the same variances -homoscedastic populations are those with the same variances, heteroscedastic populations are those with different variances Most of the time, we do not have random selection, we usually do not know the population distribution is normally distributed but data would indicate if it was skewed or not. Also, we will check the homoscedasticity when we calculate the test statistic by checking whether the largest variance is not more than twice the smallest (must be <2 for homo)
Pearson correlation coefficient assumptions
1. Random selection is used 2. Underlying population distribution for the two variables must be approximately normal 3. Each variable should vary equally, no matter the magnitude of the other variable
Hypothesis Test for Chi-Square Goodness of Fit
1. State the populations (there are always two: 1) the population that matches the frequencies of participants like those we observe and 2) another population that matches the frequencies that matches the frequencies of participants like those we would expect according to the null hypothesis); the comparison distribution is for a chi-square distribution; there is only 1 nominal variable, so we will use a chi-square goodness of fit test, the participants were not randomly selected (generalize with caution), there is a minimum # of participants in every category, and the observations are independent from each other 2. State the null and research hypotheses in words only 3. Determine the characteristics of the comparison distribution by finding the degrees of freedom based on the numbers of categories, or cells, in which participants can be counted (it is the number of categories minus 1) -use this value to find the chi-square comparison distribution 4. Determine the critical value, or cutoff, for the chi-square statistic by using the appropriate degrees of freedom calculated using the number of categories and a p level of 0.05 -the chi-square critical value can never be negative, only positive 5. Calculate the test statistic using the observed and expected frequencies. The expected frequencies are determined from the information hat we have about the general population X^2= [SUM (o-e)^2]/(e) 6. Make a decision- is it past the critical chi-square value? If yes, reject the null; if no, fail to reject the null
How does ANOVA use variance estimates to test for differences between means?
A large between groups variance and small within-groups variance for an ANOVA indicate a small degree of overlap among samples and likely a small degree of overlap among populations; a large between groups variance divided by a small within-groups variance produces a larger F stat; if the F stat goes beyond the critical cut off value, then we can reject the null hypothesis and this would indicate that the sample means are different from each other -we calculate the variance among sample means because you can't simply subtract to find how different each is spread apart; this is especially true if you have more than two means -the between groups variance indicates the difference between means that you find in the data; if the difference is much larger than the within-groups variance, then what we would expect by chance especially, then you can reject the null hypothesis -the within-groups variance reflects the difference that we would expect between means by chance; there is always variation among means within a population so we would expect this to happen When the F stat is one, this indicates that there is no difference between the variance of the between-groups means and the variance of the within-groups means; they came from the same place and you cannot infer that their means are different
What is meant by a main effect? What values are used in testing for (and interpreting) the presence of a main effect?
A main effect occurs in a factorial ANOVA (more than two IVs and one scale DV) when one of the IVs has an influence on the DV. We evaluate if there is a main effect by disregarding the influence of any other independent variable in the study -to test if there is a main effect, we would calculate the F stat for both (or more than two) of the IVs to see if their values extend beyond the critical value determined by F (between df, within df). If they do go beyond the crit value, then you know that you have a significant influence from one or both of the main effects -to get the F stat for both, you will need to divide the MSbetween/MSwithin for both rows (main effect 1) and columns (main effect 2)
When do we conduct a post-hoc test, and what does it tell us?
A post hoc test is a statistical procedure frequently carried out after the null hypothesis has been rejected; it allows us to make multiple comparisons -referred to as a 'follow up' test -it is used to determine which groups are significantly different from each other -a statistically significant F stat means that some difference exists somewhere in the study
What is meant by a two-way between-groups ANOVA? Give an example. Understand what is meant by a ____ by ______ design.
A two way between groups ANOVA looks at two or more IVs and one DV, but it also looks to see if there is an interaction effect between IVs on the DV -a between groups design is one in which participants are only exposed to one condition (whereas a within-groups design exposes participants to all conditions) -instead of the descriptor two-way, researchers refer to an ANOVA with its corresponding number of cells ex. Three levels of the IV (medication-Lipitor, Zocor, placebo) and two levels of beverage (water or grapefruit) would be a 3x2 ANOVA
How does repeated measures or within-gorups ANOVA partition variance differently as compared to a between-groups ANOVA?
A within groups design reduces errors that are due to differences between the groups, because each group includes exactly the same participants -ultimately, this enables researchers to reduce the within-groups variability associated with differences among the people in the study across groups; with lower within groups variability, this means that a smaller denominator for the F statistic and a larger F statistic makes it easier to reject the null -researchers usually require less participants because they are testing the same group and not two different groups
What is the main purpose of Analysis of Variance?
Allows researchers to focus on the most important finding by assessing one or more IVs at the same time
F distribution
An F distribution allows you to conduct a single hypothesis test with multiple groups plus they are more conservative than the z or t distributions
What is meant by an interaction? Explain why an interaction might be of interest (examples might help). What values are used in testing for (and interpreting) the presence of an interaction?
An interaction (especially a statistical interaction) means that two or more of the main effects (IV) have an effect on the DV that neither have on its own; both variables work together to create this interaction -this is important because both (or more) of the main effects might not have an impact on the DV alone, which would show in their F stat, but if you look at the F stat for their interaction, they might work together to create an effect on the DV
How does this help interpret the outcomes (in addition to information revealed by significance tests)?
By removing the subjects sum of squares, we can determine the variability explained ONLY by the between-groups differences
What advantages are present for conducting a two-way ANOVA, rather than using a one-way design? Explain how adding an extra factor might reveal an effect that would be hidden in a one-way ANOVA.
Conducting a two-way between groups ANOVA (factorial ANOVA) allows researchers to separate between-groups variance into three finer categories: two main effects and an interaction effect. The interaction effect is something that is not tested in a one-way design and it allows us to see if there is blended effect resulting from the interaction between the two IVs; it is not a separate IV ex. It is like blending chocolate syrup and milk; each variable is unique on its own and it creates something new when you mix them (interaction) -you can also test two or more IVs at the same time, rather than just testing one IV
What problems are associated with using multiple t tests to compare multiple means?
Conducting multiple t tests greatly increases your chance of making a Type I error and greatly decreases your chance of being able to reject the null. Only a F stat has the power to quantify the added complexity of having more than two samples in a comparison study
What is the difference between factors and levels?
Factors act as categories (such as males or females) and levels are subcategories within each factor (such as shortness or tallness). Factors are the independent variables and levels are the conditions within those independent variables 2 Factors with 2 Levels Male, short Male, tall Female, short Female tall
If you didn't use an ANOVA and you didn't calculate an F statistic, why is performing more than one t test bad?
Having numerous t tests greatly increases the probability of making a Type I error; with only one comparison (instead of 6, 10, 12, etc) you have a 0.05 chance of having a Type I error (rejecting the null when the null is true) and a 0.95 chance of having a Type I error in any given analysis when the null is true. -With more comparisons, your chance of making a Type I error greatly increases ex. With two comparisons, (0.95)(095)=0.903 or 90% chance of not having a Type I error and a 10% chance of having a Type I error. With three comparisons, (0.95)(0.95)(0.95)=0.857 or 86% chance of not having a Type I error and a 14% chance of having a Type I error
Within-groups (subjects) ANOVA
Hypothesis test in which there are more than two samples and each sample is composed of the same participants -Also called a Repeated-Measures ANOVA ex. Testing the same group of students every year on an aptitude test
Between-groups ANOVA
Hypothesis test om which there are more than two samples and each sample is composed of different participants ex. Comparing the freshman, sophomore, junior, and senior classes on how well each class performed on a test
One-Way ANOVA
Hypothesis test that includes both one nominal IV with more than two levels and a scale DV ex. Females are compared to each other
Two-Way ANOVA
Hypothesis test that includes two nominal IVs, a regardless number of levels, and a scale DV ex. Males to females are compared to each other
How does the Bonferroni test attempt to safeguard alpha?
It is more conservative test, which means that it is more difficult to reject the null hypothesis. It is a post hoc test that provides a more strict critical value for every comparison of the means -there is a smaller critical region (more difficult to reject null) -Divide the p level by the number of comparisons; you then run a series of independent samples t tests using the more extreme o level to determine the cutoffs; the difference between means would have to be in the extremely narrow tails of a t distrubution before we would be willing to reject the null hypothesis (VERY DIFFICULT TO REJECT THE NULL!) -it minimizes the probability of making a Type I error due to this small critical region
What does it mean when the F stat is large or when the F stat is small? Quantify overlap in ANOVAs
Large= a large numbered F stat indicates that the between-groups variance is much larger than the within-groups variance and this means that the sample means are different from one another Small= a smaller numbered F stat, or an F stat close to 1, indicates that the between-groups variance and the within-groups variance are about the same -Increased spread among the sample means and the decreased spread within each sample contribute to an increase in the F stat -greater overlap indicates that the means are closer together (between-groups variability) and greater spread indicates that the means are further apart (within-groups variability) -distributions with a lot of overlap suggest that any differences among them are probably due to chance while distributions with less overlap are less likely to have been drawn from the same population (less likely that any differences among them are due to chance)
What do the lines on a corresponding ANOVA bar graph tell us about interactions?
Lines that get stronger or weaker: quantitative Lines that reverse direction: qualitative Lines that are parallel and do no cross: no interaction effect (usually unlikely-real life data is messy) Lines that cross: interaction, significant differences in differences. They can either be a visible interaction or an interaction that would occur if you were to extend the lines further along the graph (unparallel) ex. There is no difference between taking meds with grapefruit juice or water
What would a correlation coefficient of +/-1.00 indicate about the data?
That our data falls in a perfectly straight line moving in the positive direction (+1.00) or negative direction (-1.00) -knowing the score of one variable will allow you to know the score of the other variable, regardless if you calculated it or not -this is highly unlikely to happen in real life
What does a correlation coefficient of 0.00 indicate about the data?
That our data has no correlation and that there is no association between our two variables
What does the steepness of the tell tell us?
That the DV changes as the IV increases by 1 ex. For each class we skip (IV changes by 1), we can expect out exam grade (DV) to change by either a factor of +5.39 (less skipped classes) or -5.39 (more skipped classes)
Describe key characteristics of the F distribution (what is the distribution compose of? what is the typical shape? etc.)
The F distribution is a squared version of a z or t and we only have ONE cutoff for a two tailed test. The curve is a bell curve but a greater percentage of the graph lies towards the left, where 1 would be (when the two estimates are the same) -The F distribution is also the ratio of two measures of variance: the between groups variance (MS between) and variance within samples (MS within)
F statistic
The F statistic calculates between-groups and within-groups variance to conduct the hypothesis test called an ANOVA: it is the ratio of two measures of variance: WILL ALWAYS BE A POSITIVE VALUE! 1. Between-groups variance which indicates differences among sample means 2. Within-groups variance which estimates an average of the sample variances -it is the between groups variability divided by the within groups variability The F stat can be found on the F table and it is the distribution of ever possible combination of sample size (one type of df) and the number of samples (another type of df)
What are the advantages for using a within-groups ANOVA? Given the advantages, why don't we always use this design? What concerns might be present in these designs?
The advantage of using a within-groups design over a between-groups design is that you can use less participants and you will be able to reduce the within-groups variability associated with differences among people in a study across the groups. By reducing this variability, you will make the denominator in the F stat equation smaller and this will produce a larger F stat in general which will enable you to more accurately reject the null hypothesis
Identify the source of the "subjects' effect" in a repeated measures ANOVA. When will the subjects' effect value be large?
The denominator if the effect size equation takes into account the total variability, SStotal, but removes the variability caused by differences among participants, SSsubjects -this enables us to determine variability explained only by between-groups differences -the subjects effect value will be large if we have a small number of participants in a study
Between-groups variance
The estimate of the population variance based on the differences among sample means a) a big number in the numerator indicates a great deal of distance/spread between the means suggest they came from different populations b) a smaller number in the numerator indicates very little spread between the means suggesting that the samples came from the same population -It is the estimate of the variability among the means
Within-groups variance
The estimate of the population variance based on the differences within each of the three or more sample distributions -refers to the average of the number of variances
What is meant by experiment-wise errors, and how does this relate to the risk associated with multiple t tests? What is the difference between setting an error rate per pair vs. experiment-wise?
The experiment - wise error rate is the probability of making at least one Type I error when performing the whole set of comparisons -with more comparisons you increase your risk of committing a Type I error
What criteria are used in selecting one particular regression line through the data set? Why is it considered a "best fit" line? Compare with the mean as a measure of central tendency.
The line of best fit is very specific to each data set; the line should got right through the center of the data points and it is influenced by outliers if there are any -the line leads to the least amount of errors in prediction -the mean can be used as a measure of central tendency because it is equal to the sum of all the values divided by the number of values and it represents the value that is most common in your data set; can be used with discrete and continuous data, and it includes every data point in your data set in its calculation; it is also greatly influenced by outliers -the line of best fit is better in some cases because it can predict where possible values may fall in relation to others; it is influenced by outliers in a sense but not as much as the mean; it can be used for both normal and skewed data, which is a disadvantage for the mean which is more similar to the median and mode when the data is normally distributed. The proportionate reduction in error is a statistic that quantifies how much more accurate predictions are when we use the regression line instead of the mean as a prediction tool
Why are main effects often ignored when there is an interaction?
The main effects are ignored when there is an interaction because if the main effects ARE statistically significant, they are further qualified by the interaction effect, which you will describe in the summary. The pattern seen within the cell means will tell the story -you should discuss if one main effect is statistically significant and one is not, especially if you were unable to reject the null hypothesis for the interaction
Effect Size for ANOVA, R^2
The proportion of variance in the dependent variable that is accounted for by the independent variable -SS between/SS within -R^2 tells us how big of a difference is, but we don't know which pairs of means are responsible for these effects
What does subjects degrees of freedom mean?
The subjects degrees of freedom corresponds to the sum of squares for differences across participants, the subjects sum of squares or SSsquares -the degrees of freedom is found by subtracting 1 from the total number of participants in the study, since we have a singular sample of participants
When using ANOVA, what different kinds of factors can be used? Be able to offer examples, especially for multi-way ANOVA?
The term factor is a synonym for independent variable. With a one-way design, we have one nominal independent variable with more than two levels and a scale dependent variable. With a two-way design, we have two nominal independent variables, regardless of levels and a scale dependent variable -a multi-way ANOVA, or a multifactorial ANOVA, is called such because we can add a growing number of nominal independent variables to the study; we just have to state how many variables we are testing by the number we put in front of our ANOVA -when we call an ANOVA a 3x2 or so on, we are describing the number of factors and the number of levels of each factor; this is in reference to the cells
What is meany by R^2 or n^2? How is it interpreted?
This is the effect size for an ANOVA test, it is used to determine just how big the difference are and it can either be small, medium, or large in effect -when we reject the null hypothesis, we know that at least one of the means is different from at least one other mean (we don't know where exactly the differences lie until we conduct a post hoc test such as the Tukey HSD test)
What happens when you have a larger sample size when calculating a correlation? What can you do about it?
With a larger sample size, you are likely to have more variability in your data and you are likely to get a large correlation value (one that exceeds +/- 1.00). The more people you have in your data, the more deviations get contributed to your sum; if the scores are more variable, the deviations will be larger and so would the sum of the products You can correct this problem by fixing the denominator. In the denominator of the correlation coefficient, we divide by the square root of the sum of scores for both variables. By using the sum of squares, SSx and SSy, we remove the influence of sample size in order to calculate the variance. The numerator is the sum of the products of the deviations for each variable
linear regression equation
Y (hat)= a+b (x) a=intercept, the predicted value for Y when X is equal to 0, which is the point at which the line crosses or intercepts, the y-axis b= slope, the amount that Y is predicted to increase for an increase of 1 in X ex. When a student misses 0 classes, they have a predicted exam grade of 94.30; when a student misses 1 class, they have a predicted exam grade of 88.919. The slope of the difference is negative, such that 94.30-88.919 is -5.39 Y (hat)= 94.39-5.39(X) If someone misses two classes, then 94.39-5.39(2)= a score of 83.52 on an exam -If you were to continue to input the number of missed classes based on this original linear regression equation based on 0 missed classes, you would see that you have a negative trend; as the number of missed classes increases, the score on the exam decreases
Are the assumptions for a one-way between groups ANOVA the same as a one-way within groups ANOVA? Is there any differences?
Yes! With one exception... 1. Random selection should be used, if it is not you should be cautious about generalizing it across the rest of the population 2. The population distribution should be normally distributed, but data should usually show that it is not skewed if you do not know its distribution pattern 3. Homoscedasticity should be test by comparing the largest variance to the smallest variance, making sure that it is not more than twice the smallest. 4. You must counterbalance in order to determine any outside effects; if mo counterbalance is done, there may be order effects
Is the value of Pearson correlation coefficient the same as the value of the standard regression coefficient ?
Yes, unless there was an error in rounding. Both coefficients indicate the change in standard deviations that we expect when the IV increases by 1 standard deviation -this fact is only true when there is one IV
How many critical values will you need to look up in order to analyze where your F stats fall into play for a two-way between groups ANOVA?
You will need three critical values, based of F(between df, within df) for the two main effects and the interactions -you could have all three values go beyond the critical value, or you could only have the interaction effect be significant
Relative Risk
a measure created by making the ratio of two conditional proportions; it is also called relative likelihood or relative chance -by making a ration of two conditional proportions, we can say that one group is twice as likely or three times as likely to show some outcome or that the other group is half or a third as likely to show that outcome
chi-square test for independence
a nonparametric test that is used when there are two nominal variables; like the correlation coefficient, the chi-square test for independence does not require that we identify the IV or DV, but knowing an IV/DV can help you articulate hypotheses -the chi-square test for independence is used to determine whether the two variables are independent of each other (no matter which one is considered to be the IV) -based on the chi-square distribution
chi-square goodness of fit
a nonparametric test that is used when there is one nominal variable; there is no IV or DV, just one categorical variable with two or more categories into which participants are placed -measures how good the fit is between the observed data in the various categories of a single nominal variable and the data we would expect according to the null hypothesis -if there is a really good fit with the null hypothesis, then we cannot reject the null hypothesis (we are hoping for a bad fit between the observed data and what we expect according to the null hypothesis) -based on the chi-square distribution
standard regression coefficient
a standardized version of the slope in a regression equation, is the predicted change in the DV in terms of standard deviations for an increase of 1 standard deviation in the IV; symbolized by beta and often called the beta weight; we use the slope of the regression line equation, b beta=(b) (square root SSx/ square root SSy)
standard error of the estimate
a statistic indicating the typical distance between a regression line and the actual data points; it is the amount of error around the line of best fit and it can be quantified -the standard deviation of the actual data points around the regression line
Pearson correlation coefficient
a statistic that quantifies a linear relation between two scale variables; it is a single number that is calculated to determine the direction and strength of a relationship between two variables when their overall pattern indicates a straight-line relation -it is symbolized by the Greek letter "p" (rho) or an italic letter, r -usually, you only use rho (p) when you are referring to the population parameter -it can also be used as an inferential statistic that relies on a hypothesis test to determine whether the correlation coefficient is significantly different from 0 (no correlation)
proportionate reduction in error
a statistic that quantifies how much more accurate predictions are when we use the regression line instead of the mean as a prediction tool; also called the coefficient of determination; it tells us how good the regression equation is in comparison to the using the mean as the predictor -more specifically it is a statistic that quantifies how much more accurate predictions are when scores are predicted using a specific regression equation rather than when the mean is just predicted for everyone -it is easier to use the regression line equation in place of the mean when we are making a prediction about a certain set of data points
path analysis
a statistical method that examines a hypothesized model, usually by conducting a series of regression analyses that quantify the paths between variables at each succeeding step in the model
structural equation modeling (SEM)
a statistical technique that quantifies how well sample data "fit" a theoretical model that hypothesizes a set of relations among multiple variables -encourages researchers to think of variables as a series of connections
partial correlation
a technique that quantifies the degree of association between two variables that remains when the correlations of these two variables with a third variable are mathematically eliminated; used when a third variable may be influencing the co-relation of the first two variables -removes all overlapping variability of each variable with a third ex. allows researchers to calculate the partial correlation of number of absences and grade, while correcting for percentage of homework assignments completed
analysis of covariance (ANCOVA)
a type of ANOVA in which a covariate is included so that statistical findings reflect effects after a scale variable has been statistically removed -specifically, a covariate is a scale variable that we suspect associates, or covaries, with the IV of interest -we are subtracting the possible effect of a confounding variable
stepwise multiple regression
a type of multiple regression in which computer programs determine the order in which IVs are included in the equation -the computer will first identify the IV responsible for the most variance in the DV (the IV with the highest R^2) -if this variable is statistically significant, then the computer will move onto the next IV to see if that one is statistically significant (if the first one was not statistically significant then the computer program would stop) - if both the first and second IV are statistically significant together in terms of R^2 and this value is over the value of the first IV R^2 alone, then the computer chooses the IV responsible for the next largest amount of variance -very resilient on data; results can generate hypotheses than the researcher can then test
hierarchical multiple regression
a type of multiple regression in which the researcher adds IVs into the equation in an order determined by theory -each additional IV will add to the R^2 value
positive correlation
an association between two variables such that participants with high scores on one variable tend to have high scores on the other variable as well -positive correlations describe situations in which participants tend to have similar scores with respect to mean and spread, on both variables, whether or not the scores or low, medium, or high ex. Hours spent studying on test performance- the more you study, the higher your grade
negative correlation
an association between two variables such that participants with high scores on one variable tend to have low scores on the other variable ex. Number of class absences on final exam grade- the more absences, the lower the grade; less absences, the higher the grade
chi-square test
chi-square tests are statistical tests that are used when all of our variables are nominal (both IV and DV); we can count the frequency of any event and assign each frequency to one (and only one) category, then the chi-square statistic lets us test for the independence of those categories and estimates effect size -allows us to compare what we observe with what we expect Symbol: X^2
Pearson correlation coefficient degrees of freedom
df= N-2 We subtract 2 from the sample size, which takes into account the number of participants and not the number of scores -your comparison distribution is based on an r distribution using this calculated degrees of freedom
Calculating the proportionate reduction in error and how it compares to the mean
r^2=(SStotal-SSerror)/SStotal SStotal=sum of squares total, this is the measure of error that would result if we predicted the mean for every person in the sample (it is the total error we would have if here were no regression equation SSerror=sum of squared errors, for which the error is the actual score minus the predicted score and all of the errors are squared and summed together (it represents the error that we'd have if we predicted Y using the regression equation) 1. Determine the error associated with using the mean as the predictor (SStotal) 2. Determine the error associated with using the regression equation as the predictor (SSerror) 3. Subtract the SSerror from the SStotal 4. Divide the difference between the SStotal and SSerror by the SStotal -once we add all of our data points and create a scatterplot, we connect the points to the regression line and these vertical lines represent the error that results from predicting Y for everyone using the regression equation -the word proportionate indicates that we want a proportion of the total error that we have reduced, which is why we divide the difference of SStotal and SSerror by the SStotal ex. A reduction of 0.724 or 72.4% of the original error means that we have that much better of a chance at predicting Y by using the regression equation over the mean
test-retest reliability
refers to whether the scale being used provides consistent information every time the test is taken -to calculate the measure's test-retest reliability, the measure is given twice to the same sample, typically with delay in between tests -a large correlation indicates that the measure yields the same results consistently over time (good reliability) -a small correlation would indicate that the measure yields different results over time (bad reliability)
grand mean
the mean of every score in a study, regardless of which sample the score came from GM= sum (x)/ Ntotal
nonparametric tests
we use nonparametric tests when 1) the DV is nominal 2) the DV is ordinal or 3) the sample size is small and the population of interest may be skewed -nonparametric tests allow us to distinguish between pattern -used whenever we categorize the observations, describe rank, such as class position, athletic place, preferred flavor of ice cream, or when we have a small sample size (such as the number of people that have Nobel Peace Prizes...a very select and small group)
Tukey HSD
widely used post hoc test that determines the differences between in terms of standard error; the HSD is compared to the critical value -it is also called the q test 1. calculate the difference between each [air of means 2. divide each difference by the standard error 3. compare HSD for each pair means to a critical value to determine whether the means are different enough to reject the null hypothesis