Mult. Regression Exam 3
Given an ANCOVA model with 3 treatment groups and 1 (metric) covariate, how is the overall treatment effect tested in multiple regression? What is the df for the overall treatment effect? What is the df for the covariate?
(SEE ANSWER IN STUDY GUIDE)
Depict the relationship among the variables in ANCOVA using Venn diagrams and path models in a true experiment versus in a non-experimental control group (a.k.a. observational) study. Why is there no correlation expected between the covariate and the treatment in the true experiment? With what two things can the covariate be related in this type of quasi-experiment with nonrandom assignment?
(see figure) Why is there no correlation expected between cov. and treatment in true experiment? "If subjects are randomly assigned to treatment conditions, the expectation is that there will be no correlation between any baseline covariate variable and treatment assignment (unless there is faulty randomization that assigns participants to treatment etc.)" [Categorical x Continuous handout, pg. 9] With what two things can the covariate be related in this type of quasi-experiment with nonrandom assignment? · This problem of (a) participant selection into treatment results in potential correlation between the pre-existing characteristics of the subject and assignment to treatment. (b) The covariate may also be correlated to a smaller or larger degree with the criterion Y. When both (a) and (b) occur, we have confounding: The covariate serves as an alternative explanation of the relationship between the treatment and the outcome. [Categorical x Continuous handout, pg. 10] · Note: this is shown in the venn diagram above where cov. is related to (a) treatment and (b) outcome
What is the general strategy for studying the effect of a point on the regression outcome (delete the point and rerun analysis).
1. Compute regression analysis with case included, 2. Repeat the analysis with case deleted, 3. Assess how some result in regression has changed
What are the two approaches that may be taken with data imputation with multivariate missing data? What are the primary strengths and limitations of each?
1. Data Augmentation makes a strong assumption that the complete data have a multivariate normal distribution. This implies all relationships are linear. The imputation step is conducted simultaneously for all variables having missing data. When the data are continuous and approximate multivariate normality, this approach can be very efficient. 2. Fully Conditional Specification (FCS, a.k.a. chained equations). In this approach, each variable with missing data is imputed one at a time. The variable (say X1) with the least missing data is imputed first, then the variable (say X2) with the second least missing data is imputed second, and so on. Each regression equation represents the measurement level of the outcome variable. If X1 is binary, logistic regression is used; if X2 is a count variable, Poisson regression is used, ... Thus, FCS is far more flexible than data augmentation. However, FCS is far more likely to have problems achieving a solution and may not return any results.
Describe the steps involved in conducting a propensity score analysis in an observational study comparing students in one school given an active treatment (T) and students in a second school given a control treatment (C).
All identified covariates that might be confounding the treatment-outcome base are measured at baseline · A logistic regression equation is used to estimate the probability that each participant would receive the treatment o In brief we estimate the log of odds (the "logit") that person i is a case · We identify cases in each group that we can closely match on the logit of their propensity score o If cases are properly matched on propensity scores, then they will be balanced within sampling error on all covariates that are involved in the prediction of the propensity score. This outcome mimics a randomized experiment, but only for measured covariates · Each of the covariates are checked for balance · The two matched groups are compared on the outcome
What is meant by auxiliary variables? Describe an advantage of multiple imputation in its utilization of auxiliary variables.
Auxiliary Variables - Imputation Phase can be based on all available variables that might predict missingness (e.g., distance to site). Analysis phase based only on variables of theoretical interest. The advantage of using auxiliary variables is that they can potentially make MAR more like MCAR. Additional terms can be added to the imputation phase to represent curvilinear or interactive relationships. · Imputation phase can be based on all available variables that might predict missingness (variables of theoretical interest + auxiliary variables). Analysis phase based only on variables of theoretical interest · Potentially make MAR (data Missing At Random) like MCAR (data Missing Completely At Random) · Inclusive Analysis Strategy (Collins, 2001) attempts to identify auxiliary variables that are useful for imputation, either b/c they reduce nonresponse bias or improve power o In the context of missing data handling, the goal is to control for (condition on) auxiliary variables that differentiate the complete and incomplete cases. Simple bivariate correlations can help identify potential auxiliary variables, as can path analysis models (Enders, 2017)
In coding of a two-group variable, we considered the use of unweighted effect codes (UE: +1,-1) versus the contrast codes (C: +.5, -.5). In the equation = b1 X + b2 C + b3 XC + b0 when using the contrast code versus = b1 X + b2 UE + b3 X*UE + b0 when using the unweighted effects codes, explain how the numerical values of coefficients will change when you switch coding systems. Will the significance of the coefficients change when you change between these two coding systems?
Change of numerical values: b0 and b1 will not change; b2 and b3 will be half the value in unweighted than they were in contrast Will significance change?: Overall significance will stay the same, but significance of specific coefficients (b2 and b3) may differ between models The significance of the coefficients does not change between coding schemes. Contrast codes: B0 = the unweighted average intercept B1 = the unweighted average slope B2 = the mean of the group coded .5 - mean of the group coded -.5 B3 = the slope of the group coded .5 and slope of the group coded -.5 Unweighted effects codes: B0 = the unweighted average intercept B1 = the unweighted average slope B2 = the half mean of the group coded .5 - mean of the group coded -.5 B3 = the half slope of the group coded .5 and slope of the group coded -.5
A design matrix is a set of codes used in ANOVA. Consider a 2 x 2 factorial design. There are two levels of A (low, high) and two levels of B (low, high). Write the design matrix for the 2 x 2 design in ANOVA. Otherwise stated, what do C1, C2, and C3 (code variables) look like?
Condition C1 C2 C3 Low A, Low B -1/2 -1/2 +1/2 Low A, High B -1/2 +1/2 -1/2 High A, Low B +1/2 -1/2 -1/2 High A, High B +1/2 +1/2 +1/2 For the rescaled contrast codes, b0 is the unweighted mean of the three groups. b1 is "main" effect of A (difference between high and low means) b2 is "main"effect of B (difference between high and low means) b3 is the A x B interaction
DATA LAYOUT. I will give you the layout of a categorical variable with several levels, and a different number of cases per level. For example, I might give you four groups, with n1=2, n2=4, n3=5, and n4=3. Be able to create CONTRAST CODES.
Contrast Codes: C1 C2 C3 G1 -1/4 -1/3 1/2 G2 -1/4 -1/3 -1/2 G3 -1/4 2/3 0 G4 3/4 0 0 Yhat = b0 + b1C1 + b2C2 + b3C3 B0 = the unweighted mean of all four groups B1 = the difference between the mean of G4 and the mean of G1-G3 (i.e., G1 + G2 + G3 / 3) B2 = the difference between the mean of G3 and the mean of G1 and G2 (i.e., G1 + G2 / 2) B3 = the difference between the mean of G1 and the mean of G2
What does DFBETAS measure?
DFBETAS is a measure of standardized change in regression coefficient when case is deleted (a set per case) Standardized change in a regression coefficient—by how many standard errors does a single case change each regression coefficient? One score per regression coefficient per case; regression coefficient with case in analysis versus removed from analysis Measure of influence
What does DFFITS measure?
DFFITS is a measure of standardized change based on predicted scores (one per case) Standardized change in predicted score —by how many standard errors is the predicted score changed by one case? One score per case; case deleted from predicted score and standard error Measure of inluence
What are possible solutions when you detect an outlier?
Deletion method - The initial analysis is run (analysis 1), the most problematic outlier is deleted, then the analysis is re-run (analysis 2) and new outliers are identified. This process continues until the set of outliers have been detected and deleted one by one. Note that different cases may show up as outliers in each subsequent analysis following deletion of a case. That is why it is important to delete one outlier per analysis. Report both original analysis with outlier and the analysis without outlier with explanation. Two issues with this approach: (a) From the standpoint of statistical inference, deleting a case based solely on leverage (the values of the X variables) has a different conceptual status than deleting cases based on distance or influence (involving the Y variable). (b) When outliers are deleted, exactly what is being estimated (the estimand) or tested by the statistic becomes a bit murkier. Transformations - Change the metric of the data (i.e., taking the log of Y and X and rerunning analysis). The transformation does not always make outliers disappear. Sometimes new and different outliers will appear following transformation Robust Regression - use robust estimators that are less influenced by extreme values (e.g., least absolute deviation LAD when there are outliers on Y and not X; Thiel-Sen Estimator when working with single predictor regression; Least trimmed squares; or bootstrapping).
What are deterministic outliers (a.k.a., contaminated observations)? What are some example sources? What are probabilistic outliers (a.k.a., rare cases)? What is an example source?
Deterministic outliers (i.e., contaminated observations) are errors in data collection. Some example sources include situations where interviewer misread questions, errors in recording, state of respondent (e.g., intoxicated). Probabilistic outliers (i.e., rare cases) occur when the extreme value is in fact a true possibility. An example source would be having a misspecified model (i.e., have a linear instead of quadratic model). In order to remedy this extreme value a researcher could delete the case, correctly specifying a model, or transform the data
Why are the VIF statistics typically not reported with a categorical IV with G groups?
Different coding schemes will produce different VIFs, so the standard measures of tolerance/VIF for variables comprised of multiple terms are not meaningful All G-1 variables are needed to represent the full categorical variable not just a single variable
Explain distance. Is a specific regression model required to measure distance? Does high distance necessarily mean that a point is affecting the regression outcome?
Distance: potential to affect regression outcome (How far Y_hat is from the regression line) Regression equation is required (Residual is the basis of all measures of distance) High distance means a point is affecting regression model = NOT NECESSARILY, instances where it does NOT change slope but would increase the intercept
For each coding scheme, be able to take the general regression equation and the codes and interpret what each of the coefficients in the equation is measuring. If the reference (base) group is changed for dummy, weighted, or unweighted effects codes, be able to explain how each of the coefficients would change.
Dummy codes: B0 = mean of the reference group Bi = the difference between the mean of Gi and the mean of the reference group If the reference (base) group changed then each coefficient would change. B0 would represent the mean of the new reference group and Bi would be the difference between Gi and the new reference group. Effect codes: B0 = the weighted/unweighted mean of the total number of groups G Bi = the difference between the mean of Gi and the weighted/unweighted mean of all groups G. If the reference (base) group changed then I don't think the coefficients would change.
DATA LAYOUT. I will give you the layout of a categorical variable with several levels, and a different number of cases per level. For example, I might give you four groups, with n1=2, n2=4, n3=5, and n4=3. Be able to create the DUMMY CODES.
Dummy: *G4 is the reference group C1 C2 C3 G1 1 0 0 G2 0 1 0 G3 0 0 1 G4 0 0 0 Yhat = b0 + b1C1 + b2C2 + b3C3 B0 = mean of the reference group G4 B1 = the difference between the mean of G1 and the mean of the reference group G4 B2 = the difference between the mean of G2 and the mean of the reference group G4 B3 = the difference between the mean of G3 and the mean of the reference group G4
Be able to set up contrast codes if given a specific set of a priori hypotheses.
Example: We predict we need high motivation (A) and high ability (B) to show a good effect as opposed to any other combination of the two variables so low/low, the two low/high combos, would all have -1; high/high would have +3 if there were any groups we didn't want to include in the comparison, it gets a 0
If the within class regression slopes differ as a function of the categorical variable in ANCOVA, what does this tell you about the slopes of the within class regression lines in the groups of the ANCOVA? How is this tested? What is the difficulty in coming up with an estimate of the treatment effect in ANCOVA if the within class regression slopes are not parallel?
If the within class regression slopes differ as a function of the categorical variable, the slopes differ, and we do not have an additive effect of the treatment variable on the criterion for all values of the covariate. We can see if the regression lines are parallel by fitting separate regression lines to the data in each group graphically. We can also test for a significant interaction between the treatment and covariate using: = b0 + b1 COV + b2 T + b3 T * COV Test for the interaction with the b3 coefficient The difficulty here is that the main treatment effect is not constant on all levels of the covariate (no additive effect). We need to do more investigating to understand the actual effect because it varies.
What is the problem in regression diagnostics with clusters of errant points?
If you have a cluster of points that are working together to influence the the outcome, then removing one of the cases will not alleviate the problem since the other errant points will continue to affect the outcome.
Explain influence. Is a specific regression model required to measure influence? How does influence relate to leverage and distance. Does high influence necessarily mean that the point is affecting the regression outcome?
Influence: amount by which a data point changes the regression equation (it is function of leverage and distance Influence = leverage x distance) · A point may have high leverage or high distance and not affect the position of the regression plane. A point that has both high distance and has high leverage potentially can move the regression plane, but not all such points do so. High influence change in each regression coefficient and the regression intercept and therefore change in the predicted score. Influence is the amount by which a data point changes the regression equation. Influence = leverage x distance
Explain leverage. Is a specific regression model required to measure leverage? Does high leverage necessarily mean that a point is affecting the regression outcome?
Leverage is the potential of a point to affect the regression coefficients, depending on how extreme the case is solely on the predictors (how far is the data point from the center to the data) No regression equation is required to measure leverage. High leverage = high POTENTIAL to change slope, intercept, or both (does NOT mean that a point IS affecting the regression outcome) No, we don't need a specific regression model to measure leverage The effect on the regression outcome depends on how extreme the case; if it is far from the centroid but still near the linear path, it won't affect the outcome
What are the three characterizations of errant data points? --leverage, distance, influence.
Leverage, or the extent to which the case is close to or far from the rest of the cases, in terms of scores on the predictors X1...Xp only. Distance (discrepancy), or the extent to which the score on the criterion is extreme, given the values on the set of predictors. Influence, or the extent to which a single data point changes the outcome of the regression analysis.
What are the two types of measures of influence? What does each measure?
Measures of global change in the whole regression equation. DFFITS was the measure I emphasized (also Cook's D). Measures of specific change in each regression coefficient. There is a set of measures for each case, one measure for each regression coefficient in the equation including the intercept.
I will give you computer output containing exactly the same regression diagnostics as on the computer printout from class example and will ask you to point out cases with high leverage, distance, influence and to indicate how regression coefficients are being influenced by individual points.
Memorize figure in study guide
There are two new methods of treating missing data that have become the current state of the art. One was discussed in class. What is this method? What is the second method that is commonly available in structural equation and multilevel modeling software? If one uses one of these approaches to handling missing data, what missing data mechanism must be operating to yield unbiased parameter estimates?
Multiple imputation (in class) and Maximum likelihood (in SEM): These methods assume mechanism MAR; will be more accurate & assume greater power Multiple imputation (MI): • We begin with one data set that has missing values • Multiple imputation (MI) creates multiple copies of the data (typically 20 or more in psychology), each of which has a different set of plausible replacement values • MI is preferable to traditional methods because it requires the less stringent and more realistic MAR assumption (MI analyses will be more accurate and will produce greater power)
Are unweighted effects codes orthogonal (uncorrelated)?
No, because they share the same base group.
Are the pairs of dummy codes in a dummy variable coding scheme orthogonal?
No, since they share the same base group The sum of the weights does not equal zero (e.g. 0 + 1 = 1, not 0)
Do you get different numbers in the analysis of regression summary table if you use weighted effects codes versus unweighted effects codes to code a categorical variable with unequal group size?
No, there will be identical R^2 and ANOReg tables since these are equivalent models.
Are dummy codes centered?
No.
What does it mean if two codes from a coding scheme are orthogonal (uncorrelated)? What conditions need to be met for the following relationship to hold where c1, c2, and c3 are code variables? r2multiple = r2y,c1 + r2y,c2 + r2y,c3
Orthogonal means the sets of codes are uncorrelated (independent) with each other · Two conditions to be Orthogonal codes: o Sum of the weights = 0 o Sum of the product of the weights for different code variables = 0 o For max interpretability, code should be 1-unit apart (NOT necessary for orthogonality) § Example with three groups C1 C2 C1*C2 → (product of the weights) Control 1 -1 -1 1 Control 2 -1 1 -1 Treatment 2 0 0 ∑0 ∑0 ∑0 The proportion of variation accounted for by one is not related to the proportion of variation accounted for by the others. So you can add them up if all three variables are uncorrelated, since they each account for their own unique variance to the squared multiple correlation.
What is the primary potential advantage of ANCOVA over ANOVA in randomized experiments? What is the primary purpose of the inclusion of covariates in ANCOVA in the non-equivalent control group design (a.k.a., observational study)?
Primary advantage in randomized experiments: Covariate included in the design to partial out variability from the outcome that is unrelated to the treatment, thereby increasing the statistical power of the test for treatment effect. "To the extent that the covariate removes from the criterion irrelevant variance, the statistical power of the ANCOVA will be greater than that of the corresponding ANOVA." [Categorical x Continuous handout, pg. 9] Primary advantage in observational studies: For the observational study, the primary role of ANCOVA is to partial out pre-existing between group differences on covariates that are related the criterion that should not be attributed to treatment—these differences are due to participant selection into treatment. In addition ANCOVA might also partial out error variation in the criterion (as in the true experiment). [Categorical x Continuous handout, pg. 10]
In any coding scheme for G groups, how many code variables are required to characterize the G groups?
Recall that for G groups, we need G-1 code variables to represent the full set of differences between the group means.
What is the solution to the problem described in Q. 76? How is MSresidual(i) computed? I will refer to residuals divided by their deleted standard errors as "externally studentized".
Run the regression with the cases removed and compute the predicted score as well as the standard error from the analysis with the case removed.
Why are standardized solutions NOT typically reported with categorical IVs?
Standardized solutions assume that a very good estimate of the population variance is available. With categorical variables, this depends on having a random sample from the population to which you wish to generalize so that the proportion in each category in the sample represents the proportion in the population. When we don't have a representative population, the standardized effect can be widely inaccurate.
What is meant by the statement that "the different coding schemes represent equivalent regression models?"
The R^2 and F-test of the ANOReg is identical across the coding schemes.
What is the problem with simply dividing a residual by its standard error to compute a standardized residual?
The case can move the regression plane toward itself and reduce its own residual and increase the residual for all other cases. This would create a larger MSresidual. The standard errors can be unstable, particularly if they are outliers in the data
Be able to take a 2 x 3 ANOVA (two levels of factor A and three levels of factor B) and show the design matrix. Test the hypotheses that the mean of B1 does not differ from the mean of (B2 and B3) and that the means of B2 and B3 do not differ."
The correct coding to have a 1-unit difference between for example A1B1 and the mean of A1B2 + B1B3 which facilitates interpretation of the difference as it corresponds to a 1-unit change Condition C1 C2 C3 C4 C5 A1, B1 -.67 0 A1, B2 .33 -.5 A1, B3 .33 .5 A2, B1 -.67 0 A2, B2 .33 -.5 A2, B3 .33 .5 Yhat = b0 +b1C1 + b2C2 +b3C3 + b4C4 + b5C5 The t-test of b1 is the test of the mean of B1 versus the mean of B2 plus B3. The t-test of b2 is the test of the mean of B2 versus the mean of B3. Alternatively, the term b1 could be set to 0 and the full model versus the reduced model without the b1C1 term could be compared using the test of gain in prediction. The F-test for the gain in prediction with one term dropped will equal t2 for the test of the corresponding regression. In the one predictor case (1 df in numerator of F), the gain in prediction and t-test yield identical conclusions. When the effect of b2 is dropped, the F-test for the gain in prediction (and the t-test for b2 in the full model) will yield identical results (F = t2).
What measures are on the main diagonal of the hat matrix denoted hii?
The elements on the main diagonal are the measures of leverage of each data point in the dataset.
If you have unequal sample sizes in the groups in a data set, describe the two grand means that can be computed. Then explain how the unweighted effects codes versus the weighted effects codes measure discrepancies between the group means and these two different grand means. When do unweighted and weighted effects codes yield the same results?
The grand means can be weighted or unweighted. In the unweighted mean, each group is weighted equally: (Ybar1 + Ybar2 + Ybar3 + ... + YbarG) / G In the weighted mean, each group is weighted by its sample size: [n1ybar1 + n2ybar2 + n3ybar3 + ... + ngybarg] / [n1 + n2 + n3 + ... ng] *Note the weighted effects coding scheme presumes that we have a truly random or representative sample so that the proportion of cases in each group is representative of the proportion of that group in the population. Rarely the case in psychology. The weighted and unweighted effect codes yield the same results when each group has an equal sample size (n1 = n2 = n3).
What is the centroid?
The point that represents the mean of every predictor in the data space
What weaknesses of ANCOVA do propensity score approaches address? What is meant by the region of support?
The propensity score approach addresses four weaknesses of ANCOVA. · It permits proper adjustment for a large number of covariates. · It provides good checks on the adequacy of the adjustment model. · It does not rely on properly representing the functional form of the relationship between the covariate and the outcome. · It does not allow risky extrapolation. The region of support: where there are cases from both groups. Outside the region of support, no direct comparison is possible. Propensity scores permit comparison only within the region of support.
. How does the ANOVA summary table for the data in the DATA LAYOUT relate to the analysis of regression summary table when the groups are coded with a set of unweighted effects codes?
They will be identical.
?????84. Briefly describe how the major robust approaches to multiple regression.
Thiel-Sen Estimator In the one predictor case, this estimator is highly resistant to outliers. This method estimates the slope by taking the median of the slopes that are estimated using all possible pairs of cases in stand. OLS regression. A downfall is that the standard errors may be larger than other estimators. Least Trimmed Squares When computing the OLS estimate of the regression coefficient, this method sorts the squared residuals from lowest to highest and trims them with using an apriori percentage. Bootstrapping Bootstrap sample 2000 times and estimate the regression coefficients and then trim by 2.5% of the highest and lowest values.
What are the three stages of multiple imputation.
Three Phases of Multiple Imputation 1. Imputation phase Create multiple copies of the data, each with different imputed values for missing values 2. Analysis phase Perform standard statistical analyses (e.g., MR) separately on each data set 3. Pooling phase Combine the collection of estimates and standard errors into a single set of results
Suppose you have two groups that you have dummy coded as follows: C1 C2 T1 1 0 T2 0 1 C 0 0 How would you compare the means of T1 and T2 from the regression output?
To estimate the means, mean T1 = b0 + b1 and mean T2 = b0 + b2. To test the difference between the means of T1 and T2, recode as follows: C1 C2 T1 0 0 T2 0 1 C 1 0 Then the test of b2 is the test of the difference between the means of T1 and T2.
DATA LAYOUT. I will give you the layout of a categorical variable with several levels, and a different number of cases per level. For example, I might give you four groups, with n1=2, n2=4, n3=5, and n4=3. Be able to create UNWEIGHTED EFFECTS.
Unweighted Effects Coding: *G4 is the base group C1 C2 C3 G1 1 0 0 G2 0 1 0 G3 0 0 1 G4 -1 -1 -1 Yhat = b0 + b1C1 + b2C2 + b3C3 B0 = the unweighted mean of the four groups combined B1 = the difference between the mean of G1 and the unweighted mean of the three groups combined B2 = the difference between the mean of G2 and the unweighted mean of the three groups combined B3 = the difference between the mean of G3 and the unweighted mean of the three groups combined
DATA LAYOUT. I will give you the layout of a categorical variable with several levels, and a different number of cases per level. For example, I might give you four groups, with n1=2, n2=4, n3=5, and n4=3. Be able to create UNWEIGHTED EFFECTS CODES.
Unweighted Effects Coding: *G4 is the base group C1 C2 C3 G1 1 0 0 G2 0 1 0 G3 0 0 1 G4 -1 -1 -1 Yhat = b0 + b1C1 + b2C2 + b3C3 B0 = the unweighted mean of the four groups combined B1 = the difference between the mean of G1 and the unweighted mean of the three groups combined B2 = the difference between the mean of G2 and the unweighted mean of the three groups combined B3 = the difference between the mean of G3 and the unweighted mean of the three groups combined Weighted Effects Coding: *G4 is the base group C1 C2 C3 G1 1 0 0 G2 0 1 0 G3 0 0 1 G4 -2/3 -4/3 -5/3
?????83. What is the general strategy taken by robust statistics to the problem of outliers?
Use alternative estimators to OLS regression that are robust to outliers.
Are unweighted effects codes centered for equal group size? for unequal group size?
Weighted effect sizes can be centered only if group sizes are equal.
Suppose you run a one factor analysis of variance on a data set like that in the DATA LAYOUT above, Q .4. What will be the relationship of the resulting ANOVA summary table to a regression analysis in which a set of dummy codes are used to code the four groups, and the criterion is the same as the dependent variable in the regression analysis?
When categorical variables are the independent variables, we have a special case--classic oneway ANOVA. Only one value is predicted for each category (group). SSregression is entirely due to differences between the treatment groups. SSresidual is entirely due to differences between the predicted value for the entire group and the scores of the individual subjects. These correspond to SSbetweeen groups and SSwithin groups in ANOVA. The ANOReg produces exactly the same results as the ANOVA
Suppose you have a categorical variable with four groups, e.g., four geographical regions of the US. You wish to use it as a predictor in a regression analysis. What is the general strategy for employing a categorical variable as a predictor in a regression analysis? What determines which coding scheme is used?
With four groups, the general strategy would be to create G -1 (i.e., 3) coding variables such that Yhat = b0 + b1C1 + b2C2 + b3C3. This would allow us to test the overall group effect. Then we would choose the coding scheme that allows us to test the specific focused hypothesis of interest. The following coding schemes would be used in the following situations: 1. Dummy coding would be used if the hypothesis involved comparing each group with a reference group. 3. Weighted/unweighted effects coding would be used if the hypothesis involved comparing the mean of each group with the overall mean of all the groups combined (i.e., the grand mean). The difference would be whether you would weigh the grand mean or not. Weighted effect coding scheme presumes that we have a truly representative sample such that the proportions of cases in each group is representative of the population. 4. Contrast codes would be used if the hypothesis involved comparing specific differences between groups or combinations of groups.
For unweighted effects coding, be able to take the general regression equation and the codes and explain what each of the coefficients in the equation is measuring.
Yhat = b0 + b1C1 + b2C2 + b3C3 B0 = the unweighted mean of the four groups combined B1 = the difference between the mean of G1 and the unweighted mean of the three groups combined B2 = the difference between the mean of G2 and the unweighted mean of the three groups combined B3 = the difference between the mean of G3 and the unweighted mean of the three groups combined
Consider a 2 x 3 ANOVA with 2 levels of A (low, high) and 3 levels of B (low, moderate, high). The design matrix uses five code variables as predictors of the dependent variable. Using a series of regression equations, explain how you would find SSA, SSB and SSAB. What are the numerator degrees of freedom for the A, B, and AB effects?
Yhat = b0 + b1C1 + b2C2 + b3C3 + b4C4 + b5C5 SSA: compare full model to reduced (without A effect b1C1); df= 1 SSB: compare full model to reduced (without B effect b2C2 + b3C3); df= 2 SSAB: compare full model to reduced (without interaction terms b4C4 + b5C5); df= 2
Write a regression equation for a categorical [SL1] predictor, a continuous predictor, and their interaction. Rearrange this equation into the simple regression equation for the regression of Y on X at values of the categorical variable C.
Yhat = b1 X + b2 C + b3 XC + b0 Yhat = (b1 + b3 C)X + (b2 C + b0)
Consider gender as a dummy coded variable, 1=male, 0=female. Suppose you have the coefficients for the overall regression equation, where X is continuous and D is a dummy code: Yhat = b1 X + b2 D + b3 XD + b0 Yhat = .4X + .3 D + .2XD + 1.5
b0 (1.5) is the intercept for the group coded zero (female) when X = 0 b1 (.4) is the regression of Y on X in the group coded zero (female) b2 (.3) is the difference in intercepts for the group coded one (male) minus for the group coded zero (female). b3 (.2) is the difference in slopes for the group coded one (male) minus for the group coded zero (female)
If you use dummy coding for two groups, and the dummy code interacts with the continuous variable in the equation, be able to indicate what each of the regression coefficients in the equation measures. What two tests would you perform to see whether each within group regression line differs from 0? (Located in "Categorical X Continuous Interactions" and "Example 10a SPSS Narrative")
b0 = the predicted value of Y when X = 0 (or Xc = 0, the mean of X, if centered) b1 = the effect of X on Y in group 0; the estimated slope of the reference group b2 = the difference between the intercepts of the two groups b3 = the difference in slopes between the two groups (0 and 1, usually control and treatment); if it is significant, then they have different slopes for the regression of Y on X, indicating an interaction The two tests to perform to see whether each within group regression line differs from 0 are two regression equations using the dummy codes. First, code the Control group = 0; the regression of Y on X in this equation is the effect for the Control group only (b1 coefficient). Then, reverse the coding so that the Treatment group = 0 and the Control group = 1. Then rerun the regression equation. The regression of Y on X in this second analysis tests the significance of the slope for the Treatment group only
What does the Johnson-Neyman procedure test?
identifies "regions of significance", that, to identify a value of X above which the treatment effect is significant, for which the elevation of the two simple regression lines differs procedure of testing conditional effects of treatment C at particular values of X
What are adjusted means? How are the adjusted (conditional) means estimated using multiple regression?
· Adjusted means: the predicted mean value for each group given that the covariate is held constant at a value equal to its mean in the full sample. o Estimate adjusted means by centering the covariate o Yhat = b0 + b1T + b2COVc(centered) · Typically, the equation is centered at the overall mean of the coviarate (the centroid of the covariates if there's more than 1); the predicted value of Y in each group will be a conditional mean. · Dummy coding offers the most straightforward interpretation of the adjusted means
To what problems does the inclusion of step 1 lead (testing the effect of X on Y)?
· Baron and Kenny's step 1 is a combined test of the direct c' and indirect (mediated) ac effects. Inclusion of this step does not test the specific hypothesis of interest in mediation, i.e. H0: αβ = 0, where α is the value of a in the population and β is the value of b in the population · Inclusion of step 1 can decrease the statistical power of the test of mediation
Suppose I have the regression equation = b0 + b1X + b2C + b3XC. In this regression equation, C is gender, X is height, Y is weight. Based on this regression model, how do I test the difference in weight for men and women who are 68 inches tall?
· Center the data at X (height) = 68 and apply dummy coding/contrast coding to the gender variable, then run the regression model. The b2 coefficient gives the difference in intercepts/value of weight between men and women at X = 68 inches.
What is a within class regression line in the ANCOVA? What assumption is made about within class regression lines in analysis of covariance? What other assumptions underlie the use of ANCOVA?
· Consider separating the data into two subsets—we consider the data separately within each of the two treatment conditions, T and C. We can plot the regression of the criterion on the covariate in each of the treatment conditions. Each of these regression lines is called a within class regression line. · Key assumption underlying the analysis of covariance: There is no interaction between the covariate and the treatment variable (e.g., why the lines are drawn parallel; see below) · Other assumptions: the lines are drawn parallel, the relationship between the covariate and the outcome has been properly specified (e.g., for what values of the covariate the treatment effect is significant?), homoscedasticity (see figure)
Briefly describe two alternative approaches that address the problem in Q. 46.
· Distribution of Product Method o Statistical theory to provide the exact (ugly) distribution of the confidence interval of ab. The distribution will in general not be close to normal. § Standard error will affect this distribution, so the estimate is not a pivot statistic · Percentile Bootstrapping o Percentile bootstrapping performs similarly to the distribution of the product method. A program could easily be set up in R paralleling the program we used to estimate the maximum or minimum of a quadratic regression. A bootstrap sample would be selected and equation (2) above would be used to estimate a and equation (3) would be used to estimate b in the same bootstrap sample. This process would be repeated in a large number of bootstrap samples (say 2000) so we have 2000 bootstrap estimates of ab. Then the estimates are sorted from low to high, and the lower and upper limits are at 2.5 percentile and 97.5 percentile. o The estimate of ab from the parent sample is the best point estimate of the mediated effect. If the 95% percentile bootstrap sample does not include 0, then the mediated effect is statistically significant.
In a randomized [SL1] experiment, can the test of the mediated effect be confounded? How might it be confounded? What variables are particularly important to control for to reduce this form of confounding?
· If X is randomized to treatment and control conditions, the issue is whether some other (confounding) variable might be causing both M and Y, leading to incorrect inference. · If X is not randomized (a non- randomly assigned treatment condition; an individual difference variable, e.g., X is biological sex; X is secure attachment), causal inference becomes much more problematic. Some other variable(s) may be jointly causing X and M, X and Y, and/or M and Y. A very good understanding of the entire network of potential causal relationships is needed, rarely the case in psychology · There may also be a "backdoor path" when baseline measures of the mediator and outcome are correlated with the post-treatment mediator and outcomes, making it look like there is a relationship independent of treatment effect. To reduce this you control for baseline levels of the mediator and outcome.
In the regression equation = b0 + b1X + b2C + b3XC, if you change the coding of the categorical variable (C) with two levels from dummy to contrast coding, will the regression coefficient for the interaction change or remain the same?
· It will stay the same; both will represent a 1-unit change between the slopes, so they will be the same
There are two older methods of treating missing data that have been traditionally used in standard programs within computer programs like SPSS and R. What are they? If a researcher uses one of these older approaches to handling missing data, what missing data mechanism must be operating to yield unbiased parameter estimates.
· Listwise deletion: eliminates all cases with missing values, resulting in a complete data set · Pairwise deletion: eliminates cases on an analysis-by-analysis basis (e.g., correlations, means, SDs based on different Ns) · These deletion methods assume missing completely at random (MCAR)! We can get biased estimates if we have MAR or MNAR
How can the indirect (mediated) effect be tested more directly than with the Baron and Kenny approach? What problem arises in statistically testing the indirect effect?
· MacKinnon (2008) points out that Baron and Kenny's step 1 is a combined test of the direct c' and indirect (mediated) ab effects. Inclusion of this step does not test the specific hypothesis of interest in mediation, i.e. H0: αβ = 0, where α is the value of a in the population and β is the value of b in the population. Step 1 does not test this! o Decreases statistical power of the test of mediation · You can explicitly test the indirect effect with the product estimator ab o Problem: the indirect effect ab is NOT a pivot statistic and does not have a standard sampling distribution
Donald Rubin (1976) defined three missing data mechanisms. These represent classes of missing data in terms of how propensity for a missing value on Y relates to either other variables or to the value of Y itself. By propensity for missingness (i.e., that the score is missing) is meant the probability that a score is missing. Briefly describe each of the three mechanisms.
· Missing Completely at Random (MCAR) o The probability of missing data on Y is unrelated to other measured variables and is unrelated to the values of Y itself; nothing in the data set predicts the propensity to have a missing value there · Missing at Random (MAR) o probability of missing data on Y is related to other measured variables o After controlling for other variables, there is no remaining association between propensity for missing data on Y and the would-be values of Y o Multiple imputation requires MAR assumption! o Dean's list and looking at student GPA...whether someone is on Dean's list is dependent on their GPA · Missing Not at Random (MNAR) o probability of missing data on the dependent variable Y is related to the values of Y itself o The Y score that is missing is related to the value you would have observed if the person had responded; causes substantial bias
In addition to the usual assumptions of multiple regression, what additional assumptions are needed for mediation analysis for causal inference?
· Multiple regression assumptions o Linearity o Homoscedasticity o Normality of residuals o Independence of participants (no clustering) o Independence of errors · Additional assumptions for mediation o No measurement error in the mediator o No reverse causal effects o No confounding (no omitted variables)
What is meant by the "pick a point approach" to testing group differences in a regression model for a binary treatment vs. control IV, a continuous covariate, and their interaction.
· Pick a point on the moderator (covariate, continuous) variable and center the data at this point. Then run the regression equations at that point to test the differences between groups/slopes at that point, which is now at Xc = 0.
What are the three rules for choosing contrast codes to maximize interpretability?
· Required. The sum of the weights for each code variable must equal 0. · Required. The sum of the product of the weights for each pair of code variables must equal 0. o Given the first two rules hold, if n is the same in each group, the contrast codes are orthogonal—a desirable outcome. · Optional. If there are only positive weights with the same positive value and negative weights with the same negative value, the difference in the value of the set of positive weights and negative weights should equal 1. o (This facilitates interpretation of the results by creating a 1-unit difference between the combined positive weights and the negative weights).
According to the Baron and Kenny (1986) causal steps approach, what are the four steps involved in testing mediation?
· Step 1. Test the regression coefficient c for statistical significance. Is there an overall effect of X on Y? · Step 2. Test the regression coefficient a for statistical significance. Does X have an effect on the mediator M? · Step 3. Test the partial regression coefficient b of the relationship between the mediator M and the outcome Y over and above X. Thus, the effect of X is statistically controlled in this step. · Step 4. (Optional). To show that the results are consistent with full mediation, test c' in equation 3. This effect should be non-significant (i.e., consistent with 0 effect).
What sort of data configuration lends itself most naturally to coding with dummy codes? What are the three criteria useful in choosing a reference group with dummy codes?
· When you have a reference group of interest that you want to reference against/compare the other groups to, like a control group · Criteria: o Have a meaningful control/reference group o Not a small sample size (small samples gives unstable estimates) o Reference group needs to be a well-defined category
What is meant by mediation?
· an initial variable (e.g. treatment) can lead to changes in an intermediate variable (mediator), which, in turn, causes changes in the outcome variable of interest · X affects Y through mediating variable
If you use contrast codes for two groups[SL1] , and the contrast code interacts with the continuous variable in the equation, be able to interpret what each of the regression coefficients in the equation measures. [SL1]Will we need to know the interpretation of 3 or more groups?
· b0 (intercept) is the unweighted mean of the two intercepts · b1 is the unweighted mean of the two slopes · b2 is the difference between the intercepts of the group coded higher minus group coded lower · b3 (interaction) is the difference between the slopes of the group coded higher minus group coded lower
What are the three stages of multiple imputation (in detail)
• Create multiple copies of the data, each with different imputed values for missing values • Two steps: • I-step: imputation step, uses regression equations to predict the values of incomplete variables from the complete variables • It adds random residuals to each predicted value to eliminate bias associated with regression imputation • P-step: posterior step, generates updated regression equations for the next imputation cycle and creates a sampling distribution • Use the filled-in data to update the regression coefficients • Define a plausible set of alternate regression parameters • Randomly draw a new set of regression coefficients • Iterated many times! (MCMC chain) 2) Analysis phase • Perform standard statistical analyses (e.g., MR) separately on each data set • imputation phase produces m complete data sets. The value chosen for m (typically 20 or more in psychology) depends on the size of the data set; in the analysis phase, we perform the same planned analysis separately on each data set 3) Pooling phase • Combine the collection of estimates and standard errors into a single set of results • Has a between and within imputations component • Sampling error due to missing data and complete-data sampling error variances need to be combined • Between-imputation variance quantifies the variability of the estimates across the m data sets • As m (number of imputations) increases, it decreases the weight on VARb which ultimately lowers standard error • SPSS doesn't do this automatically with standardized coefficients and squared multiple correlations
