717: Final Exam

Regarding multi-level modeling (MLM): b) What is a Level 1 model?

"Level 1" within-subject model can include coded categorical and/or continuous independent variables. Develop a regression model of purely within-subjects effects to apply independently to each participant's data. This model is separately fit to the observations of each individual subject.

Regarding multi-level modeling (MLM): c) What is a Level 2 model? Include indication of what the dependent variables are in such models

"Level 2" models are between-subjects models. In their simplest form they only involve means/intercepts, but in others, they may include between-subjects variables:

Describe what is meant by Type 3 sums of squares. Under what conditions do Type 3 SSs affect the power of tests in a factorial ANOVA, and what is that effect?

"Unique" SS = Type 3 SS accredit and eliminate variance due to multiple effects. There is a loss of power due to reduced amount of variance "credited" to each of the effects.

a) State the null and alternative hypotheses for the omnibus test for the effect of treatment. b) Create a table that partitions each score on the dependent variable into parts related to the grand mean, the group effect, and error. Label precisely the columns corresponding to each of these 3 parts. Your table may have >3 columns, but be sure to clearly label the columns that answer the question. c) Conduct a complete omnibus analysis of variance for the effect of treatment group and summarize it in a standard table. d) Given that 9.55 is the critical value of F for the correct df's, draw your conclusions. e) What is the standard but biased! index of variance accounted for by group in this ANOVA model? What is its value in this case? f) What alternative index of variance accounted for would you use to conduct a power analysis for a replication of this study? No need to compute the alternative index - just name i

(a) H0: Mu1 = Mu2 = M3 Ha: NOT Mu1 = Mu2 = Mu3 (b) picture (c) picture (d) Fcritical > Fobtained. There is insufficient evidence in these data to conclude that the 2 drugs and placebo are differentially effective in treating depression. (e) eta^2 is the standard (biased) index of variance accounted for by group in this ANOVA model. eta^2 = SSgroup/SSTC (f) I would use w^2 or epsilon^2 which are standard estimates of variances accounted for by group in the population. (1 - MSerror/MStc)

Assuming that you have a factorial ANCOVA with 2 levels of Factor A, 4 levels of Factor B, continuous covariate X, and 10 subjects per cell. a) List sources of variance and degrees of freedom for the ANCOVA. b) What additional analyses should be conducted to determine whether the two main assumptions of ANCOVA are met? Be specific about independent and dependent variables in these analyses. List the sources and degrees of freedom for each of these additional analyses. c) What would be an appropriate course of action if Assumption #1 was violated, assuming that A and B are experimental factors? d) What would be an appropriate course of action if Assumption #2 was violated?

(b) Assumption #1: Groups don't differ on the covariate. Assumption #2: No interaction between covariate and grouping variables (homogeneity of slopes) (c) At least, proceed with caution. Tell reader we had a violation of the assumption. Analyze the data to see how much of a violation we have. If we decide to use it, we must be able to make a case for why we did that. Look at overlap among the groups on the covariate. Be cautious in your interpretation. Other option: drop the covariate. Is it really necessary? (d) You don't have a valid ANCOVA, but results may still be interpretable. The test of the assumption may lead to an interesting and important discovery! The thing you identified as a covariate actually influences the results of your experiment.

Describe the advantages of using continuous variables as predictors, as opposed to blocking such variables to obtain groups.

1. Better precision in describing the relationship to the d.v. 2. Possibility of learning more about the shape of the relationship between the continuous variable (e.g. age) and the dependent variable 3. Avoid having results that depend on arbitrary cutpoints In general, blocking moves systematic variance from the numerator to the denominator of the F-ratio.

Repeated measures ANOVA makes an assumption that is not present in between-subjects ANOVA. e) Name two procedures for computing ε and indicate which provides the most accurate results?

1. Huynh-Feldt - less conservative, more appropriate/accurate 2. Greenhouse-Geisser

State two reasons to avoid stepwise regression analysis with automated variable selection in scientific research?

1. It yields R-squared values that are badly biased high. 2. The F and chi-squared tests quoted next to each variable on the printout do not have the claimed distribution. 3. The method yields confidence intervals for effects and predicted values that are falsely narrow. 4. It yields p-values that do not have the proper meaning, and the proper correction for them is a difficult problem. 5. It gives biased regression coefficients that need shrinkage (the coefficients for remaining variables are too large). 6. It has severe problems in the presence of collinearity. 7. It is based on methods that were intended to be used to test pre- specified hypotheses. 8. Increasing the sample size does not help very much. 9. It allows us to not think about the problem. 10. It uses a lot of paper.

State the two main assumptions of ANCOVA

1. The groups should not differ on the covariate. 2. "homogeneity of regression slopes": There is no interaction between the covariate(s) and the grouping variables.

Repeated measures ANOVA makes an assumption that is not present in between-subjects ANOVA. f) State two reasons why the Mauchley test is not recommended.

1. inaccurate (false positives and negatives) 2. not needed --> most packages correct for an sphericity violation.

What is a type I error?

A Type I error occurs you reject the null when the null is true. False positive. In plain terms...this means, that the researchers conclude that significant results exist in a study, when really, they don't.

What is a type II error?

A Type II error is when you fail to reject the null when the null is false. False negative. In plain terms...this means the researchers conclude that the study results are non-significant when in reality...the results were significant!

In what circumstances would you use dummy coding?

A design for which dummy coding would be appropriate would be comparing medication, CBT, and art therapy to a control group receiving no treatment. Because dummy coding is useful when there is reference group, and each treatment would be compared to a control group not receiving any treatment, this would be appropriate.

What is the difference between a fixed and a random factor? What random factor appears in all repeated-measures designs?

A fixed factor is a factor for which the levels in your design are the actual levels about which you want to draw inferences. (e.g., "Drug: represents only the rugs in the study). A random factor is a factor for which the levels in your design represent a sample of the population of levels about which you want to draw inferences (e.g., "Subject" represents all subjects within the population of interest).

What are the characteristics of a "good" MCP?

A good MCP controls family wise error rate without costing (reducing) too much power.

What information is needed to conduct and use power analysis to determine the required sample size for a study?

A power analysis is typically conducted at the stage of study planning, e.g. in a grant application, to determine the necessary sample size given the alpha you choose and your expected effect size relative to variability. Effect size is usually estimated based on previous research or your own rational for the minimum effect you hope to detect.

What does it mean to say that a test is robust with respect to an assumption?

A test is robust with respect to an assumption to the extent that violation of the assumption does not substantially alter the distribution of the test statistic.

Describe two advantages and two disadvantages of repeated-measures designs.

Advantages: 1. Each subject provides information about all treatments, so fewer subjects needed. 2. individual differences do not confound treatment differences like they do in between-subjects designs. Disadvantages: 1. Carryover effects: response to one treatment is influence by exposure to other treatment(s). 2. an additional assumption: sphericity

List the factors that affect power, and the effect that each of them has on power. Be clear about the direction of these effects.

Alpha: as alpha decreases, power decreases. researchers generally set alpha to .05 as a standard. N (sample size): as N increases, power increases. effect size: as effect size increases, power increases. variability in the effect: as variability n the effect increases, power decreases.

Define "interaction"

An interaction occurs when the effect of one factor varies among levels of another factor.

State the three main assumptions of every GLM-based test.

Assumption 1: (a) residuals are normally distributed (b) Statistically (Shapiro-Wilks test), graphically (like Q-Q plots), descriptively (Skewness and kurtosis) (c) Yes the F-test is generally robust with respect to this assumption. Assumption 2: (a) homogeneity of variance (b) graphically or statistically (Brown-Forsythe [B-F] modification of the Levene test for ANOVA or White's test for regression) or variance ratio rule of thumb (result should be less than or equal to 9) (c) Yes, it is robust except when group sizes are unequal. Assumption 3: (a) Errors are independent of each other. (b) statistically: Durbin-Watson test (c) No, the F-test is not robust with respect to this assumption, so it is important to ensure independence of observations between participants.

Why are assumptions important?

Assumptions are important because they are the basis of critical value tables and p-values. If assumptions are not met, then those values are not the correct numbers to base your results on.

For each of the following designs, list all sources of variance and their associated degrees of freedom. If there are multiple error terms in the analysis, list each error term immediately after the systematic sources of variance (i.e., the effects to be tested) for which it is used, as illustrated in class. Assume 10 subjects per group or between-subjects cell, or in the design as a whole if all subjects are in one group. b) C (within, 3 levels) X D (within, 2 levels) a) Now go back and mark each systematic source of variance in Parts a-d to which the special assumption of repeated-measures analysis applies? a) Consider a focused test of the C1 versus C3 difference in the design described in Part b). What would be the appropriate error term for this test?

C1 vs C3 x subj

You are interested in predictors of success in college as measured by college GPA. You test the predictive value of two variables - high school GPAs and standardized college admissions test (ACT composite) scores - in a random sample of 500 recent college graduates. You include the interaction in this analysis. (b) You could choose to run the analysis before or after centering the predictor variables, or you could run it both ways. Which of these analyses would you conduct, and why?

Centering continuous variables - in both continuous × continuous and continuous × categorical analyses - is recommended by all authoritative regression sources. We center to make our b coefficients interpretable. Do it before running our analyses. The only reasoning not to center is to graph. So, center for analyses, not for graphing.

Define adjusted means for ANCOVA. Why should they be used?

Covariate-adjusted Means = estimated means, based on correcting for any differences among the groups on the covariate. This procedure yields the means that would theoretically have occurred if the groups were exactly equated on the covariate. It only makes sense to use adjusted means in situations in which it makes sense to use ANCOVA.

In what circumstances would you use effect coding?

Effect coding makes sense when you have no natural reference group, but just want to determine whether any of the means have unusual means relative to the overall mean. Comparing four different SSRIs for treatment of depression.

Consider the following multiple comparison situations and state which of the MCPs presented in class would be the most appropriate choice for each. Assume equal-n and equal variance in all cases. b) running comparisons of activation between two conditions in 800 voxels in an fMRI study

FDR (False Discovery Rate)

Define family-wise alpha and contrast it with per-comparison alpha.

Family-wise alpha (αFW) is the probability of at least 1 type I error in a set (family) or comparisons. Per-comparison alpha (αPC) is the probability of a type I error for each individual comparison. MCPS reduce αPC to control αFW

Regarding polynomial regression: b) What characteristics should the independent or predictor variables have in order to use this method?

Generally, both the independent/predictor variable and dependent variable should be on an interval or ratio scale.

What is the characteristics of a situation when the Fisher test would be the best choice among all of the MCPs presented in class?

Good for MCPs with no more than 3 groups

What negative consequences can result from under-powered studies.

Low power increases the risk of Type II error. It should be avoided because it can have negative impacts by misinforming policy, future research theory, and clinical practice. It is also wasteful of time, money, and resources, and places research subjects and animals through needless procedures and other inconveniences that can be unethical.

Regarding mediational analysis: a) What is the purpose of this method of analysis?

Mediation analysis is an attempt to test hypotheses about why variables are related. The relationship between some variable X (independent/predictor variable) and some variable Y (dependent variable) is mediated by some variable M (the potential mediator).

On what primary basis di Meehl criticize the null hypothesis significance test?

Meehl criticized the NHST on the basis that it is almost always false, and therefore whether or not you reject the null hypothesis depends almost entirely on the sensitivity (power) of your design, and with enough subjects you will almost certainly reject H0 regardless of the truth of the underlying theory.

What is the primary implication of the Meehl paper for the conduct of scientific research, as discussed in class?

Meehl emphasized the importance of making predictions that are as specific as possible, making and testing specific directional hypotheses. He also recommends caution when using and interpreting large tables with asterisks indicating significance due to the assumption that H0 is quasi-always false.

Regarding multi-level modeling (MLM): a) To what form of ANOVA is MLM most closely related? Give one reason why one might one choose MLM rather than ANOVA in a particular research project.

Multi-level modeling resembles repeated-measures ANOVA: 1. Main effects of between- and within-subjects factors/variables are tested, as are between Χ between and between Χ within interactions. 2. The error term is partitioned. Effects are tested only on the error variability that they could potentially account for. Advantages of multi-level models over repeated-measures ANOVA include: 1. Greater flexibility in the ways that within-Ss variability can be accounted for (e.g., continuous within-subjects substantive variables and covariates, within-subjects autoregressive effects, etc.). 2. Missing within-Ss observations are easily accommodated

Centering of continuous variables is often recommended, especially in designs that include interactions. What is centering and why is it recommended? For what purpose(s) is centering not recommended?

Multicolliniarity problem nearly disappears if the predictor variables are deviated from their means (i.e., "centered") before forming the cross-product. (1) Centering won't totally eliminate the correlations between "main effect" continuous variables and their interactions, but they will greatly reduce them. (2) Interpretation of partial slopes. Centering based on the obtained predictor (X, Z) means will yield partial slopes that are sample-specific. Not recommended: (1) Plots should generally be based on uncentered data so they relate to "raw" scores, at least if those scores are meaningful to your audience. But again, center continuous variables for analysis.

In what circumstances would you use orthogonal contrast coding?

Orthogonal contrast coding would be useful if you wanted to answer (a) if the experimental treatments better than the control group (CBT, medication, and art therapy vs. no treatment); (b) are medications better than CBT or art therapy; (c) is CBT more effective than art therapy (you can answer multiple independent questions). Specific, directional hypotheses.

You are interested in predictors of success in college as measured by college GPA. You test the predictive value of two variables - high school GPAs and standardized college admissions test (ACT composite) scores - in a random sample of 500 recent college graduates. You include the interaction in this analysis. (d) What would you do to visualize the predictive effects of GPA and ACT scores, and their interaction, on high school GPA?

Plot the expected value of college GPA( college GPA') for various combinaitions of HS GPA and ACT comp.

Regarding polynomial regression: a) What is the purpose of this method of analysis?

Polynomial regression is a general method for linearizing non-linear relationships, including those with >1 inflection point. It is a form of interactive model in which the interaction is between the variable and itself.

Consider the following multiple comparison situations and state which of the MCPs presented in class would be the most appropriate choice for each. Assume equal-n and equal variance in all cases. a) testing all possible pairwise differences among group means by computer

REGW

Regarding Lee J. Cronbach's 1957 Presidential Address to APA: What research design, attributed to Cronbach, was described in this course, and what is the generic name for designs that could be analyzed similarly?

Research design: aptitude x treatment continuous x categorical

Consider the following multiple comparison situations and state which of the MCPs presented in class would be the most appropriate choice for each. Assume equal-n and equal variance in all cases. c) testing all possible pairwise differences among group means, plus two additional complex comparisons

Scheffe or Holm

You are interested in predictors of success in college as measured by college GPA. You test the predictive value of two variables - high school GPAs and standardized college admissions test (ACT composite) scores - in a random sample of 500 recent college graduates. You include the interaction in this analysis. (e) What tests might you conduct as a basis for understanding the effects of these variables. Name the type of test and also describe the test using the variables in this analysis.

Simple effects tests is the same as for the continuous x categorical design, but because there are no groups, they involve estimating the simple slope for one predictor at various points along the range of the other predictor. Test simple slops of GPA at different values of ACTcomp, or vice versa, but only if HS GPA x ACT comp intrxn is significant.

What are the differences between a simultaneous and a sequential MCP?

Simultaneous MCPS: all tests are compared to the same critical value of the test statistic and the outcome of one test has no impact on whether the others are conducted. These are useful because they can be used as the basis for confidence intervals. Sequential MCPS: proceed in a particular order, and subsequent tests are not run if prior tests were not significant. (stopping rule). Sequential tests cannot produce Cis, but are generally more powerful than simultaneous alternatives.

"Standardization" refers to a linear transformation in which the mean of a variable is subtracted and the result is divided by its standard deviation. What is the value of standardizing the dependent and independent/predictor variables in multiple regression?

Standardizing predictor variables: 1. Predicted values of Y can easily be found for unit-scaled values of X and Z. 2. Standardizing the predictors also allows the regression coefficients to be meaningfully compared to each other. Standardizing dependent variables: 3. If the dependent variable (Y) is not in physical or other meaningful units, it may make sense to convert this variable to z-scores as well.

Regarding polynomial regression: c) Give an example of a polynomial regression equation, in error-free form.

Test X using error from ... Y' = a + b1X, Test X^2 using error from ... Y' = a + b1X + b2X^2 , and Test X^3 using error from the full model: Y' = a + b1X + b2X^2 + b3X^3

What does the variance inflation factor (VIF) indicate? What is a recommended cutoff for acceptable VIFs and what should you do if you have VIFs that are too high?

The "variance" in Variance Inflation Factor is simply the square of the SEb, or (SEb)^2. VIF refers to the factor by which the (SEb)^2 is enlarged (multiplied) as a result of the correlation of Xi with other X's. Recommended upper limits for the VIF in regression analysis vary from about 2.5 to 10. The most trustworthy sources suggest that VIF's > 2.5 should be addressed in some way. How to reduce the maximum VIF in a regression model: a. Eliminate the variable with the highest VIF, or eliminate variables that correlated with it. b. Combine correlated variables using a data reduction method like Principal Components Analysis.

What is the expected value of the F-statistic under the null hypothesis, and why is this so?

The expected value of F under the null hypothesis 1. When H0 is true and there is no relationship between X and Y in the population, and/or no effect of group. Mserror/residual and Msregression/group are both unbiased estimates of the population variance and independent of each other. Since they both are estimates of population variance and equal each other on average, the F ratio Msreggroup/Mserrorresidual = 1.

In what circumstance is ANCOVA most frequently misused? *will be asked on final exam*

The main purpose of ANCOVA is NOT to statistically control for confounds in correlational research.

State the primary purpose of analysis of covariance

The main purpose of ANCOVA is to reduce error variance (using a covariate to account for some of that variance) and thereby increase power and/or reduce the size of the required sample for an adequately-powered research design.

In general for MCP's, what is the trade-off between the number of tests conducted and the power of each test?

The more tests you run, the more power is reduced for each test. Bonferroni (αPC) = (αFW)/# of contrasts 3 contrasts αPC = .167 5 contrasts αPC = .01 10 contrasts αPC = .005 More tests = less power.

Regarding Lee J. Cronbach's 1957 Presidential Address to APA: What were the two "historic streams of method, thought, and affiliation" to which his address referred?

The past and future place within psychology of two historic streams of method, thought, and affiliation—experimental psychology and correlational psychology—is discussed in this address of the President at the 65th annual convention of the APA.

Regarding polynomial regression: a) What type of sums of squares is used to for this method, that is rarely used in other regression analyses?

This is a case where we use tests based on the Type 1 SS. The partitioning of variance is always incremental in polynomial regression, so the effect of X^2 isn't assessed until the effect of X has been partialed.

Indicate whether the "main-effects only" or "full model" error term would be correct for testing main effects in each of the following situations that involve a continuous predictor variable: Consider the same situation as in Part a, but now assume that the Method X Pre-class Math Knowledge interaction is significant. You again test the main effect of teaching method: (a) You are conducting a study of two teaching methods for undergraduate statistics, and you administer a math test at the beginning of the term for use as a covariate. Before conducting the ANCOVA on final exam scores, you test the Method X Pre-class Math Knowledge interaction, which is non-significant. You then conduct the ANCOVA to determine whether teaching method has an effect after controlling for pre-class math knowledge.

Use the MSE from the full model for all tests (main effects and interaction) if the interaction test was a principle reason for conducting the research OR the interaction is significant.

Indicate whether the "main-effects only" or "full model" error term would be correct for testing main effects in each of the following situations that involve a continuous predictor variable: (c) You are interested in whether individuals who differ in "PA" (a personality trait) interacts with the effectiveness of contingency-based treatments for substance abuse. You conduct a study to assess this interaction. The interaction is n.s., but you still want to test the significant main effects of positive affectivity and contingency versus control treatment.

Use the MSE from the full model for all tests (main effects and interaction) if the interaction test was a principle reason for conducting the research OR the interaction is significant.

Indicate whether the "main-effects only" or "full model" error term would be correct for testing main effects in each of the following situations that involve a continuous predictor variable: (a) You are conducting a study of two teaching methods for undergraduate statistics, and you administer a math test at the beginning of the term for use as a covariate. Before conducting the ANCOVA on final exam scores, you test the Method X Pre-class Math Knowledge interaction, which is non-significant. You then conduct the ANCOVA to determine whether teaching method has an effect after controlling for pre-class math knowledge.

Use the MSE from the main-effects-only model to test the main effects if the interaction is not a principle focus of the research, AND it is nonsignificant.

Regarding mediational analysis: b) What is the purpose of the bootstrap confidence interval in mediational analysis?

Used to test indirect effects in mediation analysis. A bootstrap confidence interval for the indirect effect is constructed by repeatedly (1000's of times) and randomly resampling cases from the data with replacement, and computing the indirect effect ab each time.

Regarding mediational analysis: c) What is the difference between mediation and moderation?

Variable M mediates the relationship between X and Y if the effect of X on Y depends on X influencing M and M in turn influencing Y. M moderates the relationship between X and Y if X and M interact to determine Y. "Moderation" and "interaction" are essentially synonyms.

Repeated measures ANOVA makes an assumption that is not present in between-subjects ANOVA. d) Describe procedures for dealing with violations of the assumption, using epsilon (ε)

Violations are dealh with by adjusting the numerator and denominator df downward, therefore increasing the critical value/threshold for significance. We do this by multiplying df by E(epsilon). Adjustment is sensitive to the degree of violation, and E will be approx 1 with little/no violation, and greater than 1 to the degree that the assumption is violated.

You are interested in predictors of success in college as measured by college GPA. You test the predictive value of two variables - high school GPAs and standardized college admissions test (ACT composite) scores - in a random sample of 500 recent college graduates. You include the interaction in this analysis. (c) You could choose to standardize the variables in this analysis. Would you do this, and why (or why not)?

We want to be able to compare our b coefficients. Will also help with graphing if our variables are not meaningful in raw form.

What considerations are involved in interpreting interactions when there are main effects?

When interpreting interactions when main effects are present, must consider that the interaction may be partly due to scale sensitivity (ceiling and floor effects), or choice of units (can create or destroy an interaction if alternate units are non-linear transformations).

What issues are involved in interpreting main effects when there are interactions?

When interpreting main effects when interactions are present, you must be careful not to make general statements about the main effects unless they are true. In general, when both main and interaction effects are present you focus on and interpret the interaction. The exception is if the interaction is ordinal, in which case it may be reasonable to interpret the main effect(s).

What is the characteristics of a situation when would no MCP test would be the best choice?

When you have planned and orthogonal comparisons.

You are interested in predictors of success in college as measured by college GPA. You test the predictive value of two variables - high school GPAs and standardized college admissions test (ACT composite) scores - in a random sample of 500 recent college graduates. You include the interaction in this analysis. a) State the regression model in its error-free form.

Y' = a + b1X +b2Z + b3XZ (college GPA)' = a + b1(HS GPA) + b2(ACT comp) + b3(HS GPA x ACT comp)

State the preferred statistical tests for violations of the main assumptions of between-subjects GLM analyses.

You can use transformations to correct for violations of assumptions. If this is not appropriate for your analyses, you can use a corresponding nonparametric test or perform resampling procedures.

You are conducting a study of two different drug treatments, compared to a placebo. You want to know whether drug treatment is effective in general, and whether the two drugs differ in effectiveness from each other. Do you need an MCP, and if so, what would be the best MCP to use among those that were presented in class?

You do not need an MCP. The questions are orthogonal, and you don't need an MCP when the tests are planned or orthogonal.

Consider the following multiple comparison situations and state which of the MCPs presented in class would be the most appropriate choice for each. Assume equal-n and equal variance in all cases. computing all possible pairwise comparisons among condition means in a 1-way repeated measures design with 4 conditions

You need an MCP, best choice, Fisher-Hayter

Consider the following multiple comparison situations and state which of the MCPs presented in class would be the most appropriate choice for each. Assume equal-n and equal variance in all cases. d) computing confidence intervals for all possible pairwise differences among group means, plus two additional complex comparisons

You need an MCP; best choice, Bonferroni.

You want to compute confidence intervals for APP differences among 5 means, plus an additional complex difference. Do you need an MCP, and if so, what would be the best MCP to use among those that were presented in class?

You need an MCP; best choice, Bonferroni.

You are conducting a study of two different drug treatments, comparing a placebo. You want to know whether each of the drug treatments differs in effectiveness from placebo. Do you need an MCP, and if so, what would be the best MCP to use among those that were presented in class?

You need an MCP; best choice, Dunnett.

In an fMRI study you are comparing peak brain activation in 1000 comparable voxels between two groups of individuals, with and without dementia. Do you need an MCP, and if so, what would be the best MCP to use among those that were presented in class?

You need an MCP; best choice, FDR.

You are conducting all-possible pairwise tests among a set of means but do not have access to a statistical package that implements MCPs. Do you need an MCP, and if so, what would be the best MCP to use among those that were presented in class?

You need an MCP; best choice, Fisher-Hayter.

You want to conduct APP tests among a set of means, plus two additional complex comparisons. Do you need an MCP, and if so, what would be the best MCP to use among those that were presented in class?

You need an MCP; best choice, Holm.

You want to compute confidence intervals for all-possible-pairwise differences among a set of means. Do you need an MCP, and if so, what would be the best MCP to use among those that were presented in class?

You need an MCP; best choice, Tukey.

For each of the following research designs or questions, name the type of design, the independent and dependent variables, and any covariates. d) You are interested in studying circadian changes in cortisol in men and women with high- and moderate-stress jobs. For this purpose you sample blood cortisol hourly over 5 consecutive workdays.

a. Mixed-design b. IV = job stress-level, sex DV = cortisol changes c. covariate = none test w/ repeated measures ANOVA or multilevel model

What is the characteristics of a situation when the Holm test would be the best choice among all of the MCPs presented in class?

a. Pairwise comparison good for blended family/custom test, very flexible without the consequence of Scheffe.

Describe each of the following procedures and indicate when they would be used:a) Simple main effects analysisb) Simple contrastsc) Simple interactions tests

a. Simple main effects: An interaction occurs when the effect of a factor varies among levels of another factor. SME analyzes how the effect of one factor varies among levels of the other factor because knowing that an interaction is present does not give you specific information about the direction or degree that the effects vary. Use this if the interaction is significant in a factorial ANOVA b. Simple contrasts: Test the differences in specific combinations of the levels of one factor at the specific level of the second factor. Used to followup significant SMEs, involving > 2 means. c. A simple interaction test is used when you want to see how an interaction varies among levels of an additional factor.

What is the characteristics of a situation when the Scheffe test would be the best choice among all of the MCPs presented in class?

a. The ultimate fishing license - good for performing unlimited pairwise and complex tests

Given graphs or tables of means for a factorial design, indicate whether there appears to be a main effect of A, a main effect of B, and/or an AxB interaction.

a. main effect of A&B, no interaction b. main effect of A & B, AxB interaction c. main effect of B, AxB interaction, no main effect of A d. no main effect B&A, AxB interaction

Repeated measures ANOVA makes an assumption that is not present in between-subjects ANOVA. c) Describe the set of tests to which the assumption applies.

all F-tests involving a repeated-measures factor with > 1 numerator df

For each of the following designs, list all sources of variance and their associated degrees of freedom. If there are multiple error terms in the analysis, list each error term immediately after the systematic sources of variance. Assume 10 subjects per group or between-subjects cell, or in the design as a whole if all subjects are in one group. d) A (between, 3 levels) X B (between, 4 levels) X C (within, 5 levels) a) Now go back and mark each systematic source of variance in Parts a-d to which the special assumption of repeated-measures analysis applies? b) Assuming that you found a significant A X C interaction in the design described in Part d). What next set of tests might you conduct to help you understand this interaction? c) Assuming that you found a significant A X C X D interaction in the design described in Part d). What next set of tests might you conduct to help you understand this interaction?

b) simple main effects c) simple interaction analysis

Why do we partition the error term into between- and within-subjects components in the ANOVA for repeated measures research designs.

e partition the error term into between and within subjects components because the within subjects factor (eg drug) cannot also account for differences between subjects in their mean level. Condition and subjects are crossed, or orthogonal, and therefore, no matter how powerful the condition effect is, it cannot account for differences due to each subject. When a factor can't, by design, account for unexplained variance, it does not make sense to include that variance as error when testing the effect of that factor (within subjects condition). This is the rationale for partitioning the error as such and removing between-subjects components from the error term of the F-test for the within subjects effect.

For each of the following designs, list all sources of variance and their associated degrees of freedom. If there are multiple error terms in the analysis, list each error term immediately after the systematic sources of variance (i.e., the effects to be tested) for which it is used, as illustrated in class. Assume 10 subjects per group or between-subjects cell, or in the design as a whole if all subjects are in one group. c) A (between, 4 levels) X C (within, 3 levels) d) Now go back and mark each systematic source of variance in Parts a-d to which the special assumption of repeated-measures analysis applies? e) Consider tests of all possible pairwise comparisons among levels of Factor A in Part c). How many such tests would there be and what would the error term(s) be for these tests? f) Consider tests of all possible pairwise comparisons among levels of Factor C in Part c).

e. 6 tests. subj|A f. C1 vs C2 x subj|A C1 vs C3 x subj|A C2 vs C3 x subj|A

What probability does the False Discovery Rate aim to control? How is this different from the probability that other MCPs aim to control?

p (rejecting H0 I H0 is true) (Type I error) Attempts to control the proportion of rejected null hypotheses for which the null is actually true, other MCPs aim to control FW alpha

Define power as a probability.

power is p(rejecting H0 | H0 is false) = 1 - Beta Power is the probability of correctly rejecting the null hypothesis when it is false.

Repeated measures ANOVA makes an assumption that is not present in between-subjects ANOVA. a) Name the assumption in its current form. b) Define the assumption in words or a formula.

a & b) Sphericity: true sphericity assumption = theta^squared of (Yi - Yj) is a constant for all i,j pairs (i DNE J) --> the variance of the difference of all i,j pairs is a constant/the same across all levels

For each of the following research designs or questions, name the type of design, the independent and dependent variables, and any covariates. c) You are comparing 2 drugs for controlling blood pressure to a placebo. You are including men and women in your study but only to reduce within-group error variability.

a. Between-Subjects One-Way Design b. IV = drug DV = blood pressure c. covariate = sex

For each of the following research designs or questions, name the type of design, the independent and dependent variables, and any covariates. a) You are comparing 3 educational methods for teaching statistics. You measure each students knowledge before and after the class.

a. Between-Subjects Pre-Post Design b. IV = method DV = post-test knowledge c. covariates = pre-test knowledge

Why do we have MCPs?

a. Control increase in Type I error rate that comes from multiple tests

For each of the following research designs or questions, name the type of design, the independent and dependent variables, and any covariates. b) You are comparing 2 drugs for controlling blood pressure to a placebo, and hypothesize that one of them is more effective in women than in men. You measure blood pressure before and after a 4-week period of taking the drug.

a. Factorial Between-Subjects Pre-Post Design b. IV = drug condition, sex DV = blood pressure c. covariate = pre-intervention blood pressure

What is the characteristics of a situation when the Dunnett test would be the best choice among all of the MCPs presented in class?

a. Good for comparing groups to a common control

What is the characteristics of a situation when the False Discover Rate (FDR) test would be the best choice among all of the MCPs presented in class?

a. Good for genetics or fMRI studies, can test hundreds or thousands of genes/voxels between conditions.

What is the characteristics of a situation when the Tukey test would be the best choice among all of the MCPs presented in class?

a. Good for getting Confidence Intervals for all possible pairwise

What is the characteristics of a situation when the Bonferroni test would be the best choice among all of the MCPs presented in class?

a. Good for pairwise confidence intervals, assuming g > 3. Tailored for a specific set that you want to test.

What is the characteristics of a situation when the REGW test would be the best choice among all of the MCPs presented in class?

a. Good for performing all possible pairwise by computer

What is the characteristics of a situation when the Fisher-Hayter test would be the best choice among all of the MCPs presented in class?

a. Good for performing all possible pairwise by hand

For each of the following research designs or questions, name the type of design, the independent and dependent variables, and any covariates. e) You are interested in studying post-stress recovery of blood cortisol in men and women following exposure to a stressful laboratory task. For this purpose you sample blood cortisol every 30 minutes for 3 hours after completion of the task.

a. Mixed Sex x Time Design (analysis of variance) b. IV = sex, time DV = cortisol c. covariate = none