Research methods II final study
The error-term (i.e., denominator) for the F-test for one-way within-subjects ANOVA has _______ degrees of freedom.
(N-1) x (k-1)
Assume that you are taking the correlational approach and you already know that X and Y are correlated. Now you are trying to establish the directionality of the relationship: whether X causes Y or if Y causes X. To do this, you have collected measures of both X and Y at both Time 1 and Time 2, here referred to as: X1, Y1, X2, and Y2. The correlation between X1 and X2 is _______ . A. an autocorrelation B. a cross-lag correlation C. a cross-sectional correlation D. impossible to calculate
A
Q4: Assume that you believe that depression (D) causes anxiety (A). To test this claim, you have measured depression and anxiety (in a large number of participants) at two different times (1 & 2). You have already conducted the three preliminary tests (i.e., D1-A1, D1-D2, A1-A2) and all of these required correlations were large and significant. You are now about the conduct the two critical tests. In order for your data to support your claim that depression causes anxiety, which of the following needs to be true? A. the correlation between depression at Time 1 and anxiety at Time 2 (D1-A2) is significant and larger than the correlation between anxiety at Time 1 and depression at Time 2 (A1-D2) B. the correlation between anxiety at Time 1 and depression at Time 2 (A1-D2) is significant and larger than the correlation between depression at Time 1 and anxiety at Time 2 (D1-A2)
A
The decision whether to use a between-subjects design or a within-subjects design involves _______ . A. a trade-off between Construct Validity and Statistical Conclusion Validity B. a trade-off between Internal Validity and External Validity C. a trade-off between External Validity and Construct Validity
A
cross-lagged panel design
A method in which the same set of behaviors or characteristics are measured at two separate points in time (often years apart); six different correlations are computed, and the pattern of correlations is used to infer the causal direction.
One-way between-subjects ANOVA
A statistical procedure used to test hypotheses for one factor with two or more levels concerning the variance among group means. This test is used when different participants are observed at each level of a factor and the variance in a given population is unknown. Look at Levene's, alternative = Welch's
paired samples t-test
A test used to compare two means for a within-groups design, a situation in which every participant is in both samples; also called a dependent-samples t test
Q2: There are trade-offs between the four types of validity (plus Ethics, Practicality, & Efficiency). One of the most important implications of these trade-offs is that ___. A. while some studies or experiments might be better than others, there is no such thing as a "perfect" study or experiment. B. the amount of each validity is either zero or 100%. C. any given study or experiment is either perfect or useless. D. all research is useless.
A. No perfect experiment
According to the rule that is used in psychology (and most other empirical sciences), when the p-value is less than .05, we _______ . A. reject the null hypothesis (H0) B. retain the null hypothesis (H0) C. reject the alternative hypothesis (H1) D. retain the alternative hypothesis (H1)
A. Reject null hypothesis
If you reject a null hypothesis that is actually true (which is know as a "false alarm"), you _______ . A. have made a Type I error B. have made a Type II error C. owe it an apology [please don't choose this option]
A. Type I error
In the diagram below, all causal relationships are shown as solid-line arrows (as is typical). If this diagram is accurate, then the partial correlation between X & Y with respect to Z [prYX.Z] will be __________ . (If it matters, you may assume that the simple, bivariate correlation between X & Y [rYX] is strong and significant.) A. very small or zero and not significant B. just as large is the correlation between X & Y [rYX] and also significant C. much larger than the correlation between X & Y [rYX] and even more significant
A. Very small and not zero
In the diagram below, all causal relationships are shown as solid-line arrows (as is typical). If this diagram is accurate, then the partial correlation between X & Y with respect to Z [prYX.Z] will be __________ . (If it matters, you may assume that the simple, bivariate correlation between X & Y [rYX] is strong and significant.) A. very small or zero and not significant B. just as large is the correlation between X & Y [rYX] and also significant C. much larger than the correlation between X & Y [rYX] and even more significant
A. Very small or zero and not significant
Which of the following needs to be true in order for you to use the data to answer the critical question of whether X causes Y or if Y causes X? (Note that you are not yet trying to answer the directionality question; you are only verifying that the data can be used to answer this question.) X1 and Y1 are correlated X1 and X2 are correlated Y1 and Y2 are correlated all of the above
All of the above
Which of the following would meet the first criterion for establishing causation? a significant bivariate correlation a significant paired-samples t-test a significant independent-samples t-test all of the above
All of the above
What is a mixed-factor design
An experiment with at least one between-subjects factor and at least one within-subjects factor
A "design confound" is _______ . A. an aspect of the experiment that covaries with the dependent variable B. an aspect of the experiment that covaries with the independent variable C. whether one is using a within- or between-subjects design D. the same thing as a "subject confound"
B
A manipulation check is ________ . A. an extra measure at the very beginning of the experiment, used to verify successful and/or selective influence of the target theoretical construct B. an extra measure in the middle of the experiment (after the IV has been set but before the DV has been measured), used to verify successful and/or selective influence of the target theoretical construct C. an extra measure at the very end of the experiment, used to verify successful and/or selective influence of the target theoretical construct
B
If you test the same IV-DV relationship (as the original experiment), but make a few changes that shouldn't make a difference according to the current theory (e.g., switch from using red and green stimuli to using blue and orange stimuli), then you have run _______ . A. a direct replication B. a conceptual replication C. an extension D. a totally worthless experiment
B
Which of the following is one of the core tenets (or guiding concepts) of all empirical sciences, including psychology? A. Everyone must agree on the data and which theories are most-likely correct. B. Everyone must agree on the data, but are free to disgaree about which theories are most-likely correct. C. Everyone is free to disagree about the data, but must agree about which theories are most-likely correct. D. Everyone is free to disagree about both the data and which theories are most-likely correct.
B
Which of the following is still a potential problem even when you use a two-group design? A. history effects B. biased attrition effects C. maturation effects D. all of the above
B
The defining attribute of "experiment" (i.e., the thing that all true experiments have and non-experiments don't have) is _______ . A. at least one dependent variable B. at least one independent variable C. at least one extraneous variable D. at least one potential confound
B. At least one independent variable
Q3: Which of the following is not one of the three criteria for establishing (i.e., providing evidence in support of) a causal claim? A. covariation (association) B. external validity C. internal validity D. temporal precedence
B. External validity
Cross-Lagged Analysis
Before- 3 preliminaries Main analysis- two cross lagged Follow up tests- none
one-way within-subjects ANOVA analysis
Before- means & SEs (caveat), check sphericity Main analysis- repeated-measures ANOVA Follow up tests- optional pairs (Bon or D-S)
One-way between-subjects ANOVA analysis
Before- means & SEs, check equal variance Main analysis- ANOVA, get paired t-tests Follow up tests- optional pairs (Tukey's HSD)
Two-way ANOVA analysis
Before- means & SEs, check relevant assumptions Main analysis- repeated measures ANOVA Follow up tests- interaction (significant = parse, non-significant = run SMEs), additive: maybe some pairs
Simple (bivariate) correlation analysis
Before- nothing Main analysis- correlation (pearsons' R) Follow up tests- none
partial correlation analysis
Before- verify X-Y correlation (validate internal validity) Main analysis- X-Y*Z partial correlation Follow up tests- optional panel study on X & Z
A "mixed-factor" design is best defined as an experiment that has _______ . at least one between-subjects factor at least one within-subjects factor both of the above
Both
In order for the results from a quasi-experiment to be used to establish causation (without the need for additional conditions or measures), _______ . (Note: some people refer to quasi-experiments that don't need additional conditions or measures as "classic quasi-experiments.") (A) the value of the quasi-IV should be determined or set long before the experiment is run (B) the value of the quasi-IV should be determined or set after the experiment starts (C) the value of the quasi-IV should be set by a random process both (A) & (C) both (B) & (C)
Both A and C
More specifically, manipulation checks are used to verify _______ . successful manipulation of the target construct selective manipulation of the target construct both of the above neither of the above
Both of the above
A "p-value" is _______ . A. the probability that the null hypothesis is true B. the probability that the null hypothesis is false C. the probability of the getting the data, assuming that the null hyposthesis is true D. the probability of the getting the data, assuming that the null hyposthesis is false
C
Assume that the experiment used a two-way design and that, when the data were analyzed, the interaction was NOT significant. Which of the following is true. A. You now have to conduct some simple main effects. B. You have the option of conducting some simple main effects. C. You aren't allowed to conduct any simple main effects, but you might need to run some pairwise comparisons. D. The analysis ends immediately ... you never do anything else when the interaction isn't significant.
C
Assume that you have run a replication-with-extension experiment. The results in the replication condition are the same as in the original experiment, but the results in the extension condition are quite different (and the interaction is significant). What does this mean? A. That something went very wrong and all work should stop until you figure out the problem. B. You have evidence of generaliztion (of the original result). C. You have evidence of moderation (of the original result). D. You have evidence of both generalization and moderation (of the original result).
C
The Pre-test/Post-test, Two-group design ________ . A. is just as bad as the Pre-test/Post-test, One-group design B. uses a control group to eliminate the problems of a Pre-test/Post-test, One-group design C. uses a control group to measure the effects of the problems of a Pre-test/Post-test, One-group design D. all of the above
C
The beliefs of an experimenter become a threat to Internal Validity _______ . A. when the experimenter thinks about them B. when the experimenter tries to hide them C. when they cause the experimenter to behave differently when running the different conditions D. when they cause the experimenter to avoid vaccines
C
Which of the following is the correct way to express the null hypothesis for a (simple) bivariate correlation? A. H0: r = 0 B. H0: r ≠ 0 C. H0: ρ = 0 D. H0: ρ ≠ 0
C.
If a theory makes a certain prediction ... but your experiment fails to produce the predicted results ... then _______ . you can conclude that the theory is correct you can conclude that the theory is wrong you can't come to any logical conclusion
Can conclude theory is wrong
If a theory makes a certain prediction ... and your experiment produces the predicted results ... then _______ . you can conclude that the theory is correct you can conclude that the theory is wrong you can't come to any logical conclusion
Can't come to any logical conclusion
In general, manipulation checks are used to verify the _______ (of the independent variable).
Construct Validity
Between-subject designs usually have more _______ than within-subject designs. Construct Validity External Validity Internal Validity Statistical Conclusion Validity
Construct validity
Assume that the experiment used a two-way design and that, when the data were analyzed, the interaction is significant. Which of the following is true. A. You should probably ignore the two main effects, as they can be very misleading. B. You must choose a parsing before moving on to the next step of the analysis. C. You should probably draw some causal diagrams (if you haven't already). D. All of the above
D
By the strict definition (from lecture), a confound in an experiment is _______ . A. anything, including a belief, that covaries with dependent variable B. an objective (directly observable) thing that covaries with dependent variable C. anything, including a belief, that covaries with independent variable D. an objective (directly observable) thing that covaries with independent variable
D
Note that this question concerns SS values. The first step to making the transition from between-subjects ANOVA to within-subjects ANOVA is to change the detailed label for the "good" variance from SSbetween-groups to SSbetween-conditions. The next step is to change the detailed label for the "bad" variance from SSwithin-groups to SSwithin-conditions. The third step is to _______ . A divide SSbetween-conditions into two pieces, both of which will be used later B. divide SSbetween-conditions into two pieces, only one of which will be used later C. divide SSwithin-conditions into two pieces, both of which will be used later D. divide SSwithin-conditions into two pieces, only one of which will be used later
D
Placebo Effects _______ . A. are a potential problem for most tests of treatment efficacy B. are probably best viewed as a threat to Construct Validity (even if some texts refer to them as threats to Internal Validity) C. are a lot like the problem of demand-driven reactivity in other experiments D. all of the above
D
The "domain" of a theory is _______ . A. where it "lives" in the brain B. the journal in which the theory was first published C. the original situation for which the theory was developed D. the list of all situations to which the theory applies
D
The Pre-test/Post-test, One-group design _________ . A. was sometimes used to test the efficacy (effectiveness) of an intervention B. should actually be avoided (as it is terrible) C. has four threats to Internal Validity due to the fact that pre-test vs post-test cannot be counter-balanced D. all of the above
D
The decision whether to use a between-subjects design or a within-subjects design involves _______ . A. a trade-off between Construct Validity and Statistical Conclusion Validity B. a trade-off between Internal Validity and External Validity C. a trade-off between External Validity and Construct Validity D. All of the above
D
The issues related to Construct Validity _______ . A. have nothing to do with experiments B. only apply to measurement C. only apply to manipulations D. apply to both measurement and manipulations
D
Which of the following is not an accurate definition of "quasi-experiment"? A. An experiment in which subjects were not randomly assigned to the conditions of a between-subjects factor B. A correlational study in which a qualitative measure is treated as if it were a between-subjects factor C. An experiment with a between-subjects factor with values (or conditions) that are attributes of the subjects D. A correlational study in which one variable is measured more than once, like a within-subjects factor
D
When is it OK to provide or accept a best guess about a population value (such as the true mean, μ) without an associated estimate of how wrong this guess might be? A. always B. when the sample standard deviation is smaller than the sample mean C. on Tuesdays in March D. Never
D never
Q1: The quality or usefulness of any given study or experiment depends on ___. A. only one of the four types of validity, selected at random. B. two of the types of validity, selected by a committee of experts. C. three of the types of validity, with the researcher(s) choosing which to ignore. D. all four of the types of validity.
D. All four
The standard error of the mean _______ . A. is equal to the sample standard deviation divided by the square-root of sample size B. gets larger when the sample standard deviation increases (all else being equal) C. gets smaller when the size of the sample is increased (all else being equal) D. All of the above
D. All of the above
Assume that you are interested in the possible effects of two different factors on the same dependent measure (e.g., the effects of handedness and Stroop Condition on response time). What is the best reason to run one experiment using a two-way design instead of two separate experiments, each using a one-way design?
Detect moderation
Now assume that someone has suggested that the relationship between X and Y might actually be due to a third variable, Z. So you have collected a new set of data with measures of all three (at the same point in time). If the partial correlation between X and Y when controlling for Z (prYX∙Z) is the same as the (plain) correlation between X and Y (rYX) and both are statistically significant, then the true relationship between X and Y _______ . Must be spurious must be mediated by Z must be moderated by Z does not involve Z
Does not involve Z
Assume, again, that someone has suggested that the relationship between X and Y might actually be due to a third variable, Z, and you have collected a new set of data with measures of all three (at the same point in time). As before, the correlation between X and Y (rYX) is statistically significant, but this time the partial correlation between X and Y when controlling for Z (prYX∙Z) is small and not significant. In this case, the true relationship between X and Y is _______ . directly causal indirectly causal, mediated by Z spurious either spurious or indirectly causal, mediated by Z (one or the other ... not sure which yet)
Either spurious or indirectly causal
What assumption should be tested (i.e., checked) before conducting a between-subject ANOVA? equal variance sphericity both of the above
Equal variance
When conducting research concerning a causal claim, the decision to take the correlational approach vs the experimental approach involves a trade-off between _______ .
External and internal validity
The validity advantage of between-subject designs [see previous question] comes mostly from _______ . a larger effect a smaller standard error fewer demand characteristics Less chance of experimenter or observer bias
Fewer demand characteristics
Which type of validity is threatened by all of the effects in the previous two questions?
Internal validity
Which of the following equations is NOT true for one-way between-subjects ANOVA? SStotal = SSbetween + SSwithin dftotal = dfbetween + dfwithin MStotal = MSbetween + MSwithin none ... those are all correct
MStotal = MSbetween + MSwithin
In the diagram below, all causal relationships are shown as solid-line arrows (as is typical). If this diagram is accurate, then the correct label for the relationship between X and Y is __________ . Mediated vs spurious
Mediated
Another example of when you aren't free to choose the design-type is when the critical measure involves a pattern of results across two or more conditions (such as in the Stroop Task). In this case _______ .
Must use a within-subjects design
This helps make it clear why, when the null hypothesis is being (exactly) obeyed, the value of F will be _______ .
One
The data in the plot below provide an example of _______ . one main effect (only) two additve main effects an over-additive interaction an under-additive interaction
Over additive interaction
Which of the following threats to the validity of a one-group, pre-test/post-test design is NOT dealt with or addressed by the addition of a control group?
Regression effects
You believe that X causes of Y, but the partial correlation between X and Y when controlling for Z (prYX∙Z) is small and not significant. What do you need to do next?
Run a longitudinal or "cross-lagged" study that includes measures of X and Z.
This formal: E(X - X)2 is
SS total
The validity advantage of within-subject designs [see previous question] comes mostly from _______ . a larger effect a smaller standard error fewer demand characteristics less chance of experimenter or observer bias
Smaller standard error
What assumption should be tested (i.e., checked) before conducting a repeated-measures (within-subjects) ANOVA? equal variance sphericity both of the above
Sphericity
In the diagram below, all causal relationships are shown as solid-line arrows (as is typical). If this diagram is accurate, then the correct label for the relationship between X and Y is __________ . Mediated vs spurious
Spurious
Within-subject designs usually have more _______ than between-subject designs. Construct Validity External Validity Internal Validity Statistical Conclusion Validity
Statistical Conclusion Validity
When conducting an experiment, we usually try to hold every variable in which we're not interested either constant or equal-on-average. This decision -- i.e., constant vs equal-on-average -- involves a trade-off between _______ .
Statistical Conclusion Validity and External Validity
one-way within-subjects ANOVA
Statistical procedure used to evaluate a within-subjects experiment with three or more levels of a single independent variable. Look at Mauchly's, correction = Huynh-Feldt
Simple (Bivariate) Correlation
The relationship between two variables in terms of how they are related to or associated with one another
Here is the formula for sample variance (for a single set of values): The numerator (i.e., upper part) of the right side of this equation is _______ .
The sum of the squared deviations from the mean
The data in the plot below provide an example of _______ . one main effect (only) two additive main effects an over-additive interaction an under-additive interaction
Two additive main effects
Two-way ANOVA
a hypothesis test that includes two nominal independent variables, regardless of their numbers of levels, and a scale dependent variable
Assume that you've decided to conduct an experiment. The first thing that you must do is come up with _______ .
a method of manipulating the independent variable (IV) that doesn't create any confounds
Experimenter bias _______ .
becomes a threat to Internal Validity when it causes the experimenter to behave differently when running the different conditions of an experiment
three criteria for causation
covariance, temporal precedence, internal validity
The degrees of freedom associated with the standard error of mean of a single set of values (such as that used for point estimation or a univariate or paired t-test), is _______ .
equal to one less than the number of subjects ... i.e., N−1
The standard error of the mean (of a single set of values), such as that used for point estimation, is _______ .
equal to the standard deviation divided by the square-root of N ... i.e., s/√N
Assume that you have just finished running an experiment. Which of the following should you do first?
get the means and standard errors for a plot of the data
Which two threats to the validity of a one-group, pre-test/post-test design are both directly tied to the passage of time between the pre-test and post-test?
history effects and maturation effects
When planning an experiment that will have more than one factor (e.g., a two-way design), you should _______ .
make separate decisions with regard to between- vs within-subjects for each factor
Assuming that you have decided to use a within-subjects design, what threat to Internal Validity is specific to this design and what do we usually do to deal with this threat?
order effects ... counter-balance
Assuming that you have decided to use a between-subjects design, what threat to Internal Validity is specific to this design and what do we usually do to deal with this threat?
subject confounds (also called selection effects) ... use random assignment or matching
Assuming that the data can be used to answer the directionality question, which of the following patterns provides the best support for the idea that X is the cause and Y is the effect?
the X1-Y2 correlation is significant and larger than the Y1-X2 correlation
partial correlation
the correlation between two variables with the influence of a third variable statistically controlled for
In statistics, the term "power" refers to _______ .
the probability of rejecting a false null hypothesis
Imagine the (hypothetical) situation where you have taken an infinite number of independent samples (all of size N) from the same population and calculated the mean of each sample. Which of the following is NOT true (according to Central Limit Theorem)? A. the mean of the sample means is equal to the population mean B. the variance of the sample means is equal to the population variance C. the standard deviation of the sample means is equal to the population standard deviation divided by the square-root of N D. the shape of the distribution of sample means is Normal [i.e., bell-shaped]
the variance of the sample means is equal to the population variance
The _______ of the difference between two independent events (e.g., separate dice-rolls) ______ .
variance [first slot] ... equals the sum of their variances [second slot]
Assume that you are testing a new treatment for depression using a two-group, pre-test/post-test design and several participants in the control group either drop out of the experiment or refuse to return for the post-test. What should you do?
verify that the remaining participants in the control group were equal-on-average to the participants in the treatment group
In general, when do we use a common or global error-term and when do we use unique or specific error-terms?
we use common or global error-terms when it's a between-subjects design and we use unique or specific error-terms when it's a within-subject design
A "floor effect" occurs _______ .
when the values of the data become more similar (constricted) because they cannot go any lower
Sometimes you aren't free to choose whether to use a between- or within-subjects design. One example of this is when it is either impossible or unethical to manipulate the variable that you think is the cause because it is built into the subject (such as right- vs left-handed). In this case _______ .
you must use a between-subjects design
Which of the following is the null hypothesis for a one-way ANOVA when the independent variable has three levels?
μ1 = μ2 = μ3
The (true) mean of the population is μ and the (true) variance of the population is σ^2. You take a random sample of 10 values from this population and calculate the mean. Then, you take a new random sample 10 values from this population and calculate the mean. Then, you take another new random sample 10 values from this population and calculate the mean. What is the expected variance across the three means?
σ^2 / 10
Assume that the true mean of the population is μ and that the true variance of the population is σ^2. You take a random sample of 20 values from this population and calculate the mean. Then, you take a new random sample 20 values from this population and calculate the mean. Then, you take another new random sample 20 values from this population and calculate the mean. What is the expected variance across the three means?
σ^2/ 20
