Lab Methods In Psych Exam 4 (Chap. 8 and 9)

· Be able to identify the language used to describe results of a regression analysis.

- "Even when age is controlled for" "Even when holding constant"

sentences to watch for

- "controlled for" = regression analysis. - "taking into account" = researchers conducted multiple-regression analyses. "even when factors were considered" "adjusting for" = multiple regression

· Under what circumstance can statistical significance testing lead to an incorrect conclusion about the population?

- A weak correlation based on a small sample is more likely to be the result of chance variation and is more likely to be judged "not significant."

· Explain what it means for an association claim to have external validity. To whom are the results of a correlational study likely to generalize?

- An association claim has external validity when you ask whether the association can generalize to other people, places, and times. To interrogate the external validity of the association, the first questions would be who the participants were and how they were selected. Generalizes to target area of interest. - Association between groups is the same. (Both negative etc.)

· Explain why a bar graph is the best way to depict an association claim involving a categorical and a quantitative variable.

- Bar graph is the best way to depict an association claim involving a categorical and a quantitative variable because each person is not represented by one data point; instead, the graph shows the mean marital satisfaction rating for all the people who met their spouses online and the mean marital satisfaction rating for those who met their spouses in person.

· Be able to interpret standardized (beta) and unstandardized (b) coefficients in a regression analysis.

- Beta basics: column labeled beta is standardized beta. There will be on beta value for each predictor variable. Beta is similar to r, but reveals more than r does. Positive beta indicates positive relationship between that predictor and the criterion variable, when the other predictor variables are statistically controlled for. Same for negative. Beta that is zero, or not significantly different from zero, represents no relationship (when other predictors controlled for). Betas are similar to correlations in that they denote the direction and strength of a relationship. If a regression table has the symbol b instead of beta, the coefficient is an unstandardized coefficient. - Similar to beta in that sign of b (+ or -) denotes a positive or negative association (when the other predictors are controlled for). Unlike two betas, we cannot compare two b values within the same table to each other. B values are computed from original measurements of the predictor variables (dollars, centimeters, inches), whereas betas are computed from predictor variables that have been changed to standardized units. Predictor variable that shows a large b may not actually denote a stronger relationship to the criterion variable than a predictor variable with a smaller b.

· Explain how to tell if a study used a bivariate correlational design.

- Bivariate Association: an association that involves exactly two variables. An analysis of bivariate correlations looks at only two variables at a time.

· Explain how mediators are different from third variables.

- Both involve multivariate research designs, and researchers use the same statistical tool (multiple regression) to detect them but they function differently. - Third-variable: proposed third variable is external to the two variables in the original bivariate correlation; might even be seen as an accident - a problematic "lurking variable" that potentially distracts from the relationship of interest. Ex. if we propose that education level is a third variable responsible for the deep talk/well-being relationship, we're saying deep talk and well-being are correlated with each other only because each one is correlated separately with education. The relationship between deep talk and well-being is there only because both of those variables happen to vary with the outside third variable, education level. - Mediator: interested in isolating which aspect of the presumed causal variable is responsible for that relationship. Is internal to the causal variable and often of direct interest to the researchers, rather than a nuisance. Ex. researchers believe stronger social ties is the important aspect, or outcome, of deep talk that is responsible for increasing well-being.

· Usually the stronger a correlation (the larger its effect size), the more likely it will be statistically significant. The stronger an association is, the more rare it would be in a population in which the association is zero. We can't tell whether a particular correlation is statistically significant by looking at its effect size alone. We have to look for p values associated with it. Under what conditions are small effect sizes likely to be important?

- Conditions that have life-or-death implications are when small effect sizes are likely to be important. Ex. Aspirin a day associated with a lower rate of heart attacks. R = .03, very weak association, but in terms of number of lives saved, even this small association was substantial.

· Which of the three causal criteria do all association claims meet?

- Covariance: the results must show a correlation, or association, between the cause variable (independent variable) and effect variable (dependent).

· Be able to distinguish predictor and criterion variables.

- Criterion Variable: (dependent variable) The variable researchers are most interested in. (pregnancy) - Predictor Variable: (independent variable) rest of the variables measured in a regression analysis.

· Which of the three causal criteria do cross-lag correlations help to address? Which criterion do they not help to address?

- Cross-lag correlations help to address temporal precedence because they show whether the earlier measure of one variable is associated with the later measure of the other variable. - Do not help address internal validity.

· Define cross-sectional correlation, autocorrelation, and cross-lag correlation. Which of these do researchers typically find most interesting and why? Which of these can be examined in both simple bivariate designs and in longitudinal designs?

- Cross-sectional correlations: test to see whether two variables, measured at the same point in time, are correlated. Can be examined in both bivariate and longitudinal designs. - Autocorrelations: determine the correlation of one variable with itself, measured on two different occasions. (time 1 compared to time 2 etc.) - Cross-lag correlations: show whether the earlier measure of one variable is associated with the later measure of the other variable. Researchers are usually more interested in these kinds of correlations. Address the directionality problem and help establish temporal precedence.

· Given a set of example correlations, be able to classify each one as cross-sectional, autocorrelation, or cross-lag.

- Cross-sectional: study reports that the correlation between mothers' overvaluation at Time 4 and children's narcissism at Time 4 was r=.099. This is a weak correlation, but consistent with hypothesis. Both variables were measured at the same time, cannot establish temporal precedence. - Autocorrelations: asked whether mothers' overvaluation at Time 1 was associated with mothers' at Time 2, Time 3 etc. and asked whether children's narcissism at Time 1 was associated with their scores at Time 2, Time 3, etc. - Cross-lag: show how strongly mothers' overvaluation at Time 1 is correlated with child narcissism later on, compared to how strongly child narcissism at Time 1 is correlated with mothers' overvaluation later on. Only one set of the cross-lag correlations was statistically significant; the other set was not significant. Mothers who overvalued their children at one time period had children who were higher in narcissism 6 months later.

· What is a curvilinear relationship? Explain why r cannot be used to describe curvilinear relationships. What value of r is likely to be obtained when the relationship between two variables is curvilinear?

- Curvilinear association: the relationship between two variables is not a straight line; it might be positive up to a point, and then become negative. R is designed to describe the slope of the best-fitting straightline through the scatterplot. When the slope goes up and then down (or vice versa), r does not describe the pattern very well. If we looked at only r and not the scatterplot, we might conclude no relationship. When researchers suspect a curvilinear association, the statistically valid way to analyze it is to compute the correlation between one variable and the square of the other.

Explain how r gives information about the strength and direction of a relationship

- Direction refers to whether the association is positive, negative, or zero. Strength refers to how closely related the two variables are - how close r is to 1 or -1.

· Explain why temporal precedence is also known as the directionality problem.

- Directionality problem: we don't know which variable came first.

· Describe the five steps used to test mediation.

- Establish that the predictor variable is related to the dependent variable. - Establish that the predictor variable is related to the mediating variable. - Establish that the mediating variable is related to the dependent variable. - Run a regression model - More definitive evidence for mediation is established only when the proposed causal variable is measured before the proposed mediator, which is measured before the dependent variable. Measured not manipulated. Does not establish causality.

· Explain what happens when you add more predictors to a regression analysis.

- Even when there are many more predictor variables in the table, beta still means the same thing. Adding several predictors to a regression analysis can help answer two kinds of questions: First, helps control for several third variables at once. Second, by looking at the betas for all the other predictor variables, we can get a sense of which factors most strongly predict chance of pregnancy.

· Be able to recognize examples of questions that would be asked about the construct validity of an association claim.

- How well were the two variables measured? Does the measure have a good reliability? Is it measuring what it's intended to measure? What is the evidence for its face validity, its concurrent validity, its discriminant and convergent validity?

For each of the three types of correlations in a longitudinal correlational design, be able to provide an interpretation of significant and nonsignificant correlations in a cross-lag model

- In cross-lag model: positive statistical values (with arrows) indicate that one came before the other. If it says n.s. not significant.

· Explain why it is not always possible to conduct an experiment. If it isn't possible to conduct an experiment because of practical or ethical reasons, what can the researcher do instead?

- In some cases, people cannot be randomly assigned to a causal variable of interest. Ex. We cannot manipulate personality traits, such as narcissism in children. Could be unethical to assign some people, especially children, to a condition in which they receive a certain type of praise, especially over a long time period. - Researchers develop ethical experiments to study such as studying reactions over a short time period to learn more about the effects of praise on children. By randomly assigning children to receive praise for who they are vs how hard they worked, researchers have produced some solid evidence that children really do change their behavior and attitudes in response to adult praise. Much more challenging to do an ethical experimental study of the effects of long-term exposure to potentially maladaptive praise at home which makes longitudinal correlational designs an attractive alternative.

· What terms are used to describe independent and dependent variables in the context of regression analysis?

- Independent variables are Predictor variables. - Dependent variable is Criterion variable.

· When asking questions about construct validity, how is interrogation of an association claim different from interrogation of a frequency claim?

- Interrogation of an association claim is different than the interrogation of a frequency claim because it focuses on the construct validity of each variable and how well each of the two variables were measured.

longitudinal vs multiple regression satisfy which criteria of causal claims?

- Longitudinal correlational designs can satisfy temporal precedence criterion. Multiple-regression analyses statistically control for some potential internal validity problems.

Longitudinal vs multiple regression vs pattern and parsimony

- Longitudinal designs allow researchers to establish temporal precedence in their data; multiple-regression analyses help researchers rule out certain third variable explanations; and the "pattern and parsimony" approach is when the results of a variety of correlational studies all support a single, causal theory. All are measured, none manipulated.

· Explain the features of longitudinal designs. What makes a study longitudinal?

- Longitudinal designs: can provide evidence for temporal precedence by measuring the same variables in the same people at several points in time. Used in developmental psychology to study changes in a trait or an ability as a person grows older. Can be adapted to tests causal claims. - A study is longitudinal when researchers measure the same variables in the same group of people across time.

· Define the term mediator.

- Mediator: (mediating variable) Researchers may propose a mediating step between two of the variables when questioning the relationship of why. Such as why watching sexual content on TV predicts a higher pregnancy risk. Study does not have to be correlational to include a mediator; experimental studies can also test them. Mediation analyses often relies on multivariate tools such as multiple regression.

· What is a moderator variable? How do moderator variables provide information about external validity?

- Moderator: When the relationship between two variables changes depending on the level of another variable, that other variable is the moderator. When an association is moderated by residential mobility, type of relationship, day of the week, or some other variable, we know it does not generalize from one of these situation to the others.

What does it mean to control for a third variable in a regression analysis? How is controlling for a third variable similar to identifying subgroups?

- Multiple regression: (multivariate regression) can help rule out some third variables, thereby addressing some internal validity concerns. - By conducting a multivariate design, researchers can evaluate whether a relationship between two key variables still holds when they control for another variable. The phrase "control for" such as controlling for age is to talk about proportions of variability. Researchers are asking whether, after they take the relationship between age and pregnancy into account, there is still a portion of variability in pregnancy that is attributable to watching sexy tv. Easier way to understand "controlling for" is to recognize that testing a third variable with multiple regression is similar to identifying subgroups. We start by looking only at the oldest age group and see whether viewing sexual TV content and pregnancy are still correlated. Then we move to the next oldest etc. We ask whether the bivariate relationship still holds at all levels.

Explain how a study can be correlational even if it doesn't use a correlation coefficient to describe the relationship between two variables

- No matter what kind of graph, when the method of the study measured both variables, the study is correlational, and therefore it can support an association claim. An association claim is not supported by a particular kind of statistic or a particular kind of graph; it is supported by a study design - correlational research - in which all the variables are measured.

· What is an outlier? How do outliers influence r?

- Outlier: an extreme score - a single case (or a few cases) that stand out from the pack. Depending on where it sits in relation to the rest of the sample, a single outlier can have an effect on the correlation coefficient r. Depending on where the outlier is, it can make a medium-sized correlation appear stronger, or a strong one appear weaker, than it really is.

Explain why outliers are more influential on effect size estimates in small samples as compared to large samples.

- Outliers matter the most when a sample is small because if there are 500 points in a scatterplot, one outlier is not going to have as much impact. If there are only 12 points, an outlier has much more influence.

· Explain why it is more difficult to interrogate statistical validity of a regression model than of a simple bivariate design.

- P value gives the probability that the beta came from a population in which the relationship is 0. When p is less than .05 the beta is considered statistically significant. When p is greater than .05, beta is considered not significant, meaning we cannot conclude beta is different from zero. - Simple bivariate relationship isn't enough to show causation. There can be temporal precedence problems and third variables present that cause concern for internal validity. A multiple-regression analysis could hold for example parental involvement constant and see if family meal frequency is still associated with academic success. Relationships can go away when potential third variables are controlled for. - More difficult to detect subgroups, outliers, and curvilinear relationships when you have more than 2 variables in your statistical model.

What does it mean to look for pattern and parsimony in research?

- Parsimony: The degree to which a scientific theory provides the simplest explanation of some phenomenon. In the context of investigating a causal claim, parsimony means the simplest explanation of a pattern of data - the theory that requires making the fewest exceptions or qualifications. - "pattern and parsimony": there's a pattern of results best explained by a single, parsimonious causal theory. Researchers can investigate causality by using a variety of correlational studies that all point in a single, causal direction. - Researchers commonly use a variety of methods and many studies to explore the strength and limits of a particular research question. - Strong basis for establishing causal relationships between variables

· Be able to describe the results of the study involving two quantitative variables if given a scatterplot depicting those results.

- Positive r means relationship is positive. High scores on one variable go with high scores on the other. In other words, high percentages of substantive conversation go with high levels of well-being.

· Be able to identify the magnitude of a correlation coefficient's effect size based on Cohen's guidelines.

- R of +/- .1 = small, or weak - R of +/- .30 = medium, or moderate - R of +/- .50 = large, or strong

· Why should journalists report on the previous body of research when writing about a newly published scientific study?

- Reporting on the latest study without giving full context can make it seem as though scientists conduct unconnected studies on a whim. Might give the impression that one study can reverse decades of previous research. Skeptics might find it easier to criticize the results of a single, correlational study.

· What is restriction of range and how does it affect r?

- Restriction of range: In a correlational study, if there is not a full range of scores on one of the variables in the association, it can make the correlation appear smaller than it really is.

· What is a spurious correlation? Explain how subgroups in the data can create spurious correlations between two variables.

- Spurious association: the original association. bivariate correlation is there, but only because of some third variable. - If we study a scatterplot of just the green dots, we see that within the subgroup of well-educated people, there is no positive relationship between deep talk and happiness. Spread out and has no positive slope at all. If we study the pattern of just the blue dots, the same thing occurs; less-educated people are lower on both happiness and deep talk, and within the group, the cloud shows no positive relationship between the two. Means that only reason deep talk and happiness are correlated is because well-educated people are higher on both of these variables. Education presents a third variable problem.

· Explain what it means to say that statistical significance depends on effect size and sample size.

- Statistical significance is related to effect size; usually the stronger a correlation, the more likely it will be statistically significant. Have to look for p values. A small effect size will be statistically significant if it is identified in a very large sample. A weak correlation based on a small sample is more likely to be the result of chance variation and is more likely to be judged "not significant."

Explain the type of variable (measured or manipulated) needed to make an association claim and why this is the case.

- The variables would be measured because researcher's can measure where people meet their spouses, but can't reasonable assign people to meet their spouse either online or in person. They can measure marital satisfaction, but they can't assign people to be satisfied or not.

· What is a third variable? How is a third variable related to each of the variables in an established association?

- Third variable problem: internal validity criterion. When we can come up with an alternative explanation for the association between two variables, that alternative is some lurking third variable. Can create spurious correlations (sometimes). - Covariance of cause and effect: We already know deep talk is associated positively with well-being. As the percentage of deep talk goes up, well-being goes up, thus showing covariance of the proposed cause and proposed effect. - Temporal precedence: study's method meant that deep talk and well-being were measured during the same, short time period, so we cannot be sure whether people used deep talk first, followed by an increase in well-being, or whether people were happy first and later engaged in more meaningful conversations. - Internal Validity: between deep talk and well-being could be attributable to some third variable connected to both deep talk and well-being. Ex. Busy, stressful life might lead people to report lower well-being and have less time for substantive talks. Or in this college sample, having a strong college-prep background is associated with deep conversations and having higher levels of well-being in college (Bc they are better prepared). Not any third variable will do. Must correlate logically with both of the measured variables in the original association. Might propose income is an alternative explanation, arguing that people with higher incomes will have greater well-being.

Explain why it is important to examine the sampling technique when asking about the external validity of an association claim.

- To interrogate the external validity of the association, the first questions would be who the participants were and how they were selected. See if it's generalizable.

· Why are experiments better than multiple-regression designs for controlling for third variables?

- Unknown third variable problem is one reason that a well-run experimental study is ultimately more convincing in establishing causation than a correlational study. An experimental study on TV, for example would randomly assign a sample of people to watch either sexy TV shows or programs without sexual content. Power of random assignment would make two groups likely to be equal on any third variables the researchers did not happen to measure such as religiosity, social class, or parenting styles. - Because participants are randomly assigned to independent variables, experiments are superior because random assignment ensures that the only difference between groups is the level of the independent variable to which they were assigned. If the groups are equal except for the independent variable, we can infer that the independent variable caused any differences between the groups on the dependent variable. - In regression, not possible to account for every third variable. Sometimes however, it's impractical or unethical to manipulate a variable. In the absence of random assignment, there are usually differences between groups. Regression allows us to control for some of these differences while examining the relationship we're interested in.

· Explain what might happen to coefficients in a regression model if you add or remove predictors from the model.

- Unlike r, there are no quick guidelines for beta to indicate effect sizes that are weak, moderate, or strong. Betas change, depending on what other predictor variables are being used - being controlled for - in the regression.

· Explain how controlling for third variables helps to address the issue of internal validity.

- We can determine internal validity between the variables when analyze and control for third variables. If the results on a scatterplot show a positive correlation with the three variables, we can tell that the relationship exists even when we hold that variable constant. Ex. Label age groups on graph of pregnancy risk vs exposure to sexual tv content. High positive correlation. All three age groups positive correlation. In a case where the third variable was the reason for the relationship, there would be no positive correlation within the age groups. There would be clusters of 16, 18, 20, no overall correlation. - Regression analysis helps address internal validity because it helps us rule out some potential third variables.

· If given a description of a hypothesis, or a figure representing a hypothesis, be able to identify the mediator.

- We know conscientious people are more physically healthy than less conscientious people. Why? The mediator might be the fact that conscientious people are more likely to follow medical advice and instructions and that's why they're healthier. Following doctor's orders would be the mediator of the relationship between the trait, conscientiousness, and the outcome, better health. - Association between having deep conversations and feelings of well-being. Researchers might next propose a reason - a mediator of this relationship. One likely mediator could be social ties: deeper conversations might help build social connections, which in turn can lead to increased well-being. Social ties mediate the relationship between deep talk and well-being in hypotheses.

· Explain why it is problematic when a journalist reports on a single study.

- When journalists report only one study at a time, they selectively present only a part of the scientific process. They might not describe the context of the research such as what previous studies have revealed, or what theory the study was testing.

· Explain how mediators are different from moderators.

- When researchers test for mediating variables, they ask: why are these two variables linked? When they test for moderating variables, they ask: Are these two variables linked the same way for everyone, or in every situation? Mediators ask: why? Moderators ask: Who is most vulnerable? For whom is the association strongest? - Mediation hypothesis could propose medical compliance is the reason conscientiousness is related to better health. Moderation hypothesis could propose that the link between conscientiousness and good health is strongest among older people (perhaps because their health problems are more severe, and most likely to benefit from medical compliance) and weakest among younger people (whose health problems are less serious anyway). - Mediating variable comes in middle of other two variables. Moderate can mean "to change" and can change the relationship between the two other variables (less intense and more intense).

· When looking at a bar graph depicting the relationship between a categorical variable and a quantitative variable, how can you tell if an association between the two variables exists?

- You examine the difference between the group averages to see whether there is an association.

· Be able to distinguish between an effect size and its statistical significance. What does it mean for an effect to be statistically significant? What is the relationship (if any) between the size of an effect and its statistical significance?

- effect size: describes the strength of a relationship between two or more variables. (whichever r is closest to 1). - Statistical significance: refers to the conclusion a researcher reaches regarding the likelihood of getting a correlation of that size just by chance, assuming there's no correlation in the real world. P value is the probability that the sample's association came from a population in which the association is zero. If probability (p) associated with the result is very small (less than 5%) we know that the result is very unlikely to have come from a zero-association population. Correlation considered statistically significant. R of of -.19 has low probability of being from a zero-association population (p<.05) so we can conclude their result is statistically significant. If p is large and it's greater than .05 we cannot rule out the possibility of a type 1 error.

· Explain why larger effect sizes enable more accurate predictions.

- effect size: describes the strength of a relationship between two or more variables. (whichever r is closest to 1). - When two variables are correlated, we can make predictions of one variable based on information from another. The more strongly correlated two variables are (larger the effect size), the more accurate our predictions can be. - - Errors of prediction get larger when associations get weaker.

Multivariate designs

- involving more than two measured variables (longitudinal, multiple-regression, and pattern and parsimony). Extremely useful and widely used tools, especially when experiments are impossible to run.

· Given a particular hypothesis, be able to identify which correlations in a longitudinal design would and would not need to be significant in order for the hypothesis to be supported.

????

· If an association claim is made based on a biased (unrepresentative) sample, what must be done in order to determine the external validity of the claim?

Can accept the study's results and leave the question of generalization to the next study, which might test the association between these two variables in some other population. Many do generalize - even to samples different from original. Imagine study of college students in the U.S. that found men to be taller than Americans? Most likely it would. Still find same association between sex and height because Dutch men are taller than Dutch women.

· Describe the kind of research questions for which pattern and parsimony in research is especially valuable.

Can converge multiple findings to one central principle.

Internal validity -> we can claim one variable caused a change in another.

Manipulated variables necessary for causal claims (internal). Internal is low priority for association claims. Can be tempting to conclude an association claim is causal. Just bc two variables are related, doesn't mean that one variable caused the other. Other possible explanations including third-variables.

What statistical test is commonly used to evaluate association claims where one variable is categorical and the other is quantitative?

Occasionally use r, it is more common to test whether the difference between means is statistically significant, usually by using a statistic called the t test, or other statistical tests.

· Explain how to tell which variable in the model has the largest effect on the criterion variable.

The higher the beta is, the stronger the relationship between that predictor variable and the criterion variable. Smaller beta is, the weaker the relationship

· *** When describing associations between two quantitative variables, we use scatterplots, and r to test the correlation.

Use scatterplots for two quantitative??

Association claims are focused on

construct and statistical validity

longitudinal designs can provide some evidence for causation by fulfilling two of the three criteria

covariance and temporal precedence

randomized experiment vs multiple regression

· A randomized experiment is the gold standard for determining causation. Multiple regression, in contrast, allows researchers to control for potential third variables, but only for the variables they choose to measure.