Research Methods 2

Be knowledgeable as to what a mediator variable is

- it explains the relationship between the dependent variable and the independent variable.

You should be aware of the one group, pretest-posttest design and why it is such a bad design.

-A researcher recruits one group of participants, measures them on a pre- test, exposes them to a treatment, intervention, or change, and then measures them on a posttest. This design differs from the true pretest/posttest design you learned in Chapter 10, because it has only one group, not two. There is no com- parison group. Therefore, a better name for this design might be "the really bad experiment." Understanding why this design is problematic can help you learn about threats to internal validity and how to avoid them with better designs.

How do outliers affect correlation coefficients?

-An outlier is an extreme score—a single case (or a few cases) that stands out from the pack. -Depending on where the outlier is, it can make a medium-sized correlation appear stronger, or a strong one appear weaker, than it really is. Outliers can be problematic because even though they are only one or two data points, they may exert disproportionate influence. -Outliers matter the most when a sample is small

control variable

-Any variable that an experimenter holds constant on purpose is called a control variable.

Be familiar with the different types of correlations that can exist in longitudinal studies

(autocorrelations, cross-sectional, and cross-lag).

Multiple regression analysis has multiple predictors and a single criterion

-Criterion variable- to choose the variable they are most interested in under- standing or predicting; this is known as the criterion variable, or dependent variable. -The rest of the variables measured in a regres- sion analysis are called predictor variables, or independent variables -We use this analysis to predict the outcome (criterion variable), but we also use this analysis to determine which predictors are significant predictors of the outcome and which are not -Multiple regression is not a foolproof way to rule out all kinds of third variables.

You should be familiar with independent groups designs versus within-groups designs in terms of measurement of the DV, number of participants needed, and advantages/disadvantages of each.

-Independent-groups design (aka between-subjects design or between-groups design)which different groups of participants are placed into different levels of the independent variable. •Within-groups design (aka within-subjects design) here is only one group of participants, and each person is presented with all levels of the independent variable. Mueller and Oppenheimer (2014) might have run their study as a within-groups design if they had asked each participant to take notes twice—once using a laptop and another time handwritten.

Longitudinal studies - through cross-lag correlations - help us establish temporal precedence

-Longitudinal research is used in developmental psychology to study changes in a trait or an ability as a person grows older. In addition, this type of design can be adapted to test causal claims.

How is probability related to correlation coefficient?

-Probability, or significance of the correlation coefficient, is determined by the correlation coefficient. -the probability of a correlation not being attributed to chance is derived from the correlation coefficient. Probability is commonly referred to as the P-value, and when this value is less than .05, it means the results are statistically significant (the results are not attributed to chance). -larger effect size means more accurate predictions

Be familiar with the different types of replication (direct, conceptual, conceptual with extension). What information do they provide us? How are they similar? How are they different?

-direct replication, researchers repeat an original study as closely as they can to see whether the effect is the same in the newly collected data. Direct replication makes good sense. However, if there were any threats to internal validity or flaws in construct validity in the original study, such threats would be repeated in the direct replication too. In addition, when successful, a direct replication confirms what we already learned, but it doesn't test the theory in a new context -conceptual replication, researchers explore the same research question but use different procedures. The conceptual variables in the study are the same, but the procedures for operationalizing the variables are different. -replication-plus-extension study, researchers replicate their original experiment and add variables to test additional questions.

How does restricting the range of one or both of the variables affect the correlation coefficient?

-if there is not a full range of scores on one of the variables in the association, it can make the correlation appear smaller than it really is. This situation is known as restriction of range. -restriction of range makes correlations appear smaller, we would ask about it primarily when the correlation is weak. When restriction of range might be a problem, researchers could either use statistical techniques that let them correct for restriction of range, or, if possible, recruit more people at the ends of the spectrum. -Ex: student scores on the SAT currently range from 400 to 1600. But our selective College S admits only students who score 1200 or higher. Therefore, the true range of SAT scores is restricted in College S; it ranges only from 1200 to 1600 out of a possible 400 to 1600.

Be familiar with the defining characteristic of the simple experiment -

-one manipulated variable (independent variable) and one measured variable (dependent variable). -Keep in mind that simple experiments help us to determine cause-and-effect relationships between the independent and dependent variables. They do this through features/processes like having treatment and control groups, random assignment to these groups, experimental control, and pre- posttreatment measurement of the DV.

dependent variable

-outcome variable

Be able to identify the type of correlation (positive, negative, no correlation) from a scatterplot or correlation coefficient.

-strength refers to how closely related the two variables are—how close r is to 1 or −1. -a positive r means that the relationship is positive -Recall that r has two qualities: direction and strength. Direction refers to whether the association is positive, negative, or zero; strength refers to how closely related the two variables are how close r is to 1 or −1. -qualitative is numbers/ categorical is bar graph because you can't measure that

Be familiar with the similarities, as well as differences between mediators and third variables.

-the proposed third variable is external to the two variables in the original bivariate correlation; it might even be seen as an accident—a problematic "lurking variable" that potentially distracts from the rela- tionship of interest. -•Similarities- -Both involve multivariate research designs. -Both can be detected using multiple regression -Differences- -Third variables are external to the bivariate correlation (problematic) .-Mediators are internal to the causal variable (not problematic).

Realize the importance of correlational research with more than 2 measured variables .

-we can get closer to determining the true cause of a particular outcome. - correlation is not causation, what are the options? Researchers have developed some techniques that enable them to test for cause. The best of these is experimentation: Instead of measuring both variables, researchers manipulate one variable and measure the other.

multiple baseline design (ABA)

A type of single-subject design in which a treatment is instituted at successive points in time for two or more persons, settings, or behaviors.

Be familiar with selection effects - what are they, how do arise, how are they dealt with?

-when the kinds of participants in one level of the indepen- dent variable are systematically different from those in the other - A selection effect may result if the experimenters assign one type of person (e.g., all the women, or all who sign up early in the semester) to one condition, and another type of person (e.g., all the men, or all those who wait until later in the semester) to another condition.

Be familiar with multiple regression analysis and how we can use this type of analysis to deal with the third variable problem.

-which can help rule out some third variables, thereby addressing some internal validity concerns. -By measuring all these variables instead of just two (with the goal of testing the interrelationships among them all), they conducted a multivariate correlational study. -conducting a multivariate design, researchers can evaluate whether a relationship between two key variables still holds when they control for another vari- able. To introduce what "controlling for" means, let's focus on one potential third variable: age. Perhaps sexual content and pregnancy are correlated only because older teens are both more likely to watch more mature shows and more likely to be sexually active.

What is the correlation coefficient in curvilinear relationships?

-which the relationship between two variables is not a straight line; it might be positive up to a point, and then become negative. -ex:when we compute a simple bivariate correlation coefficient r on these data, we get only r = −.01 because r is designed to describe the slope of the best- fitting straight line through the scatterplot. When the slope of the scatterplot goes up and then down (or down and then up), r does not describe the pattern very well. The straight line that fits best through this set of points is flat and horizontal, with a slope of zero. Therefore, if we looked only at the r and not at the scatterplot, we might conclude there is no relationship between age and use of health care. When researchers suspect a curvilinear association, the statistically valid way to analyze it is to compute the correlation between one variable and the square of the other.

3 causal criteria

1. covariance-The results must show a correlation,or association, between the cause variable and the effect variable. 2. temporal precedence- The cause variable must precede the effect variable; it must come first in time. 3. internal validity (no 3rd variables)-There must be no plausible alternative explanations for the rela- tionship between the two variables.

Be familiar with the different threats to internal validity and how they are dealt with in experimental studies.

1.Maturation threats to internal validity-, a change in behavior that emerges more or less spontaneously over time. 2.History threats to internal validity-result from a "historical" or external factor that systematically affects most members of the treatment group at the same time as the treatment itself, making it unclear whether the change is caused by the treatment received. 3.Regression threats to internal validity-refers to a statistical concept called regression to the mean. 4.Attrition threats to internal validity-becomes a problem for inter- nal validity when attrition is systematic; that is, when only a certain kind of participant drops out. If any random camper leaves midweek, it might not be a problem for Nikhil's research, but it is a problem when the rowdiest camper leaves early. His departure creates an alternative expla- nation for Nikhil's results: Was the posttest aver- age lower because the low-sugar diet worked, or because one extreme score is gone? 5.Testing threats to internal validity-a specific kind of order effect, refers to a change in the partic- ipants as a result of taking a test (dependent measure) more than once. People might have become more practiced at taking the test, leading to improved scores, or they may become fatigued or bored, which could lead to worse scores over time. Therefore, testing threats include practice effects 6.Instrumentation threats to internal validity-when a measuring instrument changes over time. In observational research, the people who are coding behaviors are the measuring instrument, and over a of time, they might change their stan- dards for judging behavior by becoming more strict or more lenient. Thus, maybe Nikhil's campers did not really become less disruptive; instead, the people judging the campers' behavior became more tolerant of shoving and hitting.

comparison group

enables us to compare what would happen both with and without the thing we are interested in

interrupted time series design

A design in which the effectiveness of a treatment is determined by examining a series of measurements made over an extended time period both before and after the treatment is introduced. The treatment is not introduced at a random point in time.

stable-baseline design (AB)

A small-N design in which a researcher observes behavior for an extended baseline period before beginning a treatment or other intervention; if behavior during the baseline is stable, the researcher is more certain of the treatment's effectiveness.

How are correlational designs and quasi-experiments similar to one another and in what ways are they different?

Both may examine relationship between two variables of interest. •Neither use random assignment.•Neither* use manipulated variables.-*Quasi-experiments use techniques like "active selection."

Why does replication sometimes fail?

Contextually sensitive effects •Number of replication attempts •Problems with original study -Sample Size-In large samples, extreme cases will almost certainly be cancelled out by individuals at other extremes. But in small samples, this is less likely to happen. Small samples can accidentally lead to significant findings that can't be replicated because there probably wasn't a real effect in the first place -Harking-hypothesising after results are known -P-Hacking- p-hacking, in part because the goal is to find a p value of just under .05, the traditional value for significance testing (Simmons, Nelson, & Simonsohn, 2011). Researchers may not intentionally analyze their data in a biased way, but cognitive biases can creep in (Nuzzo, 2015). The result is that a reported effect is not really there, and therefore cannot be replicated.

The results of the experiment by van Kleef and her colleagues did show covariance between the causal (independent) variable (size of bowl) and the outcome (depen- dent) variable (amount of pasta eaten)

Experiments Establish Covariance

What is an interaction effect? What is a crossover interaction effect? What is a spreading interaction effect?

Interaction effect-adding an additional independent variable allows researchers to look for an interaction effect (or interaction)—A person's score on the GRE measure can be represented with the following whether the effect of the original independent variable (cell formula:phone use) depends on the level of another independent variable (driver age). Therefore, an interaction of two independent variables allows researchers to establish whether or not "it depends." Crossover effect- Notice that the lines cross each other; this kind of interaction is some- times called a crossover interaction, and the results can be described with the phrase "it depends." People's preferred food temperature depends on the type of food. Spreading interaction effect-Notice that the lines are not parallel, and they do not cross over each other. This kind of interaction is sometimes called a spreading interaction, and the pattern can be described with the phrase "only when." My dog sits when I say "Sit," but only when I'm holding a treat.

What's the importance of replication?

It makes sense that a finding should be replicated in order to be considered important. If a scientist claimed to have discovered evidence of life on Mars, but other scientists could not find similar evidence, nobody would believe there is life on Mars. If research found that playing violent video games increased aggressive behavior but other studies did not yield the same result, the first result cannot be considered accurate. Replication gives a study credibility, and it is a crucial part of the scientific process.

What is measurement error and situational noise and how do they affect our ability to detect significant findings?

Measurement Error-a human or instrument factor that can inflate or deflate a person's true score on the dependent variable. For example, a man who is 160 centimeters tall might be measured at 160.5 cm because of the angle of vision of the person using the meter stick, or he might be recorded as 159.5 cm because he slouched a bit. Situational noise-external distractions—is a third factor that could cause variability within groups and obscure true group differences.

When is it less important for a study? (external validity)

Sometimes you want to know whether a lab situation created for a study generalizes to real-world settings. For example, you might ask whether the stud- ies on lying about playing cards, which took place in the laboratory on a computer, would generalize meaningfully to real-world lies. A study's similarity to real- world contexts is sometimes called its ecological validity, or mundane realism (explained in detail later in this chapter). Many psychologists consider ecological validity to be one aspect of external validity

What information is provided in the name given to a factorial design? For example, a 2 x 3 factorial design.

Suppose these researchers ran their study again, but this time they compared how high or low self-esteem people responded to three kinds of statements: Positive self-statements, negative self-statements, and no statements. What kind of design would this be?

What is a moderator variable and how is it related to an interaction effect?

The process of using a factorial design to test limits is sometimes called testing for moderators. factorial design language, a moderator is an independent variable that changes the relationship between another independent variable and a dependent variable. In other words, a moderator results in an inter- action; the effect of one independent variable depends on (is moderated by) the level of another independent variable.

these are a method for addressing the external validity of the conclusions of a small-N study?

Triangulate by comparing results with other research. Specify a limited population to which to generalize. Specify that the result applies only to the participant studied.

Reversal Baseline Designs

When might it not be appropriate? -when reversing a treatment might be harmful -when the treatment is unlikely to have a short term impact

How does external validity play into factorial designs?

When researchers test an independent variable in more than one group at once, they are testing whether the effect generalizes.

How do factorial experiments differ from simple experiments?

When researchers want to test for interactions, they do so with factorial designs. A factorial design is one in which there are two or more independent variables (also referred to as factors). In the most common factorial design, researchers cross the two independent variables; that is, they study each possible combination of the independent variables. EX:They used two independent variables (cell phone use and driver age), creating a condition representing each possible combination of the two.

Non equivalent control group design (Independent-Groups Quasi-Experiments)

a quasi-experimental study that has at least one treatment group and one comparison group, but participants have not been randomly assigned to the two groups

What are meta-analyses and why are they important?

a way of mathematically averaging the results of all the studies (both published and unpublished) that have tested the same variables to see what conclusion that whole body of evidence supports. -In a meta-analysis, researchers collect all possible examples of a particular kind of study. They then average all the effect sizes to find an overall effect size. Using meta-analysis, researchers can also sort a group of studies into categories (i.e., moderators), computing sepa- rate effect size averages for each category. From these follow-up analyses, researchers can detect new patterns in the literature as well as test new questions.

What is power and what factors affect the strength of the power that we have?

an aspect of statistical validity, is the likelihood that a study will return a statistically significant result when the independent variable really has an effect. the easiest way to increase power is to add more participants.Studies with low power can find only large effects. Studies with high power can find both large and small effects

autocorrelations

because they determine the correlation of one variable with itself, measured on two different occasions. -For example, the Brummelman team asked whether mothers' overvaluation at Time 1 was associated with mothers' overvaluation at Time 2, Time 3, and so on; they also asked whether children's narcissism at Time 1 was associated with their scores at Time 2, Time 3, and so on. -temporal and covariance not internal

remember that correlation does not equal causation and this is because...

correlation does not help us to 1.) determine temporal precedence; 2.) covariance of cause and effect; and 3) whether we have a third variable problem (can't eliminate alternative explanations)

cross-lag correlations

cross-sectional correlations and autocorrelations are gen- erally not the researchers' primary interest. They are usually most interested in cross-lag correlations, which show whether the earlier measure of one variable is associated with the later measure of the other variable. Cross-lag correlations thus address the directionality problem and help establish temporal precedence. and the begging of internal validity

bivariate only association claims

not causal

Be able to contrast a pretest-posttest design with a posttest only design

posttest-only design dependent-groups experimental designs-participants are randomly assigned to independent variable groups and are tested on the dependent variable once pretest/posttest design- participants are randomly assigned to at least two different groups and are tested on the key dependent variable twice—once before and once after exposure to the independent variable

ceiling effect

refers to the point at which an independent variable no longer has an effect on a dependent variable,

restricting range

restrict the correlation coefficient

Be able to identify the strength of the correlation by examining the correlation coefficient. Remember that strength has nothing to do with the sign of the coefficient.

strength refers to how closely related the two variables are—how close r is to 1 or −1

When and why is it important? (external Validity)

the degree to which a study's results are generalizable, to other par- ticipants and other settings. The more settings and populations in which a study is conducted, the better you can assess the generalizability of the findings. . External validity comes from how the sampleis obtained, rather than sample size.

internal validity

the degree to which changes in the dependent variable are due to the manipulation of the independent variable

statistical validity

the degree to which test scores are interpreted correctly and used appropriately

construct validity

the extent to which variables measure what they are supposed to measure

What is a statistically significant main effect? How is it determined? Be able to determine whether main effects are present given some data.

the overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable. In other words, a main effect is a simple difference. In a factorial design with two independent variables, there are two main effects.

association claims important validities

the two most important validities to interrogate are construct validity and statistical validity. You might also ask about the exter- nal validity of the association. Although internal validity is relevant for causal claims, not association claims, you need to be able to explain why correlational studies do not establish internal validity

In experiments where null effects/findings are discovered, what can explain this? Weak manipulation of the IV. Insensitive measurement of the DV (ceiling and floor effects). Keep in mind that both of these can be determined in a pilot study.

there is no significant covariance between the two? That outcome is known as a null effect, also referred to as a null result.

What are small-N designs? What are their strengths and weaknesses?

they obtain a lot of information from just a few cases. They may even restrict their study to a single animal or one person, using a single-N design. -Each participant is treated separately. Small-N designs are almost always repeated-measures designs, in which researchers observe how the person or animal responds to several systematically designed conditions. -Data for each individual are presented. -small-N designs can eliminate alternative explanations, thereby enhancing internal validity.

In within-groups designs, you should be familiar with order effects (practice/fatigue & carryover effects) and how they're dealt with - counterbalancing. What results from counterbalancing would let you know there is a true effect of the IV versus an order effect?

they present the levels of the independent variable to participants in different sequences. With counterbalancing, any order effects should cancel each other out when all the data are collected.

-cross-sectional correlations;

they test to see whether two variables, measured at the same point in time, are correlated. For example, the study reports that the correlation between mothers' overvaluation at Time 4 and children's narcissism at Time 4 was r = .099. This is a weak cor- relation, but consistent with the hypothesis. However, because both variables in a cross-sectional correlation were measured at the same time, this result alone cannot establish temporal precedence. Either one of these variables might have led to changes in the other.

Mediators vs. Moderators

•Mediators ask "why," and moderators ask "for whom" or "when." -moderators they tell us when we have a strong correlation between two variables and when we have a weak one. Let's consider a study on the correlation between professional sports games attendance and the success of the team. -mediators- -The word moderate can mean "to change," and a moderating vari- able can change the relationship between the other two variables

What are the advantages of quasi-experiments? What are their disadvantages?

•Real-world opportunities •External validity •Ethics •Construct validity and statistical validity in quasi-experiments