Research methods exam 3
What can an outlier affect
mean and correlation, it can make a correlation appear stronger or weaker depending on its placement, especially for smaller sample sizes
Cross-sectional correlations
only looking at correlations for one time they test to see whether two variables, measured at the same point in time, are correlated
What happens if one of the variables in a cross lag correlation is not not significant
it then cannot show causation, because they are mutually exclusize
beta
used to test for third variables
Independent variables answer the question
"compared to what"
Predictor variables
independent variables
Dependent variable
measured, outcome variable
confound
third variable :p
Phrases that indicate regression in popular press articles
• "Controlled for" • "Taking into account" • "Correcting for" • "Adjusting for"
Questions to ask when assessing for construct validity
• Ask about the construct validity of each variable. • How well was each of the variables measured?• Does the measure have good reliability?• Is it measuring what it's intended to measure?
Mediators vs third variables--similarities
• Both involve multivariate research designs. • Both can be detected using multiple regression
Why not just do an experiment to determine temporal precedence
• In many cases participants cannot be randomly assigned to a variable. • Cannot be assigned to preferences • Unethical to assign participants ex: looking at narcissim in children when being praised, it is unethical to make a child narcissistic.
Pattern, Parsimony, in the Popular Press-- how do journalists present these ideas
• Journalists do not always fairly represent pattern and parsimony. • When journalists report only one study at a time, they are selectively presenting only part of the scientific process -always look at many different studies for a claim to get the whole picture
Mediators vs third variables-- differences
• Third variables are external to the bivariate correlation(problematic). • Mediators are internal to the causal variable (notproblematic)
Questions to ask to assess for statistical validity?
• What is the effect size? • Is the correlation statistically significant? • Could outliers be affecting the association?• Is there restriction of range? • Is the association curvilinear?
Placebo group
(placebo control group) when the control group is exposed to an inert treatment such as a sugar pill in an attempt to eliminate mental affects
Multivariate designs
(such as longitudinal and multiple regression designs) involve more than two measured variables.
mediators vs moderators
- mediators ask "WHY" (why a variable is changing, what is causing this change) and moderators ask "For whom" or "when"--(what group/type of population is chaning the relationship ex:gender,race)
advantages of within subjects design
1. Participants in your groups are equivalent because they are the same participants and serve as their own controls. 2. These designs give researchers more power to notice differences between conditions. 3. Within-groups designs require fewer participants than other designs. (because each person is exposed to every variable)
Distadvantages of within groups designs
1. Potential for order effects 2. Might not be practical or possible 3. Experiencing all levels of the independent variable (IV) changes the way participants act (demand characteristics)
a statistically significant p value
<.05
Restriction of range
A situation involving a bivariate correlation, in which there is not a full range of possible scores on one of the variables in the association, so the relationship from the sample underestimates (makes correlation seem smaller) the true correlation.
Multiple regression
A statistical technique that computes the relationship between a predictor variable and a criterion variable, controlling for other predictor variables, thereby addressing some internal validity concerns.
Outlier
A value much greater or much less than the others in a data set--
mediators (with example)
A variable that helps explain the relationship between two other variables. Also called mediating variable ex: amount of deep talk and one's welbeing may have a mediator of quality of social ties
Internal validity
Can we make a causal inference from association?
Three causal criteria for internal validity (list)
Covariance • Temporal precedence (the directionality problem) • Internal validity (third-variable problem)
For this claim, list the criterion variables and predictor variables At year 25, the participants took three well-known cognitive tests: the DigitSymbol Substitution Test, which measures speed of mental processing; the Stroop Test, which measures processing speed and attention; and the Rey Auditory Verbal Learning Test, which assesses how well we remember information delivered orally.Participants who showed a long-term pattern of high TV viewing scored significantly lower, on average, on the cognitive tests...This held true after taking into consideration a range of factors, including age, race, education, alcohol use, and body mass index.
Criterion variables: digit symbol sub test, stroop test, rey authority test predictor variables: amount of TV,age, race, education, alcohol use, BMI
do cross lag correlations show temporal precedence
Cross-lag correlations thus address the directionality problem and help establish temporal precedence
Internal validity (third-variable problem)
Is there a third variable(C) that is associated with variables A and B independently? If so,then we can't infer causation.
Repeated measures design
DV is measured more than once after each participant was exposed to the DV
Getting at Causality with Pattern and Parsimony
Ex: cigarettes and lung cancer. Cannot straight up say that cigarrette usage causes lung cancer because companies won't like that. However, you can use parsimony to show that multiple claims that show how cigarette use is bad for health all tie together to show that it increases liklihood of lung cancer. Utilizing many claims that all have the same underlying variable of the toxicity of cigarettes prevents alternative claims of a third variable and come to a parsimonious conclusion
"control for"
Holding a potential third variable at a constant level (statistically or experimentally) while investigating the association between two other variables.
Statistical validity:
How well do the data support the conclusion?
Construct validity
How well was each variable measured?
Three criteria for causation for longitudinal studies
Longitudinal designs can provide SOME evidence for causation by fulfilling three criteria • Covariance • Temporal precedence • Internal validity
Manipulated vs. measured variables
Manipulated- independent Measured-dependent
Temporal precedence (the directionality problem):
The causal variable (A) must come before the effect variable (B).
parsimony
The degree to which a theory provides the simplest explanation of some phenomenon. In the context of investigating a claim, the simplest explanation of a pattern of data; the best explanation that requires making the fewest exceptions or qualifications.
effect size, sample size, and significance
The larger the sample size, can have a larger effect size that is more likely to be significant. The smaller the sample size, the more likely it is to be effected by outliers and a small effect size is not significant. In a large sample, a small effect size can be significant
Describe the value of pattern and parsimony, in which a variety of research results support a single, parsimonious causal theory.
The value of this allows for individuals to refute the argument of a third variable causing a relationship. For instance, with cigarette smoking causing lung cancer, some people lay say a third variable could be coffee consumption. Although valid, parsimonious statements allow for multiple, similar experiments to prove the same general point, allowing for little room for debate for a causal theory
Covariance
There must be an association between the cause variable (A) and the effect variable (B).
External validity:
To whom can the association be generalized?
order effects
When being exposed to one condition affects how participants respond to other conditions
moderator
When the relationship between two variables changes depending on the level of another variable, that other variable is called a moderator. (third variable)
When can restriction of range apply
When, for any reason, one of the variables has very little variance. For example, if researchers were testing the correlation between parental income and child school achievement, they would want to have a sample of parents that included all levels of income. If their sample of parents was entirely upper middle class, there would be restriction of range on parental income, and researchers would underestimate any true correlation.
exception when a small effect size is more important than a large effect size
a clinical study about aspirin's effect on heart attacks, found a small effect size. However, even though it was small, it was 8 fewer heart attacks, saving 8 lives.-->life or death situations allow effect sizes to be smaller
parsimonious conclusion
a conclusion come to from relationships between similar variables and experiments being drawn to form an overlying conclusion Ex: longitudinal, different ages, experimental, survey, etc studies coming to the conclusion that watching violence on TV makes one more agressive
Explain the function of a mediating variable.
a mediating variable can explain the process in which two variables are connected. They are not problematic, they just bring insight into why variables can be connected. Furthermore, you can establish temporal precedence, which can let you find causation. It is of direct interest of the researchers, rather than a third variable which distracts from a relationship. It adds depth to the relationship
pilot study
a small study before conducting your actual study with more participants
t test
a statistic to test the difference between two group averages
design confound
a variable that varies systematically with the IV and is therefore an alternative explanation--ex: people with a bigger bowl get penne pasta and people with a smaller bowl get bowtie, this may give a problem in the fsining
manipulation check
additional measure to assess how the participants perceived and interpreted the manipulation and/or to assess the direct effect of the manipulation and see if it worked
Independent groups design aka between-subjects design or between-groups design
an experimental design in which different groups of participants are exposed to different levels of the independent variable, such that each participant experiences only one level of the independent variable
Within-groups design (aka within-subjects design)
an experimental design in which each participant is presented with all levels of the independent variable
Matched groups
an experimental design technique in which participants who are similar on some measured variable are grouped into sets; the members of each matched set are then randomly assigned to different experimental conditions Ex: putting people in groups based on GPA and putting one person of each GPA group in a group.
practice effect
any change or improvement that results from practice or repetition of task items or activities
carryover effects
any lingering effects of a previous experimental condition that are affecting a current experimental condition. ex: caffeine right before an exam and taking an exam a couple hours later, the caffiene effect may carry over
Control variable
any variable that an experimenter holds constant
mean
arithmetic average
moderators ask...
ask "for whom" or "when"-- for a category of individuals/things ex: gender (male/female/etc) or type of relationship (close friends/strangers)
Bivariate correlations:
associations that involve exactly two variables
What type of variable are level of happiness and days spent on vacation?
both quantitiative
Regression does not establish
causation--Multiple regression is not a foolproof way to rule out all kinds of third variables.
temporal precedence
cause comes before effect
Solution for order effects
counterbalencing the order in which they recieve each IV EX: having the people eat the chocolate with the confederate and having some eating them alone
Comparison group
comparison condition, allows for the effect of a variable to be compared to a different group to make the effect more clear
Three criteria for causation
covariance, temporal precedence, internal validity
demand characteristics
cues in an experiment that tell the participant what behavior is expected-- happens in within groups designs because when exposed to multiple levels of the variable, you may be able to conclude what the study is about
Criterion variables
dependent variables
When is it good to use a pretest/post test study
depends on the situation! If you want to determine any differences in participants before participating in the study. IF you need to control for a certian type of individual participating (ex: wanting people interested in stem)Ex: getting a baseline for a student's GRE performance and then having them go through a mindfulness course/nutrition course and assessing after. If random assignment, there shouldn't be differences between the individuals assigned to each group.
Effect size
describes the strength of an association
When wouldn't you want to do a pre test/post test study
ex: pasta study. YOu wouldn't want people to eat before because they would be they wouldn't eat muich for the actual experiment. No need to do a pre test/post test if participation in the experiment will have an effect on results.
Treatment group(s)
experimental group, given the variable. One or more treatment conditions
internal validity
extent to which we can draw cause-and-effect inferences from a study
external validity
extent to which we can generalize findings to real-world settings
How to accurately describe associations with categorial data
find the mean value of each category and create a bar graph of each group, with one score value (the mean) for each category. Compare the means here to see association
statistical validity
how strong is the association? is the study statistically significant?
Selection effect
if participants in one condition are systematically different from the other participants, it is problematic. Ex: if each person can chose what condition they want to be in
unsystematic variability
in an experiment, when levels of a variable fluctuate independently of experimental group membership, contributing to variability within groups. You have certain types of people in one condition but also int he other condition. Ex: people with the smaller or large bowl can have spagetti or penne
Curvilinear association
in which the relationship between two variables is not a straight line; it might be positive up to a point and then become negative.
what is an order effect considered
is considered a design confound
Describing associations with categorical data, why isnt it great
isn't the greatest way to illustrate an association. Shows the values for each category, but doesn't really show a trend in data
when r= +/- .50 the effect size is
large, or strong
When everything is equal, a _______ effect size is more important than a _______ effect size
large, small
effect size trend
larger effect size, increase strength of association, give more accurate predicitons
conditions
level of an independent variable
Examples of bivariate correlation
level of happiness and days spent on vacation
Explain how longitudinal correlational designs can establish temporal precedence
longitudinal correlational designs can establish temporal precedence because temporal precedence is stating that A causes B. Longitudinal studies can prove this because it is can record the same variable at two different times, showing that one causes the other because one comes before the other.
How to interpret a regression table
look and see which variables are being controlled for in the table, they are all considered when considering each beta value. Beta values are dependent on each variable present in the table
independent variable
manipulated
For this claim and study, list the Criterion, predictor, and mediator "Active sex life may lead to improved job satisfaction and engagement in work"To understand the impact of sex on work, the researchers followed 159married employees over the course of two weeks, asking them to complete two brief surveys each day. They found that employees who engaged in sex reported more positive moods the next day, and the elevated mood levels in the morning led to more sustained work engagement and job satisfaction throughout the workday
mediator: positive mood criterion variables: work engagement, job satisfaction predictor variables: sexual intercourse
when r=+/- .30 the effect size is
medium or moderate
Example of a non linear relationship
money and happiness, it starts as a positive correlation, but then changes
effect of nature of single regression table on beta value
more criterion variables can change the beta value because you are accounting for so many varaibles
Explain how multiple-regression analyses help address internal validity (the third-variable problem)
multiple-regression analyses help address internal validity by assessing for more than two variables and looking at their interrelations (specifically variables that they think could have an effect) to determine what is actually causing the effect on the dependent variable. They asses these variables using beta values, which are dependent on the variables being observed to see which variable has the largest effect.
Can autocorrelation establish temporal precedence
no
can cross secitonal correlation establish temporal precedence?
no
Control group
no treatment
concurrent-measures design
participants are exposed to all the levels of an independent variable at roughly the same time. Ex; does one prefer a female face or a male face more, and a person is exposed to both faces at the same time and asked to chose their preference
Example of beta not signifigant: Predictor variable: family meal frequency• Criterion variable: academic success
possibility of parental involvement being a third variable-- if relationship is still significant after controlling for third variable, you can say relationship between family meal frequency and academic success is likely If when controlling for parental involvement the relationship is not statistically significant, you can say parental involvement causes the effect--non significant beta!
types of order effects
practice and carry over effects
How to avoid selection effects
random assignments and matched groups
random assignment
randomly assigning participants to a variable to avoid selection effects,
Ways to describe a nonsignificant beta
relationship between variables can be explained by third variable relationship is not significant when controlling for third variable relationship between variables goes away when third variable is held constant
Similarities and differences between beta and r
similarities: both tell us direction of relationship and strength of relationship, Difference: beta has no cut off for effect size
when r= +/-.10, the effect size is
small or weak
Purpose of a placebo group
so one could know that the changes are not due to just being in the study rather than just receiving the ingredients
systematic variability
something else is different about the groups, not just the IV
what if beta is not significant?
suggests third variable is explaining relationship between variables
Which is more of a problem-- systematic or unsystematic variabilty
systematic
Longitudinal Designs
testing the same sample now and at a different time and seeing the trend in data. can provide evidence for temporal precedence by measuring the same variables in the same people at several points in time
Power
the ability of a study to show statistically significant results
construct validity
the extent to which variables measure what they are supposed to measure
Why are simple bivariate correlations are not sufficient for establishing causation.
they are not sufficient because it only examines the relationship between two variables, does not take into consideration a third variable that could be affecting it. Furthermore, they dont always have temporal precedence which is needed for causation
autocorrelations
they determine the correlation of one variable with itself, measured on two different occasions
Example of curvilinear association
use of healthcare system and age in years
The Pearson correlation
used to measure the strength of a linear association between two variables.
Cross lag correlations
which show whether the earlier measure of one variable is associated with the later measure of the other variable.
When may random assignment not work?
with a small sample size
What can you compare beta strengths with
within a single regression table
