Research methods exam 3

What can an outlier affect

mean and correlation, it can make a correlation appear stronger or weaker depending on its placement, especially for smaller sample sizes

Cross-sectional correlations

only looking at correlations for one time they test to see whether two variables, measured at the same point in time, are correlated

What happens if one of the variables in a cross lag correlation is not not significant

it then cannot show causation, because they are mutually exclusize

beta

used to test for third variables

Independent variables answer the question

"compared to what"

Predictor variables

independent variables

Dependent variable

measured, outcome variable

confound

third variable :p

Phrases that indicate regression in popular press articles

• "Controlled for" • "Taking into account" • "Correcting for" • "Adjusting for"

Questions to ask when assessing for construct validity

• Ask about the construct validity of each variable. • How well was each of the variables measured?• Does the measure have good reliability?• Is it measuring what it's intended to measure?

Mediators vs third variables--similarities

• Both involve multivariate research designs. • Both can be detected using multiple regression

Why not just do an experiment to determine temporal precedence

• In many cases participants cannot be randomly assigned to a variable. • Cannot be assigned to preferences • Unethical to assign participants ex: looking at narcissim in children when being praised, it is unethical to make a child narcissistic.

Pattern, Parsimony, in the Popular Press-- how do journalists present these ideas

• Journalists do not always fairly represent pattern and parsimony. • When journalists report only one study at a time, they are selectively presenting only part of the scientific process -always look at many different studies for a claim to get the whole picture

Mediators vs third variables-- differences

• Third variables are external to the bivariate correlation(problematic). • Mediators are internal to the causal variable (notproblematic)

Questions to ask to assess for statistical validity?

• What is the effect size? • Is the correlation statistically significant? • Could outliers be affecting the association?• Is there restriction of range? • Is the association curvilinear?

Placebo group

(placebo control group) when the control group is exposed to an inert treatment such as a sugar pill in an attempt to eliminate mental affects

Multivariate designs

(such as longitudinal and multiple regression designs) involve more than two measured variables.

mediators vs moderators

- mediators ask "WHY" (why a variable is changing, what is causing this change) and moderators ask "For whom" or "when"--(what group/type of population is chaning the relationship ex:gender,race)

advantages of within subjects design

1. Participants in your groups are equivalent because they are the same participants and serve as their own controls. 2. These designs give researchers more power to notice differences between conditions. 3. Within-groups designs require fewer participants than other designs. (because each person is exposed to every variable)

Distadvantages of within groups designs

1. Potential for order effects 2. Might not be practical or possible 3. Experiencing all levels of the independent variable (IV) changes the way participants act (demand characteristics)

a statistically significant p value

<.05

Restriction of range

A situation involving a bivariate correlation, in which there is not a full range of possible scores on one of the variables in the association, so the relationship from the sample underestimates (makes correlation seem smaller) the true correlation.

Multiple regression

A statistical technique that computes the relationship between a predictor variable and a criterion variable, controlling for other predictor variables, thereby addressing some internal validity concerns.

Outlier

A value much greater or much less than the others in a data set--

mediators (with example)

A variable that helps explain the relationship between two other variables. Also called mediating variable ex: amount of deep talk and one's welbeing may have a mediator of quality of social ties

Internal validity

Can we make a causal inference from association?

Three causal criteria for internal validity (list)

Covariance • Temporal precedence (the directionality problem) • Internal validity (third-variable problem)

For this claim, list the criterion variables and predictor variables At year 25, the participants took three well-known cognitive tests: the DigitSymbol Substitution Test, which measures speed of mental processing; the Stroop Test, which measures processing speed and attention; and the Rey Auditory Verbal Learning Test, which assesses how well we remember information delivered orally.Participants who showed a long-term pattern of high TV viewing scored significantly lower, on average, on the cognitive tests...This held true after taking into consideration a range of factors, including age, race, education, alcohol use, and body mass index.

Criterion variables: digit symbol sub test, stroop test, rey authority test predictor variables: amount of TV,age, race, education, alcohol use, BMI

do cross lag correlations show temporal precedence

Cross-lag correlations thus address the directionality problem and help establish temporal precedence

Internal validity (third-variable problem)

Is there a third variable(C) that is associated with variables A and B independently? If so,then we can't infer causation.

Repeated measures design

DV is measured more than once after each participant was exposed to the DV

Getting at Causality with Pattern and Parsimony

Ex: cigarettes and lung cancer. Cannot straight up say that cigarrette usage causes lung cancer because companies won't like that. However, you can use parsimony to show that multiple claims that show how cigarette use is bad for health all tie together to show that it increases liklihood of lung cancer. Utilizing many claims that all have the same underlying variable of the toxicity of cigarettes prevents alternative claims of a third variable and come to a parsimonious conclusion

"control for"

Holding a potential third variable at a constant level (statistically or experimentally) while investigating the association between two other variables.

Statistical validity:

How well do the data support the conclusion?

Construct validity

How well was each variable measured?

Three criteria for causation for longitudinal studies

Longitudinal designs can provide SOME evidence for causation by fulfilling three criteria • Covariance • Temporal precedence • Internal validity

Manipulated vs. measured variables

Manipulated- independent Measured-dependent

Temporal precedence (the directionality problem):

The causal variable (A) must come before the effect variable (B).

parsimony

The degree to which a theory provides the simplest explanation of some phenomenon. In the context of investigating a claim, the simplest explanation of a pattern of data; the best explanation that requires making the fewest exceptions or qualifications.

effect size, sample size, and significance

The larger the sample size, can have a larger effect size that is more likely to be significant. The smaller the sample size, the more likely it is to be effected by outliers and a small effect size is not significant. In a large sample, a small effect size can be significant

Describe the value of pattern and parsimony, in which a variety of research results support a single, parsimonious causal theory.

The value of this allows for individuals to refute the argument of a third variable causing a relationship. For instance, with cigarette smoking causing lung cancer, some people lay say a third variable could be coffee consumption. Although valid, parsimonious statements allow for multiple, similar experiments to prove the same general point, allowing for little room for debate for a causal theory

Covariance

There must be an association between the cause variable (A) and the effect variable (B).

External validity:

To whom can the association be generalized?

order effects

When being exposed to one condition affects how participants respond to other conditions

moderator

When the relationship between two variables changes depending on the level of another variable, that other variable is called a moderator. (third variable)

When can restriction of range apply

When, for any reason, one of the variables has very little variance. For example, if researchers were testing the correlation between parental income and child school achievement, they would want to have a sample of parents that included all levels of income. If their sample of parents was entirely upper middle class, there would be restriction of range on parental income, and researchers would underestimate any true correlation.

exception when a small effect size is more important than a large effect size

a clinical study about aspirin's effect on heart attacks, found a small effect size. However, even though it was small, it was 8 fewer heart attacks, saving 8 lives.-->life or death situations allow effect sizes to be smaller

parsimonious conclusion

a conclusion come to from relationships between similar variables and experiments being drawn to form an overlying conclusion Ex: longitudinal, different ages, experimental, survey, etc studies coming to the conclusion that watching violence on TV makes one more agressive

Explain the function of a mediating variable.

a mediating variable can explain the process in which two variables are connected. They are not problematic, they just bring insight into why variables can be connected. Furthermore, you can establish temporal precedence, which can let you find causation. It is of direct interest of the researchers, rather than a third variable which distracts from a relationship. It adds depth to the relationship

pilot study

a small study before conducting your actual study with more participants

t test

a statistic to test the difference between two group averages

design confound

a variable that varies systematically with the IV and is therefore an alternative explanation--ex: people with a bigger bowl get penne pasta and people with a smaller bowl get bowtie, this may give a problem in the fsining

manipulation check

additional measure to assess how the participants perceived and interpreted the manipulation and/or to assess the direct effect of the manipulation and see if it worked

Independent groups design aka between-subjects design or between-groups design

an experimental design in which different groups of participants are exposed to different levels of the independent variable, such that each participant experiences only one level of the independent variable

Within-groups design (aka within-subjects design)

an experimental design in which each participant is presented with all levels of the independent variable

Matched groups

an experimental design technique in which participants who are similar on some measured variable are grouped into sets; the members of each matched set are then randomly assigned to different experimental conditions Ex: putting people in groups based on GPA and putting one person of each GPA group in a group.

practice effect

any change or improvement that results from practice or repetition of task items or activities

carryover effects

any lingering effects of a previous experimental condition that are affecting a current experimental condition. ex: caffeine right before an exam and taking an exam a couple hours later, the caffiene effect may carry over

Control variable

any variable that an experimenter holds constant

mean

arithmetic average

moderators ask...

ask "for whom" or "when"-- for a category of individuals/things ex: gender (male/female/etc) or type of relationship (close friends/strangers)

Bivariate correlations:

associations that involve exactly two variables

What type of variable are level of happiness and days spent on vacation?

both quantitiative

Regression does not establish

causation--Multiple regression is not a foolproof way to rule out all kinds of third variables.

temporal precedence

cause comes before effect

Solution for order effects

counterbalencing the order in which they recieve each IV EX: having the people eat the chocolate with the confederate and having some eating them alone

Comparison group

comparison condition, allows for the effect of a variable to be compared to a different group to make the effect more clear

Three criteria for causation

covariance, temporal precedence, internal validity

demand characteristics

cues in an experiment that tell the participant what behavior is expected-- happens in within groups designs because when exposed to multiple levels of the variable, you may be able to conclude what the study is about

Criterion variables

dependent variables

When is it good to use a pretest/post test study

depends on the situation! If you want to determine any differences in participants before participating in the study. IF you need to control for a certian type of individual participating (ex: wanting people interested in stem)Ex: getting a baseline for a student's GRE performance and then having them go through a mindfulness course/nutrition course and assessing after. If random assignment, there shouldn't be differences between the individuals assigned to each group.

Effect size

describes the strength of an association

When wouldn't you want to do a pre test/post test study

ex: pasta study. YOu wouldn't want people to eat before because they would be they wouldn't eat muich for the actual experiment. No need to do a pre test/post test if participation in the experiment will have an effect on results.

Treatment group(s)

experimental group, given the variable. One or more treatment conditions

internal validity

extent to which we can draw cause-and-effect inferences from a study

external validity

extent to which we can generalize findings to real-world settings

How to accurately describe associations with categorial data

find the mean value of each category and create a bar graph of each group, with one score value (the mean) for each category. Compare the means here to see association

statistical validity

how strong is the association? is the study statistically significant?

Selection effect

if participants in one condition are systematically different from the other participants, it is problematic. Ex: if each person can chose what condition they want to be in

unsystematic variability

in an experiment, when levels of a variable fluctuate independently of experimental group membership, contributing to variability within groups. You have certain types of people in one condition but also int he other condition. Ex: people with the smaller or large bowl can have spagetti or penne

Curvilinear association

in which the relationship between two variables is not a straight line; it might be positive up to a point and then become negative.

what is an order effect considered

is considered a design confound

Describing associations with categorical data, why isnt it great

isn't the greatest way to illustrate an association. Shows the values for each category, but doesn't really show a trend in data

when r= +/- .50 the effect size is

large, or strong

When everything is equal, a _______ effect size is more important than a _______ effect size

large, small

effect size trend

larger effect size, increase strength of association, give more accurate predicitons

conditions

level of an independent variable

Examples of bivariate correlation

level of happiness and days spent on vacation

Explain how longitudinal correlational designs can establish temporal precedence

longitudinal correlational designs can establish temporal precedence because temporal precedence is stating that A causes B. Longitudinal studies can prove this because it is can record the same variable at two different times, showing that one causes the other because one comes before the other.

How to interpret a regression table

look and see which variables are being controlled for in the table, they are all considered when considering each beta value. Beta values are dependent on each variable present in the table

independent variable

manipulated

For this claim and study, list the Criterion, predictor, and mediator "Active sex life may lead to improved job satisfaction and engagement in work"To understand the impact of sex on work, the researchers followed 159married employees over the course of two weeks, asking them to complete two brief surveys each day. They found that employees who engaged in sex reported more positive moods the next day, and the elevated mood levels in the morning led to more sustained work engagement and job satisfaction throughout the workday

mediator: positive mood criterion variables: work engagement, job satisfaction predictor variables: sexual intercourse

when r=+/- .30 the effect size is

medium or moderate

Example of a non linear relationship

money and happiness, it starts as a positive correlation, but then changes

effect of nature of single regression table on beta value

more criterion variables can change the beta value because you are accounting for so many varaibles

Explain how multiple-regression analyses help address internal validity (the third-variable problem)

multiple-regression analyses help address internal validity by assessing for more than two variables and looking at their interrelations (specifically variables that they think could have an effect) to determine what is actually causing the effect on the dependent variable. They asses these variables using beta values, which are dependent on the variables being observed to see which variable has the largest effect.

Can autocorrelation establish temporal precedence

no

can cross secitonal correlation establish temporal precedence?

no

Control group

no treatment

concurrent-measures design

participants are exposed to all the levels of an independent variable at roughly the same time. Ex; does one prefer a female face or a male face more, and a person is exposed to both faces at the same time and asked to chose their preference

Example of beta not signifigant: Predictor variable: family meal frequency• Criterion variable: academic success

possibility of parental involvement being a third variable-- if relationship is still significant after controlling for third variable, you can say relationship between family meal frequency and academic success is likely If when controlling for parental involvement the relationship is not statistically significant, you can say parental involvement causes the effect--non significant beta!

types of order effects

practice and carry over effects

How to avoid selection effects

random assignments and matched groups

random assignment

randomly assigning participants to a variable to avoid selection effects,

Ways to describe a nonsignificant beta

relationship between variables can be explained by third variable relationship is not significant when controlling for third variable relationship between variables goes away when third variable is held constant

Similarities and differences between beta and r

similarities: both tell us direction of relationship and strength of relationship, Difference: beta has no cut off for effect size

when r= +/-.10, the effect size is

small or weak

Purpose of a placebo group

so one could know that the changes are not due to just being in the study rather than just receiving the ingredients

systematic variability

something else is different about the groups, not just the IV

what if beta is not significant?

suggests third variable is explaining relationship between variables

Which is more of a problem-- systematic or unsystematic variabilty

systematic

Longitudinal Designs

testing the same sample now and at a different time and seeing the trend in data. can provide evidence for temporal precedence by measuring the same variables in the same people at several points in time

Power

the ability of a study to show statistically significant results

construct validity

the extent to which variables measure what they are supposed to measure

Why are simple bivariate correlations are not sufficient for establishing causation.

they are not sufficient because it only examines the relationship between two variables, does not take into consideration a third variable that could be affecting it. Furthermore, they dont always have temporal precedence which is needed for causation

autocorrelations

they determine the correlation of one variable with itself, measured on two different occasions

Example of curvilinear association

use of healthcare system and age in years

The Pearson correlation

used to measure the strength of a linear association between two variables.

Cross lag correlations

which show whether the earlier measure of one variable is associated with the later measure of the other variable.

When may random assignment not work?

with a small sample size

What can you compare beta strengths with

within a single regression table