PSYCH 3010 Bauer - EXAM 3

preregistration

The practice of posting a study's method hypotheses, or statistical analysis publicly, in advance of data collection -Some journals have a section for replication studies --> incentive to engage the important practice or replication while still publishing papers

cell

_____ = condition -In a simple experiment, can represent the level of 1 IV; in a factorial design, it represents 1 of the possible combinations of the 2 IVs

1. ME, 2. ID, 3. ME, 4. ID, 5. SN

(Ch. 11 InQuizitive) Match the solution that would best prevent within-group variability from obscuring group differences. Situation Noise Measurement Error Individual Differences 1. increased number of measurements 2. increased number of participants 3. validated scales 4. within-groups design 5. experimental control

interaction effect

(Ch. 12 InQuizitive) Typically, the most important effect that is uncovered in a factorial design is a(n) __________ effect

1. large, 2. large, 3. small, 4. small

(Ch. 13 InQuizitive) Determine whether each example is a small-N or large-N design (answer large or small) 1. The performance of 15 early-decision college students is compared to 15 typical students 2. In 25 trials, 40 participants complete each part of a go/no-go task designed to measure decision making 3. Researchers have a patient with anterograde amnesia experience 50 trials of a memory task 4. Five patients with dissociative identity disorder are each given different treatments, and then treatment is taken away to see how effective it is

1. false, 2. true, 3. false, 4. true

(Ch. 13 InQuizitive) True or False 1. Researchers should not attempt to make causal claims with quasi-experiments 2. Quasi-experiments allow scientists to study real-world phenomena in real time 3. Correlational studies and quasi-experiments are the same thing 4. Many quasi-experiments would be unethical if studied as true experiments

file drawer problem

(meta-analysis) A problem based on published literature, which might overestimate the support for a theory because studies finding null effects are less likely to be published than studies finding significant results, and are thus less likely to be included in such reviews

covariance, temporal precedence, internal validity

3 criteria for establishing causation to experiments (support causal claims)

2x2 design

(interpreting the results) -Is there a significant main effect of A? -Is there a significant main effect of B? -Is there a significant interaction?

control variable

A potential 3rd variable that an experimenter holds constant

demand characteristics

(12 internal validity threats) Cues that lead participants to guess a study's hypotheses or goals and change their behavior in the expected direction -Prevention/what can be done: double-blind study, masked design (aka blind study)

1. true (occasionally participants could change their behavior based on seeing the dependent variable twice, so a posttest-only design can prevent that), 2. false (a pretest/posttest design can track change by measuring the dependent variable before manipulation), 3. false (either design can be superior depending on the circumstances), 4. true (here pretesting will show if any group had a concentrated level of a different variable)

(Ch. 10 InQuizitive) True or False 1. Posttest-only designs are useful if a participant's behavior needs to be spontaneous 2. Posttest-only design helps track how participants change over the course of the experiment 3. Pretest/posttest designs are always superior to posttest-only designs, as they give a baseline for the dependent variable 4. Pretest/posttest designs can help prevent selection effects:

testing threat (Because there was no comparison group, it is difficult to say if the children improved due to the intervention or simply became better at the spatial reasoning task over time.)

(Ch. 11 InQuizitive) Candice has created an intervention to see if children using their imaginations increases their spatial reasoning abilities. She randomly recruits kindergarteners from city schools and pretests them with a spatial reasoning task. After a week of her intervention training she measures the children's reasoning again using the same survey. All the children scored better on the assessment the second time compared to the first. What threat to internal validity may be present? testing threat, selection-attrition threat, instrumentation threat, selection-history threat

type II error

A "miss" in the statistical inference process, in which researcher conclude that their study has not detected an effect in a population, when there really is one

ABA design

Also known as Withdrawal of Treatment Design -Baseline -Intervention -Withdrawal of treatment Examine trend -Effect doesn't occur before IV (baseline) -Effect occurs with IV (intervention) -Effect does not occur without IV (reversibility) --If reverse effect --> rule out confounding variables (ex: history, maturation)

within-groups design

An experimental design in which each participant is presented with all levels of the IV -Repeated-measures design -Concurrent-measures design -Allow researchers to treat each participant as his/her own control -Require fewer participants than independent-group designs -Present the potential for order effects and demand characteristics

dependent variable

In an experiment, the variable that's measured to assess the effects of the IV -Outcome variable

western, educated, industrialized, rich, demoratic

WEIRD -Not representative of all the world's people and may not apply to everyone

publication bias

What gets published in scientific journals and which stories are picked up by magazines and news papers. -Most readers are more interested in IVs that matter, than in those that don't --> it's more interesting to learn that dark chocolate has health benefits rather than it doesn't. -Favor differences over similarities

experimental variables

Experimenters manipulate at lease one variable and measure another

multiple-baseline design

A Small-N design in which researchers stagger their introduction of an intervention across a variety of contexts, times, or situations

placebo effect

(12 internal validity threats) A response or effect that occurs when people receiving an experimental treatment experience a change only because they believe they are receiving a valid treatment

maturation

(12 internal validity threats) This threat occurs when an observed change in an experimental group could have emerged more or less spontaneously overtime -Changes in biological & psychological conditions that occur as time passes (development, fatigue, spontaneous remission) -People adapt to their environment -Prevention: comparison group

selection-attrition threat (only members of a particular group dropped out from the study, causing a combination threat.)

(Ch. 11 InQuizitive) Jermaine is curious if the way students take notes affects their academic ability. Jermaine recruits two groups of students from classes on campus: one group of high achievers and one group of low achievers. He then asks both groups to take notes in the exact same way to see how it will affect their scores. During the study, several students in the low-achieving group drop out from the study, as they find the note-taking task difficult. At the end of the study, the results for each group are similar. What threat to internal validity may be present? history threat, selection-history threat, attrition threat, selection-attrition threat

1. experiment, 2. independent, 3. condition, 4. dependent

(Ch. 11 InQuizitive) When researchers manipulate a variable in a study, they are carrying out a(n) __1__. The manipulated variable is often called the __2__ variable. A manipulated variable always has more than one level or __3__. Researchers measure the __4__ variable to determine the effect of the manipulated variable. Control, experiment, correlation, dependent, independent, condition

1. single-N designs, 2. small-N designs, 3. large-N designs

(Ch. 13 InQuizitive) Examples of __1__ are when researchers study a single case extensively in order to extract as much data as possible. The __2__ are similar in that they treat each individual as a separate experiment, but instead of a single case, they use a few participants. And last, __3__ recruit many participants and are concerned with the data of the sample as a whole. single-N designs, large-N designs, small-N designs, quasi-experiments

literature reviews

(scientific literature) summary of all studies on topic in a narrative

independent-groups design

Different participants are exposed to each level of the independent variable. -Posttest-only design -Pretest/Posttest design -Matched groups design

between-groups

Not enough _____________ difference. -Weak manipulation of the IV -Insensitive measures of the DV -Ceiling and floor effects

counterbalancing

(Use to control for order effects) In a repeated-measures experiment, presenting the levels of the IV to participants in different sequences to control for order effects

noise/unsystematic variability

the more unsystematic variability there is within each group, the more the scores in the two groups overlap with each other. The greater the overlap, the less apparent the average difference. As described next, most researchers prefer to keep within-group variability to a minimum, so they can more easily detect between-group differences. They keep in mind a few common culprits: measurement error, irrelevant individual differences, and situation noise.

1. true (variables like personality, intelligence, and age are all held constant when the same person is exposed to all levels of the independent variable), 2. true (because a participant is exposed to all levels of the independent variable, it is not necessary to collect separate participants for each group), 3. false (because of the controls imposed by a within-groups design, this design makes it easier to be certain of differences caused by the independent variable), 4. false (though there are definite benefits to using a within-groups design, there are many circumstances where this design would not be appropriate)

(Ch. 10 InQuizitive) True or False 1. Participants in a within-groups design serve as their own control. 2. A within-groups design usually requires fewer participants than an independent-groups design. 3. It is easier to notice differences between conditions in an independent-groups design. 4. A within-groups design should be used whenever possible, as they are more efficient and require fewer participants.

1. null effect, 2. weak manipulation, 3. insensitive measure, 4. ceiling effect, 5. floor effect

(Ch. 11 InQuizitive) If the researcher concludes that there is no difference in the dependent variable, they are concluding a(n) __1__. This can either be the truth (the variables are not related) or could be due to design flaws in the experiment. A(n) __2__ can occur when the change in the independent variable is not strong enough to affect the dependent variable. The dependent variable might not be responsive enough to detect change from the independent variable; it could be a(n)__3__. Scores from the dependent variable can also spontaneously cluster near the top of possible scores, known as the __4__, or bottom, known as the __5__, which can make covariance undetectable. instrumentation threat, ceiling effect, demand characteristics, placebo effect, insensitive measure, weak manipulation, floor effect, null effect

9 cells (the # of cells in factorial design is a function of each level of each IV multiplied by the # of levels in the other IV)

(Ch. 12 InQuizitive) Read the study and type the value that best answers the question. Tia is creating a factorial experiment in which she is testing three different types of light (red, blue, and green) on three different types of mobile devices (phone, tablet, and gaming console) to see which combination results in the least amount of eye strain for the user. In this design, how many total cells will there be? ____ cells

1. true (an interaction is only possible with more than one independent variable in the same design), 2. true (each cell has a unique combination of levels from each independent variable), 3. false (a participant variable cannot be manipulated, so it is not a true independent variable), 4. false (the number of independent variables is not restricted. However, at a certain point, the analysis becomes cumbersome)

(Ch. 12 InQuizitive) True or False 1. An interaction can be described as a difference in differences 2. A cell acts as a unique condition in factorial designs 3. A participant variable is another type of true independent variable 4. A factorial design cannot have more than three independent variables

1. false, 2. true, 3. true, 4. false

(Ch. 12 InQuizitive) True or False 1. If a design has no interaction, there will be no main effects either 2. Interactions help outline the limits of an effect 3. Adding a participant variable to a design with an existing independent variable is one way to increase external validity 4. If a design has no main effects, there should be no interaction

1. reversal design (the therapists are purposely removing a treatment that was in place, to see what the effects of not having the treatment are), 2. stable-baseline design (the patient's basic habits are observed without interference for a period of time in order to create a stable baseline from which to work), 3. multiple-baseline design

(Ch. 13 InQuizitive) Match each design to the appropriate example: stable-baseline design, multiple-baseline design, reversal design 1) A patient with antisocial personality disorder is introduced to a token economy and then removed from it several weeks later, to see how it affects behavior 2) A therapist studies a patient with OCD for several weeks before initiating treatment 3)A patient with amnesia is repetitively asked to recall facts one week and stories a week later, to see if repetition will help increase memory

within-subjects design (almost every small-N design involves exposing subjects to all levels of an independent variable --> Small-N designs = repeated measures design = within-groups design)

(Ch. 13 InQuizitive) Nearly all small-N experiments are what type of design? between-groups design, mixed design, longitudinal design, within-subjects design

p-hacking

(failing replication study) A family of questionable data analysis techniques, such as adding participants after the results are initially analyzed, looking for outliers, or trying new analyses in order to obtain a p-value of just under .05, which can lead to nonreplicable results

HARKing

(failing replication study) Scientists might craft an after-the-fact hypothesis about a surprising result, making it appear as if they predicted it all along.

insensitive measures

(interrogating a null result) -Some times researchers find a null result because the researchers have not used an operationalization of the DV with enough sensitivity. -When it comes to dependent measures, it's smart to use ones that have detailed quantitative increments--not just two or three levels -Was the DV sensitive enough to measure a difference if there was a difference?

pilot study

A study completed before (or sometimes after) the study of primary interest, usually to test the effectiveness or characteristics of the manipulations

replication

Direct (exact), Conceptual, Replication-Plus-Extension -Replicable: describing a study whose results have been reproduced when the study was repeated --> the same results have actually been reproduced

construct validity

(4 validities) How well were the variables measured and manipulated? -Manipulation check -Pilot study

placebo

(Ch. 10 InQuizitive) A group in which participants are given an inert manipulation is known as a ______ group.

demand characteristic

(Disadvantage of within-groups designs) A cue that leads participants to guess a study's hypotheses or goals; a threat to internal validity -Also: experimental demand

order effect

(Disadvantage of within-groups designs) In a within-groups design, a threat to internal validity in which exposure to one condition changes participant responses to a later on condition. -A confound, meaning that behavior at later levels of the independent variable might be caused not by the experimental manipulation, but rather by the sequence in which the conditions were experienced. -Carryover effect -Practice/fatigue effect

pretest/posttest design

(Independent-groups design) Participants are randomly assigned to one of at lease two levels of an IV, and are then measured on a DV twice--once before and once after they experience the IV

posttest-only design

(Independent-groups design) Participants are randomly assigned to one of at least two levels of an IV and then measured once on the DV

fatigue effect

(Order effect / practice effect) A long sequence might lead participants to get better at the task, or to get tired or bored toward the end.

carryover effect

(Order effect) A type of order effect in which some form of contamination carries over from one condition to the next. -ex: Imagine sipping orange juice right after brushing your teeth; the first taste contaminates your experience to the second one.

repeated-measures

(Quasi-experiments) Interrupted time-series design

individual differences

(noise) Solution 1: -Use a within-groups design, instead of an independent-groups design, to control for irrelevant differences. (can require on 1/2 as many participants) Solution 2: -Add more participants (the more people you measure, the less any single person will have on the group's average. Reduces the influence of _______ WITHIN groups, thereby enhancing the study's ability to detect differences BETWEEN groups)

situational variability (noise)

(noise) Unrelated/irregular events or distractions in the external environment that create unsystematic variability within groups -Solution: control the external environment (lab) --> control surroundings of an experiment

double-blind placebo control study

(placebo effect) A study that uses a treatment group and a placebo group and in which neither the researchers nor the participants know who is in which group

replication-plus-extension

A replication study in which researchers replicate their original study, but add variables/conditions that test additional questions -Introduce a new IV

selection effects

A threat to internal validity that occurs in an independent-groups design when the kinds of participants at one level of the IV are systematically different from those at another level -Avoid with random assignment & matched groups

manipulation check

In an experiment, an extra DV researchers can include to determine how well a manipulation worked

independent variable

In an experiment, the variable that is manipulated -Conditions

replication crisis

Problems with replication attempt, with the originals study, and with publication practices -Open Science Collaboration

three-way interaction

Tests whether two-way interactions are the same at the levels of the 3rd IV -3 IVs -ex: 2x2x2, 2x2x3, 2x3x4 Main effects: -ME for A -ME for B -ME for C Interactions -AxB, AxC, BxC -AxBxC

1. false, 2. false, 3. false, 4. true

(Ch. 10 InQuizitive) True or False? 1. Two studies that reported the same amount of change between two similar conditions should have the same effect size. 2. If an experiment contains a confound, a causal claim can still be made. 3. Replicating an experiment using a larger sample improves external validity. 4. Collecting data through additional studies is another way to improve construct validity.

CV: clothes worn, DV: appeal rating, IV: type of body spray

(Ch. 11 InQuizitive) Darius is testing popular body sprays to see if they actually make people more appealing. He separates participants into three groups and has them all wear the same pants and T-shirt. The first group wears 0.5 ounces of body spray A, the second group wears 0.5 ounces of body spray B, and the third group wears no body spray. Each participant is then brought into a room and two independent coders rate how appealing they are on a 1 to 10 scale. Match each aspect of the experiment to the variable it best represents. Control variable, dependent variable, independent variable Type of body spray, clothes worn, appeal rating

interaction

The effect of one IV depends on the effect of another IV -More important than main effects -Calculate the difference in the differences -Ask: Is there a difference in the differences? -Significant effect: "it depends" -3rd variable obtained

power

The likelihood a study will show a statistically significant result when an IV truly has an effect in the population; the probability of not making a Type II error -Increase by: larger samples (add more participants); accurate measures, strong manipulations, less situation noise

1. observer bias, 2. masked design, 3. double-blind study

(Ch. 11 InQuizitive) The influence of a researcher's own experiences upon how she views outcomes in an experiment is called __1__. This can be prevented by making sure that researchers are unaware of which participants are in which conditions, called a __2__. To further reduce this issue, the conditions can sometimes be unknown to both the experimenter and the participant, called a __3__. observer bias, placebo effect, masked design, double-blind placebo control study, double-blind study

full counterbalancing

(Ch. 10 InQuizitive) Yoshi is testing how being exposed to pictures of different types of foods affects sleepiness. Yoshi is showing each of his participants ten pictures of dessert, ten pictures of entrees, and ten pictures of salads, and he measures sleepiness after each group of ten pictures. To eliminate order effects, Yoshi is presenting all three types of food as well as all ten pictures in each group in a random order to each participant. What technique has Yoshi used to prevent order effects?

1. false (the way the sample is collected is more important than the number collected), 2. true (if a finding replicates in a new setting, it can strengthen the idea that the finding will replicate in other settings), 3. false (how well a sample represents a population is typically referred to as generalizability. Mundane realism refers to how similar an experimental situation is to real life), 4. true (failing to do so can inadvertently call into question the generalizability of the study)

(Ch. 14 InQuizitive) True or False 1. The larger the sample size, the stronger a study's external validity is 2. Conceptual replications can improve external validity 3. Researchers should specify to which population their sample generalizes 4. Mundane realism refers to how well a sample represents a population

control variable (controls are one of the key components of an experiment)

*(Ch. 10 InQuizitive)* In all studies, experimenters keep certain factors constant in order to be sure the independent variable is what is affecting the dependent variable. Though these factors are not true variables, they are often referred to as what type of variables?

two-way design

2 IVs -ex: 2x2, 2x3, 3x3 Main effects: -ME for A -ME for B Interaction -AxB

laboratory setting

A study setting that is high in experimental realism -Priority: INTERNAL validity

cultural psychology

A subdicipline of psychology focusing on how cultural contexts shape the way a person thinks, feels, and behaves -Work in generalization mode

internal validity

Does the study design rule out/control alternative explanations for the results? -A study must ensure that the causal variable, and not other factors, is responsible for the change in the outcome -Control for design confounds, selection effects, and order effects

double-blind study

(demand characteristics) Neither the participants nor the researchers who evaluate them know who is in the treatment group and who is in the comparison group

latin square

A formal system of partial counterbalancing to ensure that every condition in a within-groups design appears in each position at least once

floor effect

An experimental design problem in which IV groups score almost the same on a DV, such that all scores fall at the low end of their possible distribution

posttest only

Nonequivalent control group _______________ design -Only measure the DV once after exposure to the IV O = Observation 1 = 1st observation Exp x O1 Control group O1

concurrent-measures design

(Within-groups design) Participants are exposed to at least two levels of an IV at the same time, and then indicate a preference for one level (the DV) ex: -----> female face one group |-------------> looking pref. -----> male face

type I error

A "false positive" result in the statistical inference process, in which researchers conclude that there is an effect in a population, when there really isn't

null effect

A finding that the IV didn't make a difference in the DV; there is no significant covariance between the two. -the study was not designed well enough (no enough variability between levels --> IV doesn't effect DV, but some factor prevented it from being detected) -The IV really does not effect the DV

direct replication

A replication study in which researchers repeat the original study as closely as possible to see whether the original effect shows up in the newly collected data -Use the same variables and same operationalizations -Does not test the theory in a new context --> so, researchers value other types of replication more -If the original study shows threats to internal validity/flaws in construct validity, they would appear in this type of replication study

ceiling effect

An experimental design problem in which IV groups score almost the same as the DV, such that al scores fall at the high end of their possible distribution.

design confound

An experimenters mistake in the research design in that a second variable happens to vary systematically -Systematic variability -Unsystematic variability

experimental realism

Extent to which a lab experiment is designed so that participants experience authentic emotions, motivations, and behaviors -Even though the situation is in a controlled environment, it still draws out behavior and emotions that are realistic -Research doesn't necessarily have to be conducted in a field setting to have ecological validity. Lab studies conducted in theory-testing mode might have strong ___________ even if they don't resemble real-world situations outside the lab. Yet the data from such artificial settings help researchers test theories in the more internally valid way possible, and the results may still be important and apply to real world circumstances -Feel emotionally real

manipulation check

In an experiment, an extra DV researchers can include to determine how well a manipulation worked. -ex: Separating people with different levels of anxiety into different groups (high, med, low) -Detect weak manipulations -If manipulation was effective, look for other reasons for null effect --> ceiling/floor effects; there really is no difference

covariance

Is the IV related to the DV? -Manipulated an IV (having at least 2 levels) enables researchers to ask "compared to what?" -All experiments need a comparison group --> control group/treatment group. -2 variables change/fluctuate together --> the IV changes the DV

pretest/posttest

Nonequivalent control group _______________ design An independent-groups quasi-experiment that has at lease one treatment group and one comparison group, in which participants have not been randomly assigned to the 2 groups, and which at least one pretest, and one posttest are administered -Treatment and control group that measures DV before and after treatment

interrupted time-series design

Same group measured repeatedly over time on some DV before, during, and after the "interruption" caused by some event OOOOOOXOOOOOO (X = event) -Single group of participants that are measured multiple times -Also, Repeated Measures Design

generalization mode

The intent of researchers to generalize the findings from the samples and procedures in their study to other populations or contexts -Always frequency claims -Sometimes association and causal claims) -Goal is to make a claim about a population -Ecological validity -Real world matters -EXTERNAL validity is essential

noise

Unsystematic variability among members of a group in an experiment, which might be caused by situation variability, individual differences, or measurement error. -The greater the overlap, the smaller the effect size, and the less likely the 2 groups means will be statistically significant; that is, the less likely the study will detect covariance.

main effect

(Factorial Design) The overall effect of the 1 IV on the DV, averaging over the levels of the other IV -Take the average of each cell --> __+__ --> /2 -A "simple difference" -2 IVs = 2 ________ -Calculating the marginal means -- the means for each level of an IV, averaging across the levels of the other IV -Ask: Is there an overall difference? -May not be statistically significant - = overall effect 2x2 factorial design = 4 cells

matched groups design

(Independent-groups design) An experimental design technique in which participants who are similar on some measured variable are grouped together into sets; the members of each set are then randomly assigned to different experimental conditions. -Has the advantage of randomness; because each member is randomly assigned, the technique prevents selection effects which establishes internal validity. -Disadvantages: requires an extra step --> more time and more resources than random assignment

independent-groups

(Quasi-experiments) -Nonquivalent control group posttest only design -Nonquivalent control group pretest/posttest design

disadvantages of small-n designs

-Generalizability -Ethic issues w/ reversing treatment --> Some might question ethics of withdrawing a treatment that appears to be working; might be harmful to a patient to withdraw effective treatment

design confound, selection effect, order effect, maturation, history, regression to the mean, attrition, testing, instrumentation, observer bias, demand characteristics, placebo effects

12 Threats to Internal Validity

reverse effect

A Small-N Design in which a researcher observes a problem behavior both before and during the treatment, and then discontinues the treatment for awhile to see if the problem behavior returns -Also known as: ABAB Design or Replication Design -Can test for internal validity by observing how the behavior changes as the treatment is removed and reintroduced -Causal Statement -If the treatment is really working, behavior should improve only when the treatment is applied -Focus on individual performance -Avoid ethical problems w/ control/placebo group -Appropriate mainly for situations in which a treatment may not cause long lasting effects

field setting

When a study takes place in the real world -High ecological validity -Priority: EXTERNAL validity

advantages

_______ of Factorial Designs -More than one IV can be manipulated -- enables researchers to test and establish multiple influences of behavior -Can test limits (a form of external validity, can test for moderators -Can test theories

full counterbalancing

A method of counterbalancing in which all possible condition orders are represented

partial counterbalancing

A method of counterbalancing in which some, but not all, of the possible condition orders are represented

regression to the mean

(12 internal validity threats) A phenomenon in which any extreme finding is likely to be closer to its own typical, or mean, level the next time it's measured (with or without the experimental treatment/intervention) -Tendency for extreme scores to be less extreme on subsequent tests -Works at both extremes: an unusually good performance/outcome is likely to regress downward (toward its mean) the next time; and an unusually bad performance/outcome is likely to regress upward (toward its mean) the next time -Only occurs when a group is measured twice and only when the group has an extreme score at pretest -Prevention: comparison group that was equally extreme that did not receive the treatment

instrumentation (aka instrument decay)

(12 internal validity threats) A threat to internal validity that occurs when a measuring instrument changes over time. -Any changes that occur when a measuring instrument changes over time from having been used before (coders/interviewers) -People coding behaviors might change their standards for judging behavior by becoming more lenient/strict. --When a researcher uses different forms for the pretest and posttest, but the 2 forms are not sufficiently equivalent. -Prevention: train interviewers/coders; consider using posttest only design (make sure pretest/posttest measures are = ); counterbalance tests used at pretest and posttest

testing

(12 internal validity threats) In a repeated-measures experiment or quasi-experiment, a kind of order effect in which scores change over time just because participants have taken the test more than once; includes practice effects -Prevention: Do not use pretest; if a pretest is used, use alternative forms of the test for pretest and posttest; comparison group--can be ruled out if the treatment groups shows a larger change than the comparison group

attrition (aka mortality)

(12 internal validity threats) In pretest/posttest, repeated-measures, or quasi-experimental study, a threat to internal validity that occurs when a systematic type of participant drops out of the study before it ends -A differential loss of participants from the various experimental group -Prevention: remove drop outs' data from the pretest group average

observer bias

(12 internal validity threats) Observers expectations influence the participants interpretation of the participants behaviors or the outcome of the study -Can affect: internal validity (b/c an alternative explanation exists for the result); & construct validity (b/c it doesn't represent the true values/ levels of the DV)

history

(12 internal validity threats) This threat occurs when it is unclear whether a change in the treatment group is caused by the treatment itself or by an external or historical factor that affects most members of the group. -Events that occur outside the lab that affect everyone or almost everyone (significant events/seasons) -Systematically affects most members of the treatment group at the same time as the treatment itself, making it unclear what caused the change -Prevention: comparison group

statistical validity

(4 validities) How well does your data support you causal conclusion? -Is the difference statistically significant? (p < .05) --Yes = covariance exists between the variables in the population from which the sample is drawn --No = no covariance = the study doesn't support a causal claim -How large is the effect? --If a study used a very large sample, even tiny differences might be statistically significant --The larger the effect size, the more important, and stronger, the causal effect

external validity

(4 validities) To whom or to what can you generalize the causal claims -Generalize to other people --> if participants in a study are sampled randomly from the population of interest, you can be relatively sure the results can be generalized at least to the population of participants from which the sample came from -Generalize to other situations --> sometimes necessary to consider the results of other research -Research prioritize internal validity over external validity

1. independent-groups, 2. posttest-only, 3. pretest/posttest

(Ch. 10 InQuizitive) Experiments that compare different participants in different conditions are known as __1__. There are two basic forms of this design. The first is the __2__design where participants are tested on the dependent variable only once following the manipulation. The second is the __3__design where participants are tested on the dependent variable before and after the manipulation. Independent-groups, repeated-measures, pretest/posttest, within-group, posttest-only

stable-baseline design

A Small-N design in which a researcher observes behavior for an extended baseline period before beginning a treatment/other intervention, and continues observing behavior after the intervention -Also known as Simple Comparison or an AB Design -Measure baseline -Introduce treatment -If behavior during ____ is stable, more certain of the treatments effectiveness -Worry about history and maturation

1. order, 2. carryover, 3. practice, 4. demand characteristic

(Ch. 10 InQuizitive) In a within-groups design, exposing participants to one level of the independent variable can change how they respond to the other levels of the independent variable. This problem is generally referred to as __1__ effects, of which there are various types. One specific type is known as a __2__ effect. This occurs when exposure to one level of the independent variable contaminates how other levels of the independent variable are perceived. Another type is a __3__ effect, which occurs when participants change their responses based on repetition of the independent variable rather than on the influence of the variable itself. It is also possible that repeated exposure to the independent variable may cause participants to guess the hypothesis of the experiment, also known as __4__. demand characteristics, power, practice, counterbalancing, order carryover

type II error: unsystematic variability (unsystematic variability can obscure the experiment's effects, but is not considered a confound), type I error: systematic variability (systematic variability can often make a manipulation appear to have an effect where there is actually none)

(Ch. 10 InQuizitive) Match each variability type to the error associated with it: systematic variability, unsystematic variability (type II error: , type I error:)

1. manipulation check, 2, pilot study, 3. construct

(Ch. 10 InQuizitive) Norbert wonders about the relationship between room temperature and feelings of belonging. He wonders if reducing room temperature by 1* would have an effect on feelings of belonging. To establish that the different levels of the independent variable are sufficiently different, he chooses to conduct a __1__ by asking participants at the end of the study to report the perceived temperature in the room. He finds that the participants all report that the room is 'comfortable.' He wonders if 1* is simply not a large enough change to be noticed. To further his research, he runs a __2__ by recruiting a small number of participants and measuring how many degrees the room temperature needs to be changed before it is noticeably less comfortable. These two techniques help to establish the __3__ validity of the independent variable. statistical, pilot study, external, construct, manipulation check

latin square

(Ch. 10 InQuizitive) Rebecca is interested in how solving different types of puzzles influences creativity. She will have participants try four different puzzles (word, 3-D, 2-D, and number) and measure creativity after each puzzle. To prevent order effects, Rebecca will sort participants into groups and have each group do the puzzles in a different sequence. The first sequence will be word, 3-D, 2-D, number; while the second will be 3-D, 2-D, number, word; and so forth, until each possible condition falls in each position in the experiment. What technique has Rebecca used to prevent order effects? matched groups, partial counterbalancing, full counterbalancing, Latin square

1. selection effect, 2. random assignment, 3. matched groups

(Ch. 10 InQuizitive) When a researcher inadvertently creates a condition with a fundamentally different type of participant than another condition, this can create a__1__. You can combat this by arbitrarily assigning participants to each level, known as __2__. In some cases, especially with smaller sample sizes, researchers will assign participants to groups so that each group has a similar makeup for a particular attribute they may be concerned about. This is known as creating __3__. matched groups, selection effect, design confound, random assignment, systematic variability

applies: matched groups, independent-groups design, posttest-only design; does not apply: within-groups design, random assignment, pretest/posttest design

(Ch. 10 InQuizitive) Read the study and determine whether or not each description applies. Bruno is testing a new diet plan to see if it is effective in reducing the frequency of eating cancer-causing foods. Bruno creates two groups of people—one that will receive his new diet plan and one that will receive a traditional diet plan. Bruno knows that culture influences food choices so he creates separate lists of participants from different cultures and ensures that each group has an equal representation of different ethnicities. He puts the names of all of the participants from one ethnicity into a hat and draws out half of them to be in one group and the other half in the other group. He does the same for each of the ethnic groups. Bruno gives the two groups their diet plans and measures participants on their eating habits three months later. random assignment, pretest/posttest design, matched groups, posttest-only design, within-groups design, independent-groups design -applies: ; does not apply:

1. unsystematic variability, 2. systematic variability, 3. systematic variability, 4. unsystematic variability

(Ch. 10 InQuizitive) Unsystematic Variability vs. Systematic Variability 1. Marcos is testing how performance in video games affects aggression; he randomly assigns participants to play either an easy video game or a difficult video game. Some of his participants have never played video games before. 2. Shilpa wants to measure the positive effects of shopping online versus shopping in person. She recruits participants from a message board focused on online shopping and randomly assigns them to either buy three things online or three things in a store and then complete her shopping attitude scale. 3. Travis is comparing the effects of upbeat music and slow music on task productivity. His research assistants tell him they really enjoy the upbeat music condition. 4. Imani is doing a study on sociability in college students, with participants either one-on-one or in small groups. She later finds out that one of her research assistants is much chattier than the others.

1. design confound, 2. systematic variability, 3. unsystematic variability

(Ch. 10 InQuizitive) When researchers inadvertently create a flaw in their experiment that is a threat to internal validity, it is known as a __1__. If these flaws coincide with the experimental manipulation and call into question whether or not the manipulation or the flaws affected the dependent variable, this is known __2__. If the flawed part of the experiment does not coincide with a specific group, and is introduced at random, this is known as __3__ and does not necessarily pose a threat to internal validity. -random sampling -selection effect -matched group -unsystematic variability -design confound -systematic variability

2 main effects (type of video and a main effect for sex), 1 interaction (sex and type of video shown)

(Ch. 12 InQuizitive) Amir is conducting a study in which he shows participants one of three television commercials (charity, political, or control) in order to see which causes the most helping behavior. In this study, Amir is also interested in seeing how sex (male versus female) may interact with the type of commercial watched and affect helping behavior. Identify the number of main effects and interactions Amir is likely to find in his study. Main effects or Interactions: 4, 3, 2, 1, 6

ceiling effect (most people will say they plan to volunteer in the future regardless of the manipulation, so it is likely scores in this study would be high regardless)

(Ch. 11 InQuizitive) Antonio is studying how different pamphlets promoting local charities may increase volunteer behavior in high school students. He creates one pamphlet that shows students having fun and being social while volunteering, and another that discusses the benefits volunteering has for college and future careers. Antonio has participants read one pamphlet or the other and then fill out a measure about volunteering intentions for the future. Antonio's measure asks how likely participants are to volunteer in the future from 1 ("I will not volunteer at all") to 5 ("I will likely volunteer in the future"). When looking at his results, Antonio finds that both groups scored very high on his measure and there is no difference between the two groups. What problem is most likely causing a null effect in the study? weak manipulation, within-group variability, floor effect, ceiling effect

CV: gender of victim, events; DV: helping intentions, victim blaming; IV: environment

(Ch. 11 InQuizitive) Doris is interested in whether there is a difference in victim-blaming and victim-helping attitudes when it comes to bullying based on environment (in person versus online). Doris creates a fictional scenario in which someone is bullied in a classroom or on a social networking site. In each case, the events and gender of the victim are not changed. Doris then measures how much people blame the bullied person and how willing they are to step in and help the person. Match each aspect of the experiment to the variable it best represents. Control Variable - Dependent Variable - Independent Variable helping intentions, environment, victim blaming, gender of victim, events

1. attrition threat (attrition is common; participants who do not complete all parts of a study should be thoroughly scrutinized or removed), 2. regression threat (researchers should compare participant scores at various times within the experiment while also comparing them to what average scores are), 3. history threat (experiments are not conducted in a vacuum; outside events can often influence behavior), 4. instrumentation threat (for example, when coding behavioral data, researchers can sometimes inadvertently change how they code certain behaviors), 5. testing threat (measures must be reviewed carefully to be certain they do not have lasting effects on participants), 6. maturation threat (also known as spontaneous remission, this problem can be prevented by using comparison groups)

(Ch. 11 InQuizitive) MATCH: maturation threat, attrition threat, regression threat, testing threat, history threat, instrumentation threat 1. More participants decline to continue from one group or condition than from another 2. Extreme scores return towards average. 3. Participants' responses are altered by an event outside the control or interest of the study 4. The way the researcher measures a variable changes over the course of an experiment 5. Participant's future performance is changed because of interaction with a measure 6. Participant behavior changes spontaneously over time

1. regression threat (the original scores were extreme, it was likely they would return to average regardless of the manipulations), 2. maturation threat (participant behavior is not static and can often change throughout a study, regardless of the manipulation), 3. attrition threat (the participants who withdrew were not random, this presents a threat to validity), 4. history threat (the occurrence of the clearance sale likely influenced shopping behavior of the participants outside of what would have been typical), 5. instrumentation threat (with this threat, the instrumentation changes rather than the dependent variable), 6. testing threat (the instrument is changing how the participants behave)

(Ch. 11 InQuizitive) Match: attrition threat, regression threat, testing threat, instrumentation threat, maturation threat 1. Participants showed very high stress scores at pretest and returned to average at posttest 2. Students in a study on grade performance naturally improve during the study. 3. The three healthiest participants withdraw from a study on metabolic syndrome. 4. A large clearance sale takes place during a study on shopping behavior. 5. Observers code social behavior as less friendly over time 6. Participants change their answers on a racism scale after seeing it a second time.

weak manipulation (the logo alone may not have been enough of a difference to make participants feel that they were on different social networking sites)

(Ch. 11 InQuizitive) Nikki created an experiment to investigate if viewing pictures on different types of social networking sites would cause people to feel more or less social. She had participants look at pictures on a web page that had either a Facebook logo or a Google+ logo, but everything else about the website was the same. Nikki measured sociability on a scale of 1 to 10 that asked several questions, such as "How interested are you in meeting other people right now?" and "How interested are you in talking to your friends right now?" Nikki found that both groups had sociability averages around 6, and there was no difference between the groups. What problem is most likely causing a null effect in the study? weak manipulation, insensitive measure, ceiling effect, floor effect

1. noise, 2. measurement error, 3. individual differences, 4. situation noise

(Ch. 11 InQuizitive) Null effects can be caused by excessive amounts of unsystematic variance, or __1__. This within-group variance can be caused by a dependent variable with poor reliability or in which insufficient data is collected. This is referred to as __2__. It can also be caused by __3__ if extreme scores within the participants obscure changes brought on by the manipulation. Finally, outside factors in the experimental setting can influence participants, otherwise referred to as __4__. too many participants, measurement error, ceiling effect, manipulation checks, individual differences, situation noise, noise

1. false, 2. false (if the comparison group does not experience the same history threat as the experimental group, then no comparison should be made. This is also known as a selection-history threat), 3. true, 4. true (the data must be carefully scrutinized to be sure that the regression effect is not driving a significant finding)

(Ch. 11 InQuizitive) True or False 1. A study typically only has one threat to internal validity 2. To prevent a history threat, the comparison group should be followed at a different time or in a different location than the treatment group 3. If participants withdraw from a study in an unsystematic way, there is likely no attrition threat 4.A regression threat can lead to a significant finding, resulting in a Type I error

1. true, 2. false (increased within-group variance will likely result in a null effect), 3. false, 4. true

(Ch. 11 InQuizitive) True or False? 1. Not enough between-group differences can result in a null effect: 2. Increases in within-group variability can lead to the illusion of covariance: 3. Null effects are very uncommon in experiments: 4. Null effects can occur in any experiment.

three-way interaction (in this ex, 3 IVs - race, type of language, and news watching - are all described on how they affect racial bias)

(Ch. 12 InQuizitive) Imagine you are reading the following example from a popular press article. Researchers tested a series of different newspaper articles and found that articles containing abstract language and an African American perpetrator increased racial bias in those who watched the news on a regular basis. Which of the following best describes this design? three-way interaction, two-way interaction, not a factorial design, no interaction

two-way interaction (in this ex, 2 IVs - type of classroom and technology experience - are described)

(Ch. 12 InQuizitive) Imagine you are reading the following example from a popular press article. Researchers tested different forms of textbooks and found that interactive classrooms were more effective only for students who had high levels of previous technology experience. Which of the following best describes this design? no interaction, not a factorial design, two-way interaction, three-way interaction

1. interaction effect, 2. main effects, 3. marginal means

(Ch. 12 InQuizitive) In a factorial design with two independent variables, researchers will need to review one __1__ and two __2__. This design will also yield four __3__, which are the averages of each level of the independent variable collapsed across the levels of the other independent variable. main effect(s), marginal mean(s), factorial effect(s), interaction effect(s)

CV: orange juice, DV: aggressive behavior, IV: amount of alcohol

(Ch. 12 InQuizitive) In your text, a study on alcohol is reviewed, in which participants were given either trace amounts of alcohol in orange juice or enough alcohol in the juice to get them legally drunk. After the drink, researchers then measured aggressive behavior of the participants with a game in which they were allowed to administer shocks to people they thought were other participants. Match each aspect of the experiment to the variable it best represents. control variable, dependent variable, independent variable

applies: random assignment, between-groups design (aka independent-groups design) posttest-only design; does not apply: pretest/posttest design, within-groups design, matched groups

(Ch. 12 InQuizitive) Read the study and determine whether or not each description applies. Tyrel is testing whether exposure to political news causes people to be more opinionated. For each participant, Tyrel flips a coin and places all the heads in one group and all the tails in another. The heads group watches 20 minutes of political news, while the tails group watches 20 minutes of local nonpolitical news. Afterward, all participants write a persuasive essay and Tyrel measures how strongly and how many times they state their opinions. random assignment, pretest/posttest design, between-groups design, within-groups design, posttest-only design, matched groups

FD: interaction, differences in differences, mixed design; Not FD: main effect, independent variable, within-groups

(Ch. 12 InQuizitive) Sort the terms as related to either a factorial design or a single independent variable design. FD or Not FD main effect, mixed design, within-groups, interaction, independent variable, difference in differences

1. false, 2. false, 3. true, 4. true

(Ch. 12 InQuizitive) True or False 1. Most popular media articles will use terminology similar to that of a science journal 2. Regression is typically an indicator of factorial designs 3.Researchers tell whether interactions are significant or not in the results section 4.One way to identify a factorial design is via the A × B design (e.g., 3 × 2)

1. factorial , 2. interaction effect, 3. crossover interaction, 4. spreading interaction

(Ch. 12 InQuizitive) When an experiment tests all possible combinations of more than one independent variable, it is often referred to as a __1__ design. These designs can show that the effect of one independent variable depends on the level of another independent variable, also known as a(n) __2__. These effects typically have two types. If you were to display the data in a line graph, one type would show the data from one independent variable intersect with the data from the other, or a(n) __3__. The second type graphed in the same way would show the data moving away from each other as the dependent variable increased, or a(n) __4__. Factorial, spreading interaction, interaction effect, correlational interaction, longitudinal, crossover interaction, main effect

history threat (the presence of all the schoolchildren may have increased traffic in the new area, which might not have happened otherwise)

(Ch. 13 InQuizitive) A group of researchers is measuring foot traffic in a museum. Recently one half of the museum was redone in order to encourage more foot traffic. Researchers are counting the number of attendees in this new zone and comparing it to other zones that have not been redone. On the first day that researchers are recording foot traffic, several buses of schoolchildren arrive at the museum unexpectedly for a field trip. After a day of testing, the new zone appears to be getting just as much foot traffic as the other zones. Which problem is most likely affecting this quasi-experiment? history threat, selection efforts, maturation threat, testing threat

1. stable-baseline, 2. multiple-baseline, 3. reversal

(Ch. 13 InQuizitive) A(n) __1__ design monitors participants for a long period of time before a treatment begins, to be certain of what behaviors are typical. A(n) __2__ design records more than one type of behavior for various lengths of time before applying treatments to each particular behavior. Finally, a(n) __3__ design introduces a treatment and then removes it to see if behavior will return without the treatment interrupted time-series, quasi-experimental, multiple-baseline, reversal

observer effects (children are not doing what they otherwise would because of Dana's presence)

(Ch. 13 InQuizitive) Dana is recording playground behavior of students to investigate the differences between third, fourth, and fifth graders. For each session, Dana sits on the swings with a notebook and counts the size of groups and types of activities for each grade. She notices that several children in each grade come over to the swings, but then choose to walk away once they notice Dana. Because they walked away, they are not counted in the study. Dana finds that the lower grades tend to do more group activities, while the higher grades tend to be more individualistic. Which problem is affecting this quasi-experiment? instrumentation threat, observer effects, attrition threat, observer bias

yes: sick days, recess, texting behavior; no: video game, mindfulness

(Ch. 13 InQuizitive) Determine whether or not each example is a quasi-experiment. (respond --> yes: ; no: ) 1) Sick days are tracked for a company after each occasion that the company assigns mandatory overtime. (sick days) 2) Volunteers for a video game study are randomly assigned to play either a violent game or a nonviolent game for six months. (video game) 3) Researchers look at bad classroom behavior following recess at one school that has one recess period versus another that has two. (recess) 4) Researchers randomly assign two separate groups different mindfulness strategies to see which results in less stress. (mindfulness) 5) Researchers look for differences in texting behavior between males and females (texting behavior)

wait-list (this technique is specific to quasi-experiments and can help prevent selection effects when groups cannot be randomly assigned to a condition)

(Ch. 13 InQuizitive) Kanchi is interested in studying how water aerobics improves attitudes toward exercise over time. She has three types of patients: those who are overweight, those who are obese, and those who are of normal weight. For each weight range, she has half of the group begin treatment immediately and staggers other participants in each group to begin the treatment 2 weeks later. Kanchi is using what type of design strategy?

1. interrupted time-series design, 2. nonequivalent control group design, 3. nonequivalent control group interrupted time-series design, 4. nonequivalent control group pretest/posttest design

(Ch. 13 InQuizitive) Match each design to the appropriate description: nonequivalent control group design, nonequivalent control group pretest/posttest design, interrupted time-series design, nonequivalent control group interrupted time-series design 1. a quasi-experiment with a single group of participants that are measured after each of several events 2. a quasi-experiment with one treatment and one control group that measures the dependent variable only once 3. a quasi-experiment with two independent groups that are both measured after each of several events 4. a quasi-experiment with a treatment and a control group that measures the dependent variable before and after the treatment

1. selection effects (if participants are grouped using a participant variable, it is important to understand potential differences between those groups besides what is obvious), 2. placebo effect (creating a cover story about the true nature of the experiment can prevent a control group from having placebo effects), 3. maturation threat (a control group can help detect spontaneous change in an experiment), 4. testing threats (unobtrusive observational measures can help quasi-experiments avoid testing threats), 5. attrition threat (if participants are leaving a study in a certain way, it may reveal something about a group that was previously unaccounted for) 6. history threat (when measuring groups multiple times, it is imperative to account for any major events that could affect responses)

(Ch. 13 InQuizitive) Match each internal validity threat to the appropriate description: selection effects, maturation threat, testing threats, placebo effect, attrition threat, history threat 1) Groups vary systematically in ways other than how the independent variable was defined 2) The thought of treatment causes the participant to report changes, rather than the treatment itself 3) Participants show spontaneous change 4)Participants change their responses based on past assessment 5)Participants leave a study in a systematic way 6)An event, rather than the independent variable, changes scores on the dependent variable

1. false, 2. false, 3. true, 3. true

(Ch. 13 InQuizitive) True or False 1. Small-N designs cannot be used to make inferences about human behavior as a whole 2. Because there are so few subjects, it is impossible to control for most parts of a small-N design 3. Comparing small-N designs in humans with those in animals can increase external validity 4. Small-N experiments have poor external validity

internal validity (small-N designs are often carefully created to rule out alternative explanations)

(Ch. 13 InQuizitive) Type the term that best answers the question. Though many question the validity of small-N designs, if carefully designed and rigorously conducted they are often very strong in what type of validity?

1. experiment, 2. quasi-experiment, 3. with-in groups design, 4. between-groups design

(Ch. 13 InQuizitive) While a(n)__1__ manipulates an independent variable in order to see the change in a dependent variable, a(n)__2__ typically uses the same structure but with an independent variable that cannot truly be manipulated by the experimenter. Even though the independent variable cannot be truly manipulated, this type of research can be done by having the participants experience all levels of the independent variable (__3__) or by having each participant experience only one level of the independent variable (__4__). Within-groups design, longitudinal study, experiment, between-groups design, quasi-experiment, small-N design

HARKing (Anita removed a hypothesis and emphasized another based on her results, which can be a major problem in how others interpret the results. Anita should have reported her first hypothesis and said that it was not supported while still talking about her new ideas in her discussion)

(Ch. 14 InQuizitive) Anita is conducting a study on sleep and video games. When she began her study, she was not sure if playing video games caused people to lose sleep, or if when people cannot sleep they often turn to video games. Once she analyzed her data, she discovered that it was more likely that participants were losing sleep because they were staying up playing video games. After realizing this, Anita removed any mention of the idea that those who can't sleep turned to video games from her paper before submitting it for publication. Anita could be accused of which poor research practice: There was no issue with her research practices, p-hacking, HARKing, publication bias

1. scientific literature, 2. meta-analysis, 3. effect size

(Ch. 14 InQuizitive) Another way to increase the strength or importance of a particular finding is to review the __1__ around that finding and conduct a study the compiles the results of similar studies into one statistic. This type of study is called a(n) __2__, and the statistic typically calculated is a(n)__3__. meta-analysis, effect size, scientific literature

1. direct replication , 2. conceptual replication , 3. replication-plus-extension

(Ch. 14 InQuizitive) If a researcher wants to reproduce a previous study, she can do so in multiple ways. A(n)__1__ involves re-creating previous work as closely as possible to the original. Scientists can also conduct a(n)__2__, in which a study uses different procedures to re-create a similar finding to the original. Finally, a(n)__3__ involves copying the original experiment and including additional variables to answer new questions. statistical replication, direct replication, conceptual replication, replication-plus-extension, external replication

1. theory-testing mode (this study is not attempting to generalize its findings to any particular population), 2. generalization mode (these researchers are primarily concerned with generalizing their findings to a new population), 3. theory-testing mode (these researchers are trying to test particular parts of a theory and are therefore in theory-testing mode), 4. generalization mode (this project is attempting to expand its target population to all college students)

(Ch. 14 InQuizitive) Matching: Generalization or Theory-Testing Mode 1. A study is designed to understand what, specifically, compels people to help in emergency situations 2. Two researchers are attempting to replicate a finding from the United States in Japan 3. A lab is conducting a series of studies to understand the effects of emotion in the written word 4. A researcher is re-creating previous work from liberal arts colleges at large public universities

1. replication (if those reproducing a finding use a setting that is dramatically different from the original, certain results may not replicate), 2. original (the original study may have used a specific type of participant without realizing it, leading to an ungeneralizable finding), 3. publication practice (even though the finding has been shown to be false many times in the past, those studies were never published, leading other researchers to be unaware that the finding may have been by chance), 4. original (not reporting all of the dependent variables in a study can cause people to highly value an outcome that may have originally occurred by chance), 5. replication (reducing the sample size in the replication can often result in a small original effect being lost)

(Ch. 14 InQuizitive) Matching: Original, Replication, Publication Practice 1. A replication failed because it was conducted in a setting theoretically incongruent with the study 2. An original finding failed to replicate in a population that was not intended to be used by the original theory 3. A finding is shown to not be replicable even though many studies done in the past showing null results were never published 4. A replication failed because the original study used multiple dependent variables and only reported one 5. A replication failed because it used fewer subjects than the original study

1. open science, 2. preregistration, 3. HARKing

(Ch. 14 InQuizitive) Since the replication crisis, many practices have been implemented to improve research and promote __1__, or the practice of sharing data and materials so they can be reviewed and replicated by other researchers. One of these practices involves submitting the proposed methods and analysis plan to journals in advance for approval or __2__. This practice allows scientific journals to approve experiments before they are done, with the guarantee that any finding will be published regardless of whether or not it is significant. This process also prevents researchers from __3__, or creating theories or hypotheses that explain the results of a study after the data has been collected. Preregistration, HARKing, open science, p-hacking. meta-analysis

1. field setting, 2. ecological, 3. experimental realism

(Ch. 14 InQuizitive) Some studies are conducted in real-world situations, or a __1__. These studies are often high in __2__ validity, or the idea that these situations are likely to be encountered in everyday life. Even though some researchers believe this realism is paramount, many others will claim that laboratory studies can have __3__—in other words, even though the situation is in a controlled environment, it still draws out behavior and emotions that are realistic. cultural realism, ecological, statistical, construct, experimental realism, field setting

crisis (this even caused widespread uncertainty about the validity of many important findings in the field, which has led to many changes in scientific practices)

(Ch. 14 InQuizitive) The controversy that surrounded psychology after a massive replication attempt indicated that less than half of all psychology studies were replicable is often referred to by scientific journalists as the replication _______________.

1. false (a literature review does not calculate effects, but rather reviews the findings of others from a conceptual standpoint), 2. false (because a meta-analysis reviews many studies at once, it can draw conclusions that may not have been noticed in a single study), 3. true (this practice can help prevent the file drawer problem), 4. true (his prevents studies with null effects from being published and can cause them to be left out of a meta-analysis)

(Ch. 14 InQuizitive) True or False 1. A literature review is basically the same thing as a meta-analysis 2. A meta-analysis can only review the strength of previous findings and cannot draw new conclusions 3. The author of a meta-analysis should contact colleagues to see if they have null findings that were not published 4. The file drawer problem refers to only the results of significant studies being published

1. true (regardless of how replicable the creators designed the study, until it is re-created it cannot be considered replicable), 2. false (a conceptual replication can strengthen a finding more, by showing that a finding is so strong it occurs in settings beyond that of the original experiment), 3. true (while researchers should strive to make replications as close as possible, it is often impossible to replicate every detail), 4. false (a significant result can occur by chance and therefore should be replicated)

(Ch. 14 InQuizitive) True or False 1. A study is only replicable if it has actually been replicated 2. Conceptual replications are less useful in assuring that a finding is reliable than direct replications are 3. A direct replication can never exactly replicate an original study 4. If a study gets significant results, it does not need to be replicated

1. true (one of the obstacles of cultural psychology is being able to collect data from multiple different cultures. This typically requires larger collaborations), 2. true (cultural psychology helps illuminate differences in human behavior between cultures), 3. false, (more than just generalizability and ecological validity determine how valuable a study is), 4. false (a large majority of participants are WEIRD, or Western, educated, industrialized, rich, and democratic)

(Ch. 14 InQuizitive) True or False 1. Obtaining samples from many cultures can be very challenging 2. Cultural psychology studies how people's cultures change who they are 3. Studies that take place in the real world are more valuable than those conducted in a laboratory 4. The majority of participants in published psychology journals are representative of the world's population:

1. false (many journals do not publish direct replications because they do not add novel thoughts or ideas to the scientific literature), 2. true (any single study can find results by chance that are not indicative of the true nature of the effect; therefore, multiple replications should be conducted), 3. false (failure to replicate may be due to many different factors that warrant further investigation, but it does not serve as unequivocal evidence that the findings were false), 4. true (though psychology is central to the debate, many other fields have begun practices to replicate or make improvements in scientific rigor)

(Ch. 14 InQuizitive) True or False 1. Direct replications of previous studies are published regularly to strengthen findings 2. A single replication has the same odds of producing a result by chance as the original study 3. If a study does not replicate, it means that the original findings were false 4. The replication crisis involves other fields outside of psychology

1. theory-testing mode, 2. internal, 3. external, 4. generalization mode

(Ch. 14 InQuizitive) When a researcher attempts to test specific, nuanced aspects of a hypothesis or theory, this process is called __1__, and the researcher typically prioritizes __2__ validity. When a researcher prioritizes __3__ validity in an attempt to apply her findings to a larger population, this process is called __4__. Statistical, theory-testing mode, internal, replication mode, generalization mode, construct, external

practice effect

(Order effect) A type of order effect in which participants' performance improves over time because they become practiced at the dependent measure (not because of the manipulation or treatment)

mixed design

(Quasi-experiments) Nonquivalent groups interrupted time-series design

repeated-measure design

(Within-groups design) Participants are tested on the DV after each exposure to an IV condition ex: one group --> interact w/ own toddler --> measure oxytocin --> interact w/ new toddler --> measure oxytocin

ecological validity

(also external validity) The extent to which the tasks and manipulations of a study are similar to real-world/real life contexts --> the kinds of situations participants might encounter in their everyday lives

masked design (aka blind study)

(demand characteristics) The observers are unaware of the experimental conditions to which participants have been assigned -Participant is blind

unsystematic variability

(design confound) In an experiment, a description of when the levels of a variable fluctuate independently of experimental group membership, contributing to the variability within groups

systematic variability

(design confound) Levels of a variable coincide in some predictable way with experimental group membership (need to use controls to help with it) -Using pesto for the medium size bowl, and marinara for the large size bowl --> cant tell if results are because of the sauce type, or the bowl size

weak manipulation

(interrogating a null result) It is important to ask how the researcher operationalized the IV -You have to ask about construct validity

measurement error

(noise) The degree to which the recorded measure for a participant on some variable differs from the true value of the variable for that participant. May be random, such that scores that are too high/low cancel each other out; or they may be systematic, such that most scores are biased too high/too low. -Can inflate or deflate a participant's true score on a DV. ex (memory): One participant was drowsy; another studied vocab words right before taking the test. -- DV = true score +/- random error of measurement 2 Solutions: (1) Reliable measurements (i.e. internal, interrater, test-retest); good construct validity ( = lower error rate); precision (accurate measurements = less error) (2) Measure more instances -- can cancel out many errors simply by including more people in the sample

meta-analysis

(scientific literature) A way of mathematically averaging the effect sizes of all the studies that have tested the same variables to see what conclusion that whole body of evidence supports -Strengths: published papers are peer reviewed; find an overall effect size; can look for moderators -Limitation: file drawer problem

one-group pretest/posttest design

(the really bad experiment) An experiment in which a researcher recruits one group of participants; measures them on a pretest; exposes them to a treatment/intervention/change; and then measures them on a posttest -differs from the true pretest/posttest design because it only has one group not two --> no comparison group.

conceptual replication

A replication study in which researchers examine the same research question (same concept variables), but use different procedures for operationalizing the variables -Can improve external validity --> if a finding replicates in a new setting it can strengthen the idea that the finding will replicate in other settings

theory-testing mode

A researchers intent for a study, testing association claims or causal claims to investigate support for a theory -"Nuanced aspects of a hypothesis" -Goal is to test a theory -Isolate variables -Prioritizes INTERNAL validity -Artificial situations may be required -Real world comes later -External validity is NOT the priority

scientific literature

A series of related studies conducted by various researchers that have tested similar variables -literature reviews -meta-analysis

small-n designs

A study in which researchers gather information from just a few cases -Study one or a few participants -Typically observe and collect baseline data -Examine trend to determine effect -Instead of gathering a little info from a larger sample, they obtain a lot of info from just a few cases

factorial designs

A study in which there are 2 or more IVs, or factors

quasi-experiments

A study similar to an experiment except that the researchers do not have full experimental control (ex: may not be able to randomly assign participants to the IV conditions) -May not be able to manipulate the IV -Independent-groups -Repeated-measures -Mixed-Design

moderator

A variable that changes the relationship between two other variables. In factorial design language it is an independent variable that changes the relationship between another independent variable and a dependent variable. --> In other words, it results in an interaction; the effect of one independent variable depends on the level of another independent variable

nonequivalent control group designs

An independent-groups quasi-experiment that has at least one treatment group and one comparison group, but participants have not been randomly assigned into the 2 groups -Most common type of quasi-experiment -Experimental group -Control group -Participants are not randomly assigned

open science

As a part of a study's publication process, the practice of sharing one's data and materials freely so others can collaborate, use, and verify the results (collaboration) -Attempt replication on a larger scale

temporal precedence

Does the study design ensure that the causal variable comes before the effect/outcome variable in time? -Manipulate the causal (independent) variable to ensure that it comes 1st in time --> measure the DV -Experiments unfold over time, and the experimenter makes sure the IV comes 1st