Methods Exam #2
In the results section, (1) begin with a statement that describes the __________ performed AND upon what type of _______. Make sure to start with ___________ and build up to the ______________. (2) Follow the __________ statement with a list of the __________ findings ONLY (i.e., say what the effect was, followed by the __________, and then a description of the effect). Begin with ________, then proceed to _____________ interactions. Do NOT try to explain why you got what you found and do NOT try to explain or comment on what effect(s) __________ to make statistical significance. Also, do NOT use tables and/or figures unless the data are particularly __________ (not easy to describe in text).
analysis; data; main effects; interaction; analysis; significant; statistics; main effects; two-way; failed; complex
__________ is a protection of individual identity in which the identity of a participant remains unknown throughout the study, even to those involved in a study. _____________ is a protection of individual identity in which the identity of a participant is not made available to anyone who is not directly involved in a study. Those involved in a study, however, ARE able to identity participant information.
anonymity; confidentiality
The five general guidelines of ethical research is that the IRB must (1) ___________ research based upon __________, (2) participants must be provided with ______________, (3) participants must be informed that they have the freedom to ___________ at anytime WITHOUT penalty, (4) participants must have _________ and ___________, and (5) a __________ must occur immediately after research concludes.
approve; risk; informed consent; withdraw; privacy; confidentiality; debriefing
Suppose we ask participants to rate the personality of people based on being shown pictures of attractive and unattractive faces. Here, we have one factor (_____________) with two levels (_____________, _____________). However, suppose some of the faces shown to participants are faces of men, and some are of women. In this case, we have a SECOND factor (__________) also with two levels (__________, ___________). By crossing the levels of both factors, we create four unique groups. This is known as a _________________, which is used to compare differences between groups created by combining the levels of two or more factors.
attractiveness; attractive; unattractive; gender; man; woman; factorial design
Single-case designs are typically structured by alternating _________ and __________ phases over many trials or observations.
baseline; treatment
An ____________ is a mean difference or discrepancy between what was observed in a sample and what was expected to be observed in the population (stated by the null hypothesis).
effect
________ is a source of variance that cannot be attributed to having different groups or treatments. Two sources of this are between-persons and within-groups variability.
error
_____________ is a numeric measure of the variability in scores that can be attributed to or is caused by the individual differences of participants in each group.
error variance
The sum of within-groups and between-persons variability is the total value of __________. In a within-subjects design, however, we assume that the value of the between-persons variability is equal to _________. Hence, the within-groups variability is the ONLY measure of __________ in a within-subjects design.
error; zero; error
A significant F does NOT tell you WHERE the significant difference is, only that it IS there. You MUST examine the data directly in order to determine the nature of the effect. To be significant, F MUST ________ 1 by some critical amount (see published tables of critical values).
exceed
The minimum "r" or "t" or "F" (etc.) statistic WITH the probability of alpha (.05 or .01) that you calculated statistic MUST __________ the critical value in order to reject the null hypothesis. NOTE: Refer to p. 536 for table of critical values.
exceed
The ____________ is a condition in an experiment in which participants are treated or exposed to a manipulation, or level of the independent variable, that is believed to cause a change in the dependent variable.
experimental group
An _____________ is the identification of an independent variable and the creation of two or more groups that constitute the levels of that variable.
experimental manipulation
The variance attributed to error in a factorial design is associated with differences in ____________ within each cell, called _______________.
participant scores; within-groups variability
The change in a dependent variable across the combined levels of two factors is called an __________________.
interaction
Which effect, a main effect or an interaction, is identified in the cell means of a table summary?
interaction
A _______________ (________) is a single-case experimental design in which a single participant is observed before (A), during (B), and after (A) a treatment or manipulation.
reversal design; ABA design
The _________ design is typically conducted in applied areas of research to investigate possible solutions that can benefit individuals or society.
reversal single case
True or false: When p is less than or equal to .05, we conclude that a manipulation caused an effect because the likelihood that anything else cause the effect is less than 5%.
True
An ____________ is a source of variation associated with how the effects of one factor are influenced by, or depend on, the levels of of a second factor. In a table summary, an _____________ is a measure of how cell means at each level of one factor change across the levels of a second factor.
interaction; interaction
A ________________ is a quasi-experimental research design in which a dependent variable is measured for an interval of time before a treatment, then during the treatment, then again following the treatment. For example, the effects of caffeine on waking up: Mon- observe how long to alertness Tues- give caffeine and observe how long Wed - no caffeine and observe again
basic time series design
A ____________ involves a manipulated treatment, whereas the _______________ involves a naturally occurring treatment.
basic time series design; interrupted time series design
An _____________ reflects the difference between the cell means in the table. There is one __________ with two ways to interpret it.
interaction; interaction
True or false: The between-subjects design has GREATER power to detect an effect than the within-subjects design.
False.
True or false: In order to manipulate the levels of each factor in a factorial design, we must select a sample from at least _______ or more different populations.
False. We must select one sample from a SINGLE population
What are the df for Main effect A, for Main effect B, Main effect C, Main effect AB, Main effect AC, Main effect BC, and for Interaction of AxBxC for a 2 x 3 x 6 design? (1) df for Main effect A = (2) df for Main effect B = (3) df for Main effect C = (4) df for Main effect AB = (5) df for Main effect AC = (6) df for Main effect BC = (7) df for Interaction of AxBxC =
(1) 2-1 = 1 (2) 3-1 = 2 (3) 6-1 = 5 (4) 1(2) = 2 (5) 1(5) = 5 (6) 2(5) = 10 (7) 1(2)(5) = 10
What are the df for Main effect A, Main effect B, and for Interaction of AxB for a 2 x 6 design? (1) df for Main effect A= (2) df for Main effect B= (3) df for interaction of AxB=
(1) 2-1 = 1 (2) 6-1 = 5 (3) 5(1) =5
F = ___________________ / _______________
(treatment & error); (error)
Alpha is typically either ______ or ______. If p < alpha, then you _______ the null hypothesis (Ho). If p > alpha, results you obtained are TOO likely to have happened by _________ alone. NOTE: The null hypothesis CANNOT be proven; we either _______ it or FAIL to _______ it.
.05; .01; reject; chance; reject; reject
If there is NO treatment effect, then the F-value is ________, because the ________ in the numerator and the denominator are left and are divided by each other. HOWEVER, if there is a BIG treatment effect, then the ________ terms CANCEL out, leaving some number ________ than _______.
1; error; error; larger; 1
Researchers will often limit the number of factors in a factorial design to ________ or __________ to avoid the possibility of observing higher-order interactions with 4 or more factors.
2, 3
Even though df means are typically, "n-1" this is NOT always the case. For example, consider a 2 x 3 design: Sex (male, female) by Topic (History, Art, French). Since there are 2 levels for the Sex factor, n is __________=_____. Since there are 3 levels for the Topic factor, n is ___________ = ______. You would then multiply both n's from each factor (S x T) to get the degrees of freedom. In this case, S x T = _______ = _______.
2-1; 1; 3-1; 2; 2•1; 2
Suppose we hypothesize that ratings of athletic ability by an athlete will vary depending on the type of feedback (positive, negative) and the type of feedback delivery (written, in-person) he or she receives following a staged exercise. We create the groups by combining the levels of each independent variable. In this example, we have TWO levels of each factor, so we have ____ X _____ = _____ groups. We identify a factorial design by the number of _________ for each ________.
2; 2; 4; levels; factor
To figure out a # to represent variability: Use the formula: ∑x /n = m (mean). In other words, sum the values of the sources (e.g., 3+6+2+5) and divide this value by the # of values (e.g., 4) and you get a mean of ______. Next, use the formula ∑ (x-m) ^2. In other words, subtract the mean (4) by each individual value (e.g., 3, 6, 2, or 5) and then square the result (e.g., ∑ (3-4) ^2 = 1). Next, use the formula ∑x^2 - ∑ (x) ^2---all x's squared-- divided by n. For example, ∑ (16) ^2 - ∑ (3 ^2 + 6^2 + 2^2 + 5^2) divided by 4.
4;
For a simple 2 x 2 design, there are _______ possible combinations of statistically significant outcomes: (1) There may be _________ significant effects in which neither Factor A nor Factor B have a main effect AND there is no interaction. (2) There is a main effect of Factor _______ in which the averages are different for Factor A but they are identical for Factor B. (3) There is a main effect of Factor _________ in which the averages are different for Factor B but they are identical for Factor A. (4) There is an ___________ with _______ main effects, in which the averages are identical for both Factor A and Factor B, AND the differences among the differences are different (the lines are NOT parallel). (5) There are _______ main effects but no ____________, in which the averages are different for both Factor A and Factor B, BUT the differences among the differences are NOT different (the lines are parallel). (6) There is a main effect of Factor _____ AND an _________ but no effect of Factor ______, in which the averages are different for Factor A but they are identical for Factor B, AND the differences among the differences are different (the lines are NOT parallel). (7) There is a main effect of Factor _______ AND an __________ but no effect of Factor ______, in which the averages are different for Factor B but they are identical for Factor A, AND the differences among the differences are different (the lines are NOT parallel). (8) There are ________ main effects AND an _________, in which the averages for both Factor A and B are different AND the differences among the differences are different (the lines are NOT parallel).
8; zero; A; B; interaction; zero; two; interaction; A; interaction; B; B; interaction; A; two; interaction
A researcher conducts a 4x2 factorial design with 10 participants in each group. How many participants are needed to conduct this experiment if he or she conducts each of the following designs: A. A between-subjects factorial design B. A within-subjects factorial design C. A mixed factorial design in which the first variable is a within-subjects factor D. A mixed factorial design in which the second variable is a within-subjects factorial design
A. 10 x 4 x 2 = 80 participants B. 10 participants (equal to the sample size) C. 10 x 2 = 20 participants D. 4 x 10 = 40 participants
Which source of variability is attributed to each of the following? A. Individual differences B. The manipulation C. Participant responding within each group
A. Between-persons variability B. Between-groups variability C. Within-groups variability
Identify whether each of the following is an example of a reversal design, a multiple-baseline design, or a changing-criterion design: A. A researcher gives a child successively greater levels of positive reinforcement after an initial baseline phase to reduce how often the child bites her nails. The successive treatments are administered until the child has reached a level where she is no longer biting her nails. B. A researcher records the duration of time a participant stays on task in a dance recital four days before, four days during, and four days after a behavioral intervention strategy is implemented. C. A researcher records the quality of artistic strokes made by three participants. Each participant was given a treatment phase after three, four, or five days of a baseline phase; no baseline phase was given after the treatment was administered.
A. changing-criterion design B. reversal design C. multiple-baseline design
State the developmental research design that is described by each of the following phrases: A. Observing participants over time B. Observing groups at one time only C. Prone to testing effects D. Prone to cohort effects
A. longitudinal B. cross-sectional C. longitudinal D. cross-sectional
State whether each of the following is an example of a natural or a staged manipulation: A. A researcher measures the alertness of a student while the lights are off, dimmed, or turned on during a lecture. B. A research employs a confederate to act calm (Group Calm) or act concerned (Group Concerned) during a mock fire drill. The stress response of participants in each group is measured.
A. natural manipulation B. staged manipulation
State the type of quasi-experimental research design described in each of the following examples: A. A researcher records the time (in seconds) it takes a group of participants to complete a computer-based task following an online "how-to" course B. A researcher records the rate of traffic accidents on a section of highway each month for two years before and two years after the speed limit on that section of highway is reduced. C. A researcher records employee satisfaction before and after a training seminar. He compares satisfaction scores for employees at a local branch compared to the scores for those at the main branch who did not receive the seminar.
A. one-group posttest-only design B. Interrupted time series design C. Nonequivalent control group pretest-posttest design
State whether each of the following is an example of a sampling method used with quasi-experiments or one used with experiments: A. a researcher selects a sample of Olympic athletes from the summer and winter Olympic teams to compare differences in motivational styles between these groups of athletes. B. A researcher selects a sample of college students, then randomly assigns each student to one of two reading groups and compares comprehension scores in each group.
A. quasi-experiment B. experiment
State whether each of the following between-subjects designs is an example of a quasi-experiment or an experiment: A. A researcher compares differences in resilience among single, divorced, and married women. B. A researcher measures job satisfaction among employees randomly assigned to receive a small, moderate, or large bonus during the holidays.
A. quasi-experimental B. experimental
In a Solomon four-group design, the experimental group is Group _______, and the ________ control groups are included to account for possible confounds or extraneous factors that could be causing differences pre- to post treatment.
A; 3
________ can be figured out by following a logical path of reasoning about the goal. We want to describe a relationship among ALL members of a set considered together (i.e., if we were to change ONE member of the set, then that would have to affect the ____________ score of the distribution).
ANOVA; variability
_________ (analysis of variance) literally is an analysis of changes in data, based on the fact that __________ only deals with "differences" or "relative changes" between/among numbers.
ANOVA; variability
Single-case designs allow for greater freedom to ask innovative or new questions about treatments with unknown causes or with unknown costs or benefits. Why can a single-case design be an ethical appropriate research design to test the effectiveness of such treatments?
Because single-case designs are used with SMALL samples, thereby testing the treatment without exposing such a treatment to large groups of participants.
The ________ is a published document that recommended three principles for the ethical conduct of research with human participants: respect for persons, beneficence, and justice.
Belmont Report
___________ is an ethical principle listed in the Belmont Report that states that it is the researcher's responsibility to minimize the potential risks and maximize the potential benefits associated with conducting a research study.
Beneficence
A statistic used on significance tests for ANOVA is _______.
F
An _____________ is a review board that consists of at least one veterinarian, one scientist with experience using animals, and one public member from the community. These members review for approval animal research protocols
IACUC Institutional animal care and use committees
Research must be approved by an __________, who provides legal and ethical protections for researchers/subjects, before any data is collected.
IRB (Institutional Review Board)
The steps in the research protocol that MUST be first submitted to an IRB for approval include (1) _______________, (2) ___________, (3) ______________, and (4) _____________.
IRB approval; informed consent; conduct procedures; debriefing
An _____________ is a review board with at least FIVE members, one of whom comes from OUTSIDE the institution. These members review for approval research protocols submitted by researchers prior to the conduct of any research. EVERY institution that receives federal funding MUST have an __________.
Institutional Review Board (IRB); IRB
____________ is an ethical principle listed in the Belmont Report that states that all participants should be treated fairly and equitably in terms of receiving the benefits and bearing the risks in research.
Justice
A ______________ is a matrix design in which a limited number of order sequences are constructed such that (1) the number of order sequences equals the number of treatments, (2) each treatment appears equally often in each position, and (3) each treatment precedes and follows each treatment ONE time.
Latin square
Which type of factorial design includes the manipulation of a between-subjects factor AND a within-subjects factor?
Mixed factorial design
The _____________ is the first international code of research ethics consisting of 10 directives aimed at the protection of human participants.
Nuremberg Code
__________ is an ethical principle listed in the Belmont Report that states that participants in a research study must be autonomous agents capable of making informed decisions concerning whether to participate in research.
Respect for Persons
The ____________ is an experimental research design in which different participants are assigned to each of four groups in such a way that comparisons can be made to (1) determine if a treatment causes changes in posttest measure and (2) control for possible confounds or extraneous factors related to giving a pretest measure and observing participants over time.
Solomon four-group design
What is the biggest ethical challenge that researchers face?
The difficulty of anticipating what will happen in a study
In the notation, 2 x 3 x 4, what does this mean?
There are three factors (independent variables). The first factor has two levels, the second factor has three levels, and the third factor has four levels.
True or False: When we use random assignment, we can assume that the individual differences of participants are about the same in each group.
True.
Academic rank from lowest to highest in research is (1) __________ (part time), (2) _________ (full time, but non-tenure track), (3)_____________ (most new PhDs start here), (4)______________ (usually comes with tenure and is determined by how much research one has done), (5)_____________ (many schools stop with this rank), and (6)______________.
adjunct; lecturer; assistant professor; associate professor; full time professor; distinguished/endowed professor
Debriefing must occur immediately __________ research/experiment. It must describe the __________ of the research and reveal any __________. It must ________ potential danger or harm and provide contact information and IRB information.
after; purpose; deception; undo
_________ is where you set a p-value for accepting/rejecting the null hypothesis.
alpha
If you conducted a t-test and obtained a t-score of 3.91 (with 29 df), you could find multiple critical values in the table. Which one you use depends on 3 things: (1): what ________ is (.05 or .01) (2) whether it is 1 vs. 2 tailed (___________ tests or not) (3) need to draw the __________.
alpha; directional; degrees of freedom
The outcome of your study that indicates that you IV DID have an effect is the _________________ ( HA OR H1, H2, etc.). For example, if your null hypothesis is Red exam scores = Blue exam scores for testing room color (red vs. blue) affects on algebra performance, then your ____________ is: Red exam scores do NOT equal Blue exam scores, OR, Red exam scores are GREATER THAN Blue scores, OR, Red exam scores is LESS THAN Blue exam scores.
alternative hypothesis; alternative hypothesis
A _____________ is a type of measurement in which researchers directly observe and record the behavior of subjects or participants.
behavioral measure
Researchers often generalize observations in single-case experiments across __________, across _______ or ________, and across _________.
behaviors; subjects; participants; settings
A two by three by four (2 x 3 x 4) factorial design means there are 3 IV's, the first variable has two levels, the second has three levels, and the third has four levels. The factors are listed so that __________ subjects are first (so that you may see designs such as 3 x 4 x 2).
between
In a within-subjects design we can measure the mean difference caused by the manipulation as ______________, which is a source of variance in a dependent measure that is caused by or associated with the manipulation of the levels (or groups) of an independent variable.
between-groups variability
In a within-subjects design we can measure individual differences as _____________, which is a source of variance in a dependent measure that is caused by or associated with individual differences or differences in participant responses across all groups.
between-persons variability
We assume that the _________ variability is equal to zero in a within-subjects design because the SAME participants are observed in each group.
between-persons variability
In the one-way within-subjects ANOVA, we compute the ___________ variability and remove it, thereby leaving only ______________ variability as a measure of ________ in the denominator of the test statistic.
between-persons; within-groups; error
To conduct a _____________ design, participants are selected in a certain way, and a particular test statistic for two groups is used.
between-subjects
A _____________ is a research design in which different participants are observed one time in each group or at each level of a factor.
between-subjects design
A _______________________ is an experimental research design in which levels of a between-subjects design are manipulated, then different participants are randomly assigned to each group or to each level of that factor, and observed one time.
between-subjects experimental design
A ______________ is a type of factor in which different participants are observed in each group, or at each level of the factor.
between-subjects factor
A _______________ is a research design in which the levels of two or more between-subjects factors are combined to create groups, meaning that DIFFERENT participants are observed in each group.
between-subjects factorial design
The three types of factorial design are ______________ (a design in which all factors are between-subjects factors), a _______________ (a design in which all factors are within-subjects factors), and a ______________ ( a design with at least one between-subjects factor and at least one within-subjects factor).
between-subjects factorial design; within-subjects factorial design; mixed factorial design
For a ______________ design, a DIFFERENT set of participants is observed in each group. However, it is not always possible or practical to conduct such a study. Therefore, an alternative research design, called the _________________, also called a repeated-measures design, is a research design in which the SAME participants are observed ONE time in each group of a research study.
between-subjects; within-subjects design
In a _________-subjects research design, to be an experiment (allowing causal conclusions), a researcher must: (1) manipulate at least __________ independent variable(s) and have at least ________ levels. (2) ___________ assign subjects to conditions
between; one; two; randomly
Traditionally, __________ subjects factors are listed BEFORE ___________ subjects factors. When the number decreases (e.g., 2x3x2), the design has changed to a ________ design. If, however, the design does not decrease (e.g., 2x2, 3x4, 3x3x4, etc.), we do NOT know whether it is mixed or not.
between; within; mixed
Reasons for ADDING factors to an experiment include ________ on __________ research, ___________ for threats to validity, and enhancing the __________ of interpretation.
building; previous; controlling; informativeness
A ___________ is a possibility that there will be a lasting effect of one level of an independent variable on subjects' behavior in another level.
carryover effect
____________ are a threat to internal validity in which participation in one group "carries over" or causes changes in performance in a second group.
carryover effects
The __________ of an estimate is determined by the level of confidence: The _________ the level of confidence, the more __________ the estimate.
certainty; larger; certain
A ____________ is a single-case experimental design in which a baseline phase is followed by successive treatment phases in which some criterion or target level of behavior is changed from one treatment phase to the next. The participant must meet the criterion of one treatment phase before the next treatment phase is administered. This is for research situations in which we want to change a criterion or treatment AFTER the participant meets an initial criterion or responds to one particular treatment.
changing-criterion design
At baseline, Sam ate no bites, then Sam spun an arrow that displayed different criteria for a reward. He began with 1-2 bites, then 2-3 bites, 3-5 bites, 4-6 bites, and finally 5-6 bites in order to receive the reward. As the criterion was increased, so did Sam increase the number of bites he took of nonpreferred food. This demonstrates ___________________.
changing-criterion design (single-case)
According to the ____________, any risks to participants should be minimized and any benefits MUST outweigh any risks.
code of ethics
A ___________ is a group of individuals who share common statistical traits or characteristics, or experiences within a defined period (e.g., group of people born in the same year, served in the same war, attended the same school, etc.).
cohort
A ___________ is a threat to internal validity in which differences in the characteristics of participants in different cohorts or age groups confound or alternatively explain an observed result.
cohort effects
A disadvantage of the cross-sectional design is the possibility of _____________, which occur when preexisting differences between members of a cohort can explain an observed result. For example, suppose we use a cross-sectional design to measure how often 20-year-olds, 40-year-olds, and 80-year-olds send text messages. We are likely to find that texting decreases with age. However, there is also a ____________ due to the generational gap in advances of technology (i.e., differences in experience of familiarity with texting across the life span).
cohort effects; cohort effect
A ______________ is a quasi-experimental research design that combines longitudinal and cross-sectional techniques by observing different cohorts of participants over time at different or overlapping ages.
cohort-sequential design
You figure out a research topic by using ____________. In other words, you do not take things for granted or rely on _____________ or tradition for understanding the world. For example, testing tenacious ideas like "you should not should not go swimming for at least an hour after eating." Just because everyone "knows" something, does NOT mean that it is ___________.
common sense; familiarity; known
____________ is a procedure in which ALL possible order sequences in which participants receive different treatments or participate in different groups are balanced or offset in an experiment. This is used when the number of treatments or different groups is SMALL. In contrast, ____________ is a procedure in which SOME, but NOT all, possible order sequences in which participants receive different treatments or participate in different groups are balanced or offset in an experiment. This is used when there are THREE or MORE groups.
complete counterbalancing; partial counterbalancing
A _______________ is a factorial design in which each level of one factor is combined or crossed with each level of the other factor, with participants observed in each cell or combination of levels.
complete factorial design (completely crossed design)
__________ counterbalancing involves using _________ possible orders (e.g., with 2 conditions, one would have 2 orders--2x1--with 3 conditions, one would have _______ orders--3x2x1--with 4 conditions, one would have ________ orders--4x3x2x1--and so on).
complete; all; 6; 24
The "3 C's" of an effective literature review is be _____________, _____________, and ____________.
comprehensive; critical; clever
Degrees of freedom for the F test are as follows: For treatment it is (__________ -_____); for error, it is (__________ in the comparison - ______). F ( df T, df E) = ??? , p < .05
conditions; 1; scores; 1
A ____________ is a coresearcher or actor who pretends to be a participant in a research study for the purposes of scientific investigation.
confederate
_____________ refers to data that is tied with IDENTIFYING information (e.g., a name with a score). ____________, on the other hand, refers to data that is SEPARATE from identifying information; therefore, scores/results CANNOT be attributed to any specific individual.
confidentiality; anonymity
___________ in a research design is (a) the manipulation of a variable and (b) holding all other variables constant. When this is LOW, neither criterion are met. When this is HIGH, both criteria are met.
control
_______________ is a type of restricted random assignment in which we limit which participants are included in a sample based on characteristics they exhibit that may otherwise differ between groups in a study.
control by holding constant
________________ is a type of restricted random assignment in which we assess or measure the characteristic we want to control, group or categorize participants based on scores on that measure, and then use a random procedure to assign participants from each category to a group in the study. For example, children may first be separated into groups based on intelligence and then children from each intelligence group are then randomly assigned to a group in the experiment.
control by matching
The _____________ is a condition in an experiment in which participants are treated the same as participants in an experimental group, EXCEPT that the manipulation believed to cause a change in the dependent variable is OMITTED.
control group
One-group quasi-experimental research designs lack a _____________, because only one group is observed. Therefore, the study does not demonstrate ________ and is a __________________.
control group; cause; quasi-experiment
A _____________ is a basic or interrupted time series quasi-experimental research design that also includes a nonequivalent control group that is observed during the same period of time as a treatment group, but does NOT receive treatment.
control time series design
The ____________ is the same as the interrupted time series design, except there is a group NOT receiving the treatment. The idea is that, if groups differ AFTER the treatment, the difference is likely due to the TREATMENT rather than outside influences (outside influences should happen to both group regardless of the treatment).
control time series design
Using ________________ ensures that the number of possible order sequences is the SAME in each group. For example, with two groups (A and B), half participate first in Group A, then in Group B; and half participate first in Group B, then in Group A. The order of participation is NOT a threat to internal validity in this case because the SAME number of participants in each group have each possible order sequence. HOWEVER, there may be a reason to believe that one order sequence is associated with LARGER changes than the other.
counterbalancing
Using a ____________ procedure, the order in which participants receive different treatments or participate in different groups is balanced or offset in an experiment. Two types of this are __________ and ______________.
counterbalancing; complete counterbalancing; partial counterbalancing
The ______________ design involves observing different age groups, called __________, at one time. Unlike a longitudinal design, this design does NOT require using the same participants over time, because participants are only observed once.
cross-sectional design; cohorts
A ____________ is a quasi-experimental research design in which participants are grouped by their age and participant characteristics are measured in each age group. This design is similar to a _________________ in that the different age groups act as nonequivalent control groups, however, NO treatments are given. Each age group is called a __________, which is any group of individuals who share common statistical traits or characteristics, or experiences within a defined period. The goal is to see if differences exist between ____________. Beware of ___________, which are pre-existing differences between groups beyond age difference (e.g. expertise--comfort with technology: as people get older they are LESS comfortable with technology).
cross-sectional design; nonequivalent control group quasi-experimental design; cohort; cohorts; cohort effects
A _________ is the full disclosure to participants of the true purpose of a study, and is given at the END of a study.
debriefing
_____________ is concealment of study purpose or keeping certain information from the participants. This can be active (deliberately untruthful) or passive (omission of key information). This must be crucial to the research a
deception
_________ are one less than the sample size, or n-1.
degrees of freedom (df)
_________ almost literally means something like freedom of MOVEMENT. All scores EXCEPT the _______ are "free to vary" (n-1).
degrees of freedom (df); last
One way to minimize problems related to having no control or comparison group is to measure the same __________ variable in one group of participants before (pretest) and after (posttest) a treatment. Using this type of research design, called a __________________, we measure scores before and again following a treatment, then compare the difference between pretest and posttest scores. This design is susceptible to threats to _________ validity because the one-group design does not include a _____________.
dependent; one-group pretest-posttest design; internal; comparison group
For a 2 x 2 factorial design, we can illustrate main effects in the row and column totals of a 2 x 2 table summary. For example, Factor A (row) is whether students studied for a quiz (no, yes) and Factor B (column) is their class attendance (high, low). The main effect for each factor reflects the _____________ between the row and column totals in the table.
difference
The nonequivalent control group pretest-posttest design, like the posttest-only design, demonstrates that a treatment is associated with _____________ between groups, and NOT that a treatment CAUSED _______________ between groups, if any were observed.
differences; differences
In a between-subjects design, are the same or different participants observed at each level of a factor?
different participants
Predicting ANY difference is fine in an __________ study (non-directional). Non-directional tests are "________" distributions. Typically, studies are designed to see if a specific outcome will occur, in which case you should be making DIRECTIONAL predictions (specifying HOW groups will differ rather than just that they WILL differ). Directional tests can use "_________" distributions.
exploratory; two-tailed; one-tailed
In research, ___________ is to concoct methods or data that misrepresent aspects of a research study with the intent to deceive others.
fabrication
A _________________ is a research design in which participants are observed across the combination of levels of two or more factors.
factorial design
A _____________ is a research design in which groups are created by manipulating the levels of two or more factors, then the same or different participants are observed in each group using experimental procedures of randomization (for a ____________ factor) and using control for timing and order effects (for a __________ factor).
factorial experimental design; between-subjects; within-subjects
The advantage of using a within-subjects design is that _________ participants are required overall (economizing) because the SAME participants are observed in each group. Another advantage is that the test statistic has greater power to detect significant __________ between groups. In other words, using a within-subjects design, if an effect exists in the population, then we are _______ likely to detect it than if we ran a _________ analysis on the same data. Finally, the test statistic for a within-subjects design can detect __________ differences between groups because the denominator of the test statistic is _________, thereby making the value of the test statistic ______ than it would have been if we have included between-persons variability in our calculation of __________.
fewer; differences; more; between-subjects; smaller; smaller; larger; error
When an ODD number of conditions is used for systematic counterbalancing (i.e., five conditions that typically have 120 orders), ____________ it over in order to reduce the number of orders to __________.
flip; ten
Consider a class of 5 research methods students who get a combined exam score of 400; 400/5 = 80% average. What are their INDIVIDUAL scores? You must know the first ________ scores in order to know all the scores (i.e., with 5 scores, there are _______ df). All scores, except the last, are "free to vary" (n-1).
four; 4
Many reversal designs are at least _______ phases, or _______, so as not to return to baseline to end an experiment.
four; ABAB
The advantage of including a preexisting factor (participant weight) in a factorial design is to test the _____________ of the findings across the levels of the quasi-independent variable. For example, to interpret the interaction, researchers concluded that overweight children ate _________ when eating alone compared to in a group; in contrast, lean children ate ________ in a group than when eating alone. Hence because the researchers included the quasi-independent factor (participant weight), they determined that their manipulation (eating alone or in a group) ____________ to _________ children (who ate more in a group than alone) and differently to overweight children (who ate more alone than in a group).
generality; more; more; generalized; lean
A researcher uses a single-case design to record the number of minutes spent studying in a baseline phase and a calming music treatment phase with a student who studied in a library and the same student who studied in a college dormitory room. Based on this description, can the researcher generalize across behaviors, across participants, or across settings?
generalize across settings
In a factorial design, the LARGER the number of levels combined to create groups, the ____________ the demands on the participants. For a 3x3 within-subjects factorial design, we would observe the same participants in ________ groups. This can place undue burden on a participant.
greater; nine
The GREATER the demands in each group, the __________ the burden on the participants. For example, if we observe the same participants in different physical exercise groups, then we would be concerned for ______________.
greater; participant fatigue
If the data in baseline are stable, and there is a change in trend from baseline (A) to treatment (B), a graph will illustrate a study with ___________ internal validity.
high
An immediate change in level indicates _________ internal validity. No immediate change in level indicates ___________ internal validity. A clear change in pattern or trend from baseline to treatment indicates _______ internal validity. A continuation of the pattern or trend observed in baseline indicates __________ internal validity.
high; low; high; low
In a certain study, researchers recorded how well participants comprehended a passage that varied by type of highlighting (none, appropriate, inappropriate) and text difficulty (low, high). Researchers predicted that participants would have _________ comprehension scores for passages that were low in difficulty with no highlighting and predicted that comprehension scores would be __________ for passages that were high in difficulty with inappropriate highlighting. The hypothesis was supported in that comprehension scores were ___________ in Group None-Low and were _________ in Group-Inappropriate-High. Using the between-subjects factorial design, we can determine the total number of participants (N) in an experiment by multiplying the ____________ by the number of _________ for each ____________. In this experiment, we had ___________ factors, one with _________ levels and one with ________ levels. Suppose that 10 participants were observed in each group. In this case, the total number of participants in the experiment would be as follows: 10 x _____ x _____ 2 = _________ participants.
high; lowest; highest; lowest; sample size; levels; factor; two; three; two; 3; 2; 60
A _____________ is a research design in which the levels of more than two factors are combined or crossed to create groups.
higher-order factorial design
A _____________ is an interaction for the combination of levels of three or more factors in a factorial design.
higher-order interaction
In a three-way factorial design we could observe any combination of 3 main effects, 2 two-way interactions, and 1 three-way interaction. The three-way interaction is called a ________________ because it shows that the combination of levels of more than two factors are associated with changes in a dependent variable.
higher-order interaction
Publication requirements vary (most research comes from college or university settings). A ___________ level indicates _________ publishing such as "publish or perish schools."
higher; more
If the averages are __________, the factor has NO main effect. However, if the averages are __________, the factor DOES have a main effect. ALSO, When the differences among the differences are NOT different (and when lines plotted ARE parallel), there is NO __________. However, when the differences among the differences ARE different (and when lines plotted are NOT parallel), there IS an ______________.
identical; different; interaction; interaction
Although it seems simple to interpret behaviors as the result of single factors (e.g., the "he/she" is a jerk" syndrome), in reality there are MANY factors that contribute to behavior at the same time. Because behavior is the result of MANY influences in the real world, ______________ the number of independent variables USUALLY increases __________ validity.
increasing; external
__________ the interval between treatments or groups, while also __________ the total duration of an experiment, is often the MOST effective strategy for minimizing common threats to internal validity that are associated with observing the SAME participants over time (____________ experimental design).
increasing; minimizing; within-subjects
Using the between-subjects design, different participants are observed in each group. When participants are observed in this way, the sample is called an _______________.
independent sample
The purpose of a quasi-experimental design is to increase confidence that the results are due to the _______________ (not a confound). Ideally, researchers strive to use the strongest quasi-experimental design (sometimes there are constraints beyond a researcher's control such as not having enough subjects).
independent variable
The steps in a research design and execution are (1) select an ______________, consider its _________ to the research question, and consider the number of _________ (treatments) (2) Decide if you need a ____________ group (determine _________ strength and decide HOW to __________ the independent variable--____________ subjects or not).
independent variable; relevance; levels; control; manipulation; manipulate; between
Whatever # we come up with for ANOVA should: Be _____________ of the mean. In other words, we only care about the __________ of the #'s. Consider the following sets of #'s: 886, 887, 888, 889 VS. 6, 7, 8, 9. Both sets of #'s are equally "__________." So the variability measure should be __________--NOT bigger for the first set of numbers just because their mean is bigger.
independent; scatter; scattered; the same
When scores in each group DO overlap, ____________ become a likely explanation for an observed difference between two or more groups in an experiment.
individual differences
In a two-way factorial design, error variance is a measure of the ____________ differences, or differences in _______________, within each cell of the table.
individual; participant scores
A within-subjects design can minimize __________ differences between groups and reduce ___________ in an experiment.
individual; variability
With ______________, participants must be informed about ANY part of the research that may affect his or her willingness to volunteer. Not all information must be given, as you don't want to give away the hypothesis or bias subjects.
informed consent
A significant _____________ indicates that group means significantly vary across the combined levels of two factors.
interaction
The advantage of using a cohort-sequential design (a combination of longitudinal and cross-sectional designs) is that researchers can account for threats to ____________ validity associated with ____________ participants over time because part of the sample is a ____________ of age groups. Also, it can account for ______________ because part of the sample includes the same participants observed over time in each age group or cohort.
internal; observing; cross section; cohort effects
An __________________ is a quasi-experimental research design in which a dependent variable is measured at MANY different points in time in one group before and after a treatment that NATURALLY occurred. Examples of a naturally occurring treatment or event include a scheduled medical procedure, a wedding, a natural disaster, a change in public policy, a new law, and a political scandal. The events are BEYOND THE CONTROL of the researcher.
interrupted time series design
Researchers can control timing by controlling the ___________ between treatments or groups and the total ____________ of an experiment.
interval; duration
Whatever # we come up with for ANOVA should: Be _____________ (i.e, the variability # should be ________ when #'s are CLOSE together and ________ when #'s are spread apart). For example, the group of #'s: 1, 50, 800 VS. the group of #'s 1, 5, 8. The variability # for the first group of #'s should be ________ while the variability # for the second group of #'s should be __________.
intuitive; small; large; large; small
The single-case design is regarded as an experimental research design, because:
it meets the 3 key elements of control required to demonstrate cause-and-effect: randomization, manipulated, and comparison
AN IRB must demonstrate ____________ for risk, must consider ___________ methods to reduce risk, and must consider ___________ to _________ ratio.
justification; alternative; benefits; risks
When you see a p-value, think of it as representing the # of "minority" marbles out of 100. The ________ the p-value, the more likely you could have pulled that marble out of the jug by CHANCE. Therefore, the ________ the p-value, the LESS CONFIDENT you are that results are real (instead of by chance). Conversely, the _______ the p-value, the MORE CONFIDENT your outcome was NOT due to chance, but WAS due to the independent variable.
larger; larger; lower
In systematic counterbalancing, "N" is the ________ condition. Therefore, in a study using 6 conditions (ABCDEF), and using the formula C1, C2, Cn, C3, Cn-1, C4, Cn-2, C5, etc., Cn would be considered condition _________.
last; F
To establish a criterion for a decision, we state a ____________ for a test. For most studies, this is _________. To determine the likelihood or probability of obtaining a sample outcome, if the value stated in the null hypothesis is TRUE, we compute a __________, which is used to find the _________, which is the actual probability of obtaining a sample outcome if the null hypothesis is TRUE. We reject the null hypothesis when p is ____________ to .05; an effect _________ significance. We retain the null hypothesis when p is ______________ .05; an effect __________ significance.
level of significance; .05; test statistic; p-value; less than or equal to; reached; greater than; failed to reach
When the levels of the dependent variable in the baseline phases are obviously less than or greater than the levels of the dependent variable in the treatment phase, a change in ____________ is displayed graphically. When the direction or pattern of change in the baseline phases is different from the pattern of change in the treatment phase, a change in _________ is displayed graphically. In the typical case, a dependent variable gradually increases or decreases in the ______________ phase, but is stable or does not change in the ______________ phases.
level; trend; treatment; baseline
One drawback of having only two levels of an independent variable (factor) is that we can only express ___________ relationships (not _____________). When more than one independent variable is manipulated within a single experimental context, we have a ____________ design.
linear; curvilinear; factorial
The ____________ design involves observing the same group of participants over time.
longitudinal design
A ____________________ is a quasi-experimental research design used to study changes across the life span by observing the SAME participants at different points in time and measuring the same dependent variable at each time. This design is similar to the ______________ in that one group of participants is observed over time, however, NO treatments are given. The goal is only to see what CHANGES over time. Beware of ______________ (drop-outs change the sample) and _______________ (learn/familiar with the measure), and _______________ (time and money).
longitudinal design; one-group pretest-posttest quasi-experimental research design; attrition; testing effects; expensive
Dukes and McGuire used a multiple-baseline design to measure the effectiveness of a sex education intervention, which they administered to multiple participants with a moderate intellectual disability. The researchers recorded participant knowledge of sexual functioning using the Sexual Consent and Education Assessment on which higher scores indicate greater ability to make decisions about sex. Each participant was given a baseline phase for a different number of weeks. Scores were ___________ in this baseline phase. Only after the baseline period ended and the intervention was administered did scores on the scale _____________. Scores remained ________ for four weeks after the program ended. Hence, the results showed a change in level from baseline to intervention for each participant. Each participant in the sex education study received the intervention (or the treatment) in successive weeks. Because the treatment was administered at different times, and changes in the ________ variable only occurred once the treatment was administered, the pattern showed that the __________ caused the observed changes in the scores.
low; increase; high; dependent; treatment
A single-case design is typically associated with __________ population validity. In other words, it is not possible to know whether the results in the sample would also be observed in the population from which the sample was selected because single-case experimental designs are associated with very _______ sample sizes. However, the results in a single-case design can have ___________ external validity in terms of generalizing across behaviors, across subjects or participants, and across settings.
low; small; high
___________ is the size of the change in a dependent measure observed between phases of a design. The larger the ________ of changes in a dependent measure between phases, the higher the internal validity of a research design.
magnitude; magnitude
A researcher tests whether the lighting (dull, bright) and background noise (low, high) in a classroom influence student performance on an exam. He conducts a two-way factorial design to analyze the data and finds a significant effect of lighting, with higher scores when the lighting was bright. Is the effect described in this study an example of a main effect or an interaction?
main effect
A significant ___________ indicates that group means significantly vary across the levels of one factor.
main effect
The changes in a dependent variable across the levels of a single factor are called a _______________.
main effect
Which effect, a main effect or an interaction, can be identified in the row means and column means of a table summary?
main effect
Treatment differences among the levels of a single factor (independent variable) are ______________. When results for any level of one factor DEPEND on the level(s) of another factor OR "when the differences among the differences are different," this indicates an ______________.
main effect; interaction
A ___________ is a source of variation associated with mean differences across the levels of a single factor. In a table summary, a _____________ is a measure of how the row and column means differ across the levels of a single factor.
main effect; main effect
Consider a factorial design in which we manipulate whether the setting (familiar, unfamiliar) and the duration (1 hour, 2 hours) of a treatment influence its effectiveness to reduce (lower) social anxiety. When the ____________ are significant, we look at the row and column means to describe the effect.
main effects
Using a two-way factorial design, we can identify three sources of variation: two ______________ (one for Factor A and one for Factor B), one _____________ ( the combination of levels of Factor A and B), and ________________ (variability attributed to individual differences).
main effects; interaction; error variance
For a within-subjects factorial design to be an experiment, the researchers must have _________ the levels of each factor, __________ the levels of the two factors to create the groups, and _________ for ______ effects due to observing the same subjects in each group or in each environment, then compare group differences in the dependent variable.
manipulated; crossed; controlled; order
In a certain study, researchers recorded how well participants comprehended a passage that varied by type of highlighting (none, appropriate, inappropriate) and text difficulty (low, high). For the between-subjects factorial design to be an experiment, the researchers must have ____________ the levels of each factor (i.e., type of highlighting and text-difficulty), __________ the levels of the two factors to create the groups (i.e., 3x2 = 6 groups), and randomly assigned ____________ participants to each group (and compare group differences in the dependent variable---________________).
manipulated; crossed; different; comprehension
For a mixed factorial design to be an experiment, researchers must have _____________ the levels of each factor, __________ the levels of the two factors to create the groups, randomly assign __________ participants to each level of the between-subjects factor, and _________ for ________ effects due to observing the same participants at each level of the within-subjects factor, then compare group differences in the dependent variable.
manipulated; crossed; different; controlled; order
If groups are different, then the ___________ is causing the effect. However, if people are different, then the _____________ are causing the effect.
manipulation; individual differences
The __________ design is a within-subjects research design in which participants are matched, experimentally or naturally, based on preexisting characteristics or traits that they share. This type of design uses a ____________ sampling method.
matched-samples design; quasi-experimental
To figure out a # to represent variability: (1) find the _______ of the sample source(s) (e.g., the mean of 3, 6, 2, & 5 is 4). (2) _________ the mean from EACH independent source of the sample (NOTE: do not ADD the resulting differences, because you will get ________). THEREFORE: (3) To get rid of the negative values, we __________ the differences instead and add all of that up E.g. • 3 (source 1) - 4 (mean) = -1. Now, square (-1): (-1) ^2 = 1 • 6 (source 2) - 4 (mean) = 2. Now, square 2: 2^2 = 4 • 2 (source 3) - 4 (mean) = -2. Now square (-2): (-2) ^2 = 4 • 5 (source 4) - 4 (mean) = 1. Now square 1: 1 ^2 = 1 _______ all squared values and you get 10.
mean; subtract; zero; square; add
The main purpose for ethics/IRB is to ____________ the rights of researchers versus the rights of participants. Participant rights ALWAYS _____________ researchers rights.
mediate; supercede
_______________ still must be concerned about treatment of participants.
minimal risk research
A ________________ is a research design in which different participants are observed at each level of a between-subjects factor and also repeatedly observed across the levels of the within-subjects factor.
mixed factorial design
When more than one independent variable is manipulated AND at least one is manipulated within-subjects and at least one other is manipulated between-subjects, this is called a ____________ design. It is statistically ________ complicated than between or within designs, and experimentally tends to take the ________ characteristics of both designs.
mixed; more; best
The _________ conditions used, the more subjects you will need to recruit and test. In a __________-subjects design, no matter how much you control and randomly assign subjects to conditions, there is ALWAYS going to be group ___________ above and beyond the independent variable. This means more _____________ which makes it harder to find a statistically significant difference (if there is one).
more; between; differences; error variability
Factorial designs are ___________ informative in that they allow us to observe the ______________ of _____ or more factors.
more; interaction; two
Analysis of variance (ANOVA) for a WITHIN subjects design is statistically ___________ powerful than BETWEEN subjects. This means that an analysis based on a BETWEEN might ___________ show significant outcomes.
more; not
________________ is used when it is not possible for changes in a dependent variable to return to baseline.
multiple baseline design
___________, a type of single-subject (low-n) designs occurs when you can't (when it is impossible or unethical) reverse behaviors due to treatment. This may occur across __________ (examine several participants over time & introduce treatment at different times for different participants), across ___________ (examine many behaviors in an individual and give treatments targeting each behavior, one at a time), and across ___________ (give treatment at different times per setting, in which a change in behavior should ONLY occur as the treatment is applied).
multiple baseline designs; participants; behaviors; situations
A ____________ is a single-case experimental design in which a treatment is successively administered over time to different participants, for different behaviors, or in different settings. This is for situations in which it is not possible for changes in a dependent variable to return to baseline levels following a treatment phase.
multiple-baseline design
The start of a treatment phase varies using the _____________ to determine if the changes in a dependent variable begin only after each baseline phase ends.
multiple-baseline design
The strength of a Solomon four-group design is that ________ comparison(s) can be made to eliminate the possibility that confounds or extraneous factors are causing differences from pre- to post treatment. To demonstrate that the treatment caused changes in posttest scores, we should find a significant difference for Comparison ______ and zero difference for the remaining comparisons.
multiple; one
The number of total conditions is determined by ____________ the levels ( 2 x 2 = 4; 2 x 3 = 6; 3 x 3 = 9; etc.).
multiplying
A ____________ is the manipulation of a stimulus that can be naturally changed with little effort.
natural manipulation
__________ matched pairs match participants based on PREEXISTING characteristics inherent to each individual. For example, researchers observe pairs of identical twins. The twins then participate in a study in which the differences in scores for each pair of identical twins are compared. In CONTRAST, ___________ matched pairs match participants based on their SCORES on a dependent measure for SOME characteristics of interest. For example, researchers measure intelligence in a sample of children, then matches them based on their scores. Participants with the two highest scores are paired, then the next two highest scores are paired, and so on. Children then participate in a study in which the differences in scores for each matched pair are compared.
naturally occurring; experimentally
If a subject cannot understand the aspects of a study, there is __________ consent. Children and mentally impaired CANNOT give informed consent. However, ___________ is the consent of minor or other legally incapable person to agree to participate in research ONLY after receiving an appropriate explanation in reasonably understandable language.
no; assent
Suppose a professor gives a new lecture method to your research methods class and gives a traditional method in another research methods class, then compares grades on the topic lectured. The classes are matched on certain characteristics: Both classes are on the same topic (research methods), offered at the same school, and taught by the same professor. However, the class taught using the traditional method is a _______________ because students in that class chose to enroll in the class, so they were not randomly assigned to that class.
nonequivalent control group
A _____________________ is a quasi-experimental research design in which a dependent variable is measured following a treatment in one group and also in a nonequivalent control group that does not receive the treatment. The ______________ will have characteristics similar to the treatment group, but participants will NOT be randomly assigned to this group. Therefore, selection of a good comparison group is CRITICAL. This method is susceptible to _____________, because we do not know how similar or different the groups were to start, so differences between groups may or may not mean that the intervention had an effect. Also, students enrolled in their college classes, not the researcher, so students controlled which class they enrolled in.
nonequivalent control group posttest-only design; nonequivalent control group; selection differences
A __________________ is a quasi-experimental research design in which a dependent variable is measured in one group of participants before (pretest) and after (posttest) a treatment and that same dependent variable is also measured at pretest and posttest in another _____________ that does not receive treatment.
nonequivalent control group pretest-posttest design; nonequivalent control group
When a participant becomes tired of participating further in an experiment, they are experiencing _______________. This is one of many _________-related factors (others include maturation, regression toward the mean, testing effects, and attrition).
participant fatigue; time
The outcome of your study that indicates that your independent variable had NO effect is the _____________ (Ho). There can still be a _________, but there's either an ______ or not an ________). For example, if you test room color (red vs. blue) affects on algebra performance, then your _______________ is: Red exam scores = Blue Exam scores. This MUST be proven wrong.
null hypothesis; difference; effect; effect; null hypothesis
The ___________ is a statement about a population parameter, such as the population mean, that is ASSUMED to be TRUE, but CONTRADICTS the research hypothesis. This is a starting point. We will test whether the value stated in the __________ is likely to be true.
null hypothesis; null hypothesis
Time series quasi-experimental research designs involve many ____________ made before and after a treatment.
observations
You figure out a research topic by ___________ the world. You look for _________ relationships that appear to exist in your daily living patterns and/or the events that surround you (e.g., is there a connection between cell phones and loneliness?). You also figure out a research topic by using __________ problems. ________ research areas are hot nowadays (e.g., man-machine interface, ergonomics, etc.). For instance, is there a better design for TV universal remotes?
observing; causal; practical; applied
To be statistically significant, your __________ value must BEAT (be greater than) the ____________ value. ALWAYS look at .05 (both) on a critical value table. One signals ___________ (one-tail) and one signals ___________ (two-tail). Typically, one-tail is _________ to beat.
obtained; critical; directional; non-directional; easier
To select participants for an experiment, it is most appropriate to sample from ___________ population(s), and then randomly assign participants to different groups.
one
A _______________ is a quasi-experimental research design in which a dependent variable is measured for ONE group of participants FOLLOWING a treatment. This is also called the "one-shot case study." For example, after a professor gives a lecture (the treatment), he may record students' grades on an exam out of 100 possible points (the dependent variable) to test their learning. This design is susceptible to many threats to _____________ validity and has NO ____________ group. With this design, one cannot be certain that a change even occurred (or if it did, whether it was positive or negative). E.g. personality test for remaining WWII veterans to see if they have any lingering effects of the war.
one-group posttest-only design; internal; comparison
What test statistic is used to compare differences in interval or ratio data between two or more groups to which different participants were assigned?
one-way between-subjects ANOVA
When data are interval or ratio scale, the appropriate test statistic for comparing differences among two or more independent samples is the __________________, which is a statistical procedure used to test hypotheses for one factor with two or more levels concerning the variance among group means. The test is used when different participants are observed at each level of a factor and the variance in a given population is UNKNOWN.
one-way between-subjects analysis of variance (ANOVA)
The _______________ is a statistical procedure used to test hypotheses for one factor with two or more levels concerning the variance among group means. The test is used when the SAME participants are observed at EACH level of a factor and the variance in a given population is unknown.
one-way within-subjects ANOVA
In the ratings-of-wine experiment, we would use the __________ to determine if the manipulation (describing the wine as inexpensive, moderately expensive, or expensive) caused mean ratings of liking to vary between groups. We would then use ________ to determine which pairs of group means were different.
one-way within-subjects ANOVA; post hoc tests
The ___________ informs us ONLY that at least one group is different from another group---it does NOT tell us which pairs of groups differ. In situations in which we have MORE than two groups in an experiment, we compute _____________ tests, which are used to evaluate all possible pairwise comparisons, or differences between ALL possible pairings of two group means.
one-way within-subjects ANOVA; post hoc tests
A related-samples t test reduces the data to _________ column(s) of _______ scores, thereby eliminating the between-persons variation.
one; different
A between-subjects design allows for participants to be randomly assigned to ________ group(s), but can require __________ sample sizes, particularly with _________ group(s).
one; large; many
A statement in the results section of "no other outcomes were significant" is __________, but can ________ uncertainty.
optional; reduce
Counterbalancing does NOT eliminate ___________; rather, it balances these effects, such that they are EQUAL or the SAME in each treatment or group.
order effects
To control for time-related factors, researchers make efforts to control for ____________, which are a threat to internal validity in which the order in which participants receive different treatments or participate in different groups causes changes in a dependent variable.
order effects
__________ is the between-subjects variable (i.e., half the participants experienced Group X, then Y, and half experienced the reverse order: Group Y, then X. However, ___________ is the within-subjects variable (the same participants were observed in each group).
order; groups
The problem when we observe the SAME participants in each group, and do not randomly assign participants to one group, is that we must control for effects resulting from the __________ that participants are observed, which can threaten _______ validity; this concern is NOT a problem using a __________ design.
order; internal; between-subjects
Imagine we are measuring liking for an unsweetened drink and a sweetened drink in a within-subjects design. One possible threat to internal validity would be the __________ in which participants are given the drinks (sweetened, then unsweetened; vs. unsweetened, then sweetened). To control for this threat to internal validity, we include ___________ as a ____________-subjects variable. Results show that there is NO mean __________ in ratings at each level of _______. Hence, we controlled for ________ effects (showed no difference), while also showing a mean __________ between the groups of interest.
order; order; between; difference; order; order; difference
To control for order effects and carryover effects (two threats to internal validity), researchers can use two strategies: control ___________ (counterbalancing, partial counterbalancing) and control ____________.
order; timing
With a methodological level of control, _________ and ________ is controlled. However, with a statistical level of control, computing a ___________ is controlled.
order; timing; test statistic
The probability you would get a result by chance is the __________ (e.g., imagine a container that has 100 marbles in it: 99 marbles are one color & 1 marble is a different color; the two colors are red & white; you take a marble out & it is red). You should feel that 99 of the marbles are probably _______ because it doesn't seem likely that you got the ONLY red marble by chance alone.
p-value; red
A ____________ is a statistical comparison for the difference between two group means. A post hoc test evaluates all possible pairwise comparisons for an ANOVA with any number of groups.
pairwise comparison
To identify an interaction, we graph the cell means and look to see if the lines in the graph are ____________. A possible interaction is evident when the lines are NOT ______________.
parallel; parallel
A state of physical or psychological exhaustion resulting from intense research demands typically due to observing participants too often, or requiring participants to engage in research activities that are too demanding is ________________.
participant fatigue
Often, researchers will include a quasi-independent variable called a _____________ , which is related to or characteristic of the personal attributes of a participant (i.e., when it relates to the hypothesis being tested). Examples include intelligence level, age, gender, race, ethnicity, education level, body weight, etc.
participant variable
You figure out a research topic by using __________ research. Many articles point out "the next step" or the possible implications of their results to other theories and/or situations. These are ___________ and are often worth researching. You might think the researcher(s) missed something critical or did something wrong in their study that YOU can follow up on.
past; freebies
The reversal single-case design is structured into _____________, represented alphabetically with an A or a B. Each consists of many observations or trials. The researcher begins with a _______________, in which no treatment is given, then applies a treatment in a second phase (B), and again returns to a _______________ in which the treatment is removed.
phases; baseline phase (A); baseline phase (A)
A ______________ is a type of measurement in which researchers record physical responses of the brain and body in a human or an animal.
physiological measures
A ____________ is a statistical procedure computed following a significant ANOVA to determine which pair or pairs of group means significantly differ. These tests are needed with more than two groups because multiple comparisons must be made.
post hoc test
The _________ of an estimate is determined by the range of the confidence interval: The __________ the range of an interval, the more ___________ the estimate.
precision; smaller; precise
___________ (written) must be first obtained before releasing data that contains any potential identifiers of participants.
prior consent
Theory based research is used as you develop expertise and familiarity with the relevant literature. You get to know what needs to be done next to contribute to the field. This is called _____________ research in that you follow an underlying "program" of study.
programmatic
A ___________ is the tendency for editors of peer-reviewed journals to preferentially accept articles that show positive results and reject those that show only negative results. This is also referred to as the __________ because researchers have a tendency to file away studies that show negative results.
publication bias; file drawer problem
A ________________ resembles between-subjects manipulations except participants are NOT randomly assigned to groups. These designs do NOT have the necessary controls to rule out threats to INTERNAL validity and are used to approximate (via inference) control features of true experiments. However, they ARE good for cases in which a true manipulation is either IMPOSSIBLE or UNETHICAL.
quasi-experimental design
A _________________ is the use of methods and procedures to make observations in a study that is structured similar to an experiment, but the conditions and experiences of participants lack some control because the study lacks random assignment, includes a preexisting factor (i.e., a variable that is not manipulated), or does not include a comparison/control group.
quasi-experimental research design
When a factor in a study is not manipulated (i.e., quasi-independent), this typically means that the study is a type of ___________________.
quasi-experimental research design
When all factors in a factorial design are _____________, then design is NOT an experiment because no factor is manipulated. For example, if we measure health scores among participants of different sex (male, female) and health insurance status (insured, uninsured), then the study is a _____________ because the participants determined their group assignment; both factors were ______________.
quasi-experimental; quasi-experiment; preexisting
A ________________ is a preexisting variable that is often a characteristic inherent to an individual, which differentiates the groups or conditions being compared in a research study. Because the levels of the variable are preexisting, it is not possible to randomly assign participants to a group.
quasi-independent variable
A quasi-experiment resembles an experiment, but includes a ______________ and/or lacks a ____________. Also, additional unique characteristics of participants could be causing differences between groups. When other possible causes cannot be ruled out, the design does not demonstrate case. Therefore, it is not an experiment.
quasi-independent variable; control group
When a __________ variable is included in a factorial design, we do NOT show cause and effect for any effect that involves the ____________ factor. INSTEAD, we conclude that the levels of the ___________ are related to or associated with changes in a dependent variable.
quasi-independent variable; quasi-independent; quasi-independent variable
___________ allows researchers to assume that the individual differences of participants are about the same between groups.
random assignment
For a single-case design to be an experimental design, it must meet the following three key elements of control required to draw cause-and-effect conclusions: ______________; _____________ (of variables that operate in an experiment); _____________.
random assignment; manipulation; comparison/control group
____________ is measured by determining the extent to which scores in each group overlap. The more that scores in each group overlap, the ____________ the error variance; the less that scores overlap between groups, the ____________ the error variance.
random variation; larger; smaller
_____________ counterbalancing involves letting CHANCE "pick" the ordering per participant. One needs to use a _________ number of orders to reduce the influence of ___________ effects. There is no guarantee of an even distribution of ___________ effects across conditions and one cannot evaluate the effects of the order ______________. This is a good method of counterbalancing when there are too many cases to randomize any other way.
random; large; carryover; carryover; statistically
For a within-subjects experimental design, the SAME participants are observed in each group or at each level of an independent variable. Which requirement of an experiment (manipulation, randomization, or comparison/control) is NOT met using this design?
randomization
In a __________, also called a dependent sample, the same or matched participants are observed in each group.
related sample
A ___________, also called a paired-samples t test, is a statistical procedure used to test hypotheses concerning the difference in interval or ratio scale data for two related samples in which the variance in one population is unknown.
related-samples t test
Whatever # we come up with for ANOVA should: NOT be _________ to the # of items in the set. For example, the group of #'s 222 VS. the group of #'s 22222. We would NOT want the second group of #'s to be _________ variable than the first group.
related; more
Make sure your research idea is "interesting" (__________) within the area you with to publish. You can only publish NEW research that shows a ____________ worthwhile outcome. You have to have an article that is ____________ (statistically, as well as ______________).
relevant; significant; significant; meaningful
Which sampling design, repeated measures (within-subjects design) or matched samples, can include the manipulation of an independent variable and is therefore MORE appropriate for use in an experiment?
repeated measures (within-subjects design)
ONLY the __________ design can be used to observe participants in more than TWO groups. A _________ design CANNOT be used because it is limited to situations in which participants are paired into two groups.
repeated measures (within-subjects design); matched samples
To illustrate the ____________ design with more than two groups, suppose we test the hypothesis that participants will judge the taste of wine based on its price. To test this hypothesis, we ask older adults to taste and rate three wines to see if what they are told about the price of those wines will affect their ratings for how much they like the taste of those wines. Adults drink the SAME wine three times, but are given different information about the price of each wine before each tasting---they are told that the cost of wine is inexpensive, moderately expensive, or expensive before each tasting. To be an experiment, we must control for ____________, or the order of presenting the manipulation (whether the wine is described as inexpensive, moderate, or expensive). We hypothesize that ratings for how much the participants like the taste of the wine will be highest for the wine described as being ___________ even though all adults actually will consumer the SAME wine each time.
repeated measures (within-subjects design); order effects; expensive
Factorial designs allow you to build on previous research by ____________ a previous result while ALSO demonstrating a _________ result in the same design.
replicating; new
___________ identifies the actions that researchers must take to conduct responsible and moral research.
research ethics
A ____________ is a proposal, submitted by a researcher to an IRB, outlining the details of a study he or she wishes to complete and how he or she will address potential ethical concerns. Only upon approval by an IRB is a researchers allowed to conduct his or her study, and all researchers are bound to follow the protocol once it is approved.
research protocol
Researchers can take steps to control the assignment of participant characteristics to each group by using _________________, which is a method of controlling differences in participant characteristics between groups in a study by first restricting a sample based on known participant characteristics, then using a random procedure to assign participants to each group. Two strategies of restricted random assignment are ______________ and ____________.
restricted random assignment; control by matching; control by holding constant
In _________ random assignment (control by _________), researchers will identify characteristics that should be balanced among groups and randomly assign from these mini-pods of subjects (e.g., class of 26 women and 4 men--randomly assign 2 men to each group, 13 women to each group). The researcher must also control by holding ____________. In other words, the researcher should identify characteristics that could _________ and affect the research outcome, then screen these subjects out of the sample pool (e.g., if we want to know which RMU advertisement seems most effective to POTENTIAL applicants, the researcher should screen out subjects who are too young or already in college).
restricted; matching; constant; vary
If the null hypothesis were INSIDE the confidence interval (the range of possible values for a population parameter/mean at a specified level of confidence), the decision would have been to __________ the null hypothesis (NOT significant). However, if the null hypothesis were OUTSIDE the confidence interval, the decision would have been to _________ the null hypothesis (significant).
retain; reject
Two types of single-subject (low-n) designs are _____________ and ___________.
reversal (withdrawal ABA); multiple baseline design
__________, a type of single-subject (low-n) designs demonstrate that a treatment HAD an effect by showing that you can __________ the behavior. For instance, Baseline (A)----Treatment (B)-----Baseline (A) can be improved by extending it to an ABABA (etc.) design because one ___________ is NOT as strong or as convincing as many. Also, it isn't usually _________ to end with the removal of a potentially effective treatment (hence: ABAB).
reversal (withdrawal ABA); reverse; reversal; ethical
In the ___________ single-case design, if the treatment in Phase B causes a change in the dependent variable, then the dependent variable should change from baseline to treatment, then return to baseline levels when the treatment is removed. For example, to test if giving encouragement to students who are at risk for dropping out of school will keep them on task in the classroom, we measured the time in minutes that an "at-risk" student spent on task in a class with no encouragement (___________) for a few trials, then with encouragement (_______________) for a few trials, and again with no encouragement (________________) for a few more trials. If the encouragement (the treatment) was successful, then the time (in minutes) spent on task would be _____________ when the encouragement was given but ___________ during the observation periods before and after when no encouragement was given.
reversal; baseline, A; treatment, B; baseline, B; higher; lower
Any alteration to the physical, mental, or emotional well-being of subjects that is derived from participation in the research is __________.
risk
A type of analysis in which the researcher anticipates or weighs the risks and benefits in a study is called a _____________.
risk-benefit analysis
Using the WITHIN-subjects factorial design, the total number of participants is equal to the ____________ because the SAME participants are observed in each group. For instance, if the study was a 2x2 within-subjects design, and 8 subjects were observed in each group, then the total number of subjects would be ____.
sample size; 8
Using the mixed factorial design, the total number of participants (N) in an experiment is equal to the ___________ times the number of levels of the _____________ factor. Suppose that we have a 2x2 mixed factorial design and 30 participants were randomly assigned to each level of the between-subjects factor. In this case, the total number of participants would be ___________.
sample size; between-subjects; 60 participants each observed two times
Using the mixed factorial design, the total number of participants is equal to the ___________ X the number of ___________ of the between-subjects factor.
sample size; levels
__________ is the extent to which a researcher is honest and truthful in his or her actions, values, methods, measures, and dissemination of research.
scientific integrity
Any preexisting differences, not controlled by the researcher, between individuals who are selected from preexisting groups or groups to which the researcher does not randomly assign participants are called _____________.
selection differences
A ____________ is a type of measurement in which participants respond to one or more questions or statements to indicate their actual or perceived experiences, attitudes, or opinions.
self-report measure
A ________________ is unique in that a single participant serves as his or her own control; multiple participants can also be observed as long as each individual serves as his or her own control. In addition, the ____________ variable measured in this type of design is analyzed for each individual participant, and is NOT averaged across groups or across participants.
single-case design; dependent
A ______________ is an experimental research design in which a participant serves as his or her own control and the dependent variable measured is analyzed for each individual participant, and is not averaged across groups or across participants.
single-case experimental design
The ____________, which is also called the single-subject, single-participant, or small "n" design, is most often used in applied areas of psychology, medicine, and education.
single-case experimental design
The purpose of a ______________ (low-n) design is to determine effectiveness of a particular treatment on an individual (or small group). We can't simply evaluate a person over time before a treatment and then just introduce a treatment to observe the resulting behavior changes. Observed changes in behavior could be due either to the ___________, an _________ coincidental with the __________, or BOTH (i.e., a confound). The logic here, as with quasi-experimental designs, is to rule out alternative _______________.
single-subject;; treatment; event; treatment; explanations
When scores in each group do not overlap, this means that the error variance is small or large?
small
The ___________ the error variance, the more likely we are to include that a manipulation of an independent variable caused an observed difference between two or more groups.
smaller
Whatever # we come up with for ANOVA should: Reflect the ________ of the #'s in the distribution (the more _______, the MORE variable. For example, the group of #'s: 99, 100, 101 VS. the group of #'s: 0, 100, 200. The first set of numbers should be "_______ variable" than the second group, EVEN THOUGH they have the same means (100).
spread; spread; less
We could subtract all scores from the mean and sum up the value, however, using this procedure will always result in a solution equal to ZERO. Therefore, we _________ each deviation, and then ________ the _______ deviations, which gives the smallest solution greater than ______ for determining the distance that scores deviate from the mean. To avoid a ______ solution, one must _______ each deviation, then _______, which is represented by the _____________.
square; add; squared; zero; zero; square; add; sum of squares
__________ is the consistency in the pattern of change in a dependent measure in each phase of a design. The more consistent changes in a dependent measure are in each phase, the higher the internal validity of a research design.
stability
The _______ and _______ of change across phases in a single-case design determine the extent to which a researcher has established control. The greater the control, the higher the _________ validity.
stability; magnitude; internal
A ___________ is the manipulation of an independent variable that requires the participant to be "set up" to experience some stimulus or event. This often requires the help of a confederate, who is a coreseracher in cahoots with the researcher.
staged manipulation
_____________ (Balanced Latin Square) counterbalancing involves using only those orders that allow every condition to precede and follow the other conditions one time (the formula: C1, C2, Cn, C3, Cn-1, C4, Cn-2, C5, etc.). "N" is the _________ condition. The benefit of this type of counterbalancing is that it significantly _________ the number of orders, BUT, you must DOUBLE the orders tested when there's an _________ number of conditions. Therefore, use _________ numbers of conditions for this type of counterbalancing. (REFER to notes for examples).
systematic; last; reduces; odd; even
The F-test is virtually identical to a ________ when there are only ______ groups (t ^2 = F). The # of statistical tests that are performed by an F-test depends on how many ________ (NOT _________ of factors). You should stick with a maximum of ________ factors, which has ________ outcomes (A, B, C, AB, AC, BC, ABC).
t-test; two; factors; levels; 3; 7
To compare differences between groups, we will compute a ___________, which allows us to determine whether the manipulation or error attributed to individual differences is likely to explain group differences in ratings of liking.
test statistic
To measure error variance, and differences between groups, we identify a _______________, which is a mathematical formula that allows researchers to determine the extent to which differences observed between groups can be attributed to the manipulation used to create the different groups.
test statistic
In some situations researchers observe one or two preexisting groups at many points in time before and after a treatment, and not just at one time, using designs called the ____________________.
time series quasi-experimental designs
Why do researchers compute post hoc tests?
to determine which pairs of group means significantly differ following a significant ANOVA
In a multiple-baseline design, if the treatment causes an effect following a baseline phase for each case, then the change in level or pattern should begin only when the baseline phase ends, which is different for each case. If this occurs, then we can be confident that the ______________ is causing the observed change.
treatment
When data are interval or ratio scale, the appropriate test statistic for comparing differences between two independent samples is the ___________________, which is a statistical procedure used to test hypotheses concerning the difference in interval or ration scale data between two group means, in which the variance in the population is UNKNOWN.
two-independent-sample t test
A _______________ is a statistical procedure used to analyze the variance in a dependent variable between groups created by combining the levels of two factors.
two-way analysis of variance (ANOVA)
When the levels of two factors (A and B) are combined to create groups, the design is specifically called a ______________.
two-way factorial design
T-tests are designed for making comparisons between ______ groups. The # of t-tests needed for various factorial designs depends on how many __________. With multiple t-tests, you INCREASE the likelihood a _________ significant effect is found (i.e., more chances for ________ to occur). Alpha levels (probability of finding a difference by chance) ________ with the number of tests done.
two; conditions; false; chance; increases
In a between-subjects research design, the evaluation of a treatment effect relies on a comparison of performance between __________ or more groups of subjects. ___________ subjects are evaluated in each level of the factor (independent variable) and NO subject is listed in more than __________ condition. It is critical that groups are __________ except for the manipulation of interest. It is NOT likely with _________ numbers.
two; different; one; identical; small
Even though we may know the mean score in a distribution, we know NOTHING of the remaining scores. Therefore, we must compute measures of ____________ to determine how scores vary around the mean and how scores vary in general.
variability
__________ is a measure of the dispersion or spread of scores in a distribution and ranges from 0 to +∞.
variability
The examination of INDEPENDENT (main) effects of factors (IV) and interactions AMONG factors is an analysis of ____________ (_________).
variance; ANOVA
The # for ANOVA needs to reflect _________, a measure of variability that is quantified. We need this # to track the ________ of as much variability as possible.
variance; source(s);
Since researchers may be in a position of power, there can be no coercion to try and make participants stay in the research if they have elected to stop participation. Participation must remain ___________ and ___________, even if the entire task is not completed. What is promised to a subject must be rewarded to them.
voluntary; compensated
The "________" of a factorial design indicates the number of factors being combined or crossed to create groups.
way
In a within-subjects design we can measure the variability in participant responding within each group as ___________, which is a source of variance in a dependent measure that is caused by or associated with observing different participants within each group.
within-groups variability
In a ___________ design, ONE sample of participants is selected from a population and is observed in EACH group or at each EACH level of the independent variable.
within-subjects design
Because the same participants are used in a _______________, this research design does NOT use random assignment and therefore does not use randomization, which is a requirement in an experiment. Therefore, an added measure to control for possible _________ and _________-related factors associated with observing the same participants over time should be included in this research design. Otherwise, _________-related factors may covary with the levels of an independent variable and these factors can then threaten the ________ validity of an experiment.
within-subjects experimental design; order; time; time; internal
A ______________ is an experimental research design in which the levels of a ____________ factor are manipulated, then the SAME participants are observed in EACH group or at EACH level of the factor. To qualify as an experiment, the researcher MUST (1) ____________ the levels of the factor and include a comparison/control group, AND (2) make added efforts to control for ________ and ________-related factors.
within-subjects experimental design; within-subjects design; manipulate; order; time
A _________ is a type of factor in which the SAME participants are observed in EACH group, or at EACH level of the factor.
within-subjects factor
A _________________ is a research design in which the levels of two or more within-subjects factors are combined to create groups, meaning that the same participants are observed in each group.
within-subjects factorial design
The strength of of a Solomon four-group design is that a _________ design (pre-post comparisons) is combined with a __________ design (between-group comparisons) to establish control. However, its main limitation is the ________ of the design itself. Also, the ________ needed to make all five comparisons can be rather complex.
within-subjects; between-subjects; complexity; statistical analysis
Carryover effects in a _______-subjects design include ___________ (participants' behavior improves as they become familiar with the task), _____________ (participants' behavior declines as they become bored or tired with the task), and __________ carryover ( performance is influenced by previous conditions). One fix is to use a _____________ design instead. Another fix is to use _____________, which is a means of controlling the effects of order-of-presentation of treatments (i.e., if order might affect performance, then you need to examine/control those potential ______________).
within; practice; fatigue; general; between-subjects; counterbalancing; order effects
In a __________-subjects research design, the evaluation of a treatment effect relies on a comparison of performance within a __________ group of subjects. The subject receives ________ level(s) of _______ factor(s). Therefore, a researcher can be certain that "groups" are ___________ prior to manipulating the independent variable. Groups are made up of the ___________ subjects. Because each subject serves in _________ condition, groups are perfectly _________ prior to the manipulation of any independent variable. The total number of subjects necessary will be ________ than for ___________-subjects designs (e.g., only 30).
within; single; all; all; equivalent; same; each; equal; less; between
________-subjects designs are more sensitive (statistically) to _________ effect sizes. A significant outcome ________-subjects might not have been significant ________-subjects.
within; smaller; within; between
In a ________-subjects design, researchers cannot randomly assign subjects. Even though the goal of random assignment is to equate groups, in ___________-subjects designs, groups are automatically equal because they use the SAME subjects. HOWEVER, because you re-use subjects, there is a risk that things OTHER THAN the independent variable could change between uses.
within; within
