Exam 3
reversal design (ABAB design)
a single-case experimental design in which a single participant is observed before (A), during (B), and after (A) a treatment or manipulation
The stability of a measure in each phase is important because
When a measure is unstable, changes are occurring in a dependent variable even when the researcher is not manipulating the behavior
Cohorts in a cross sectional analysis are related in terms of
When participants were born
between-person variability
a source of variance in a dependent measure that is caused by or associated with individual differences or differences in participant responses across all groups
between-group variability
a source of variance in a dependent measure that is caused by or associated with the manipulation of the levels (or groups) of an independent variable
Error
a source of variance that cannot be attributed to having different groups or treatments. Two sources of error are between-persons and within groups variability
participant fatigue
a state of physical or psychological exhaustion resulting from intense research demands typically due to observing participants participants too often, or requiring participants to engage in research activities that are too demanding
pairwise comparison
a statistical comparison for the difference between two group means. A post hoc test evaluates all possible pairwise comparisons for an ANOVA with any number of groups
two independent-sample t test (independent-sample t test)
a statistical procedure used to test hypotheses concerning the difference in interval or ration scale data between two group means, in which the variance in the population is unknown
related-samples t test (paired-samples t test)
a statistical procedure used to test hypotheses concerning the different in interval or ratio scale data for two related sample in which the variance in once population is unknown
Error Variance (Error)
numeric measure of the variability in scores that can be attributed to or is caused by the individual differences of participants in each group
stability
the consistency in the pattern of change in a dependent measure in each phase of a design. The more stable or consistent changes in a dependent measure are in each phase, the higher the internal validity of a research design.
experimental manipulation
the identification of an independent variable and the creation of two or more groups that constitute the levels of that variable
natural manipulation
the manipulation of a stimulus that can be naturally changed with little effort. Typically involves the manipulation of a physical stimulus
staged manipulation
the manipulation of an independent variable that requires the participant to be "set up" to experience some stimulus or event. It often requires the help of a confederate.
2 types of counterbalancing
1) complete counterbalancing 2) partial counterbalancing
2 strategies of restricted random assignment
1) control by matching 2) control by holding constant
To analyze the data with a visual inspection we look for what two types of patterns that indicate the treatment caused an observed change
1. A change in level 2. A change in trend
The advantage of using the cohort sequential design is that researchers can
1. Account for threats to internal validity associated with observing participants over time because part of the sample is a cross-section of age groups 2. Account for cohort effects because part of the sample includes the same participants observed over time in each age group or cohort
Three types of Time series quasi-experimental designs are
1. Basic time series design 2. Interrupted time series design 3. Control time series design
Two types of non-equivalent control group quasi-experiments
1. Non-equivalent control group posttest only design 2. Non-equivalent control group pretest-posttest design
For a single case design to be an experimental design, it must meet the following three key elements of control required to draw cause-and-effect conclusions
1. Randomization. Using single case designs, each participant can be randomly assigned to experience many phases or treatments 2. Manipulation 3. Comparison/control group- Edward has of an access their own control
Three types of single case experimental research designs
1. Reversal design 2. Multiple baseline design 3. Changing criterion design
The level of control and therefore the internal validity of a single case design can be determined when the following two features are observed in a graph using a visual analysis
1. The stability in the pattern of change across phases 2. The magnitude or size of the change across phases
key limitation to nonequivalent control group posttest only
no random assignment between groups
interrupted time series design
A quasi-experiment in which a dependent variable is measured at many different points in time in one group before and after a treatment that naturally occurred. Coronary heart disease before and after a smoking ban
Examples of a naturally occurring treatment or event
A scheduled medical procedure, a wedding, a natural disaster, a change in public policy, a new law, and a political scandal
The quasi-independent variable in developmental designs is
Age
Single case Designs are typically structured by
Alternating baseline and treatment phases is over many trials or observations
within-subjects experimental design
An experimental research design in which the levels of a within-subjects factor are manipulated, then the same participants are observed in each group or at each level of the factor. To qualify as an experiment, the researcher must (1) manipulate the levels of the factor and include a comparison/control group, and (2) make added efforts to control for order and time-related factors.
When a dependent measure is stable, we can
Be confident that any changes in level or trend or caused by the manipulation, because changes only occurred between each phase and were otherwise stable or consistent with in each phase
The only design that can meet all three requirements of an experiment: randomization, manipulation, and inclusion of a comparison/control group to demonstrate cause-and-effect
Between subjects design
For research situations in which we want to change a criterion or treatment after the participant meets in initial criterion or response to one particular treatment, we can use
Changing-criterion design
A disadvantage of the cross-sectional design is the possibility of
Cohort effects or generation effects
The more stable a measure, the greater the
Control and the higher the internal validity in an experiment
The more that scores in each group overlap, the larger the
Error variance; the less that scores overlap between groups, the smaller the error variance
The results in a single case design can have high
External validity in terms of generalizing across behaviors, across subjects or participants, and across settings
What is the difference between a control group and a comparison group?
In a control group, participants are treated exactly the same way as the treatment group just minus the manipulation but in the comparison group they are not treated exactly the same
With changing criterion design researchers should be cautious to not
Increase or decrease the criterion too soon or by too much, which may impede the natural learning rate of the participant being observed
An advantage of analyzing the data one participant at a time is that
It allows for the critical analysis of each individual measure, where as averaging scores across groups can give a spurious appearance of orderly change
One advantage of the reversal design is that
It can be used to apply treatments that are beneficial to participants which means researchers will be asked by ethics committees to end their study with a treatment phase which was the phase that was beneficial to the participants and so therefore many reversal designs are at least four phases so as not to return to baseline to end an experiment
Two advantages of the changing criterion design are that
It does not require a reversal to baseline of an otherwise effective treatment and that it enables experimental analysis of a gradually improving behavior
When a measure is stable within each phase, we look at the _________ of changes between phases
Magnitude
What is a practical way to control for factors that you anticipate may influence differences over time in a longitudinal study?
Measuring participant characteristics
For situations in which it is not possible for changes in the dependent variable to return to baseline levels following a treatment phase, researchers can use
Multiple-baseline design
The cross-sectional design is similar to
Nonequivalent Control group quasi-experimental design in that the different age groups act as non-equivalent control groups
Which research design is particularly susceptible to the threat of selection differences
Nonequivalent control group posttest-only design
The type of quasi-experiment most susceptible to threats to internal validity, such as history effects and maturation effects, is
One group posttest-only design which is also called the one shot case study
What does statistic do you use for multiple independent samples
One way between subjects analysis of variance
A longitudinal design is similar to
One-group protest-posttest quasi-experimental research design in that one group of participants is observed over time
The advantage of a cross-sectional design is that
Participants are observed one time in each cohort
A single case design is typically associated with low
Population validity, which is a subcategory of external validity. In other words, it is not possible to know whether the results in the sample would also be observed in the population from which the sample was selected because single case experimental designs are associated with very small sample sizes
within-groups variability
Source of variance in a dependent measure that is caused by or associated with observing different participants within each group
A limitation of the reversal design is that
The change in a dependent variable and a treatment phase must return to baseline levels when the treatment is removed. However in many areas of research, such as studies on learning, a return to baseline is not possible. In these cases a reversal design cannot be used
The greater the magnitude of changes between phases, the greater
The control and the higher the internal validity in a single case experiment
The limitation of a multiple baseline design is that
The design is used when only a single type of treatment is administered. This same limitation applies to the reversal design
Internal validity is
The extent to which we can demonstrate that a manipulation or treatment causes a change in a dependent measure
Cohort-sequential design Requires only that
The longitudinal observations overlap across the cohorts
A limitation of changing criterion design is that
The target behavior must already be in the participant's repertoire
Multiple baseline design: if the treatment causes an effect following a baseline phase for each case,
Then the change in level or pattern should begin only when the baseline phase ends which is different for each case
Since one group designs do not include a no treatment control group, they are prone to what internal validity threats?
They are prone to internal validity threats that are associated with observing the same participants over time, such as testing affects and regression toward the mean
Observing participants at one time eliminates
Threats to internal validity associated with observing participants overtime such as attrition, testing effect, and regression toward the mean
What are the disadvantages of a longitudinal design?
Threats to internal validity such as attrition(participants dropping out), testing effect, and regression toward the mean. They can also require substantial resources, money, recruitment efforts, and time to complete
What does the does the do you use for two independent samples
Two independent sample t-test
An advantage of the interrupted time series design is that
We can identify if the pattern of change in a dependent variable changes from before to following a naturally occurring treatment or event
control time series design
a basic or interrupted time series quasi-experimental research design that also includes a nonequivalent control group that is observed during the same period of time as a treatment group, but does not receive the treatment
the experimental group (treatment group)
a condition in an experiment in which participants are treated or exposed to a manipulation, or level of the independent variable, that is believed to cause a change in the dependent variable
control group
a condition in an experiment in which participants are treated the same as participants in an experimental group, except that the manipulation believed to cause a change in the dependent variable is omitted
nonequivalent control group
a control group that is matched upon certain preexisting characteristics similar to those observed in a treatment group, but to which participants are not randomly assigned. In a quasi-experiment, a dependent variable measured in a treatment group is compared to that in the nonequivalent control group
Latin square design
a counterbalancing strategy where each experimental condition appears at every position in the sequence order equally often
cross-sectional design
a developmental research design in which participants are grouped by their age and participant characteristics are measured in each age group
cohort sequential design
a developmental research design that combines longitudinal and cross-sectional techniques by observing different cohorts of participants over time at overlapping times
cohort
a group of individuals who share common statistical traits or characteristics, or experiences within a defined period
test statistic
a mathematical formula that allows researchers to determine the extent to which difference observed between groups can be attributed to the manipulation used to create the different groups
restricted random assignment
a method of controlling differences in participant characteristics between groups in a study by first restricting a sample based on known participant characteristics, then using a random procedure to assign participants to each group.
baseline phase (A)
a phase in which a treatment or manipulation is absent
quasi-independent variable
a preexisting variable that is often a characteristic inherent to an individual, which differentiates the groups or conditions being compared in a research study. Because the levels of the variable are preexisting, it is not possible to randomly assign participants to groups
complete counterbalancing
a procedure in which all possible order sequences in which participants receive different treatments or participate in different groups are balanced or offset in an experiment
partial counterbalancing
a procedure in which some, but not all, possible order sequences in participants receive different treatments or participate in different groups are balanced or offset in an experiment
nonequivalent control group pretest-posttest design
a quasi-experiment research design in which a dependent variable is measured in one group of participants before (pretest) and after (posttest) a treatment and that same dependent variable is also measured at pretest and posttest in another nonequivalent control group that does not receive the treatment
one-group pretest-posttest design
a quasi-experimental design in which the effect of an independent variable is inferred from the pretest-posttest difference in a single group. The dependent variable is measured before and after a treatment is administered.
one-group posttest-only design
a quasi-experimental design that has no control group and no pretest comparison; a very poor design in terms of internal validity
basic time series design
a quasi-experimental research design in which a dependent variable is measured at many different points in time in one group before and after a treatment that is manipulated by the researcher is administered
nonequivalent control group posttest-only design
a quasi-experimental research design in which a dependent variable is measured following a treatment in one group and also in a nonequivalent control group that does not receive the treatment
between-subjects design
a research design in which different participants are observed one time in each group or at each level of a factor
within-subjects design (repeated measure design)
a research design in which the same participants are observed one time in each group of research study
phase
a series of trials or observations made in one condition
multiple baseline design
a single-case experiment in which a treatment is successively administered over time to different participants, for different behaviors, or in different settings. Example: sex education intervention, baseline phase for a different number of weeks
Changing Criterion Design
a single-case experimental design in which a baseline phase is followed by successive treatment phases in which some criterion or target level of behavior is changed from one treatment phase to the next. The participant must meet the criterion of one treatment phase before the next treatment phase is administered
One-way between-subjects ANOVA
a statistical procedure used to test hypotheses for one factor with two or more levels concerning the variance among group means, This test is used when different participants are observed at each level of a factor and the variance in a given population unknown.
one-way within-subjects ANOVA
a statistical procedure used to test hypotheses for one factor with two or more levels concerning the variance among group means. This test is used when the same participants are observed at each level of a factor and the variance in a given population is unknown.
carryover effects
a threat to internal validity in which participation in one group "carries over" or causes changes in performance in a second group
order effects
a threat to internal validity in which the order in which participants receive different treatments or participate in different groups causes changes in a dependent variable
between-subjects factor
a type of factor in which different participants are observed in each group, or at each level of the factor
within-subjects factor
a type of factor in which the same participants are observed in each group, or at each level of the factor
self-report measure
a type of measurement in which participants respond to one or more questions or statements to indicate their actual or perceived experiences, attitudes, or opinions
behavioral measure
a type of measurement in which researchers directly observe and record the behavior of subjects or participants
physiological measure
a type of measurement in which researchers record physical responses of the brain and body in a human or an animal
control by matching
a type of restricted random assignment in which we assess or measure the characteristic we want to control, group or categorize participants based on scores on that measure, and then use a random procedure to assign participants from each category to a group in the study
control by holding constant
a type of restricted random assignment in which we limit which participants are included in a sample based on characteristics the exhibit that may otherwise differ between groups in a study
matched samples design (matched-pairs design)
a within-subjects research design in which participants are matched, experimentally or naturally, based on preexisting characteristics or traits that they share
The Solomon four-group design
an experimental design in which different participants are assigned to each of four groups in such a way that comparisons can be made to 1) determine if a treatment causes changes in posttest measure and 2) control for possible confounds or extraneous factors related to giving a pretest measure and observing participants over time.
single-case experimental design
an experimental research design in which a participant serves as his or her own control and the dependent variable measured is analyzed for each individual participant and is not averaged across groups or across participants also called single-subject, single participant, or small n design most often used in applied areas of psychology, medicine, and education
between-subjects experimental design
an experimental research design in which the levels of a between-subjects factor are manipulated, then different participants are randomly assigned to each group or to each level of that factor, and observed one time
placebo
an inert substance, surgery, or therapy that resembles a real treatment but has no real effect
selection difference
any differences, which are not controlled by the researcher, between individuals who are selected from preexisting groups or groups to which the researcher does not randomly assign participants
key limitation to nonequivalent control group pretest-posttest
no random assignment between groups
longitudinal design
developmental research design used to study changes across the life span by observing the same participants at different points in time and measuring the same dependent variable at each time
in an independent sample,
different participants are independently observed on time in each group
control
in research design is a) the manipulation of a variable and b) holding all other variables constant.
key limitation to basic time series design
no control group for comparison
key limitation to interrupted time series design
no control group for comparison
key limitation to one-group posttest only
no control group for comparison
key limitation to one-group pretest-posttest
no control group for comparison
key limitation to control series design
no random assignment between groups
post hoc test
statistical procedure computed following a significant ANOVA to determine which pair or pairs of group means significantly differ. These tests are needed with more than two groups because multiple comparisons must be made
related sample (dependent sample)
the same or matched participants are observed in each group
magnitude
the size of the change in d dependent measure observed between phases of a design. The larger the magnitude of changes in a dependent measure between phases, the higher the internal validity of a research design
quasi-experimental research design
the use of methods and procedures to make observations in a study that is structured similar to an experiment, but the conditions and experiences of participants lack some control because the study lacks random assignment, includes a preexisting factor, or does not include a comparison/control group
cohort effects (generation effects)
threat to internal validity in which differences in the characteristics of participants in different cohorts or age groups confound or alternatively explain an observed result, Ie texting and age
counterbalancing
using this procedure, the order in which participants receive different treatments or participate in different groups is balanced or offset in an experiment.