Chapter 8
Practice Effect
A practice effect is a beneficial effect on a DV measurement caused by previous experience with the DV.
Convenience Sampling
A researcher's sampling of participants based on ease of locating the participants; often does not involve true random selection. e.g. psychology students for class credits (overrepresented in the literature)
Multiple-Treatment Interference
A threat to external validity that occurs when a set of findings results only when participants experience multiple treatments in the same experiment ( repeated measures designs). E.g. one training section group v.s. multiple training section group
External validity
A type of evaluation of your experiment that asks whether your experimental results apply to populations and situations that are different form those of your experiment.
Replication with extension
An experiment that seeks to confirm (replicate) a previous finding but does so in a different setting or with different participants or under different conditions.
Generalization
Applying the results from an experiment to a different situation or population.
Diffusion or Imitation of Treatments
Diffusion or imitation of treatments can occur if participants in one treatment group become familiar with the treatment of another group and copy that treatment.
Demand characteristics
Features from the experimenters that inadvertently lead participants to respond in a particular manner. E.g. experiments in the psychology department must be related to psychoanalysis
History
History refers to events that occur between the DV measurements in a repeated measures design.
Selection
If we choose participants in such a way that our groups are not equal before the experiment, we cannot be certain that our IV caused any difference we observe after the experiment.
Threats to External Validity (Based on Methods)
Interaction of Testing and Treatment Interaction of Selection and Treatment Reactive Arrangements Demand characteristics Multiple-Treatment Interference Selection Bias
How Important is Internal Validity?
It is the most important property of any experiment. If you do not concern yourself with the internal validity of your experiment, you are wasting your time. CAREFULY DESIGN YOUR EXPERIMENT BEFORE IMPLEMENT!
Three customary types of generalization in which we are interested.
Population generalization Environmental generalization Temporal generalization
Reactive Measures
Reactive measures are DV measurements that actually change the DV being measured. Many attitude questionnaires are reactive measures.
Statistical Regression
Statistical regression occurs when low scorers improve or high scorers fall on a second administration of a test due solely to statistical reasons.
Testing
Testing is a threat to internal validity that occurs because measuring the DV causes a change in the DV.
Instrumentation (Instrument Decay)
The equipment or human measuring the DV changes the measuring criterion over time.
Internal/external validity relationship
The external validity of an experiment is concerned with "To whom and under what circumstances can the results be generalized?" The internal validity is concerned with the question, " Is the treatment, in this particular case, responsible for the observed changes(s)?"
Replication
When we replicate an experimental finding, we are able to place more confidence in that result.
Protecting Internal Validity
You can (and should) implement the various control procedures discussed in this book. Use a standard procedure Experimenters use standard procedures called experimental designs to help ensure internal validity.
Selection Bias
Your sample is not representative to the population. Small response rate Convenient sampling Failing rate (successful subjects) The Infamous White Rat
Threats to Internal Validity
history maturation testing Practice EffectReactive MeasuresInstrumentation (Instrument Decay) Statistical Regression Selection Mortality Diffusion or Imitation of Treatments Interactions among multiple threats
Reactive Arrangements
A threat to external validity caused by an experimental situation that alters participants' behavior, regardless of the IV involved. We cannot be sure that the behaviors we observe in the experiment will generalize outside that setting because the artificial conditions of the experiment do not exist in the real world
Interaction of Selection and Treatment
A threat to external validity that can occur when a treatment effect is found only for a specific sample of participants. E.g. older adults: only those who are physically active may come and participate the study E.g. Patient population: early stage of patients (hard for lock-in stage patients) E.g. Patients: take medication when have symptom
Interaction of Testing and Treatment
A threat to external validity that occurs when a pretest sensitizes participants to the treatment yet to come. E.g. Examine the effect of exercise on health Pretest sensitizes participants to be aware of health issues Participants may change some behavior (e.g. diet) during the training section
Maturation
Maturation refers to changes in participants that occur over time during an experiment.
Mortality
Mortality can occur if experimental participants from different groups drop out of the experiment at different rates.