Chapter 8

Practice Effect

A practice effect is a beneficial effect on a DV measurement caused by previous experience with the DV.

Convenience Sampling

A researcher's sampling of participants based on ease of locating the participants; often does not involve true random selection. e.g. psychology students for class credits (overrepresented in the literature)

Multiple-Treatment Interference

A threat to external validity that occurs when a set of findings results only when participants experience multiple treatments in the same experiment ( repeated measures designs). E.g. one training section group v.s. multiple training section group

External validity

A type of evaluation of your experiment that asks whether your experimental results apply to populations and situations that are different form those of your experiment.

Replication with extension

An experiment that seeks to confirm (replicate) a previous finding but does so in a different setting or with different participants or under different conditions.

Generalization

Applying the results from an experiment to a different situation or population.

Diffusion or Imitation of Treatments

Diffusion or imitation of treatments can occur if participants in one treatment group become familiar with the treatment of another group and copy that treatment.

Demand characteristics

Features from the experimenters that inadvertently lead participants to respond in a particular manner. E.g. experiments in the psychology department must be related to psychoanalysis

History

History refers to events that occur between the DV measurements in a repeated measures design.

Selection

If we choose participants in such a way that our groups are not equal before the experiment, we cannot be certain that our IV caused any difference we observe after the experiment.

Threats to External Validity (Based on Methods)

Interaction of Testing and Treatment Interaction of Selection and Treatment Reactive Arrangements Demand characteristics Multiple-Treatment Interference Selection Bias

How Important is Internal Validity?

It is the most important property of any experiment. If you do not concern yourself with the internal validity of your experiment, you are wasting your time. CAREFULY DESIGN YOUR EXPERIMENT BEFORE IMPLEMENT!

Three customary types of generalization in which we are interested.

Population generalization Environmental generalization Temporal generalization

Reactive Measures

Reactive measures are DV measurements that actually change the DV being measured. Many attitude questionnaires are reactive measures.

Statistical Regression

Statistical regression occurs when low scorers improve or high scorers fall on a second administration of a test due solely to statistical reasons.

Testing

Testing is a threat to internal validity that occurs because measuring the DV causes a change in the DV.

Instrumentation (Instrument Decay)

The equipment or human measuring the DV changes the measuring criterion over time.

Internal/external validity relationship

The external validity of an experiment is concerned with "To whom and under what circumstances can the results be generalized?" The internal validity is concerned with the question, " Is the treatment, in this particular case, responsible for the observed changes(s)?"

Replication

When we replicate an experimental finding, we are able to place more confidence in that result.

Protecting Internal Validity

You can (and should) implement the various control procedures discussed in this book. Use a standard procedure Experimenters use standard procedures called experimental designs to help ensure internal validity.

Selection Bias

Your sample is not representative to the population. Small response rate Convenient sampling Failing rate (successful subjects) The Infamous White Rat

Threats to Internal Validity

history maturation testing Practice EffectReactive MeasuresInstrumentation (Instrument Decay) Statistical Regression Selection Mortality Diffusion or Imitation of Treatments Interactions among multiple threats

Reactive Arrangements

A threat to external validity caused by an experimental situation that alters participants' behavior, regardless of the IV involved. We cannot be sure that the behaviors we observe in the experiment will generalize outside that setting because the artificial conditions of the experiment do not exist in the real world

Interaction of Selection and Treatment

A threat to external validity that can occur when a treatment effect is found only for a specific sample of participants. E.g. older adults: only those who are physically active may come and participate the study E.g. Patient population: early stage of patients (hard for lock-in stage patients) E.g. Patients: take medication when have symptom

Interaction of Testing and Treatment

A threat to external validity that occurs when a pretest sensitizes participants to the treatment yet to come. E.g. Examine the effect of exercise on health Pretest sensitizes participants to be aware of health issues Participants may change some behavior (e.g. diet) during the training section

Maturation

Maturation refers to changes in participants that occur over time during an experiment.

Mortality

Mortality can occur if experimental participants from different groups drop out of the experiment at different rates.