Research

Threats to internal validity

1. History 2. Maturation 3. Testing 4. Instrumentation 5. Statistical Regression 6. Selection Bias 7. Experimental Mortality 8. Selection-maturation interaction

Threats to external validity

1. reactive/interactive effect of testing 2. interaction effects of selection bias and experimental variables 3. 11. reactive effects of experimental arrangements 4. multiple treatment interference

internal validity

extent to which we can accurately state that the independent variable produced the observed effect

Multiple Treatment Interference

Threat to external validity. As multiple treatments are given to the same subjects, it is difficult to control for the effects of prior treatments.

reactive effects of experimental arrangements

Threat to external validity. Exposure to experimental conditions may prevent generalizations about effects upon those in non-experimental settings.

Interaction effects of selection biases and the experimental variable

Threat to external validity. This becomes more problematic as selecting/recruiting participants becomes more difficult (e.g., if the 10th school finally agrees to participate, highly likely that they are different from the previous 9 and may therefore respond differently to the experimental variable)

Reactive/interactive effect of testing

Threat to external validity. pretest may increase/decrease a subject's sensitivity/responsiveness to the experimental variable (e.g., priming). Therefore, the results from pretested population would not be generalizable to populations not pretested.

Selection bias

Threat to internal validity. Bias in selecting participants for comparison groups. Systematic differences exist in subjects' characteristics between tx groups.

Selection x Maturation

Threat to internal validity. Groups change as a function of maturation and selection bias. Experimental and control groups have naturally grown apart (developed differently); difference due to developmental change rather than treatment.

History

Threat to internal validity. Includes any events (directly in the environment; in the outside world) occurring between measurements other than the experimental variable (e.g., instruction, observational learning)

Statistical Regression

Threat to internal validity. Occurs with groups selected on basis of extreme scores; scores of subjects that are very high/low tend to move closer to average upon retesting.

Maturation

Threat to internal validity. Processes that occur as a function of time (e.g., aging, fatigue, hunger)

Experimental mortality

Threat to internal validity. Subject attrition may bias results (if differential loss of participants from comparison groups.)

Instrumentation effects

Threat to internal validity. Testing instruments/conditions are inconsistent, pre & post test not equivalent. Changes related to the measurement tools use (e.g., the instrument itself, the observers/scorers) responsible for changes in measurement (illusory change in performance)

Testing effects

Threat to internal validity. The effects that taking a test have on the scores of subsequent tests (i.e., practice/re-test effects)

external validity

relates to the ability to generalize findings from a given study to/across (1) target population, (2) tasks, (3) environments