Research
Threats to internal validity
1. History 2. Maturation 3. Testing 4. Instrumentation 5. Statistical Regression 6. Selection Bias 7. Experimental Mortality 8. Selection-maturation interaction
Threats to external validity
1. reactive/interactive effect of testing 2. interaction effects of selection bias and experimental variables 3. 11. reactive effects of experimental arrangements 4. multiple treatment interference
internal validity
extent to which we can accurately state that the independent variable produced the observed effect
Multiple Treatment Interference
Threat to external validity. As multiple treatments are given to the same subjects, it is difficult to control for the effects of prior treatments.
reactive effects of experimental arrangements
Threat to external validity. Exposure to experimental conditions may prevent generalizations about effects upon those in non-experimental settings.
Interaction effects of selection biases and the experimental variable
Threat to external validity. This becomes more problematic as selecting/recruiting participants becomes more difficult (e.g., if the 10th school finally agrees to participate, highly likely that they are different from the previous 9 and may therefore respond differently to the experimental variable)
Reactive/interactive effect of testing
Threat to external validity. pretest may increase/decrease a subject's sensitivity/responsiveness to the experimental variable (e.g., priming). Therefore, the results from pretested population would not be generalizable to populations not pretested.
Selection bias
Threat to internal validity. Bias in selecting participants for comparison groups. Systematic differences exist in subjects' characteristics between tx groups.
Selection x Maturation
Threat to internal validity. Groups change as a function of maturation and selection bias. Experimental and control groups have naturally grown apart (developed differently); difference due to developmental change rather than treatment.
History
Threat to internal validity. Includes any events (directly in the environment; in the outside world) occurring between measurements other than the experimental variable (e.g., instruction, observational learning)
Statistical Regression
Threat to internal validity. Occurs with groups selected on basis of extreme scores; scores of subjects that are very high/low tend to move closer to average upon retesting.
Maturation
Threat to internal validity. Processes that occur as a function of time (e.g., aging, fatigue, hunger)
Experimental mortality
Threat to internal validity. Subject attrition may bias results (if differential loss of participants from comparison groups.)
Instrumentation effects
Threat to internal validity. Testing instruments/conditions are inconsistent, pre & post test not equivalent. Changes related to the measurement tools use (e.g., the instrument itself, the observers/scorers) responsible for changes in measurement (illusory change in performance)
Testing effects
Threat to internal validity. The effects that taking a test have on the scores of subsequent tests (i.e., practice/re-test effects)
external validity
relates to the ability to generalize findings from a given study to/across (1) target population, (2) tasks, (3) environments