Research Methods External Validity

How are stimulus characteristics and settings a threat to external validity? And what are they?

1) Generality is limited when results are restricted to the unique stimulus characteristics of the study (e.g., don't extent from the laboratory to the clinic), or when stimulus sampling is narrow (e.g., using one therapist to provide all treatment in a psychotherapy outcome study). 2) Refers to the range of stimuli that participants are exposed to during an investigation. Important stimuli may include the setting of the experiment, the time of day that the experiment is conducted, and the experimenters themselves

How is the timing of assessment a threat to external validity?

A question of whether the results of a study extend beyond the times that the intervention was administered (e.g., time of day, time of year) or to times after the intervention is terminated. For example, psychotherapy outcome results obtained immediately after treatment may not extend to a later period . (e.g., 6 months post-treatment).

Sample Characteristics - how are they a threat to external validity?

A threat when the results of a study cannot be plausibly extended to individuals with characteristics beyond those included in the investigation. If the findings have import only for individuals whose characteristics are identical to the experimental participants then generality is absent.

How is multiple treatment interference problematic for external validity ?

Concerns the question of drawing appropriate conclusions about the effects of a particular intervention when participants are exposed to more than one treatment condition in parallel or sequentially. For example, generalizing the effects of a second intervention might be restricted because of the effects of an earlier intervention if the results obtained extend only to individuals who experience both interventions in the same order.

How is subject reactivity to experimental arrangements a threat to external validity?

Concerns the question of whether similar results would be obtained if participants were not aware they were participating in a research study (a.k.a., the Hawthorne effect).

How is reactivity to assessment a threat to external validity?

Participant responses that are biased because of their awareness that they are being assessed in some way limits the generality of findings. For example, the communication of couples observed by researchers in the laboratory may well be different than what would be observed by a fly on the wall during a conversation at their home.

How are novelty effects a threat to external validity?

Refers to limited generality due to the uniqueness or novelty of the experimental situation or treatment. Difficult to evaluate in an initial experiment, but can often be determined through experimental replication

How is test sensitization a threat to external validity?

Restricted generality due to participants responding to or evaluating treatment differently because testing instruments or procedures sensitize them to the treatment. In other words, the effects of testing are confounded with the effects of treatment.

What kinds of investigations are criticized for the external validity of their sample characteristics?

Studies which attempt to generalize treatment effects based on non-clinical samples to clinical populations or from a narrow demographic sample to the larger population

How can accusers effectively criticize the external validity of a study's sample characteristics?

offer plausible reasons for believing sample characteristics delimit external validity since there is virtually no limit to individual variation..