Threats to External Validity

Low Statistical Power

Ability to reject the null hypothesis when it is false Affected by inadequate sample size The failure to control other sources of variability

Interaction or Treatment and Setting and Reactive Effects of Experimental Arrangements are also known as:

Ecological Validity

Statistical Conclusion Validity

Error Rate Reliability and Variance

Reliability and Variance

Findings are threatened when extraneous factors increase the variability with the data

Multiple-treatment Interference

Happens in repeated measures designs Participants receive more than one treatment level If a carry-over effect is present, then that is this external validity problem Maybe need IND Groups Design

Population Validity

How well do the characteristics of the subjects tested align with the target population the researcher wishes to generalize back to How well does the sample represent the population?

As studies get older we are always concerned with the question, if we repeated this study today would we find the same results

Interaction of Treatment and History

Synonym for Interaction of Treatment and Selection

Population Validity

Reactive/Interactive Effects of Testing

Pretest may make the participants more aware of the treatment to be used

Mneumonic External Validity Sentence

Puppies Try Hard Running Interactive Sessions Engaging Eggplants

Treatment might only be effective IF pretest were given and thus the results of the study could not be generalized beyond

Reactive/Interactive Effects of Testing

Examples of reliability and variance

Unreliable measures Protocol not being administered consistently Influences of the ENV Nuisance or confounding variable

Ecological Validity

How well does the ENV/the setting the experiment is conducted in align with setting the researcher wishes to generalize about

Interaction of Selection Bias and Experimental Treatment

If treatment is effective in one school, it is hard to generalize to other more regular schools 9 schools say NO, 10th school says YES

List threats to external validity (6)

Population Interaction of Treatment and History Reactive or Interactive Effects of Testing Interaction of Selection Bias and Experimental Treatment Ecological

Methods of Controlling External Validity

Random Selection Blind and Double Blind Studies Counterbalanced designed Careful use of operational definitions

What can we do to address some of these issues?

Random Selection (when feasible) Blind and Double Blind Studies Counterbalanced designed Careful use of operational definitions

Influence on experimental control

Theoretical framework Participants Application of design Controlling IV Measurement of DV Statistical Model & Analysis Interpreting Results

Interaction of Treatment and History

This threat deals with the problems of trying to generalize results to different periods of time either past or future

Violation of the assumptions of statistical tests

Using appropriate procedures Meeting the requirements for the tests

Error Rate

With certain some analysis the researcher may have to perform a large # of statistical power