Ch.7 Experimental Research
Confounding constructs and levels of constructs threat
-construct validity threat, drawing conclusions about variables when some levels of the variables are absent
Inadequate Preoperational Explication of Constructs Threat
-construct validity threat, insufficient definition of the independent and/or dependent variables
Mono-operation bias threat
-construct validity threat, only one form of the experimental variable is implemented
Mono-method bias threat
-construct validity threat, only one form of the independent variable is tested
Hypothesis guessing within experimental conditions threat
-construct validity threat, participants behave differently when they know they are part of an experiment (the behavior of the subjects can also threaten internal validity)
Interaction Effects of Selection Biases and the Experimental Treatment Threat
-external validity threat, an effect of some selection factor of intact groups interacting with the experimental treatment that would not be the case if the groups were formed randomly
Reactive Effects of Experimental Arrangements Threat
-external validity threat, an effect that is due to the artificial or novel experimental setting (this can also threaten internal validity)
Interaction Effect of Testing Threat
-external validity threat, pretesting interacts with the experimental treatment and causes some effect such that the results will not generalize to an unpretested population.
Multiple-Treatment Interference Threat
-external validity threat, when the same subjects receive two or more treatments (As in repeated measures design). There may be a carryover effect between treatments such that the results cannot be generalized to single treatments.
Reactive Effects of Experimental Arrangements Example
An experiment in remedial reading instruction has an effect that does not occur when the remedial reading program is implemented in the regular classroom
History Threat Example
During a relatively short instructional experiment, one group of subjects misses instruction due to a power failure at the school.
Selection Threat Example
The experimental group in an instructional experiment consists of a high-ability class, while the control group is an average-ability class.
Interaction Effects of Selection Biases and the Experimental Treatment Example
The results of an experiment in which teaching method is the experimental treatment is effective with low achievers but is not as effective with high achievers
Instrumentation Threat Example
Two examiners in an instructional experiment administered the posttest with different instructions and procuedures.
Inadequate Preoperational Explication of Constructs Example
Two teachers implement individualized instruction in very different ways because they lack a precise definition of a term.
Interaction in an experiment
an effect on the dependent variable such that the effect of one independent variable changes over the levels of another independent variable
Repeated measures designs
designs in which the same subject is measured more than once on the dependent variable
Time series designs
involve repeated measurement with an experimental treatment inserted between two of the treatments
Testing Threat
-internal validity threat, The effect of taking one test on the scores of a subsequent test.
Statistical Regression Threat
-internal validity threat, an effect caused by a tendency for subjects selected on the basis of extreme scores to regress toward an average performance on a subsequent test
Instrumentation Threat
-internal validity threat, an effect due to inconsistent use of the measuring instruments
Selection Threat
-internal validity threat, an effect due to the groups of subjects not being randomly assigned to groups; a selection factor is operating such that the groups are not equivalent
Mortality Threat
-internal validity threat, in a health experiment designed to determine the effects of various exercises, those subjects finding exercise most difficult stop participating.
Selection-Maturation Interaction Threat
-internal validity threat, in a problem-solving experiment, intact groups of junior high school students and senior high students are involved. The junior high students tire of the task sooner than the older students.
Maturation Threat
-internal validity threat, processes operating within the subject as a function of time.
History Threat
-internal validity threat, unanticipated events occurring while the experiment is in progress that affect the dependent variable.
Fishing and the error rate problem
-statistical conclusion validity threat, capitalizing on chance findings
Violated assumptions of statistical tests
-statistical conclusion validity threat, failing to meet the underlying assumptions
Low statistical power
-statistical conclusion validity threat, using a sample size that is too small to detect differences between groups
Reliability of measures
-statistical conclusion validity threat, using technically inadequate measures
Maturation Threat Example
In a learning experiment, subject performance begins decreasing after about fifty minutes due to fatigue.
Interaction Effect of Testing Threat Example
In a physical performance experiment, the pretest cues the subjects to respond in a certain way to the experimental treatment that would not be the case if there were no present.
Testing Threat Example
In an experiment in which performance on a logical reasoning test is the dependent variable, a pretest cues the subjects about the posttest.
Statistical Regression Example
In an experiment involving reading instruction, subjects grouped because of poor pretest reading scores show considerably greater gains than the average and high readers.
Hypothesis guessing within experimental conditions example
Subjects in an experimental program interact differently when they know they are being studied
Solomon four-group design
a combination of the posttest-only control group design and the pretest-posttest control group design
Reliability of measures example
a psychologist finds no difference between boys' and girls' recall of memories of infancy
Fishing and the error rate problem example
a researcher compares two methods on fifty dependent variables and bases conclusions on two significant findings
Confounding constructs and levels of constructs example
a researcher concludes that music enhances exercise programs when this is only true for certain kinds of music
Low statistical power example
a researcher concludes that two instructional approaches are equally effective when an experiment with five students in each group yields no significant differences
Violated assumptions of statistical tests example
a researcher reports means and variances of nominal scaled data such as ethnicity
Mono-operation bias example
an experiment concerning the effect of feedback includes only written feedback
Mono-method bias example
an experiment on reducing test anxiety uses only a paper-pencil self-report of anxiety
Multiple-Treatment Interference Example
in a drug experiment the same animals are administered four different drug doses in some sequence. The effects of the second and fourth doses cannot be separated from possible (delayed) effects of preceding doses.