Ch.7 Experimental Research

Confounding constructs and levels of constructs threat

-construct validity threat, drawing conclusions about variables when some levels of the variables are absent

Inadequate Preoperational Explication of Constructs Threat

-construct validity threat, insufficient definition of the independent and/or dependent variables

Mono-operation bias threat

-construct validity threat, only one form of the experimental variable is implemented

Mono-method bias threat

-construct validity threat, only one form of the independent variable is tested

Hypothesis guessing within experimental conditions threat

-construct validity threat, participants behave differently when they know they are part of an experiment (the behavior of the subjects can also threaten internal validity)

Interaction Effects of Selection Biases and the Experimental Treatment Threat

-external validity threat, an effect of some selection factor of intact groups interacting with the experimental treatment that would not be the case if the groups were formed randomly

Reactive Effects of Experimental Arrangements Threat

-external validity threat, an effect that is due to the artificial or novel experimental setting (this can also threaten internal validity)

Interaction Effect of Testing Threat

-external validity threat, pretesting interacts with the experimental treatment and causes some effect such that the results will not generalize to an unpretested population.

Multiple-Treatment Interference Threat

-external validity threat, when the same subjects receive two or more treatments (As in repeated measures design). There may be a carryover effect between treatments such that the results cannot be generalized to single treatments.

Reactive Effects of Experimental Arrangements Example

An experiment in remedial reading instruction has an effect that does not occur when the remedial reading program is implemented in the regular classroom

History Threat Example

During a relatively short instructional experiment, one group of subjects misses instruction due to a power failure at the school.

Selection Threat Example

The experimental group in an instructional experiment consists of a high-ability class, while the control group is an average-ability class.

Interaction Effects of Selection Biases and the Experimental Treatment Example

The results of an experiment in which teaching method is the experimental treatment is effective with low achievers but is not as effective with high achievers

Instrumentation Threat Example

Two examiners in an instructional experiment administered the posttest with different instructions and procuedures.

Inadequate Preoperational Explication of Constructs Example

Two teachers implement individualized instruction in very different ways because they lack a precise definition of a term.

Interaction in an experiment

an effect on the dependent variable such that the effect of one independent variable changes over the levels of another independent variable

Repeated measures designs

designs in which the same subject is measured more than once on the dependent variable

Time series designs

involve repeated measurement with an experimental treatment inserted between two of the treatments

Testing Threat

-internal validity threat, The effect of taking one test on the scores of a subsequent test.

Statistical Regression Threat

-internal validity threat, an effect caused by a tendency for subjects selected on the basis of extreme scores to regress toward an average performance on a subsequent test

Instrumentation Threat

-internal validity threat, an effect due to inconsistent use of the measuring instruments

Selection Threat

-internal validity threat, an effect due to the groups of subjects not being randomly assigned to groups; a selection factor is operating such that the groups are not equivalent

Mortality Threat

-internal validity threat, in a health experiment designed to determine the effects of various exercises, those subjects finding exercise most difficult stop participating.

Selection-Maturation Interaction Threat

-internal validity threat, in a problem-solving experiment, intact groups of junior high school students and senior high students are involved. The junior high students tire of the task sooner than the older students.

Maturation Threat

-internal validity threat, processes operating within the subject as a function of time.

History Threat

-internal validity threat, unanticipated events occurring while the experiment is in progress that affect the dependent variable.

Fishing and the error rate problem

-statistical conclusion validity threat, capitalizing on chance findings

Violated assumptions of statistical tests

-statistical conclusion validity threat, failing to meet the underlying assumptions

Low statistical power

-statistical conclusion validity threat, using a sample size that is too small to detect differences between groups

Reliability of measures

-statistical conclusion validity threat, using technically inadequate measures

Maturation Threat Example

In a learning experiment, subject performance begins decreasing after about fifty minutes due to fatigue.

Interaction Effect of Testing Threat Example

In a physical performance experiment, the pretest cues the subjects to respond in a certain way to the experimental treatment that would not be the case if there were no present.

Testing Threat Example

In an experiment in which performance on a logical reasoning test is the dependent variable, a pretest cues the subjects about the posttest.

Statistical Regression Example

In an experiment involving reading instruction, subjects grouped because of poor pretest reading scores show considerably greater gains than the average and high readers.

Hypothesis guessing within experimental conditions example

Subjects in an experimental program interact differently when they know they are being studied

Solomon four-group design

a combination of the posttest-only control group design and the pretest-posttest control group design

Reliability of measures example

a psychologist finds no difference between boys' and girls' recall of memories of infancy

Fishing and the error rate problem example

a researcher compares two methods on fifty dependent variables and bases conclusions on two significant findings

Confounding constructs and levels of constructs example

a researcher concludes that music enhances exercise programs when this is only true for certain kinds of music

Low statistical power example

a researcher concludes that two instructional approaches are equally effective when an experiment with five students in each group yields no significant differences

Violated assumptions of statistical tests example

a researcher reports means and variances of nominal scaled data such as ethnicity

Mono-operation bias example

an experiment concerning the effect of feedback includes only written feedback

Mono-method bias example

an experiment on reducing test anxiety uses only a paper-pencil self-report of anxiety

Multiple-Treatment Interference Example

in a drug experiment the same animals are administered four different drug doses in some sequence. The effects of the second and fourth doses cannot be separated from possible (delayed) effects of preceding doses.