Research Methods Chapter 5-6
Lewis Terman
-First, he transformed the intelligence test created by Alfred Binet of France into the popular Stanford‐Binet IQ test. -Second, he began a longitudinal study of gifted children that continued long after he died
block randomization
a process of randomization that first creates treatment blocks containing one random order of the conditions in the experiment; subjects are then assigned to fill each successive treatment block
experiment
a systematic research study in which the investigator directly varies some factor (or factors), holds all other factors constant, and observed the results of the variation
participant bias
a tendency for research participants to respond in a certain way because they know they are being observed, or they believe they know what the researcher wants
subject selection effects
a threat to the internal validity of a study; occurs when those participating in a study cannot be assigned randomly to groups; hence the groups are nonequivalent
R.S woodworth
after introducing his now famous distinction between independent and dependent variables, R. S. Woodworth emphasized the importance of control in experimental research
subject variable
already existing characteristics of the individual participating in the study, such as gender, age, socioeconomic class, cultural group, intelligence, disorders etc.
double-blind procedure
an experimental procedure in which both the research participants and the research staff are ignorant (blind) about whether the research participants have received the treatment or a placebo. Commonly used in drug-evaluation studies.
confound
any uncontrolled extraneous variable that co‐varies with the independent variable and could provide an alternative explanation of the results. That is, a confounding variable changes in the same way that an independent variable changes (i.e., they co‐vary) and, consequently, its effect cannot be distinguished from the effect of the independent variable.
Situational variables
are features in the environments that participants might encounter
instructional variables
are manipulated by telling different groups to perform a particular task in different ways
random assignment
assigning participants to experimental and control conditions by chance, thus minimizing preexisting differences between those assigned to the different groups
progressive effect
because it is assumed that performance changes steadily (progressively) from trial to trial
Hawthorne effect
behavior is affected by the knowledge that one is in an experiment and is therefore important to the study's success
does cross sectional approach use between subjects or within
between
how to control order effects in "testing once per condition"
complete or partial counterbalancing
pretest
evaluate people prior to the experience
construct validity in experimental research
experimental research, construct validity has a related meaning, referring to the adequacy of the operational definitions for both the independent and the dependent variables used in the study
research protocols
experiments have highly detailed descriptions of the sequence of steps that experimenters should follow in every research session
external validity
extent to which we can generalize findings to real-world settings
how to manipulate test variables
give participants different kinds of problems to solve
floor effect
happens when all the scores are extremely low, usually because the test is too difficult for everyone, once again producing a failure to find any differences between conditions.
within-subjects design
if each participant receives both levels A and B, you could say both levels exist within each individual participating in the study;
demand characteristics
if participants can figure out the hypothesis, they may try to behave in a way that confirms it.
error bars
indicate the amount of variability that occurred within each condition.
instrumentation
is a problem when the measurement instrument changes from pretest to posttest. In the self-paced psychology course mentioned earlier, the pretest and posttest wouldn't be the same but would presumably be equivalent in level of difficulty.
order effects
major problem is that once a participant has completed the first part of a study, the experience or altered circumstances could influence performance in later parts of the study
what is another word for independent variable
manipulated variable or factor
posttest
measure is taken
what is the problem with complete counterbalancing
number of levels of the independent variable increases and the possible orders needed increases
ceiling effect
occurs when the average scores for the groups in the study are so high that no difference can be determined between conditions.
disadvantage of within-subjects design
order effects
does longitudinal approach use between subjects or within
takes a within‐subjects or repeated measures approach in which a single group of subjects is studied over time
statistical conclusion validity
the extent to which the researcher uses statistics properly and draws the appropriate conclusions from the statistical analysis.
regression to the mean
the first score from a subject is an extreme score, then the second or third score from the same person will be closer to whatever the mean is for the larger set of scores. This is because, for a large set of scores, most will cluster around the mean and only a few will be far removed from the mean (i.e., extreme scores).
deception
the primary purpose of deception is to induce participants to behave more naturally than they otherwise might
experiment group
treatment is present
what do all independent variables require?
two levels(conditions)
task variables
type of independent variable in which participants are given different types of tasks to perform
advantages of within subjects designs
-Fewer participants needed to detect differences -eliminate problems creating equivalent groups -convenience
between subject designs, significant experimental effects could be due to
- 1) Chance - 2) Differences in our independent variable - 3) Individual differences in our experimental groups
advantages of within subjects design
-Fewer participants needed -Eliminates possibility that differences between levels of the independent variable could be due to individual differences - So differences cannot be attributed to participants because all participants are in all conditions
when to use matching
-Small numbers per group might cause random assignment to fail -Can use IQ tests to do this to make sure smartest people don't happen to end up in one group (when there is a small number of participants) -The matching variable must correlate with the dependent variable - Measuring the matching variable is feasible
experimental bias
a mistake in the design of an experiment that makes a particular result more likely
complete counterbalancing
-Whenever participants are tested once per condition in a within‐subjects design, one solution to the order problem is to use complete counterbalancing
advantages and disadvantages of cross sectional
-advantage of the cross‐sectional approach to the experiment on language is time; data collection for a study comparing 3‐, 4‐, and 5‐year‐olds might take a month. If done as a longitudinal study, data collection would take at least 2 years - However, a potentially serious difficulty with some cross‐sectional studies is a special form of the problem of non-equivalent groups and involves cohort effects
in within-subject designs, significant effects could be due to
-chance -differences in independent variable
manipulation check
-check for demand characteristics -This can be accomplished during debriefing by asking participants in a deception study to indicate what they believe the true hypothesis to be (the "good subject" might feign ignorance, though)
how to control order effects in within-subject designs
-counterbalancing -the procedure works better for progressive effects than for carryover effects
longitudinal disadvantages
-longitudinal studies also have problems, most notably with attrition -Longitudinal studies also pose potential ethical problems. As people develop and mature, they might change their attitudes about their willingness to participate
how can the statistical conclusion validity be violated
-researchers might do the wrong analysis or violate some of the assumptions required for performing a particular analysis -Another factor reducing the statistical validity of a study concerns the reliability of the measures used. If the dependent measures are not reliable, there will be a great deal of error variability, which reduces the chances of finding a significant effect.
testing more than one per condition
-reverse counterbalancing -block randomization
what conditions are needed for matching
1) you must have good reason to believe the matching variable will have a predictable effect on the outcome of the study—that is, you must be confident that the matching variable would be correlated with the dependent variable 2) there must be a reasonable way of measuring or identifying participants on the matching variable
between-subjects design
A research design in which different groups of participants are randomly assigned to experimental conditions or to control conditions.
random selection
A way of ensuring that a sample of people is representative of a population by giving everyone in the population an equal chance of being selected for the sample
control group
In an experiment, the group that is not exposed to the treatment; contrasts with the experimental group and serves as a comparison for evaluating the effect of the treatment.
matching procedure
In matching, participants are grouped together on some subject variable such as their characteristic level of anxiety and then distributed randomly to the different groups in the experiment
evaluate apprehension
Participants want to be evaluated positively, so they may behave as they think the ideal person should behave
how can testing impact internal validity?
Testing is considered a threat to internal validity when the mere fact of taking a pretest has an effect on posttest scores. There could be a practice of repeated testing, or aspects of the pretest could sensitize participants to something about the program.
independent variable
The experimental factor that is manipulated; the variable whose effect is being studied.
ecological validity
The extent to which a study is realistic or representative of real life.
dependent variable
The outcome factor; the variable that may change in response to manipulations of the independent variable.
cohort sequential design (trying to balance cohort and attrition problems)
a group of subjects is selected and retested every few years, and additional cohorts are selected every few years and also retested over time
blocked random assignment
a method of random assignment in which participants are assigned to conditions in sequential blocks, each of which contains all of the conditions
Extrenous variable
These are variables that are not of interest to the researcher but that might influence the behaviour being studied if not controlled properly.
placebo control group
This procedure allows for comparison of those actually getting some treatment (e.g., a drug) and those who think they are getting the treatment but aren't
carry over effects
Thus, in a study with two basic conditions, experiencing the first condition before the second might affect the person much differently than experiencing the second before the first
reverse counterbalancing
When using reverse counterbalancing, the experimenter simply presents the conditions in one order and then presents them again in the reverse order
partial counterbalancing
Whenever a subset of the total number of orders is used, the result is called partial counterbalancing or, sometimes, incomplete counterbalancing -This can be accomplished by taking a random sample of orders from the complete set of all possible orders or, more simply, by randomizing the order of conditions for each subject
balanced Latin square
With a perfectly balanced Latin square, you are assured that (a) every condition of the study occurs equally often in every sequential position (b) every condition precedes and follows every other condition exactly once
x versus y variable
X=independent Y=dependent
field research
a broader term for any empirical research outside of the laboratory including both experimental studies and studies using non- experimental methods.
are demand characterisics more troublesome for within or between
with-in subject design
