Research Methods Chapter 5-6

Lewis Terman

-First, he transformed the intelligence test created by Alfred Binet of France into the popular Stanford‐Binet IQ test. -Second, he began a longitudinal study of gifted children that continued long after he died

block randomization

a process of randomization that first creates treatment blocks containing one random order of the conditions in the experiment; subjects are then assigned to fill each successive treatment block

experiment

a systematic research study in which the investigator directly varies some factor (or factors), holds all other factors constant, and observed the results of the variation

participant bias

a tendency for research participants to respond in a certain way because they know they are being observed, or they believe they know what the researcher wants

subject selection effects

a threat to the internal validity of a study; occurs when those participating in a study cannot be assigned randomly to groups; hence the groups are nonequivalent

R.S woodworth

after introducing his now famous distinction between independent and dependent variables, R. S. Woodworth emphasized the importance of control in experimental research

subject variable

already existing characteristics of the individual participating in the study, such as gender, age, socioeconomic class, cultural group, intelligence, disorders etc.

double-blind procedure

an experimental procedure in which both the research participants and the research staff are ignorant (blind) about whether the research participants have received the treatment or a placebo. Commonly used in drug-evaluation studies.

confound

any uncontrolled extraneous variable that co‐varies with the independent variable and could provide an alternative explanation of the results. That is, a confounding variable changes in the same way that an independent variable changes (i.e., they co‐vary) and, consequently, its effect cannot be distinguished from the effect of the independent variable.

Situational variables

are features in the environments that participants might encounter

instructional variables

are manipulated by telling different groups to perform a particular task in different ways

random assignment

assigning participants to experimental and control conditions by chance, thus minimizing preexisting differences between those assigned to the different groups

progressive effect

because it is assumed that performance changes steadily (progressively) from trial to trial

Hawthorne effect

behavior is affected by the knowledge that one is in an experiment and is therefore important to the study's success

does cross sectional approach use between subjects or within

between

how to control order effects in "testing once per condition"

complete or partial counterbalancing

pretest

evaluate people prior to the experience

construct validity in experimental research

experimental research, construct validity has a related meaning, referring to the adequacy of the operational definitions for both the independent and the dependent variables used in the study

research protocols

experiments have highly detailed descriptions of the sequence of steps that experimenters should follow in every research session

external validity

extent to which we can generalize findings to real-world settings

how to manipulate test variables

give participants different kinds of problems to solve

floor effect

happens when all the scores are extremely low, usually because the test is too difficult for everyone, once again producing a failure to find any differences between conditions.

within-subjects design

if each participant receives both levels A and B, you could say both levels exist within each individual participating in the study;

demand characteristics

if participants can figure out the hypothesis, they may try to behave in a way that confirms it.

error bars

indicate the amount of variability that occurred within each condition.

instrumentation

is a problem when the measurement instrument changes from pretest to posttest. In the self-paced psychology course mentioned earlier, the pretest and posttest wouldn't be the same but would presumably be equivalent in level of difficulty.

order effects

major problem is that once a participant has completed the first part of a study, the experience or altered circumstances could influence performance in later parts of the study

what is another word for independent variable

manipulated variable or factor

posttest

measure is taken

what is the problem with complete counterbalancing

number of levels of the independent variable increases and the possible orders needed increases

ceiling effect

occurs when the average scores for the groups in the study are so high that no difference can be determined between conditions.

disadvantage of within-subjects design

order effects

does longitudinal approach use between subjects or within

takes a within‐subjects or repeated measures approach in which a single group of subjects is studied over time

statistical conclusion validity

the extent to which the researcher uses statistics properly and draws the appropriate conclusions from the statistical analysis.

regression to the mean

the first score from a subject is an extreme score, then the second or third score from the same person will be closer to whatever the mean is for the larger set of scores. This is because, for a large set of scores, most will cluster around the mean and only a few will be far removed from the mean (i.e., extreme scores).

deception

the primary purpose of deception is to induce participants to behave more naturally than they otherwise might

experiment group

treatment is present

what do all independent variables require?

two levels(conditions)

task variables

type of independent variable in which participants are given different types of tasks to perform

advantages of within subjects designs

-Fewer participants needed to detect differences -eliminate problems creating equivalent groups -convenience

between subject designs, significant experimental effects could be due to

- 1) Chance - 2) Differences in our independent variable - 3) Individual differences in our experimental groups

advantages of within subjects design

-Fewer participants needed -Eliminates possibility that differences between levels of the independent variable could be due to individual differences - So differences cannot be attributed to participants because all participants are in all conditions

when to use matching

-Small numbers per group might cause random assignment to fail -Can use IQ tests to do this to make sure smartest people don't happen to end up in one group (when there is a small number of participants) -The matching variable must correlate with the dependent variable - Measuring the matching variable is feasible

experimental bias

a mistake in the design of an experiment that makes a particular result more likely

complete counterbalancing

-Whenever participants are tested once per condition in a within‐subjects design, one solution to the order problem is to use complete counterbalancing

advantages and disadvantages of cross sectional

-advantage of the cross‐sectional approach to the experiment on language is time; data collection for a study comparing 3‐, 4‐, and 5‐year‐olds might take a month. If done as a longitudinal study, data collection would take at least 2 years - However, a potentially serious difficulty with some cross‐sectional studies is a special form of the problem of non-equivalent groups and involves cohort effects

in within-subject designs, significant effects could be due to

-chance -differences in independent variable

manipulation check

-check for demand characteristics -This can be accomplished during debriefing by asking participants in a deception study to indicate what they believe the true hypothesis to be (the "good subject" might feign ignorance, though)

how to control order effects in within-subject designs

-counterbalancing -the procedure works better for progressive effects than for carryover effects

longitudinal disadvantages

-longitudinal studies also have problems, most notably with attrition -Longitudinal studies also pose potential ethical problems. As people develop and mature, they might change their attitudes about their willingness to participate

how can the statistical conclusion validity be violated

-researchers might do the wrong analysis or violate some of the assumptions required for performing a particular analysis -Another factor reducing the statistical validity of a study concerns the reliability of the measures used. If the dependent measures are not reliable, there will be a great deal of error variability, which reduces the chances of finding a significant effect.

testing more than one per condition

-reverse counterbalancing -block randomization

what conditions are needed for matching

1) you must have good reason to believe the matching variable will have a predictable effect on the outcome of the study—that is, you must be confident that the matching variable would be correlated with the dependent variable 2) there must be a reasonable way of measuring or identifying participants on the matching variable

between-subjects design

A research design in which different groups of participants are randomly assigned to experimental conditions or to control conditions.

random selection

A way of ensuring that a sample of people is representative of a population by giving everyone in the population an equal chance of being selected for the sample

control group

In an experiment, the group that is not exposed to the treatment; contrasts with the experimental group and serves as a comparison for evaluating the effect of the treatment.

matching procedure

In matching, participants are grouped together on some subject variable such as their characteristic level of anxiety and then distributed randomly to the different groups in the experiment

evaluate apprehension

Participants want to be evaluated positively, so they may behave as they think the ideal person should behave

how can testing impact internal validity?

Testing is considered a threat to internal validity when the mere fact of taking a pretest has an effect on posttest scores. There could be a practice of repeated testing, or aspects of the pretest could sensitize participants to something about the program.

independent variable

The experimental factor that is manipulated; the variable whose effect is being studied.

ecological validity

The extent to which a study is realistic or representative of real life.

dependent variable

The outcome factor; the variable that may change in response to manipulations of the independent variable.

cohort sequential design (trying to balance cohort and attrition problems)

a group of subjects is selected and retested every few years, and additional cohorts are selected every few years and also retested over time

blocked random assignment

a method of random assignment in which participants are assigned to conditions in sequential blocks, each of which contains all of the conditions

Extrenous variable

These are variables that are not of interest to the researcher but that might influence the behaviour being studied if not controlled properly.

placebo control group

This procedure allows for comparison of those actually getting some treatment (e.g., a drug) and those who think they are getting the treatment but aren't

carry over effects

Thus, in a study with two basic conditions, experiencing the first condition before the second might affect the person much differently than experiencing the second before the first

reverse counterbalancing

When using reverse counterbalancing, the experimenter simply presents the conditions in one order and then presents them again in the reverse order

partial counterbalancing

Whenever a subset of the total number of orders is used, the result is called partial counterbalancing or, sometimes, incomplete counterbalancing -This can be accomplished by taking a random sample of orders from the complete set of all possible orders or, more simply, by randomizing the order of conditions for each subject

balanced Latin square

With a perfectly balanced Latin square, you are assured that (a) every condition of the study occurs equally often in every sequential position (b) every condition precedes and follows every other condition exactly once

x versus y variable

X=independent Y=dependent

field research

a broader term for any empirical research outside of the laboratory including both experimental studies and studies using non- experimental methods.

are demand characterisics more troublesome for within or between

with-in subject design