600 Chapt. 7 & 8 - Random Assignment, Control Techniques and and Types of Tests

Strengths and Weaknesses of "WITHIN" Participants / Repeated Measure Designs

(S) Because the same people participate in all experiments, participants serve as their own control and variables such as age, gender , and prior experience remain constant over the experiment. (S) Fewer research participants needed (S) If Counterbalancing is used, controls are in place for all the common threats to internal validity. (W) Taxing on participants because they have to be present for ALL treatments. (W) there is a confounding influence of a sequencing effect. Must use control techniques such as counterbalancing to refute these.

Strengths and Weaknesses of Posttest Only Between Groups

(S) includes more than one experimental group and allow us to pose and address more research questions. (W) you must have the # of participants needed for each test (There's more than just one test to perform) (W) Randomization does not provide complete assurance that the necessary equivalence has been attained. Esp. if the group is >30 participants. If there is ANY doubt that random assignment will NOT work, it is advisable to combine matching and statistical control along with randomization technique. (W) Lacks a pretest (checks the success of the randomized process)

3 Reasons Random Assignment is important during research

1. . It is the only method for controlling unknown extraneous variables by holding them constant 2. . Provides maximum assurance that systematic differences between groups, that might bias the results, are eliminated. 3. As long as a sufficient sample size is used, a researcher can reasonably assume that random assignment will produce groups that are approximately equal.

5 Advantages to having a Pretest

1. Allows the researcher to see how well the randomized process worked. (random assignment) This way a researcher can check the similarity of the participants BEFORE the experiment takes place to ensure they are equal to what is relevant for the study (motivation intelligence, attitudes) 2. Check for Ceiling Effect - occurs when the participants scores on the dependent variable are so high that they cannot go up from pretest to posttest. 3. Check for Floor effect - occurs when scores are so low that they cannot go down from a pretest or posttest. 4. If the experimental and control groups are slightly different on the dependent variable an Analysis of covariance test can be used to adjust for pretest differences. 5. To gain a perspective of an empirical demonstration of whether an overall change in response occurred from pretesting to posttesting. Researchers must be careful doing this because external validity of already taking the test could pose threats to the real test. Researchers feel that this outweighs the advantages of the pretest.

3 Types of Mixed Designs

1. Between participants/subject variable - type of independent variable where different participants receive different levels of the IV. 2. Within-participants/subject variable - type of independent variable where all participants receive all levels of the IV. 3. Pretest-Posttest control group design - Administration of posttest on two or more randomly assigned groups of participants after the groups have been pretested and administered the different levels of the IV.

4 Types of Experimenter Controls

1. Blind Technique 2. Partial Blind Technique 3. Double Blind Placebo Method 4. Automation

2 Types of GROUP Counterbalancing

1. Complete Counterbalancing - All possible sequences of treatment conditions are used. Paticipants are randomly assigned to sequence. N! = N multiplied by each number below it. For 2 treatments it is 2! 1,2 and 2,,1. For 3 the # of sequences are 3! 3x2x1 or 1,2,3;1,3,2;2,3,1;3,1,2;3,2,1 ...the possible # of sequences can be substantial for just 4! and 5! therefore, this is rarely used when the researcher has 3 or more treatment conditions. 2. Incomplete Counterbalancing - (most frequently used) whereby all possible sequences of treatment conditions are NOT listed AND each treatment condition MUST appear an equal number of times in each ordinal position (ABDC, BCAD, CDBA, DACB). SEE IMAGE for examples of INCOMPLETE COUNTERBALANCING SAMPLES in even and odd #s of sequences. -1, 2, n, 3, (n-1), 4, (n-2), 5

Advantages to Matching

1. Controls for the variables on which participants are matched 2. Increases the sensitivity of the experiment

Weaknesses to Matching

1. Groups are equated only on the matching variables which restricts the populations size. 2. Identifying the variables on which to match can be difficult to identify 3. As the # of variables on which to match increase, the # of participants increases 3. Restricts generalization to the type of participants in the study. SEE IMAGE

7 Threats to Internal Validity on strong experimental designs

1. History 2. Maturation 3. Instrumentation 4. Testing 5. Regression artifact AKA Regression to the mean (one of the trickiest threats to validity - requires extra research for this topic) 6. Attrition (no shows and dropouts) 7. Additive/interaction effects (experimenter effects)

4 MAJOR Disadvantages of Individual Matching without RANDOM ASSIGNMENT as the final setp

1. It is difficult to know which matching variables should be used or which are most critical. Ideally you're trying to find variables that comparison groups differ on but are related to the DV. The ideal situation is the variables selected should be those that show the LOWEST inter-correlation between each other, but are the highest with the dependent variable. 2. In order to match individuals on many variables you must have a large pool of individuals to choose in order to obtain a few who are matched on the relevant variables to the study. 3. Matching can limit the generalization of the results because if you have to throw out participants for whom you cannot find adequate matches then you will produce unrepresentative samples. 4. Some variables are very difficult to use in matching (ex. If you want to equate individuals on the basis of the effect of psychotherapy, you will have to measure that effect, and find people who have had psychotherapy.)

5 Controls for Reducing Experimenter Errors

1. Make researchers aware of making careful observations (may require training) 2. Use multiple data recorders (video cameras, computers, observers) 3. have participants make responses on a computer 4. control the attributes of the experimenter 5. Using the same experimenter in all treatment conditions unless a treatment condition interacts with attributes.

3 Types of Weak Designs

1. One-Group Posttest-only Design 2. One-Group Pretest-Posttest Design 3. Posttest-only Design with nonequivalent Groups

2 Types of Counterbalancing

1. Order Effect - A sequencing effect arising from the order in which the treatment conditions are administered to participants. (Ex. Participants get to Practice the experiment OR Participants get Fatigued between administrations of treatments) 2. Carryover Effect - A sequencing effect that occurs when performance in one treatment condition affects performance in another treatment condition (ex. effects of treatment A still affecting participant when engagement in treatment B)

4 Specific Types of Strong Experiment Designs

1. Posttest ONLY "Between" Group Design 2. WITHIN Participants/Repeated Measure Designs 3. Mixed Designs 4. Factorial Designs

2 Functions of the Control Group

1. Provides a Couterfactual comparison group (what the participants' responses would have been had they not received the treatment) 2. Controls for rival hypothesis - the goal is to get all variables to operate on the control and experimental groups identically, except for the one variable.

3 Types of Concurrent Verbal Reports:

1. Sacrifice Groups - each group of participants is "sacrificed" by being stopped at different point in the experiment and probed regarding the participants' perceptions of the experiment. 2. Concurrent verbal probing - requires participants to report their perceptions at the end of each trial 3. Think-aloud technique - requires participants to verbalize any thoughts or perceptions they have regarding the experiment while they are performing the experimental task.

Random Assignment

A control technique to equate groups of participants (ensuring that very member has an equal chance of being assigned to any group) -Provides maximum insurance that groups are equal --Eliminates systematic differences between groups --Doesn't eliminate extraneous variables, but randomly distributes them across groups --The most important and powerful control technique www.randomizer.org

Yolked Control (A Matching Technique)

A matching technique that matches participants on the basis of the temporal sequencing of administering an event. ex. Matching Monkeys using the Yolked Method If a monkey had to push a lever every 20 seconds in order to not receive a shock, and we wanted to know if the "stress" of that situation is what causes ulcers, then we have to have another monkey hooked up to the shocking that is NOT in control of the shock AND can't see what's going on. This way we can control the variable of "stress" inducing ulcers.

Blind Technique

A method whereby knowledge of each research participant's treatment condition is kept from the experimenter.

Concurrent Verbal Report

A participant's oral report of the experiment, which is obtained as the experiment is being performed.

One-group posttest-only design

A single group of research participants is measured on a dependent variable after having undergone an experimental treatment. Ex. Taking a survey at the end of a program to evaluate if the program was effective. This design excludes pretesting the individuals or using a separate no-treatment group to compare

Differential Carryover Effects

A treatment condition, when using repeated measures, that affects participants' performance in a later condition in one way and in another way when followed by a different condition (NO TYPE OF COUNTERBALANCING CAN CONTROL THIS SO IF THIS IS ENCOUNTERED, DON'T USE REPEATED MEASURES)

Counterbalancing

ALL participants receive ALL treatment conditions. Used only with repeated measures/within participants designs. SEE IMAGE

Posttest ONLY "Between" Group Design

Administration of a posttest to two or more randomly assigned groups of participants that receive the different levels of the IV. In this test all the groups are treated equal except for the IV. Any extraneous threats are controlled because they have an equal chance of occurring in either group. Ex. 3 groups of college students testing to see which were more likely to graduate (1) meets with professor each month (2) meets with professor every 2 months (3) do not meet with professor

"WITHIN" Participants / Repeated Measure Designs

All participants receive all conditions (uses a within-subjects IV and a posttest is administered after each condition is administered.) Uses Counterbalancing (giving the treatment in different orders) to average out any carryover affects. Ex. 3 different breakfasts administered to see the effects on cognitive performance. Different elementary students were asked on three different days to eat cereal, oatmeal, and no breakfast and each day their performance was measured. Counterbalancing was used because each one had a different order they ate the breakfasts. SEE IMAGE where X=posttest and O=treatment

Retrospect Verbal Report

An oral report in which the participant retrospectively recalls aspects of the experiment. AKA Postexperimental Inquiry questions like: what did the participant think the experiment was about? What did they think the experimenter expected to find? What type of response did the participant attempt to give, and why? How does the participant think others will respond in this situation?

Factorial Designs - Using Cells (includes cell interaction descriptions and definitions)

DEF of CELLS: Combination of two or more IVs. (SEE IMAGE) To get the total # of cells to use multiply the # of IVs by the # of categories. (ex. If you want to use 3 IVs of one study and 2 IVs for another study you will have 6 cells in the diagram - A1B1, A2B1, A3B1, A2B1, A2B2, A3B2). From here you calculate the cell and marginal mean for each group. Then the researcher can calculate the main effect and the interaction effect. Cell mean - the average score of the participants in a single cell Marginal Mean - the average score of all participants receiving one level of an IV. Main effect - The influence of one independent variable on the dependent variable Interaction effect - When the effect of two or more Ivs on the DV is more complex than indicated by the main effect. Two Way interaction - occurs when the effect of one IV on the DV varies at the different levels of other IVs. (ex. If you are studying the amount of sleep and the effects of caffeine consumption, you analyze the data for two MAIN effects (one for each IV - such as caffeine consumption and amount of sleep) and one INTERACTION effect (for the "interaction" of the two IVs.)

Experimental Designs

Designs that effectively control extraneous variables and provide strong evidence of cause and effect. Internal Validity is improved by eliminating rival hypothesis through control techniques and control groups.

Goal of Experimentation

Identify the causal effect of the IV and the DV Internal validity is derived from controlling extraneous variables and elimination of different influences (confounds)

Double-Blind placebo Method (A type of Experimenter Control)

Nether the experimenter nor the research participant is aware of the treatment condition administered to the participant. (Used extensively by drug researchers to eliminate participant bias) This type procedure eliminates the development of differential participation perception because all participants are told the same thing.

Deception (During the Control of Participant Effects)

Omission of or altering the truth of information given to the participant during a research study. Used when there is no other way to gain the knowledge and risk does not outweigh the benefit of the information must keep the false information constant for all participants

One MAJOR Advantage of Individual Matching

Participants in the various groups are equal on the matched variable(s) which rules out these extraneous variables as rival explanations that future critics might counter as the reason for the final analysis. SEE IMAGE

Intrasubject Counterbalancing

Participants take treatments in more than one order. May not be feasible with long treatment sequences. (ex. Pepsi challenge where each participant tastes A(pepsi) and then B(coke) and makes a judgement and then tastes B(coke) and then A(pepsi) and makes an analysis. (ABBA AKA 2 Treatment-condition experiment) or for AB&C it would be ABCCBA

Randomized Counterbalancing

Sequence order is randomly determined for each individual (ex. If you have three different types of therapy and each participant experiences each one but the order for the different types are randomized: 1,2,3; 1,3,2; 2,3,1; 2,3,2:3,1,2 and 3,2,1)

Automation

The technique of totally automating the experimental procedures so that no experimenter - participant interaction is required.

Matching

The use of any of a variety of techniques to equate participants in the treatment groups on specific variables. When random assignment is not possible, matching can be an effective technique to equate groups.

Factorial Designs

Two or more IVs are studied to determine their separate and joint effects on the dependent variable.

How to ELIMINATE potential rival hypothesis

Use Control Techniques

Individual / Subject matching

Where individual participants are matched with second participants and then from those groups are randomly selected to treatment groups. See Image

Blocking

building the extraneous variable into the research (ex. when testing on intelligence, divide the participants into the same IQ score groups and then from the groups do random assignment.)

Weak experimental designs

control for very few threats to internal validity. Avoid these designs when a strong design can be used.

Statistical Control

control of measured extraneous variables during data analysis

One-Group Pretest-Posttest Design

design in which a treatment condition is interjected between pretest and posttest of the dependent variable. No Control Group is used.

Posttest-only design with nonequivalent Groups

design in which the performance of an experimental group is compared with that of a nonequivalent control group at the posttest. In this experiment you could have nonequivalent groups going into the study. The problem with this is the selection. The only way to ensure that the groups are equated is to assign participants randomly to the two groups. If this is not possible the next best technique is to match on relevant variables. This is not a substitute for random assignment, though.

Randomize Control Trials

experimental designs with random assignment to both Experimental Groups and Control Groups Experimental Group def. (AKA treatment group) - a group of research participants that receive some level of the independent variable that is intended to produce an effect. Control group def. - a group of research participants that do NOT receive the active level of the independent variable; they might either receive zero amount of the IV or receive an amount that is in some sense a standard value, such as what they would typically receive if they were not participants in the research.

Experimenter Effects

the biasing influence that can be exerted by the experimenter. Each experiment requires that the experimenter attributes be minimized by the researcher for "those experimenter attributes which correspond with the psychological task" (Johnson 1976 pg.95) (Ex. On hostility-related tasks, it is necessary to hold the experimenters' hostility constant, In weight reduction experiments, the weight of the therapist might need to be approximately the same weights as the participants...)

Research Design

the outline, plan, or strategy used to investigate the research problem.

The Partial Blind Technique

whereby the experimenter is kept ignorant of the condition of the research participant is in for a portion of the study. This is only a partial solution, but it is better than the experimenter's having knowledge of the participant's condition throughout the experiment. If the experimenter can leave the room following the experiment with a 3rd party (who is also unknowledgeable about the condition) measure the DV, then the solution would come closer to approaching completeness.