Test three ch. 8 - 10

a Quasi-experimental Design with Nonequivalent Groups - a Quasi-experimental Design with Nonequivalent Groups- Chapter 10

much stronger design can be created with a small modification of the posttest-only nonequivalent control group design. The modification involves adding a pretest that obtains measurements of both groups before the treatment is administered. The result- ing design is called a pretest-posttest nonequivalent control group design and can be represented as follows: O X O (treatment group) O O (nonequivalent control group) In this design, the first step is to observe (measure) both groups. The treatment is then administered to one group, and, following the treatment, both groups are observed again. The addition of the pretest measurement allows researchers to address the problem of individual differences as a confound that exists with all nonequivalent group research. Specifically, the researcher can now compare the observations before treatment to estab- lish whether the two groups really are similar. If the groups are found to be similar before reatment, the researcher has evidence that the participants in one group are not substan- tially different from the participants in another group, and the threat of individual differ- ences is reduced. Note, however, that the pretest scores simply allow the researcher to ensure that the two groups are similar with respect to one specific variable. Other poten- tially important variables are not measured or controlled. Thus, the threat of individual differences is reduced, but it is certainly not eliminated. This type of design also allows a researcher to compare the pretest scores and posttest scores for both groups to help determine whether the treatment or some other, time-related factor is responsible for changes. In Chapter 9, we introduced a set of time-related factors such as history and maturation that can threaten internal validity. In the pretest-posttest nonequiva- lent groups design, however, these time-related threats are minimized because both groups are observed over the same time period and, therefore, should experience the same time-related fac- tors. If the participants are similar before treatment but different after treatment, the researcher can be more confident that the treatment has an effect. On the other hand, if both groups show the same degree of change from the pretest to the posttest, the researcher must conclude that some factor other than the treatment is responsible for the change. Thus, the pretest-posttest nonequivalent control group design reduces the threat of individual differences, limits threats from time-related factors, and can provide some evidence to support a cause-and-effect rela- tionship. As a result, this type of research is considered quasi-experimental. A pretest-posttest nonequivalent control group design compares two non- equivalent groups. One group is measured twice, once before a treatment is administered and once after. The other group is measured at the same two times but does not receive any treatment. Because this design attempts to limit threats to internal validity, it is classified as quasi-experimental Although the addition of a pretest to the nonequivalent control group design reduces some threats to internal validity, it does not eliminate them completely. In addition, the fact that the groups are nonequivalent and often are in separate locations creates the poten- tial for other threats. Specifically, it is possible for a time-related threat to affect the groups differently. For example, one group may be influenced by outside events that are not expe- rienced by the other group. The students in one high school may be enjoying a winning football season, whereas students in another school may be depressed because their team is losing every game. In Chapter 9, we identified the influence of outside events as history effects. When history effects differ from one group to another, they are called differential history effects. The differential effects can be a confounding variable because any differ- ences observed between the two groups may be explained by their different histories. In a similar way, other time-related influences such as maturation, instrumentation, testing effects, and regression may be different from one group to another, and these differential effects can threaten the internal validity of a nonequivalent group study.

Equivalent Groups - Chapter 8

n Chapter 7, we identified three general techniques for controlling confounding variables: randomization, matching, and holding constant. These techniques can be used to protect a study from confounding environmental variables. With a between-subjects design, how- ever, a researcher must also protect the study from individual differences between groups. Fortunately, with a between-subjects experimental design, the researcher has control over the assignment of individuals to groups. Thus, the researcher has both the opportunity and the responsibility to create groups that are equivalent. Specifically, the separate groups must be: 1. Created equally. The process used to obtain participants should be as similar as possi- ble for all of the groups. 2. Treated equally. Except for the treatment conditions that are deliberately var- ied between groups, the groups of participants should receive exactly the same experiences. 3. Composed of equivalent individuals. The characteristics of the participants in any one group should be as similar as possible to the characteristics of the participants in every other group. The techniques available for establishing equivalent groups of participants are dis- cussed in the following section

Limitations of Counterbalancing - Chapter 9

- As demonstrated in Table 9.2, counterbalancing can be used to prevent order effects (or other time-related effects) from confounding the results of a within-subjects research study. In the same way that random assignment is a routine technique for maintaining validity in between-subjects research, counterbalancing is a routine technique used in within-subjects research. However, this apparently simple and effective technique has some limitations.

the Structure of Nonexperimental and Quasi-experimental Designs - Chapter 10

- Nonexperimental and quasi-experimental studies often look like experiments in terms of the general structure of the research study. In an experiment, for example, a researcher typically creates treatment conditions by manipulating an independent variable, and then measures participants to obtain a set of scores within each condition. If the scores in one condition are significantly different from the scores in another condition, the researcher can conclude that the two treatment conditions have different effects (Figure 10.1). Similarly, a nonexperimental or quasi-experimental study also produces groups of scores to be compared for significant differences. One variable is used to create the groups or condi- tions, and then a second variable is measured to obtain a set of scores within each condition. In nonexperimental and quasi-experimental studies, however, the different groups or treatment conditions are not created by manipulating an independent variable. Instead, the groups are usually defined in terms of a specific participant variable (e.g., college graduate/no college) or in terms of time (e.g., before and after treatment). These two methods of defining groups produce two general categories of nonexperimental and quasi-experimental designs: 1. Between-subjects designs, also known as nonequivalent group designs 2. Within-subjects designs, also known as pre-post designs

threats to Internal Validity for Nonequivalent Group Designs - Chapter 10

A general example of a nonequivalent group design is shown in Figure 10.3. Notice that the groups are differentiated by one specific factor that identifies the groups. In the example evaluating in-class electronic devices, the differentiating factor was the school policy: one high school encouraged use and one banned use. Typically, the purpose of the study is to show that the factor that differentiates the groups is responsible for causing the participants' scores to differ from one group to the other. For this example, the goal is to show that the school policy concerning electronic devices is responsible for the different levels of student performance in the two schools. However, a nonequivalent group design has a built-in threat to internal validity that precludes an unambiguous cause-and-effect explanation. That threat was introduced in Chapter 6 as individual differences between groups. Recall that individual differences create a confound whenever the assignment procedure produces groups that have different participant characteristics. For example, the two high schools in the electronic device study may differ in terms of student IQs, socioeconomic background, racial mixture, student motivation, and so on. These variables are all potentially confounding variables because any one of them could explain the differences between the two groups. Because the assignment of participants is not controlled in a study using nonequivalent groups, this type of research always is threatened by individual differences. You may recognize that a non- equivalent groups study is similar to the between-subjects experimental design presented in Chapter 8. However, the experimental design always uses some form of random assign- ment or matching to ensure equivalent groups. In a nonequivalent groups design, there is no random assignment and there is no assurance of equivalent groups. In this section, we consider three common examples of nonequivalent group designs: (1) the differential research design, (2) the posttest-only nonequivalent control group design, and (3) the pretest-posttest nonequivalent control group design. The first two designs make no attempt to control or minimize individual differences as a confound and are nonexperimental designs. The third design is a modification of the posttest-only design and is classified as quasi-experimental because it does attempt to minimize the threat of individual differences as a confound.

advantages and Disadvantages of Between-Subjects Designs - Chapter 8

A main advantage of a between-subjects design is that each individual score is independent from the other scores. Because each participant is measured only once, the researcher can be reasonably confident that the resulting measurement is relatively clean and uncontaminated by other treatment factors. For this reason, a between-subjects experimental design is often called an independent-measures experimental design. In an experiment comparing per- formance under different temperature conditions, for example, each participant is exposed to only one treatment condition. Thus, the participant's score is not influenced by such factors as: • practice or experience gained in other treatments; • fatigue or boredom from participating in a series of different treatments; and • contrast effects that result from comparing one treatment to another (a 60-degree room might feel cold after a 70-degree room, but the same 60-degree room might feel warm after a 50-degree room). In addition, between-subjects designs can be used for a wide variety of research ques- tions. For any experiment comparing two (or more) treatment conditions, it is always pos- sible to assign different groups to the different treatments; thus, a between-subjects design is always an option. It may not always be the best choice, but it is always available. One disadvantage of between-subjects designs is that they require a relatively large number of participants. Remember, each participant contributes only one score to the final data. To compare three different treatment conditions with 30 scores in each treatment, the between-subjects design requires 90 participants. This can be a problem for research involving special populations in which the number of potential participants is relatively small. For example, a researcher studying preschool children with a specific learning dis- ability might have trouble finding a large number of individuals to participate.

Strengths and Weaknesses of the Longitudinal Developmental Design - Chapter 10

A major strength of the longitudinal research design is the absence of cohort effects because the researcher examines one group of people over time rather than comparing groups that represent different ages and come from different generations. Second, with longitudinal research, a researcher can discuss how a single individual's behavior changes with age. However, longitudinal research is extremely time-consuming, both for the par- ticipants (it requires a big commitment to continue in the study) and the researcher (the researcher must stay interested in the research and wait for years to see the final results). In addition, these designs are very expensive to conduct because researchers need to track people down and persuade them, when necessary, to come back to participate in the study. If the study spans many years, there is the additional expense of training new experiment- ers to take over the study. Furthermore, these designs are subject to high dropout rates of participants. People lose interest in the study, move away, or die. When participants drop out of a study, it is known as participant attrition (or participant mortality), and it may weaken the internal validity of the research. Specifically, if the participants who drop out are systematically different from those who stay, the group at the end of the study may have different characteristics from the group at the beginning. For example, if the less-motivated individuals drop out, then the group at the end is more motivated than the group at the beginning. The higher level of motivation (rather than age) may explain any changes that are observed over time. (The issue of participant attrition is discussed in more detail in Chapter 9.) A final weakness of the longitudinal research design is that the same individuals are measured repeatedly. It is possible that the scores obtained late in the study are partially affected by previous experience with the test or measurement procedure. (In Chapter 9, we discussed order effects as a threat to internal validity.) Table 10.2 summarizes the strengths and weaknesses of cross-sectional and longitudi- nal developmental research designs

nonequivalent group design - Chapter 10 def

A nonequivalent group design is a research study in which the different groups of participants are formed under circumstances that do not permit the researcher to control the assignment of individuals to groups, and the groups of participants are, therefore, considered nonequivalent. Specifically, the researcher cannot use random assignment to create groups of participants.

a Quasi-experimental pre-post Design - The Time-Series Design - Chapter 10

A simple modification of the pretest-posttest design minimizes the threats to internal validity and produces a much stronger research design. The modification consists of using a series of observations, in place of the single observation, before and after the treatment or event. The result is called a time-series design and can be represented as follows:time-series design has a series of observations for each participant before a treatment or event and a series of observations after the treatment or event. A treatment is a manipulation administered by the researcher, and an event is an out- side occurrence that is not controlled or manipulated by the researcher. The intervening treatment or event (X) may or may not be manipulated by the researcher. For example, a doctor may record blood pressure for a group of executives before and after they complete relaxation training. Or a researcher may evaluate the effect of a natural disaster such as earthquake or flood on the well-being of a group of students by recording visits to the school nurse for the months before and after the disaster. In one case, the researcher is manipulating a treatment (the relaxation training) and in the other case, the researcher is studying a nonmanipulated event (an earthquake). A study in which the intervening event is not manipulated by the researcher is sometimes called an interrupted time-series design. Occasionally, a time-series study is used to investigate the effect of a predictable event such as a legal change in the drinking age or speed limit. In this case, researchers can begin collecting data before the event actually occurs. However, it often is impossible to predict the occurrence of an event such as an earthquake, so it is impossible for research- ers to start collecting data just before one arrives. In this situation, researchers often rely on archival data, such as police records or hospital records, to provide the observations for the time-series study. In a time-series design, the pretest and posttest series of observations serve several valuable purposes. First, the pretest observations allow a researcher to see any trends that may already exist in the data before the treatment is even introduced. Trends in the data are an indication that the scores are influenced by some factor unrelated to the treatment. For example, practice or fatigue may cause the scores to increase or decrease over time before a treatment is introduced. Similarly, instrumentation effects, maturation effects, or before a treatment is introduced. Similarly, instrumentation effects, maturation effects, or regression should produce noticeable changes in the observations before treatment. On the other hand, if the data show no trends or major fluctuations before the treatment, the researcher can be reasonably sure that these potential threats to internal validity are not influencing the participants. Thus, the series of observations allows a researcher to min- imize most threats to internal validity. As a result, the time-series design is classified as quasi-experimental. It is possible for an external event (history) to be a threat to internal validity in time-series designs, but only if the event occurs simultaneously with the treatment. If the outside event occurs at any time other than the introduction of the treatment, it should be easy to separate the history effects from the treatment effects. For example, if the partic- ipants are affected by an outside event that occurs before the treatment, the effect should be apparent in the observations that occur before the treatment. Figure 10.5 shows three possible outcomes in which the treatment has no effect but instead the participants are influenced by an outside event. Notice that a problem occurs only when the treatment and the outside event coincide perfectly. In this case, it is impossible to determine whether the change in behavior was caused by the treatment or by the outside event. Thus, history effects (outside events) are a threat to validity only when there is a perfect correspon- dence between the occurrence of the event and the introduction of the treatment. Suppose, for example, that a clinical researcher uses a time-series design to evaluate a treatment for depression. Observations are made for a group of depressed clients for a week before therapy begins, and a second series of observations is made for a week after therapy. The observations indicate significant improvement after therapy. However, suppose that, by coincidence, there is an abrupt change in the weather on the same day that therapy starts; after weeks of cold, dark, rainy days, it suddenly becomes bright, sunny, and unseasonably warm. Because the weather changed at the same time as the treatment, it is impossible to determine what caused the clients' improvement. Was the change caused by the treatment or by the weather? The series of observations after the treatment or event also allows a researcher to observe any posttreatment trends. For example, it is possible that the treatment has only a temporary effect that quickly fades. Such a trend would be seen in the series of posttreat- ment observations. Figure 10.6 demonstrates how a series of observations can be more informative than single observations made before and after treatment. The figure shows a eries of scores that are consistently increasing before treatment and continue to increase in an uninterrupted pattern after treatment. In this case, it does not appear that the treat- ment has any effect on the scores. However, if the study included only one observation before treatment and only one observation after treatment (O3 and O4), the results would indicate a substantial increase in scores following the treatment, suggesting that the treat- ment did have an effect.

within-subjects experimental design, or repeated-measures experimental design - Chapter 9

A within-subjects experimental design, or repeated-measures experimental design, compares two or more different treatment conditions (or compares a treatment and a control) by observing or measuring the same group of individ- uals in all of the treatment conditions being compared. Thus, a within-subjects design looks for differences between treatment conditions within the same group of participants. To qualify as an experiment, the design must satisfy all other requirements of the experimental research strategy, such as manipulation of an independent variable and control of extraneous variables.

A Word of Caution about Multiple-Group Designs

Although a research study with more than two groups can give a clear and convincing picture of the relationship between an independent and a dependent variable, it is possi- ble to have too many groups in a research design. One advantage of a simple, two-group esign is that it allows the researcher to maximize the difference between treatments by selecting opposite extremes for the independent variable. The mirror image of this argu- ment is that a design with more than two groups tends to reduce or minimize the dif- ference between treatments. At the extreme, there is a risk of reducing the differences between treatments so much that the differences are no longer significant. Therefore, when designing a single-factor multiple-group research study, be sure that the levels used for the independent variable are sufficiently different to allow for substantial differences for the dependent variable.

Disadvantages of Within-Subjects Designs - Chapter 9

Although a within-subjects design has some definite advantages relative to a between- subjects design, it also has some disadvantages. The primary disadvantage comes from the fact that each participant often goes through a series of treatment conditions, with each treatment administered at a different time. Whenever the treatments occur at different times, there is an opportunity for time-related factors, such as fatigue or the weather, to influence the participants' scores. For example, if a participant's performance steadily declines over a series of treatment conditions, you cannot determine whether the decline is being caused by the different treatments or is simply an indication that the participant is getting tired. You should recognize this problem as an example of a confounding variable that threatens the internal validity of the experiment. Specifically, whenever there is an alternative expla- nation for the results, the experiment is confounded. In Chapter 6 (p. 150), we noted that time-related factors can threaten the internal validity of a within-subjects experiment. These time-related factors, which are discussed in Section 9.1, are the major disadvan- tages of a within-subjects experimental design. Another potential problem for a within-subjects design with different treatments administered at different times is participant attrition. In simple terms, some of the indi- viduals who start the research study may be gone before the study is completed. Because a within-subjects design often requires repeated measurements under different conditions for each individual, some participants may be lost between the first measurement and the final measurement. This problem is especially serious when the study extends over a period of time and participants must be called back for additional observation. Participants may forget appointments, lose interest, quit, move away, or even die. In addition to shrink- ing the sample size, the attrition problem may exaggerate volunteer bias if only the most dedicated volunteers continue from start to finish. As noted in Chapter 6, volunteer bias can threaten the external validity of a research study. In situations in which participant attrition is anticipated, it is advisable to begin the research study with more individuals than are actually needed. In this way, the chances are increased of having a reasonable number of participants left when the study ends.

Counterbalancing and Order Effects - Chapter 9

Although counterbalancing has exactly the same effect on time-related threats and order effects, the process of counterbalancing is usually discussed in terms of order effects. Therefore, throughout the rest of this section, we focus on counterbalancing and order effects. Keep in mind, however, that counterbalancing is just as effective for controlling factors such as history and maturation as for controlling order effects. The hypothetical data in Table 9.2 provide a numerical demonstration of counterbalancing, and how it controls threats to validity. The table shows the results from an experiment in which a researcher uses a within-subjects design to compare two treatments. The design is counterbalanced with four of the eight participants starting in treatment I and ending with treatment II, and the other four participants receiving the treatments in the reverse order. Table 9.2a shows scores as they would appear if there were no order effects. The data have been constructed to produce a 6-point difference between the two treatment conditions (mean I 5 20 vs. mean II 5 26). The modified scores in Table 9.2b show how order effects influence the data. For this example, we assume that experience in one treatment condition produces an order effect that causes a 5-point increase in scores for the next treatment. ecause the design is counterbalanced, the first four participants begin the experiment in treatment I, and the 5-point order effect adds to their scores in treatment II. The remain- ing four participants receive the treatments in the opposite order, so the order effect adds to their scores in treatment I. Notice that the result of the counterbalancing is to distribute the order effects evenly between the two treatments; that is, the order effects are balanced across the treatment conditions. Although the treatment means are affected by the order effects, they are affected equally. As a result, there is still a 6-point difference between the two treatment means, exactly as it was without any order effects. The point of this demon- stration is to show that order effects can change individual scores and can change means, but when a design is counterbalanced, the changes do not influence the mean differences between treatments. Because the treatment differences are not affected, the order effects do not threaten the internal validity of the study. The value of counterbalancing a within-subjects design is that it prevents any order effects from accumulating in one particular treatment condition. Instead, the order effects are spread evenly across all the different conditions so that it is possible to make fair, unbiased comparisons between treatments (no single treatment has any special advantage or disadvan- tage). On the other hand, counterbalancing does not eliminate the order effects; they are still embedded in the data. Furthermore, the order effects are hidden in the data so that a researcher cannot see whether they exist or how large they are. In Table 9.2, we identify and expose hypothetical order effects to demonstrate how they influence a counterbalanced design. In real life, however, all you see are the final scores, which may or may not include order effects.

Sample Size - Chapter 8

Although sample size does not affect individual differences or variance directly, using a large sample can help minimize the problems associated with high variance. Sample size exerts its influence in the statistical analyses such that some of the negative effects of high variance can be statistically overcome by use of a very large sample. However, this tech- nique has limitations because the influence of sample size occurs in relation to the square root of the sample size. The square-root relationship means that it takes a dramatic increase in sample size to have a real effect. To reduce the effects of high variance by a factor of 4, for example, the sample size must be increased by a factor of 16; a sample of 20 would need to be increased to a sample of 320. Usually, it is much more efficient to control vari- ance by either standardizing procedures or directly limiting individual differences.

Cross-Sectional Longitudinal Designs - Chapter 10

Although the term cross-sectional longitudinal design may appear to be internally contra- dictory, there are research studies for which this label is appropriate. Specifically, many research studies compare the results obtained from separate samples (like a cross-sectional design) that were obtained at different times (like a longitudinal design). Typically, this type of research is examining the development of phenomena other than individual aging. For example, Pope, Ionescu-Pioggia, and Pope (2001) examined how drug use and life- style have changed over the past 30 years by returning to the same college every 10 years to measure freshman attitudes and behaviors. Because Pope and his colleagues measured different individuals every 10 years, this research combines elements of cross-sectional and longitudinal designs. In a similar study, Mitchell, Wolak, and Finkelhor (2007) exam- ined trends in youth reports of unwanted exposure to pornography on the Internet. This study compared results from a survey of 10- to 17-year-old Internet users in the year 2000 with an equivalent survey of a different sample in the year 2005. Although both of these studies are examining development (or social evolution) over time, neither is a purely longitudinal or a purely cross-sectional design. Nonetheless, you are likely to find this type of research is occasionally described as longitudinal or cross-sectional. Because the design is not clearly one or the other, we hedge a little and classify this research cross- sectional longitudinal. The complete set of quasi-experimental and nonexperimental research designs, including developmental designs, is summarized in Table 10.3

Separating time-related Factors and Order effects - Chapter 9

Although the time-related threats to internal validity are commonly grouped together in one category, researchers occasionally distinguish between those that are related exclu- sively to time and those that are related to previous experience within the research study. Specifically, threats from history, maturation, instrumentation, and regression are related exclusively to time and are not directly connected to experience in a previous treatment. On the other hand, order effects are directly related to experience obtained by partici- pating in previous treatment conditions. For example, participants may learn new skills in one treatment that can influence future behavior, or become fatigued from participa- tion in one treatment, which then affects their scores in later treatments. Based on this distinction, researchers often separate order effects from the other time-related threats to internal validity. Throughout the remainder of this chapter, we will use both terms, order effects and time-related threats, to refer to the general set of time-related factors that can threaten the internal validity of a within-subjects experiment. Finally, you should realize that time-related effects and order effects are only threats for within-subjects experiments that compare different treatments at different times. In studies that administer the different treatments all together, there is no opportunity for these threats to exist (see Figure 9.1

Holding Variables Constant or restricting range of Variability - Chapter 8

Another method of preventing individual differences from becoming confounding vari- ables is simply to hold the variable constant. For example, if a researcher suspects that gender differences between groups might confound a research study, one solution is to eliminate gender as a variable. By using only female participants, a researcher can guar- antee that all of the groups in a study are equivalent with respect to gender; all groups are all female. An alternative to holding a variable completely constant is to restrict its range of val- ues. For example, a researcher concerned about potential IQ differences between groups could restrict participants to those with IQs between 100 and 110. Because all groups have the same narrow range of IQs, it is reasonable to expect that all groups would be roughly equivalent in terms of IQ. Although holding a variable constant (or restricting its range) can be an effective way to prevent the variable from confounding a research study, this method has a serious drawback. Whenever a variable is prevented from reaching its natural range of variation, the external validity of the research is limited. A research study that uses only young adults, for example, cannot be generalized to the entire population of all adults. Similarly, results obtained for participants within a narrow range of IQs cannot be generalized to the whole population. As we noted in Chapter 6, attempting to improve internal validity by exercising control within a research study can threaten external validity or the ability to generalize the results.

The Posttest-Only Nonequivalent Control Group Design - Chapter 10

Nonequivalent groups are commonly used in applied research situations in which the goal is to evaluate the effectiveness of a treatment administered to a preexisting group of participants. A second group of similar but nonequivalent participants is used for the control condition. Note that the researcher uses preexisting groups and does not control the assignment of participants to groups. In particular, the researcher does not randomly assign individuals to groups. For example, Skjoeveland (2001) used a nonequivalent group study to examine the effects of street parks on social interactions among neighbors. Parks were constructed in one area, and the people living in that neighborhood were compared with two con- trol groups that did not get new parks. Similarly, Goldie, Schwartz, McConnachie, and Morrison (2001) evaluated a new ethics course for medical students by comparing the group of students who took the new course with a nonequivalent group who did not take the course. This type of research is called a nonequivalent control group design A nonequivalent control group design uses preexisting groups, one of which serves in the treatment condition and the other in the control condition. The researcher does not randomly assign individuals to the groups. A posttest-only nonequivalent control group design is one common example of a nonequivalent control group design. This type of study is occasionally called a static group comparison. In this design, one group of participants is given a treatment and then is measured after the treatment (this is the posttest). The scores for the treated group are then compared with the scores from a nonequivalent group that has not received the treat- ment (i.e., the control group). This design can be represented schematically using a series of Xs and Os to represent the series of events experienced by each group. In this nota- tion system, developed by Campbell and Stanley (1963), the letter X corresponds to the treatment, and the letter O corresponds to the observation or measurement. Thus, the treat- ment group experiences the treatment first (X) followed by observation or measurement (O). The control group does not receive any treatment but is simply observed (O). The two groups are represented as follows: X O (treatment group) O (nonequivalent control group) If a design includes random assignment of participants to groups in the study, an R is placed as the first symbol in each line of notation. The absence of an R in this schematic reflects the use of preexisting groups, as in a nonequivalent control group design. A posttest-only nonequivalent control group design compares two nonequiv- alent groups of participants. One group is observed (measured) after receiving a treatment, and the other group is measured at the same time but receives no treat- ment. This is an example of a nonexperimental research design. The posttest-only nonequivalent control group design is commonly used when a treatment is given to a well-defined, isolated cluster of individuals, such as the students in a classroom or the patients in a clinic. In these situations, a separate cluster (e.g., another classroom or another clinic) is often selected as the nonequivalent control group. The neighborhood parks program discussed earlier is a good example of this type of study. The program is administered in one neighborhood, and other neighborhoods that do not receive the parks serve as a nonequivalent control group. Note that the purpose of the study is to show that the parks have an effect by demonstrating a difference in social interactions for the two groups. Although this kind of research design appears to ask a cause-and-effect question (Do the parks cause a difference?), the research design does not protect against individual differences as a confound. As we noted earlier, the people in the two neighborhoods could differ on a variety of variables (in addition to the parks), and any of these other variables could be responsible for the difference in social interactions. Because the posttest-only nonequivalent control group design does not address the threat of individual differences as a confound, it is considered a nonexperimental design.

Multiple-treatment Designs - Chapter 9

As we discussed in Chapter 8, the primary advantage of using more than two treatment conditions is that the data are more likely to reveal the functional relationship between the two variables being studied (see Figure 8.5, p. 205). A researcher can create a series of conditions (independent variable), and then observe how the participants' behavior (depen- dent variable) changes as they move through the series of treatments. A multiple-treatment design also produces a more convincing demonstration of a cause-and-effect relationship than is provided by a two-treatment design. Demonstrating repeatedly that a dependent variable responds each time an independent variable is changed produces compelling evi- dence that the independent variable is responsible for causing changes in the dependent variable. The disadvantages of using multiple treatments in a within-subjects design include the same basic problem introduced in Chapter 8 (see p. 205). If a researcher creates too many treatment conditions, the distinction between treatments may become too small to generate significant differences in behavior. In addition, multiple treatments for a within-subjects design typically increase the amount of time required for each participant to complete the full series of treatments. This can increase the likelihood of participant attrition. Finally, the ability to completely counterbalance a design becomes more difficult as the number of treatment conditions increases. With data measured on an interval or ratio scale, the typical statistical analysis consists of computing a mean for each treatment condition, then using a repeated-measures ANOVA to test for any significant differences among the treatment means (see Chapter 15). For more complex within-subjects designs, consult an advanced statistics text to verify that an appropriate analysis technique exists before beginning the research study.

Minimizing Variance within treatment - Chapter 8

As we have noted, large individual differences can lead to large variance within treatments, which can undermine the potential success of a between-subjects research study. Therefore, researchers are well-advised to take whatever steps are possible to reduce the variance inside each of the treatment conditions. The following options provide some ways to accomplish this.

Statistical Consequences of removing individual differences of within subjects design - chapter 9

As we noted in the text, the process of removing individual differences from the variance in a within-subjects design is accomplished during the statistical analysis. To demonstrate this phenomenon, we consider the statistical evaluation for the two sets of data shown in Table 9.3. Both sets of data contain exactly the same scores and produce exactly the same means: the mean for treatment I is 41, for treatment II the mean is 45, and for treatment III the mean is 49. The purpose of the statistical analysis is to determine whether these mean differences are statistically significant; that is, are the differences large enough to conclude that they are very unlikely to have occurred by chance alone, and probably represent real differences between the treatments (see Box 7.1, p. 161)? With three treatment conditions, the appropriate statistical procedure is an analysis of variance (ANOVA). The analysis first computes a variance that measures the size of the actual mean differences; the bigger the differences, the bigger the variance. The analysis then computes a second variance, called the error variance, which estimates the size of the mean differences that would be expected if there were no treatment effects. This second variance, the error variance, is the one that is influenced by individual differences. Finally, the analysis compares the two variances to determine whether the actual mean differences (variance 1) are significantly bigger than the mean differences that would be expected without any treatment effects (variance 2). For the data in Table 9.3, both designs, between-subjects and within-subjects, have exactly the same mean differences and produce exactly the same value for variance 1, V1 5 64. For the between-subjects design, the error variance includes individual differences, and the data in Table 9.3 produce a value of V2 5 334. In this case, the actual mean differences (V1 5 64) are definitely not bigger than would be expected if there were no treatment effects (V2 5 334), and we conclude that there are no significant differences. For the within-subjects design, however, the individual differences are eliminated from the error variance. As a result, the size of the error variance (V2) is substantially reduced. For the data in Table 9.3, the error variance is V2 5 1. For the within-subjects design, the actual mean differences (V1 5 64) are substantially bigger than would be expected without any treatment effects (V2 5 1), and we conclude that there are significant mean differences. Once more, the general point from this demonstration is that a within-subjects design removes the individual differences from the data, which reduces the variance and can greatly increase the likelihood of detecting significant differences between treatment conditions

Chapter Summary - Ch 10

At this point, you should review the learning objectives presented at the beginning of each sec- tion and be sure that you have mastered each objective. In many research situations, it is difficult or impossible for a researcher to satisfy completely the rigorous requirements of an experiment, particularly when doing applied research in natural settings. In these situations, a researcher may use the nonexperimental or the quasi-experimental research strategy. Nonexperimental and quasi-experimental studies always contain a threat to internal validity that is integral to the design and cannot be removed. As a result, these two research strategies cannot establish unambiguous cause-and-effect expla- nations. Quasi-experimental studies make some attempt to control threats to internal validity but nonexperimental studies typically do not. Nonexperimental and quasi-experimental studies often look like experiments because they involve comparing groups of scores. Unlike experiments, however, the different groups are not created by manipulating an independent variable; instead, the groups are defined in terms of a preexisting participant characteristic (e.g., college graduate/no college) or defined in terms of time (e.g., before and after treatment). These two methods for defining groups produce two general categories of nonexperimental and quasi-experimental designs: nonequivalent group designs and pre-post designs. In nonequivalent group designs, the researcher does not control the assignment of indi- viduals to groups because the two groups already exist. Therefore, there is no assurance that the two groups are equivalent in terms of extraneous variables and internal validity is threat- ened by individual differences between groups. Three types of nonequivalent group designs are discussed: (1) the differential research design, (2) the posttest-only nonequivalent control group design, and (3) the pretest-posttest nonequivalent control group design. The first two designs make no attempt to limit the threat of individual differences between groups and are classified as nonexperimental. The pretest-posttest nonequivalent control group design does reduce the threat of individual differences and is classified as quasi-experimental. The second general category is the pre-post design. The goal of a pre-post design is to evaluate the influence of the intervening treatment or event by comparing the observations before treatment with the observations made after treatment. Two examples of pre-post designs are considered: (1) the pretest-posttest design and (2) the time-series design. The first design makes no attempt to control time-related threats and is classified as nonexperimental. The sec- ond is quasi-experimental. Developmental research designs are another type of nonexperimental research. The pur- pose of developmental designs is to describe the relationship between age and other variables. There are two types of developmental research designs. The cross-sectional research design. compares separate groups of individuals with each group representing a different age. The obvious advantage of this design is that the researcher need not wait for participants to age to examine the relationship between a variable and age. However, the cohort or generation effect is a major weakness. In the longitudinal research design, the same group of individuals is followed and measured at different points in time; hence, cohort effects are not a problem. However, longitudinal research is extremely time-consuming for participants and researchers, and participant dropout can create a biased sample

Chapter Summary - Chapter 8

At this point, you should review the learning objectives presented at the beginning of each sec- tion and be sure that you have mastered each objective. In this chapter, we examined the characteristics of the between-subjects experimental research design. The general goal of a between-subjects experiment is to determine whether differences exist between two or more treatment conditions. The defining characteristic of a between-subjects design is that different but equivalent groups of individuals are compared. The primary advantage of a between-subjects design is the fact that each individual score is independent of the other scores because each participant is measured only once. The pri- mary disadvantage of a between-subjects design is individual differences. In between-subjects designs, individual differences can become confounding variables and produce high variance. The potential confounding influence of individual differences is a particular problem for between-subjects designs. Because a between-subjects design compares different groups of individuals, there is always the possibility that the characteristics of one group can be substan- tially different from the characteristics of another group. Techniques for establishing equivalent groups of participants include random assignment, matched assignment, and holding variables constant. Individual differences also have the potential to produce high variance in the scores within each group or treatment condition. High variance within groups can obscure any treat- ment effects that may exist. Several methods that can be used to minimize the variance (differ- ences) within treatments are discussed. In addition to individual differences, there are other threats to the internal validity of between-subjects designs. Each of these potential confounds is also discussed in this chapter. Finally, different applications of the between-subjects design are considered along with the appropriate statistical analysis.

Chapter 9 Summary

At this point, you should review the learning objectives presented at the beginning of each section and be sure that you have mastered each objective. This chapter examined the characteristics of the within-subjects experimental design. The general goal of a within-subjects experiment is to determine whether differences exist between two or more treatment conditions. The defining characteristic of a within-subjects design is that it uses a single group of individuals, and tests or observes each individual in all of the different treatments being compared. The primary advantage of a within-subjects design is that it essentially eliminates all the problems based on individual differences that are the primary concern of a between-subjects design. First, a within-subjects design has no individual differences between groups. There is only one group of participants, so the group of individuals in treatment I is exactly the same as the group of individuals in treatment II; hence, there are no individual differences between groups to confound the study. Second, because each participant appears in every treatment con- dition, each individual serves as his own control or baseline. This makes it possible to measure and remove the variance caused by individual differences. The primary disadvantage of a within-subjects design is that the scores obtained in one treatment condition are directly related to scores in every other condition. The relationship between scores across treatments creates the potential for the scores in one treatment to be influenced by previous treatments, previous measurements, or previous experience. This general problem is called an order effect because the current scores may have been affected by events that occurred earlier in the order of treatments. Order effects can be a confounding variable in a within-subjects design. In addition to order effects, other threats to the internal validity of within-subjects designs are discussed. A technique for dealing with such problems is to counterbalance the conditions. Finally, different applications of the within-subjects design are considered along with the appropriate statistical analysis.

Choosing Within- or Between-Subjects Design - Chapter 9

By now, it should be clear that a within-subjects design has some distinct advantages and some unique disadvantages compared to a between-subjects design. It should also be clear that the advantages of one design are essentially the same as the disadvantages of the other. Three factors that differentiate the designs are: 1. Individual differences. The prospect that individual differences may become confounding variables or increase variance is a major disadvantage of between- subjects designs. However, these problems are eliminated in a within-subjects design. Because the within-subjects design reduces variance, it is generally more likely to detect a treatment effect (if one exists) than is a between-subjects design. If you anticipate large individual differences, it is usually better to use a within-subjects design. 2. Time-related factors and order effects. There is often the potential for factors that change over time to distort the results of within-subjects designs. However, this problem is eliminated in a between-subjects design, in which each individual participates in only one treatment and is measured only once. Thus, whenever you expect one (or more) of the treatment conditions to have a large and long-lasting effect that may influence the participants in future conditions, it is better to use a between-subjects design. 3. Fewer participants. Although it is a relatively minor advantage, we should note once again that a within-subjects design typically requires fewer participants. Because a within-subjects design obtains multiple scores for each individual, it can generate a lot of data from a relatively small set of participants. A between-subjects design, on the other hand, produces only one score for each participant and requires a lot of participants to generate a lot of data. Whenever it is difficult to find or recruit participants, a within-subjects design is a better choice.

Applications and Statistical Analysis of Within-Subjects Designs - Chapter 9

Commonly, a within-subjects design is preferred to a between-subjects design to take advantage of one or more of the special characteristics of this type of research. For example: 1. Because the within-subjects design requires only one group, it often is used when obtaining a large group of research participants is difficult or impossible. If a researcher studies a population with a rare characteristic (Olympic athletes, people with multiple-personality disorder, or women taller than 7 feet), then a within-subjects design is more efficient because it requires fewer participants. 2. We have noted repeatedly that one big advantage of a within-subjects design is that it reduces or eliminates variability caused by individual differences. Whenever a researcher anticipates that the data will show large variability caused by differences between participants, a within-subjects design is the preferred choice.

The Pretest-Posttest Nonequivalent Control Group Design - Chapter 10

If the data consist of numerical scores, then the appropriate statistical analysis is a two- factor, mixed design analysis of variance (the pre-post factor is within-subjects and the group factor is between-subjects). This analysis is not covered in this book but is available on most statistical software programs such as SPSS. If you are comparing the pre-post means for one of the groups, then a repeated-measures t test can be used. Also, if you are comparing the two group means for either the pretest or the posttest scores, then an inde- pendent-measures t test is appropriate.

Nonexperimental and Quasi-Experimental Research Strategies - Chapter 10

In Chapter 6, we identified five basic research strategies: experimental, nonexperimental, quasi-experimental, correlational, and descriptive. In this chapter, we discuss the details of the nonexperimental and quasi-experimental strategies. (The experimental strategy is discussed in Chapter 7, the correlational strategy is discussed in Chapter 12, and the descriptive strat- egy is discussed in detail in Chapter 13.) The experimental research strategy was introduced in Chapter 7 as a means for establishing a cause-and-effect relationship between variables. Recall that the experimental strategy is distinguished from other research strategies by two basic requirements: manipulation of one variable and control of other, extraneous variables. In many research situations, however, it is difficult or impossible for a researcher to satisfy completely the rigorous requirements of an experiment. This is particularly true for applied research in natural settings such as educational research in the classroom and clinical research with real clients. In these situations, a researcher can often devise a research strategy (a method of collecting data) that involves comparing groups of scores, like an experiment, but fails to satisfy at least one of the requirements of a true experiment. Although these studies resemble experiments, they always contain a confounding variable or other threat to internal validity that is an integral part of the design and simply can- not be removed. The existence of a confounding variable means that these studies cannot establish unambiguous cause-and-effect relationships and, therefore, are not true experi- ments. Such studies are generally called nonexperimental research studies. Occasionally, a nonexperimental study is modified in an attempt to minimize the threats to internal validity. The resulting designs are called quasi-experimental studies. The distinction between the nonexperimental research strategy and the quasi-experimental research strategy is the degree to which the research strategy limits confounding and controls threats to internal validity. If a research design makes little or no attempt to minimize threats, it is classified as nonexperimental. A quasi-experimental design, on the other hand, makes some attempt to minimize threats to internal validity and approaches the rigor of a true experiment. As the name implies, a quasi-experimental study is almost, but not quite, a true experiment. In this chapter, we focus on nonexperimental designs and introduce some of the modifications that produce some closely related quasi-experimental designs. In each case, we discuss the aspect of the design that prevents it from being a true experiment. Like true experiments, the nonexperimental research strategy and the quasi-experimental research strategy typically involve comparison of scores from different groups or different conditions. However, these two strategies use a nonmanipulated variable to define the groups or conditions being compared. The nonmanipulated variable is usually a participant variable (such as college graduate vs. no college) or a time variable (such as before vs. after treatment). The distinc- tion between the two strategies is that nonexperimental designs make little or no attempt to control threats to internal validity, whereas quasi-experimental designs actively attempt to limit threats to internal validity.At the end of this chapter, we examine developmental research, which includes research designs intended to investigate how age is related to other variables. Because age is a variable that cannot be manipulated, developmental designs are not true experiments and can be included in other categories of nonexperimental research. However, developmental designs are generally presented as a separate group of research designs with their own terminology. As we introduce the basic developmental research designs, we discuss how they are related to other types of nonexperimental research.

Counterbalancing: Matching treatments with respect to time

In Chapter 7 (p. 171), we discussed the technique of matching variables across treatments to prevent the variables from becoming threats to internal validity. At that time, we also mentioned that a similar process could be used to help control time-related threats. The process of matching treatments with respect to time is called counterbalancing. In coun- terbalancing, different participants undergo the treatment conditions in different orders so that every treatment has some participants who experience the treatment first, some for whom it is second, some third, and so on. As a result, the treatments are matched, or balanced, with respect to time. With two treatments, for example, half of the participants begin in treatment I, and then move to treatment II. The other half begin in treatment II, then receive treatment I. As a result, the two treatments are matched; for both treatments, 50% of the participants experience the treatment first and 50% experience the treatment second. This procedure disrupts any systematic relationship between time and the order of treatment conditions, and thereby eliminates potential confounding from time-related threats or order effects. In the previous section, for example, we described an experiment by Stephens, Atkins, and Kingston (2009) examining the effect of swearing in response to pain (p. 212). In one condition, the participants were told to shout their favorite swear words while experiencing a painful stimulus (ice water) and in the second condition they shouted a neutral word. Half of the participants started with the swearing condition and half started with the neutral word condition. After a brief rest, the two groups switched words. Thus, the two conditions (curse and neutral) were counterbalanced, with half of the participants swearing first and half swearing second. or a within-subjects design, counterbalancing is defined as changing the order in which treatment conditions are administered from one participant to another so that the treatment conditions are matched with respect to time. The goal is to use every possible order of treatments with an equal number of individuals par- ticipating in each sequence. The purpose of counterbalancing is to eliminate the potential for confounding by disrupting any systematic relationship between the order of treatments and time-related factors. You may have noticed that counterbalancing requires separate groups of partici- pants, with each group going through the series of treatments in a different order. The existence of separate groups may appear to contradict the basic definition of a within- subjects design. The solution to this apparent contradiction is based on the observation that although the groups go through the treatments in different orders, they all receive the full set of treatments. Thus, we still have a within-subjects design, with one combined group of individuals participating in all of the different treatment conditions. In Chapter 11 (p. 287), we return to this issue when we re-examine a counterbalanced study as a com- bination of a within-subjects design (with one group in all the treatments) and a between- subjects design (with different groups receiving the treatments in different orders)

review of the experimental research Strategy - Chapter 8

In Chapter 7, we introduced the experimental research strategy, as well as its major goal, which is to demonstrate a cause-and-effect relationship between two variables. To accomplish this goal, the experimental strategy requires several basic characteristics: (1) manipulation of one variable to create a set of two or more treatment conditions; (2) measurement of a second variable to obtain a set of scores within each treatment condition; (3) comparison of the scores between treatments; and (4) control of all other variables to prevent them from becoming confounding variables. At the end of the study, the researcher compares the scores from each treatment with the scores from every other treatment. If consistent differences exist between treatments, the researcher can conclude that the differences have been caused by the treatment con- ditions. For example, a researcher may compare memory scores for a list of one-syllable words with scores for a list of two-syllable words. By showing that there are consistent differences between the two groups of scores, the researcher can demonstrate that memory is related to the number of syllables in the words (i.e., the number of syllables causes dif- ferences in memory). Two basic research designs are used to obtain the groups of scores that are compared in an experiment: 1. The different groups of scores all can be obtained from the same group of partic- ipants. For example, one group of individuals is given a memory test using a list of one-syllable words, and the same set of individuals is also tested using a list of two-syllable words. Thus, the researcher gets two sets of scores, both obtained from the same sample. This strategy is called a within-subjects design and is discussed in Chapter 9. 2. An alternative strategy is to obtain each group of scores from a different group of participants. For example, one group of individuals is given a list of one-syllable words to memorize and a separate group receives a list of two-syllable words. This type of design, comparing scores from separate groups, is called a between-subjects design. We examine the characteristics of a between-subjects research design in this chapter.

Other Confounding Variables - Chapter 8

In addition to the threat of individual differences between groups, a between-subjects design must also be concerned with threats to internal validity from environmental vari- ables that can change systematically from one treatment to another (Chapter 6, p. 149). Thus, there are two major sources of confounding that exist in a between-subjects design - 1. 2. Confounding from individual differences. Individual differences are any participant char- acteristics that can differ from one participant to another. If these characteristics are dif- ferent from one group to another, then the experiment is confounded. For example, the participants in one group may be older, smarter, taller, or have higher socioeconomic status than the participants in another group. One group may have a higher proportion of males or a higher proportion of divorced individuals than another group. Any of these variables may produce differences between groups that can compromise the research results. Confounding from environmental variables. Environmental variables are any charac- teristics of the environment that may differ. If these variables are different between groups, then the experiment is confounded by environmental variables. For example, one group may be tested in a large room and another group in a smaller room. Or one group may be measured primarily during the morning and another group during the afternoon. Any such variable may cause differences between groups that cannot be attributed to the independent variable

Within-Subjects Experiments and Internal Validity - Chapter 9

In Chapter 8, we described the basic elements of the between-subjects experimental research design. Recall that the defining characteristic of a between-subjects experiment is that it requires separate but equivalent groups of participants for the different treatment conditions compared. In this chapter, we introduce an alternative research procedure: the within-subjects design. The defining characteristic of a within-subjects design is that it uses a single group of participants and tests or observes each individual in all of the different treatments being compared. The different treatments can be administered sequentially, with participants receiving one treatment condition followed, at a later time, by the next treatment (Figure 9.1a). In Chapter 1, for example, we described an experiment by Stephens, Atkins, and Kingston (2009) examining the effect of swearing on the experience of pain (pp. 10-14). In the study, participants were asked to place one hand in icy cold water for as long as they could bear the pain. In one condition, the participants were told to repeat their favorite swear word over and over for as long as their hands were in the water. In the second condi- tion, the participants repeated a neutral word. Each participant started in one condition and, after a brief rest, repeated the ice water plunge switching words to the other condition. Thus, all the participants experienced both conditions with a brief rest period in between. The results clearly showed that swearing significantly increased pain tolerance and decreased the perceived level of pain. It also is possible that the different treatment conditions are administered all together in one experimental session (Figure 9.1b). For example, Schmidt 1994) presented participants with a list containing a mix of humorous and nonhumorous sentences, and then asked them to recall as many as possible. The results showed that sig- nificantly more humorous sentences were recalled, indicating the humor plays an important role in human memory. In this case, the researcher switched back and forth between the two treatment conditions (humorous and nonhumorous) with only a few seconds between one treatment and the next. Whether the treatment conditions are administered in a sequence over time or are presented all together, the key element of a within-subjects design is that all the individuals in one sample participate in all of the treatment conditions. In one sense, a within-subjects study is the ultimate in equivalent groups because the group in one treatment condition is absolutely identical to the group in every other condition. In the context of statistical analysis, a within-subjects design is often called a repeated-measures design because the research study repeats measurements of the same individuals under different conditions. In this chapter, we examine the within-subjects experimental design or repeated-measures experimental design; that is, the within-subjects design as it is used in experimental research comparing different treatment conditions. Using the terminol- ogy of experimental research, in a within-subjects experimental design the same group of individuals participates in every level of the independent variable so that each participant experiences all of the different levels of the independent variable. We should note, however, that the within-subjects design is also well suited to other, nonexperimental types of research that investigate changes occurring over time. For example, studies in human development often observe a single group of individuals at different ages to monitor development over time. Examples of nonexperimental within-subjects designs are examined in Chapter 10.

Between-Subjects Nonexperimental and Quasi-Experimental Designs: Nonequivalent Group Designs - Chapter 10

In Chapter 8, we introduced the between-subjects experimental design as a method of comparing two or more treatment conditions using a different group of participants in each condition. A common element of between-subjects experiments is the control of individ- ual differences by assigning participants to specific treatment conditions. The goal is to balance or equalize the groups by using a random assignment process or by deliberately matching participants across treatment conditions. Note that the researcher attempts to create equivalent groups of participants by actively controlling which individuals go into which groups. There are occasions, however, when a researcher must examine preexist- ing groups. For example, a researcher may want to compare student performance for a high school that encourages students to use their phones and tablets during class with student performance in a high school that bans the use of electronic devices. In this study, the researcher does not have control over which individuals are assigned to which group; the two groups of participants already exist. Because the researcher cannot use random assignment or matching to minimize the individual differences between groups, there is no assurance that the two groups are equivalent. In this situation, the research study is called a nonequivalent group design.

Random Assignment and Matching - Chapter 8

In Section 8.3, we also suggested that random assignment or matching groups could be used to help limit confounding from individual differences. However, these techniques have no effect on the variance within groups. If we randomly assign older and younger adults to each group, for example, then we can expect relatively little age difference between groups, but we still have a mixture of older and younger adults (age differences) within groups. In the same way, matching groups so that each group has exactly 50% older adults does not eliminate or reduce the age differences within each group

Limit Individual Differences - Chapter 8

In Section 8.3, we suggested that holding a participant variable constant or restricting its range could be effective techniques used for limiting confounding from individ- ual differences (see p. 195). This technique also reduces the variance within a group of participants. If it is known, for example, that age is a variable related to the participants' scores (e.g., older adults tend to have higher scores than younger adults), then a mixed group of older and younger adults will have higher variance than a group consisting of only younger adults. In the mixed group, the age differences (older vs. younger) will contribute to the variance within the group. By holding age constant (older only), age differences are eliminated and the variance within the group is reduced. In the same way, restricting a participant variable to a narrow range of values cre- ates a more homogeneous group and, therefore, can reduce the variability in the scores. For example, if the participants within a group are limited to those between the ages of 18 and 20, then age differences between participants make a very small contribution to the variance of scores within the group. In general, any attempt to minimize the differences between participants within a group tends to reduce the variance within the group.

Asymmetrical Order Effects - Chapter 9

In Table 9.2, we use exactly the same 5-point order effect whether participants started in treatment I or in treatment II. That is, we assume that the order effects are symmetri- cal. This assumption of symmetry is not always justified. It is definitely possible that one treatment might produce more of an order effect than another treatment. For example, one treatment condition might provide more opportunity for practice than the other conditions. Or one treatment might be more demanding and create more fatigue than the other treat- ment conditions. In such situations, the order effects are not symmetrical, and counterbal- ancing the order of treatments does not balance the order effects.

terminology in Nonexperimental, Quasi-experimental, and Developmental Designs - Chapter 10

In a true experiment, the researcher manipulates an independent variable to create treat- ment conditions and then measures a dependent variable (scores) in each condition; scores in one condition are compared with the scores obtained in another condition. In nonexperimental and quasi-experimental research, no independent variable is manipu- lated. Nonetheless, nonexperimental studies do involve comparing groups of scores. In nonequivalent group studies, for example, the scores from one group of participants are compared with the scores from a different group. In pre-post studies, the scores obtained before the treatment are compared with the scores obtained after the treatment. In general, the variable that differentiates the groups (or sets of scores) is similar to the independent variable in an experiment and is often called an independent variable. However, this vari- able is more accurately referred to as a quasi-independent variable. As in an experiment, the score obtained for each participant is called the dependent variable Within the context of nonexperimental and quasi-experimental research, the vari- able that is used to differentiate the groups of participants or the groups of scores being compared is called the quasi-independent variable, and the variable that is measured to obtain the scores within each group is called the dependent variable. In nonequivalent control group studies, for example, one group receives the treat- ment and one does not. The group difference, treatment versus nontreatment, determines the quasi-independent variable. In time-series studies, the researcher compares one set of observations (scores) before treatment with a second set of observations after treatment. For these studies, the quasi-independent variable is defined as "before versus after treatment." Note that the same terminology is used for nonexperimental research as well as quasi-experimental studies. In differential research, for example, the participant variable used to differentiate the groups is called the quasi-independent variable. In a differential study comparing self-esteem scores for children from two-parent and single-parent homes, the number of parents is the quasi-independent variable, and self-esteem is the depen- dent variable. In a developmental study (either longitudinal or cross-sectional) examining changes in memory that occur with aging, the different ages are the quasi-independent variable and the memory scores are the dependent variable

Individual Differences and Variability - Chapter 8

In addition to becoming confounding variables, individual differences have the potential to produce high variability in the scores within a research study. As we noted earlier, high variability can obscure any treatment effects that may exist and therefore can undermine he likelihood of a successful study. In general, the goal of most research studies is to demonstrate a difference between two or more treatment conditions. For example, a study may be designed to show that one therapy technique is more effective than another. To accomplish this goal, it is essential that the scores obtained in one condition are noticeably different (higher or lower) than the scores in a second condition. Usually, the difference between treatments is described by computing the average score for each treatment, then comparing the two averages. However, simply comparing two averages is not enough to demonstrate a noticeable difference. The problem comes from the fact that in some situ- ations, a 10-point difference is large, but in other circumstances, a 10-point difference is small. The absolute size of the difference must be evaluated in relation to the variance of the scores. Variance is a statistical value that measures the size of the differences from one score to another (see Chapter 15, p. 381). If the scores all have similar values, then the variance is small; if there are big differences from one score to the next, then variance is large. The following example demonstrates how individual differences influence variance and how variance can influence the interpretation of research results. We begin with two distinct populations, one in which the individual differences are relatively small, and one in which the individual differences are large. The two popula- tions are shown in Table 8.1. In the table, each number represents the score for a single individual. Notice that in population A, the scores are all very similar, indicating that the individual differences (the differences from one person to another) are relatively small and the variance is small. In population B, the differences between scores are large, indicating large individual differences and large variance. We then conduct the following hypotheti- cal research study, first with population A and then with population B. 1. We select a random sample of 20 scores from the population and randomly divide the sample into two groups with 10 in each group. 2. One group is then assigned to a control condition that has no effect whatsoever on the participants' scores. The second group is assigned to a treatment that increases each participant's score by 10 points. To simulate this treatment effect, we simply add 10 points to the original score for each individual. or population A, the results of this hypothetical research study are shown as a table and as a graph in Figure 8.3. From either the numbers in the table or the piles of scores in the graph, it is easy to see the 10-point difference between the two conditions. Remember, in population A, the individual differences are small, which means that the variance of the scores is small. With small variance, the 10-point difference between treatments shows up clearly. Next, we repeat the study using scores selected from population B. The results of this simulation are shown in Figure 8.4. This time, it is very difficult to see any differ- ence between the two conditions. With the large individual differences in population B, the variance is large and the 10-point treatment effect is completely obscured. Although Figures 8.3 and 8.4 illustrate the effects of increasing (or decreasing) variance, you should realize that variance also has a dramatic influence on the statistical interpretation of the results. Specifically, the difference between treatments in Figure 8.3 is statistically signifi- cant but the difference in Figure 8.4 is not significant. It may be helpful to think of the variance within each group as similar to interference to a cell phone or radio signal. When there is a lot of interference, it is difficult to get a clear signal. Similarly, when a research study has a lot of variance, it is difficult to see a real treatment effect. In between-subjects research, much of the variance is caused by individual differences. Remember, each individual score represents a different individual. Whenever there are large differences between individuals, there is large variance.

Differences between treatments and Variance within treatments - Chapter 8

In general, the goal of a between-subjects research study is to establish the existence of a treatment effect by demonstrating that the scores obtained in one treatment condition are significantly different (higher or lower) than the scores in another treatment condi- tion. For example, if we can demonstrate that people in a bright yellow room are con- sistently happier and have more positive moods than people in a dark brown room, then we have reason to conclude that room color (the treatment) has an effect on mood. Thus, big differences between treatments are good because they provide evidence of differential treatment effects. On the other hand, big differences within treatments are bad because the differences that exist inside the treatment conditions determine the variance of the scores, and, as we demonstrated in Figure 8.4, large variance can obscure patterns in the data. Notice that we are distinguishing differences between treatments and variance (differences) within treatments. Researchers typically try to increase the differences between treatments and to decrease the variance within treatments. For example, if we were examin- ing the effects of room color on mood, it would not be wise to compare two rooms that were slightly different shades of green. With only a subtle difference between the two colors, we would be unlikely to find a noticeable difference in mood. Instead, the best strategy would be to maximize the difference between room colors to increase our chances of finding a large difference in mood between treatments. Again, the goal is to increase the difference between treatments. At the same time, however, we would like to decrease the variance within treatments. Because a between-subjects design has a separate group of participants for each treatment condition, the variance within treatments is also the variance within groups. In the following section, we examine some of the methods that can be used to reduce or minimize the variance within treatments. In addition, we consider some of the design decisions that a researcher must make when developing a between-subjects research study and look at how those decisions affect variance within treatments.

Matching Groups (Matched assignment) - Chapter 8

In many situations, a researcher can identify a few specific variables that are likely to influence the participants' scores. In a learning experiment, for example, it is reasonable to expect that intelligence is a variable that can influence learning performance. In this case, it is important that the researcher not allow intelligence to become a confounding variable by permitting one group of participants to be noticeably more intelligent than another group. Instead of hoping that random assignment produces equivalent groups, a researcher can use matching to guarantee that the different groups of participants are equivalent (or nearly equivalent) with respect to intelligence. For example, a researcher comparing two different methods for teaching fifth-grade math wants to be sure that the two groups of participants are roughly equivalent in terms of IQ. School records are used to determine the IQs of the participants, and each student is classified as high IQ, medium IQ, or low IQ. The high-IQ participants are distributed equally between the two groups; half is assigned to one group and the other half is assigned to the second group using restricted random assignment. The medium-IQ participants and the low-IQ participants are evenly distributed between the two groups in the same way. The result is two separate groups of participants with roughly the same level of intelli- gence on average. A similar matching process can be used to equate groups in terms of proportions. If a sample consists of 60% older adults (age 40 or more) and 40% younger adults (age less than 40), restricted random assignment could be used to distribute the older adults equally among the different groups. The same process is then used to distribute the younger adults equally among the groups. The result is that the groups are matched in terms of age, with each group containing exactly 60% older and 40% younger participants. Notice that the matching process requires three steps. 1. Identification of the variable (or variables) to be matched across groups 2. Measurement of the matching variable for each participant. 3. Assignment of participants to groups by means of a restricted random assignment that ensures a balance between groups Matching groups of participants provides researchers with a relatively easy way to ensure that specific participant variables do not become confounding variables. However, there is a price to pay for matching, and there are limitations that restrict the usefulness of this process. To match groups with respect to a specific participant variable, the researcher first must measure the variable. The measurement procedure can be tedious or costly and always adds another level of work to the study. In addition, it can be difficult or impossible to match groups on several different variables simultaneously. To match groups in terms of intelligence, age, and gender could require some fairly sophisticated juggling to achieve the desired balance of all three variables. Finally, groups cannot be matched on every single variable that might differentiate participants. Therefore, researchers typically use matching only for variables that are judged to have strong potential to be confounding. In a learning experiment, for example, intelligence is a variable that is likely to affect learn- ing performance, but eye color is a variable that probably has little to do with learning. In this case, it would make sense to match groups for intelligence but not for eye color

Nonexperimental Designs with Nonequivalent Groups - The Differential Research Design- Chapter 10

In most between-subjects research, individual differences are considered to be a problem that must be controlled by random assignment, matching groups, or some other process. However, there are research studies for which individual differences are the primary interest. For example, researchers are often interested in how behavior is influenced by gender differ- ences, or how performance is influenced by age differences. In these situations, researchers deliberately create separate groups of participants based on a specific individual difference such as gender or age. Note that these studies involve no manipulation but simply attempt to compare preexisting groups that are defined by a particular participant characteristic. For example, a researcher may want to compare self-esteem scores for children from two-parent households with children from single-parent households. Note that the researcher does not control the assignment of participants to groups; instead, the participants are automatically assigned to groups based on a preexisting characteristic. For this example, the children are assigned to groups based on the number of parents in the household. Although this type of study compares groups of participants (like a between-subjects experiment), the researcher does not manipulate the treatment conditions and does not have control over the assignment of participants to groups. Again, this is not a true experiment. A research study that simply compares preexisting groups is called a differential research design because its goal is to establish differences between the preexisting groups. This type of study often is called ex post facto research because it looks at dif- ferences "after the fact," that is, at differences that already exist between groups. Because the differential research design makes no attempt to control the threat of individual differ- ences between groups, it is classified as a nonexperimental research design. For example, a study by InsuranceHotline.com (Romanov, 2006) found significant differences in the number of car accidents and tickets for people with different astrological signs. Libras and Aquarians were the worst offenders, while Leos and Geminis had the best overall records. Clearly, people who have different astrological signs form preexisting groups that were not manipulated or created by the researchers. In another somewhat bizarre study, DeGoede, Ashton-Miller, Liao, and Alexander (2001) swung a pendulum at their participants and measured how quickly the participants moved their hands to intercept the approaching object. This study examined gender differences and age differences, once again comparing scores for preexisting groups Many research questions in social psychology and personality theory are focused on differences between groups or categories of people. Personality theorists, for example, often classify people according to attachment style, and then examine differences between individuals with different styles. Many research studies have demonstrated that the style of mother/child attachment formed in infancy persists as an individual develops and is related to adult intimacy and romantic relationships (Brennan & Morris, 1997; Feeney, 2004). Differential research and correlational research, which also examines relationships between variables, are compared in Box 10.1.

Removing Individual Differences from Within-Subjects Data - Chapter 9

In the preceding example, we removed the individual differences by equalizing all the participants. In a normal research situation, this equalizing process is accomplished by statistical analysis (instead of manipulation of the data). However, the result is the same: The variance caused by individual differences is removed. The statistical removal of indi- vidual differences is demonstrated in Box 9.1. Finally, we should note that you cannot use this equalizing process to remove the individual differences from the data in a between- subjects design. In between-subjects data, every score is from a separate individual and an attempt to equalize the participants as in Table 9.4 would simply change all the scores to the same value, which would also eliminate the treatment effects.

Chapter 9 - explanations of within and between subjects

Individual differences are an integral part of a between-subjects design, and they are automatically a part of the variance in the scores. For the within-subjects data, how- ever, the treatment effects are not connected to the individual differences. To evaluate the difference between treatments I and II, for example, we never compare John to Mary. Instead we compare John (in treatment I) to John (in treatment II), and we com- pare Mary (in treatment I) to Mary (in treatment II). Because the treatment effects and individual differences are not connected, we can separate the individual differences from the rest of the variance in a within-subjects design. Once again, consider the within-subjects experiment (see Table 9.3b). Although indi- vidual differences are part of the variance in the data (e.g., John's scores are different from Mary's scores), we can determine how much of the variance is caused by the indi- vidual differences. For these data, for example, there is a consistent difference of about 10 points between John and Mary in each of the three treatments. Similarly, there is a 30-point difference between John and Bill and a 40-point difference between John and Kate. Whenever the individual differences are reasonably consistent across treatment conditions, they can be measured and separated from the rest of the variance. Thus, in a within-subjects design, the following measurements are possible: • It is possible to measure the differences between treatments without involving any individual differences. Because the same participants are in every treatment condition, the treatment effects and the individual differences are not linked. • It is possible to measure the differences between individuals. When the individual differ- ences are consistent across treatments, they can be measured and removed from the rest of the variance in the data. This can greatly reduce the negative effects of large variance. To demonstrate the actual process of separating the individual differences from the rest of the variance, consider once again the within-subjects data in Table 9.3b. For these data, Kate consistently has the highest score in each treatment. Specifically, the average score for the four participants across all three treatments is 45; however, the aver- age score for Kate is 65. This is an example of an individual difference; clearly, Kate is different from the other participants. However, we can eliminate this difference by simply subtracting 20 points from each of Kate's scores. As a result, Kate becomes a more "normal" participant. Similarly, John's average score is 20 points lower than the group average, so we can make John "normal" by adding 20 points to each of his scores. Finally, we subtract 10 points from each of Bill's scores and then add 10 points to each of Mary's scores. The resulting data are shown in Table 9.4. Notice that we have removed the individual dif- ferences by making the four individuals equal (all four participants now have an average score of 45) but we have not changed any of the treatment effects. For example, John's score still increases by 5 points as he goes from treatment I to treatment II, and increases another 5 points as he goes from treatment II to treatment III. Also, all of the treatment means are exactly the same as they were before we started adding and subtracting. Thus, the newly created scores preserve all of the important characteristics of the original scores. That is, the changes (treatment effects) that occur for the participants, individually and collectively, are the same as in the original data. However the big differences from one participant to another in Table 9.3b are now gone, and the resulting scores show only a 1- or 2-point difference between individuals. Removing the individual differences drastically reduces the variance of the scores and makes the 4-point mean differences from treatment to treatment much easier to see. The differences between treatments for the data in Table 9.4 are even more obvious when the scores are presented in a graph. Figure 9.2 shows the original within-subjects data from Table 9.3b and the adjusted data from Table 9.4. When the individual differ- ences are removed, the treatment effects are much easier to see. By measuring and removing individual differences, the within-subjects design reduces variance and reveals treatment effects that might not be apparent in a between-subjects design. In statistical terms, a within-subjects design is generally more powerful than a between-subjects design; that is, a within-subjects design is more likely to detect a treat- ment effect than a between-subjects design.

Summary and recommendations - Chapter 8

Individual differences between groups are always a potential confounding variable in a between-subjects design. Therefore, it is important for researchers to create groups of par- ticipants that are as equivalent as possible at the beginning of a research study. Most of the time, researchers attempt to create equivalent groups by using random assignment because it is relatively easy and does not require any measurement or direct control of extrane- ous variables. The number of participant variables that could produce differences between groups is essentially infinite, and random assignment provides a simple method of bal- ancing them across groups without addressing each individual variable. However, random assignment is not perfect and cannot guarantee equivalent groups, especially when a small sample is used. Pure chance is not a dependable process for obtaining balanced equivalent groups. When one or two specific variables can be identified as likely to influence the depen- dent variable, these variables can be controlled either by matching groups or by holding the variable constant. However, matching requires pretesting to measure the variable(s) being controlled, and it can become difficult to match several variables simultaneously. Holding a variable constant guarantees that the variable cannot confound the research, but this process limits the external validity of the research results.

Chapter Over view - chapter 10

It appears that there is some truth to the old adage that whatever does not kill us makes us stronger. Seery, Holman, and Silver (2010) asked participants to report their lifetime exposure to a list of negative events such as illness, injury, assault, abuse, financial difficulty, and bereavement, and they obtained a variety of measurements of mental well-being. The authors summarize their results by comparing the outcomes for three groups of participants: individuals with some history of adversity report better mental health and higher well-being compared to either people with no history or people with a high history of adversity. It appears that adversity in moderation does make us stronger. Because this study compares groups of scores, it may appear to be another example of the experimental strategy covered in Chapters 7-9. Specifically, it strongly resembles the between-subjects experiments presented in Chapter 8. However, you should also realize that the Seeley et al. study is missing one or two of the characteristics that are essential for a true experiment. Specifically, there is no manipulated independent variable. Instead, the three groups of participants are defined by the levels of adversity that they have experienced, which is not controlled or manipulated by the researchers. Also, the researchers have no control over the assignment of individuals to groups; a person who enters the study with a high level of adversity is automatically put into the high adversity group. Without manipulation and control, the study is definitely not an experiment. In fact, this kind of research is known as nonexperimental. In Chapter 6, we noted that both the nonexperimental and quasi-experimental research strategies compare groups of scores, like true experiments, but do not manip- ulate an independent variable to create the groups. As a result, these two strategies do not have the internal validity of true experiments and cannot establish unambiguous cause-and-effect relationships. The distinction between the two strategies is that quasi- experimental studies make some attempt to minimize threats to internal validity, whereas nonexperimental studies typically do not. In this chapter, we discuss details of these two strategies, as well as different types of nonexperimental and quasi-experimental designs. Developmental designs, which are closely related to nonexperimental designs, are also presented

Defintions - chapter 10

Like true experiments, the nonexperimental research strategy and the quasi-experimental research strategy typically involve comparison of scores from different groups or different conditions. However, these two strategies use a nonmanipulated variable to define the groups or conditions being compared. The nonmanipulated variable is usually a participant variable (such as college graduate vs. no college) or a time variable (such as before vs. after treatment). The distinc- tion between the two strategies is that nonexperimental designs make little or no attempt to control threats to internal validity, whereas quasi-experimental designs actively attempt to limit threats to internal validity.

Differential research and Correlational research - Chapter 10

Many researchers place differential research in the same category as correlational research. In many ways, differential research is similar to the correlational research strategy (introduced in Chapter 6 and discussed in Chapter 12). In differential and correlational studies, a researcher simply observes two naturally occurring variables without any interference or manipulation. The subtle distinction between differential research and correlational research is whether one of the variables is used to establish separate groups to be compared. In differential research, participant differences in one variable are used to create separate groups, and measurements of the second variable are made within each group. The researcher then compares the measurements for one group with the measurements for another group, typically looking at mean differences between groups (Figure 10.4a). A correlational study, on the other hand, treats all the participants as a single group and simply measures the two variables for each individual (Figure 10.4b). Although differential research and correlational research produce different kinds of data and involve different statistical analyses, their results should receive the same interpretation. Both designs allow researchers to establish the existence of relationships and to describe relationships between variables, but neither design permits a cause-and- effect explanation of the relationship

Matched-Subjects Designs - Chapter 9

Occasionally, researchers attempt to approximate the advantages of within- and between-subjects designs by using a technique known as a matched-subjects design. A matched-subjects design uses a separate group for each treatment condition, but each individual in one group is matched one-to-one with an individual in every other group. The matching is based on a variable considered to be particularly relevant to the specific study. Suppose, for example, that a researcher wants to compare different methods for teaching mathematics in the third grade. For this study, the researcher might give a math- ematics achievement test to a large sample of students, then match individuals based on their test scores. Thus, if Tom and Bill have identical math achievement scores, these two students can be treated as a matched pair with Tom assigned to one teaching method and Bill assigned to the other. If the study compares three treatments, then the researcher needs to find triplets of matched individuals. Although a matched-subjects study does not have exactly the same individuals in each treatment condition (like a within-subjects design), it does have equivalent (matched) individuals in each treatment In a matched-subjects design, each individual in one group is matched with a participant in each of the other groups. The matching is done so that the matched individuals are equivalent with respect to a variable that the researcher considers to be relevant to the study The goal of a matched-subjects design is to duplicate all the advantages of within- and between-subjects designs without the disadvantages of either one. For example, a matched-subjects design attempts to mimic a within-subjects design by having "equiv- alent" participants in all of the treatment conditions. In a within-subjects design, the equivalent participants are literally the same people, and in a matched-subjects design, the equivalent participants are matched sets of people. Thus, a researcher does not need to worry that the participants in one treatment are noticeably different from the partici- pants in another treatment. In addition, the statistics used to evaluate a matched-subjects design are the same as those used for within-subjects designs. In both designs, the variance caused by individual differences is measured and removed. The matched-subjects design also mimics a between-subjects design by using a separate group for each treatment condi- tion with each individual measured only once. Thus, there is no chance for the scores to be influenced by time-related factors or order effects. It is possible to match participants on more than one variable. For example, a researcher could match participants on the basis of age, gender, race, and IQ. In this case, for example, a 22-year-old white female with an IQ of 118 who was in one group would be matched with another 22-year-old white female with an IQ of 118 in another group. Note, however, that matching can become extremely difficult as the number of matched variables increases and the number of different groups increases. In general, a matched-subjects design attempts to eliminate the problems associated with between-subjects experiments (individual differences) and the problems associated with within-subjects experiments (order effects). However, a matched-subjects design is only a crude approximation to a repeated-measures design. The matched pairs of partici- pants in a matched-subjects design are not really the same people. Instead, they are merely "similar" individuals with the degree of similarity limited to the variable(s) that are used for the matching process. Simply because two individuals have the same IQ is no guaran- tee that they are also the same or even similar on other variables. Thus, matched-subjects designs are not nearly as effective at removing individual differences as are within-subjects designs.

Comparing Within-Subjects and Between-Subjects Design - Chapter 9

Often a research question can be addressed with either a between-subjects or a within- subjects experiment. The between-subjects design would use a different group of participants for each of the treatment conditions and the within-subjects design would use the same individuals in all of the treatments. The decision about which design to use is often based on the relative advantages and disadvantages of the two designs.

Switch to a Between-Subjects Design - Chapter 9

Often, researchers begin a research study with some knowledge or expectation of the exis- tence and magnitude of order effects. For example, if the study involves measuring skill or performance over a series of treatment conditions, it is reasonable to assume that practice gained in the early treatments is likely to affect performance in later treatments. If the study involves a tedious or boring task repeated under different conditions, the researcher can expect fatigue or boredom to develop during the course of the study. In some situa- tions, order effects are so strong and so obvious that a researcher probably would not even consider using a within-subjects design. For example, a within-subjects design is a poor choice for a study comparing two methods of teaching reading to first-grade children. After the children have been taught with method I, they are permanently changed. You cannot erase what they have learned and try to teach them again with method II. In this extreme case, the obvious strategy for avoiding order effects is to use a between-subjects design with a separate group for each of the two teaching methods. Usually, a between-subjects design (with a separate group for each treatment) is available as an alternative and com- pletely eliminates any threat of confounding from order effects. Although the potential for order effects is not always as severe as with learning to read, a between-subjects design is often the best strategy whenever a researcher has reason to expect substantial order effects.

comparing proportions for two or More Groups - Chapter 8

Often, the dependent variable in a research study is measured on a nominal or ordinal scale. In this case, the researcher does not have a numerical score for each participant and cannot calculate and compare averages for the different groups. Instead, each individual is simply classified into a category, and the data consist of a simple frequency count of the participants in each category on the scale of measurement. Examples of nominal scale measurements are: • academic major for college students • occupation Examples of ordinal scale measurements are: • • • college class (freshman, sophomore, etc.) birth order (first born, second born) high, medium, or low performance on a task Because you cannot compute means for these variables, you cannot use an indepen- dent-measures t-test or an ANOVA (F-test) to compare means between groups. How- ever, it is possible to compare proportions between groups using a chi-square test for independence (see Chapter 15, p. 406). As with other between-subjects experiments, the different groups of participants represent different treatment conditions (manipulated by the researcher). For example, Loftus and Palmer (1974) conducted a classic experiment demonstrating how language can influence eyewitness memory. A sample of 150 students watched a film of an automobile accident, and participants were then questioned about what they saw. One group was asked, "About how fast were the cars going when they smashed into each other?" Another group received the same question except that the verb was changed to "hit" instead of "smashed into." A third group served as a control and was not asked any question about the speed of the two cars. A week later, the participants returned and were asked additional questions about the accident, including whether they remembered seeing any broken glass in the accident. (There was no broken glass in the film.) Notice that the researchers are manipulating the form of the initial question and then measuring a yes/no response to a follow-up question 1 week later. Figure 8.6 shows the structure of this design represented by a matrix with the independent variable (differ- ent groups) determining the rows of the matrix and the two categories for the dependent variable (yes/no) determining the columns. The number in each cell of the matrix is the frequency count showing how many participants are classified in that category. For exam- ple, of the 50 students who heard the word smashed, there were 16 (32%) who claimed to remember seeing broken glass even though there was none in the film. By compari- son, only 7 out of 50 (14%) of the students who heard the word hit claimed to have seen broken glass. The chi-square test compares the proportions across one row of the matrix (one group of participants) with the proportions across other rows. A significant outcome means that the proportions in one row are different from the proportions in another row, and the difference is more than would be expected if there was not a systematic treatment effect. Loftus and Palmer found that participants who had been asked a leading question about the cars smashing into each other were significantly more likely to recall broken glass than participants who were not asked a leading question.

Independent Scores - Chapter 8

One additional characteristic of the between-subjects design deserves special mention. A between-subjects design allows only one score for each participant. Every individual score represents a separate, unique participant. If a between-subjects experiment produces 30 scores in treatment A and 30 scores in treatment B, then the experiment must have employed a group of 30 individuals in treatment A and a separate group of 30 individuals in treatment B, for a total of 60 participants. In the terminology of experimental research, a between-subjects experimental design uses a different group of participants for each level of the independent variable, and each participant is exposed to only one level of the independent variable. Occasionally, a researcher may combine several measurements for each individual into a single score. In particular, when the variable being measured is not particularly stable (e.g., reaction time), a researcher may choose to measure the variable several times and then average the measurements to produce a single, more reliable score. However, the net result is always one score per individual participant.

Advantages of Within-Subjects Designs - Chapter 9

One advantage of a within-subjects design is that it requires relatively few participants in comparison to between-subjects designs. For example, to compare three different treat- ment conditions with 30 participants in each treatment, a between-subjects design requires subjects design, however, requires only 30 participants (the same group of 30 participants is used in all three conditions). Because a within-subjects study requires only one group, it is particularly useful in situations in which participants are difficult to find. For example, it might be difficult to recruit a large sample of people for a study examining twins who are at least 80 years old. The primary advantage of a within-subjects design, however, is that it essentially eliminates all of the problems based on individual differences that are the primary concern of a between-subjects design. Recall from Chapter 8 that in a between-subjects design, individual differences can create two major problems for research: 1. Individual differences between groups can become a confounding variable. If the individuals in one treatment condition are noticeably different from the individuals in another treatment (e.g., smarter, faster, bigger, or older), then the individual differ- ences, rather than the treatments, may explain any observed differences. 2. The individual differences within each treatment condition can create high variance, which can obscure any differences between treatments. These problems are reduced or eliminated in a within-subjects design. First, obvi- ously, a within-subjects design has no individual differences between groups because there is only one group of participants. The group in one treatment is exactly the same as the group in every other treatment, which means that there are no individual differences between groups to confound the study. Second, because each participant appears in every treatment condition, each individual serves as his own control or baseline. This makes it possible to measure and remove the variance caused by individual differences. The fol- lowing example demonstrates how the problems associated with individual differences are reduced in a within-subjects design. Table 9.3 shows two sets of hypothetical data. The first set is from a typical between-subjects experiment and the second set represents a within-subjects experiment. Each score is labeled with the participant's name so that we can examine the effects of individual differences. For the between-subjects data, every score represents a different person. For the within-subjects data, on the other hand, the same people are measured in all three treatment conditions. The difference between the two designs has some important consequences: 1. Both research studies have exactly the same scores, and both show the same differ- ences between treatments. In each case, the researcher would like to conclude that the differences between treatments were caused by the treatments. However, with the between-subjects design (see Table 9.3a), the participants in treatment I may have characteristics that make them different from the participants in treatment II. For example, the four individuals in treatment II may be more intelligent than the participants in treatment I, and their higher intelligence may have caused their higher scores. This problem disappears in the within-subjects design (see Table 9.3b); the participants in one treatment cannot differ from the participants in another treatment because the same individuals are used in all the treatments. 2. Although the two sets of data contain exactly the same scores, they differ greatly in the way that the individual differences contribute to the variance. For the between-subjects experiment, the individual differences and the treatment effects are tied together and cannot be separated. To measure the difference between treatments, we must also mea- sure the differences between individuals. For example, John scored 5 points lower than Sue, but it is impossible to determine whether this 5-point difference is caused by the treatments or is simply a matter of individual differences (John is different from Sue)

Strengths and Weaknesses of the Cross-Sectional Developmental Design - Chapter 10

One obvious advantage of the cross-sectional design is that a researcher can observe how behavior changes as people age without waiting for a group of participants to grow older. The example in Figure 10.7 shows that we do not need to follow a group of people over the next 40 years to observe the differences that occur during 40 years of aging. With the cross-sectional design, data can be collected in a short period of time. In addition, cross-sectional research does not require long-term cooperation between the researcher and the participant; that is, the researcher does not have to incur the time and expense of tracking people down for 40 years and encouraging them to continue in the research. The cross-sectional research design is not without its weaknesses. One weakness is that a researcher cannot say anything about how a particular individual develops over time because individuals are not followed over years. A more serious problem is that factors other than age may differentiate the groups. For example, 40-year-old women not only are younger than 80-year-old women but also grew up in very different envi- ronments. Opportunities for education, employment, and social expectations were very different for these two groups of women. In general, individuals who are the same age and have lived in similar environments are called cohorts. For example, today's pre- school children, today's adolescents, and today's college students would be three sets of cohorts. In addition to being different ages, these three groups have also experienced different social and cultural environments. The environmental factors that differentiate one age group from another are called cohort effects, or generation effects, and they may be responsible for differences observed between the groups instead of age. As a result, generation effects are a threat to internal validity for a cross-sectional design. Spe- cifically, in a cross-sectional study, the generation of the participants changes from one group to another so that the apparent relationship between age and other variables may actually be caused by generation differences. For example, suppose that you compared computer literacy for three groups: one with 40-year-olds, one with 60-year-olds, and one with 80-year-olds. Almost certainly, the data would show a decline in literacy as the participants grow older. However, you should not assume that this difference should be attributed to age. Specifically, you should not conclude that losing computer literacy is a consequence of aging. The 80-year-old participants did not lose computer literacy as they aged; instead, they spent most of their lives in an environment without computers and never had computer literacy to start with. Cohorts are individuals who were born at roughly the same time and grew up under similar circumstances. The terms cohort effects and generation effects refer to differences between age groups (or cohorts) caused by unique characteristics or experiences other than age A great example of how cohort effects can influence the results of research comes from studies on the relationship between IQ and age (Baltes & Schaie, 1974). Many research studies show that IQ declines between the ages of 20 and 50. On the other hand, a separate group of studies shows little or no decline in IQ between the ages of 20 and 50. How can these two sets of data be so completely different? One answer lies in the designs of the studies. The data that show IQ declining with age are generally obtained with cross-sectional studies. The problem with cross-sectional designs is that the results may be influenced by cohort effects because the groups being compared are not only dif- ferent in age but also lived in different decades. The fact that the groups grew up and lived in different environments could affect their IQ scores and be the source of the IQ differences between the groups. Cohort effects are more likely when there are large age differences between groups. The second set of studies, showing stable IQ, monitored the same set of people over a long period of time. This type of research design is called the longitudinal research design and is discussed next. Incidentally, other researchers have raised serious questions about this interpretation of the aging and IQ relationship (Horn & Donaldson, 1976).

Order effects as a Confounding Variable - Chapter 9

Order effects can produce changes in the scores from one treatment condition to another that are not caused by the treatments and can confound the results of a research study. To demonstrate this confounding effect, we examine a hypothetical experiment in which nts; on average, the scores in treatment I are the same as in treatment II. Results for this hypothetical study are shown in Table 9.1a. Notice that some individual participants show a small increase or decrease between treatment conditions, representing error that can occur in any measurement process (see the discussion of reliability in Chapter 3). However, on average, there is no difference between the treatment conditions; both pro- duce an average score of 20. Now, consider the data shown in Table 9.1b. For these data, we assume that each participant started the experiment in treatment I and then was moved to treatment II. In addition, we assume that participation in treatment I produces an order effect (e.g., a prac- tice effect) that causes the subsequent measurements to be 5 points higher than they would be normally. Thus, we have added a 5-point order effect to each participant's score in treatment II. Notice that the 5-point increase is not caused by the second treatment but is rather an order effect resulting from earlier participation in treatment I. The resulting data in Table 9.1b illustrate two important points: 1. The order effect varies systematically with the treatments; that is, it always con- tributes to the second treatment but never to the first. Whenever something changes systematically with the independent variable, it is a confounding variable. Thus, the results of this study are confounded by the order effects. 2. In this example, the confounding from the order effects makes the data look like there is a 5-point difference between the treatments. With the help of order effects, the indi- vidual participants and the group mean show consistently higher scores in the second treatment. These data could lead the researcher to conclude that there is a significant difference between the treatments when, in fact, no such difference exists (remember, we constructed the original data so there is no difference between treatments). Thus, order effects, like any confounding variable, can distort the results of a research study. In this example, the order effect creates what looks like a treatment effect but actually is just an order effect. In other situations, order effects can diminish or exaggerate a real effect, thereby posing a real threat to the internal validity of the research.

Definitions - Chapter 9

Order effects occur when the experience of being tested in one treatment condition (participating and being measured) has an influence on the participants' scores in a later treatment condition(s). Order effects threaten internal validity because any observed differences between treatment conditions may be caused by order effects rather than the treatments. Carry-over effects occur when one treatment condition produces a change in the participants that affects their scores in subsequent treatment conditions. Progressive error refers to changes in a participant's behavior or performance that are related to general experience in a research study but not related to a specific treatment or treatments. Common examples of progressive error are practice effects and fatigue.

random assignment (randomization) - Chapter 8

Probably the most common method of establishing groups of participants is random assignment. Recall from Chapter 7 that random assignment simply means that a random process (such as a coin toss) is used to assign participants to groups. The goal is to ensure that all individuals have the same chance of being assigned to a group. Because group assignment is based on a random process, it is reasonable to expect that characteristics such as age, IQ, and gender are also distributed randomly across groups. Thus, we mini- mize the potential for confounding from individual differences because it is unlikely that any group is systematically older, or smarter, or more feminine than another. It should be obvious that assigning participants with a simple random process such as a coin toss or drawing numbers out of a hat is likely to create groups of different sizes. If it is desirable to have all groups the same size (equal ns), which is typically the case, then the process can be modified to guarantee equal-size groups. To divide 90 participants into three equal groups, for example, the researcher could start with 90 slips of paper, 30 with #1, 30 with #2, and 30 with #3, and then draw one slip for each individual to determine. the group assignment. In this case, the process is a restricted random assignment; the In restricted random assignment, the group assignment process is limited to ensure predetermined characteristics (such as equal size) for the separate groups. DeFINItION restriction is that the groups must be equal in size. The advantage of using a random process to establish groups is that it is fair and unbi- ased. Just as football teams use a coin toss to determine who receives the opening kickoff, random assignment eliminates prejudice from the decision process. However, a random process does not guarantee a perfectly balanced outcome. When tossing a coin, for exam- ple, we can expect an equal 50-50 distribution of heads and tails in the long run (with a large sample). However, in the short run (with a small sample), there are no guarantees. A sample of only 10 coin tosses, for example, can easily contain eight or nine heads and only one or two tails. With any random process, we trust chance to create a balanced out- come. In the long run, chance proves to be fair, but in the short run, anything can happen by chance. Specifically, there is always a possibility that random assignment will pro- duce groups that have different characteristics and thus confound the experiment. Because pure chance is not a dependable process for obtaining balanced and equivalent groups, researchers often modify random processes by placing some limitations on or exerting some control over the outcomes. One such modification, restriction of equal group sizes, has been discussed; two additional techniques follow.

Other Threats to Internal Validity of Between-Subjects Experimental Designs - Chapter 8

Remember that the goal of the between-subjects experimental design is to look for differences between groups for the dependent variable and to demonstrate that the observed differences are caused by the different treatments (i.e., by the manipulation of the independent variable). If the differences between the groups can be explained by any factor other than the treatments, the research is confounded and the results cannot be inter- preted without some ambiguity. Also recall from Chapter 6 that any factor that allows for an alternative explanation for the research results is a threat to internal validity. Earlier in this chapter, we discussed the two major threats that can undermine the internal validity of a between-subjects study: confounding due to individual differences between groups and confounding from environmental variables. Now, we consider additional potential con- founds that are specifically related to between-subjects designs.

Chapter Over view - chapter 9

Step 6 of the research process involves selecting a research design. In this chapter, we discuss in detail another type of experimental research design: the within-subjects design. We also consider the threats to internal validity for this design and discuss the relative advantages and disadvantages of within-subjects experiments compared to the between-subjects experiments that were presented in Chapter 8

Threats to Internal Validity of Within-Subjects experiment - Chapter 9

When a within-subjects experimental study compares different treatments that are administered at different times, it must be concerned with threats to internal validity from environmental variables and time-related factors that may change systematically from one treatment to another and may influence the participants' scores (Chapter 6, pp. 149-151). Thus, there are two major sources of potential confounding for a within- subjects design. 1. Confounding from environmental variables. Environmental variables are characteristics of the environment that may change from one treatment condition to another. For example, one treatment may be evaluated during the morning and another treatment during the afternoon. Any such variable may cause differences in scores from one treatment to another and, therefore, provides an alternative explanation for the differences between treatments. 2. Confounding from time-related variables. A serious concern of within-subjects designs comes from the fact that the design often requires a series of measurements made over time. During the time between the first measurement and the final measurement, the participants may be influenced by a variety of factors other than the treatments being investigated, and these other factors may affect the participants' scores. If this occurs, then the internal validity of the study is threatened because a change in a participant's score from one treatment to the next could be caused by an outside factor instead of the different treatments. In this section, we identify five time-related threats to internal validity. a. History: The term history refers to environmental events other than the treatment that change over time and may affect the scores in one treatment differently than in another treatment. Events that occur in participants' lives at home, in school, or at work may affect their performance or behavior in different sections of the research study. For example, a research study that extends over several days with a different treatment condition each day can be influenced by an outside event that is likely to affect some of the participants on one particular day, but not on another day, then the event may provide an explanation for unusual performance on that particular day. For example, a long power outage on campus could cause anxiety and confusion that affects the participants' performance on one specific day. Notice that a history effect becomes a confounding variable only if it influences at least one treatment condition differently and influences enough of the participants to have an effect on the overall group performance. When a group of individuals is being tested in a series of treatment conditions, any outside event(s) that influences the participants' scores in one treatment dif- ferently than in another treatment is called a history effect. History is a threat to internal validity because any differences that are observed between treatment conditions may be caused by history instead of the treatments. Definition Definition Definition b. Maturation: Any systematic changes in participants' physiology or psychology that occur during a research study and affect the participants' scores are referred to as maturation. Maturation effects are of particular concern when the research participants are young children or elderly adults. Young children, for example, can gather new knowledge and skills or simply grow bigger and stronger in a relatively short time. As a result, their performance at the end of a series of treatment conditions may be very different from their performance at the beginning, and the change in performance may not have been caused by the treatments but by maturation. In general, maturation threatens the internal validity of a research study conducted over time because it weakens our confidence that the different treatment conditions are responsible for observed changes in the participants' scores. Maturation is of particular concern in research situations in which the series of treatments extends over a relatively long time. c. Instrumentation: The term instrumentation (sometimes called instrumental bias or instrumental decay) refers to changes in a measuring instrument that occur over time. Instrumentation is much more likely to occur with behavioral observation measures (discussed in Chapters 3 and 13) than with other types of measures. Consider, for example, a researcher observing a group of children and recording occurrences of aggressive behavior. In this situation, part of the measurement depends on the subjective interpretation of the observer, whose criteria may change from one time to another. As a result, the same behavior may be judged differently at different times. Notice that the changes in the participants' scores are not caused by the treatment but by a change in the measurement instrument (the researcher). Like history and maturation, instrumentation is of particular concern in research situations in which the series of treatments extends over a relatively long time. Maturation is when a group of individuals is being tested in a series of treatment conditions, any physiological or psychological change that occurs in participants during the study and influences the participants' scores. Maturation is a threat to internal validity because observed differences between treatment conditions may be caused by maturation instead of the treatments. Instrumentation refers to changes in the measuring instrument that occur during a research study in which participants are measured in a series of treatment conditions. Instrumentation is a threat to internal validity because any observed differences between treatment conditions may be caused by changes in the measuring instrument instead of the treatments. Regression toward the mean: Statistical regression, or regression toward the mean, refers to the tendency for extreme scores on any measurement to move toward the mean (regress) when the measurement procedure is repeated. Individuals who score extremely high on a measure during the first testing are likely to score lower on the second testing, and, conversely, individuals who score extremely low on a measure during the first testing are likely to score higher on the second testing. Statistical regression occurs because an individual's score is a function both of stable factors such as skill and of unstable factors such as chance. Although the stable factors remain constant from one measurement to another, the unstable factors can change substantially. Your grade on an exam, for example, is based on a combination of knowledge and luck. Some of the answers you really know; others you guess. The student who gets the highest score on the first exam probably combines knowledge and good luck. On the second exam, this student's knowledge is still there, but luck is likely to change; thus, the student will probably score lower on the second exam. This is regression toward the mean. In research, regression is a concern whenever participants are selected for their exceptionally high (or low) scores in the first treatment condition. When the same participants are tested a second time, their scores are likely to be lower (or higher) based on regression. Notice that the change in scores is not caused by a new treatment but rather by the statistical phenomenon of regression. In general, statistical regression threatens the internal validity of a research study because it creates the possibility that the observed changes in the participants' scores are caused by regression instead of by the treatments. Regression toward the mean: Statistical regression, or regression toward the mean, refers to the tendency for extreme scores on any measurement to move toward the mean (regress) when the measurement procedure is repeated. Individuals who score extremely high on a measure during the first testing are likely to score lower on the second testing, and, conversely, individuals who score extremely low on a measure during the first testing are likely to score higher on the second testing. Statistical regression occurs because an individual's score is a function both of stable factors such as skill and of unstable factors such as chance. Although the stable factors remain constant from one measurement to another, the unstable factors can change substantially. Your grade on an exam, for example, is based on a combination of knowledge and luck. Some of the answers you really know; others you guess. The student who gets the highest score on the first exam probably combines knowledge and good luck. On the second exam, this student's knowledge is still there, but luck is likely to change; thus, the student will probably score lower on the second exam. This is regression toward the mean. In research, regression is a concern whenever participants are selected for their exceptionally high (or low) scores in the first treatment condition. When the same participants are tested a second time, their scores are likely to be lower (or higher) based on regression. Notice that the change in scores is not caused by a new treatment but rather by the statistical phenomenon of regression. In general, statistical regression threatens the internal validity of a research study because it creates the possibility that the observed changes in the participants' scores are caused by regression instead of by the treatments. Order effects (practice, fatigue, and carry-over effects): Whenever individuals are tested in a series of treatment conditions, participation in one treatment may have an influence on the participants' scores in the following treatments. For example, becoming fatigued in one treatment may lead to poorer performance in the next treatment. You should recognize this problem as a threat to internal validity. Specifically, the experience of being tested in one treatment may explain why the participants' scores are different in the following treatment. Remember, an alternative explanation for an observed difference is a threat to internal validity. In this case, the researcher does not know whether the observed change in performance is caused by the different treatments or by fatigue. Any possible change in performance caused by participation in a previous treatment is called an order effect and is a threat to internal validity because it provides an alternative explanation for the results. Common examples of order effects include fatigue effects (progressive decline in performance as a participant works through a series of treatment conditions) and practice effects (progressive improvement in performance as a participant gains experience through the series of treatment condition). It also is possible that a specific treatment causes changes in the participants so that the lingering aftereffects of the treatment carry over to the next treatment (or treatments) and alter the participants' scores. For example, participants in a memory study may learn a new rehearsal strategy in one treatment condition, and continue to use the strategy to help improve their memory scores when participating in later treatment conditions. Appropriately, these effects are called carry-over effects. Another common example of carryover is a contrast effect in which the subjective perception of a treatment condition is influenced by its contrast with the previous treatment. For example, participants entering a room with moderate lighting for their second treatment may perceive it as dark if they are coming from a brightly lit room for their first treatment. However, the same moderately lit room may be perceived as bright if participants are coming from a dimly lit room. Notice that carry-over effects are caused by experiencing a specific treatment. Other order effects, such as practice effects and fatigue, come from the general experience of being in the study. Occasionally, these other order effects are called progressive error to differentiate them from carry-over effects.

Communication between Groups - Chapter 8

Whenever the participants in one treatment condition are allowed to talk with the participants in another condition, there is the potential for a variety of problems to develop. For example, a researcher may want to test the effectiveness of a new treatment for depression. Using a between-subjects design, the researcher randomly assigns half the clients of an inpatient facil- ity to receive the new treatment and half to receive the standard treatment for depression. If the participants talk to each other, however, then those individuals receiving the old treatment may learn about the new treatment and may begin to use some elements from the new treatment. Diffusion refers to the spread of the treatment from the experimental group to the control group, which tends to reduce the difference between the two conditions. This is a threat to the internal validity of a between-subjects design because the true effects of the treatment can be masked by the shared information (i.e., it appears that there is no difference between the groups because both groups are actually getting much of the same treatment). Another risk is that an untreated group learns about the treatment being received by the other group and demands the same or equal treatment. This is referred to as compen- satory equalization. For example, in a study examining the effects of violent television. bach & Singer, 1971). This threat commonly occurs in medical and clinical studies when one group receives a treatment drug and another does not. A similar problem arises when researchers try to assess the effectiveness of large-scale educational enrichment pro- grams (involving such improvements as computers in the classrooms). Parents and teach- ers of the classes or schools that do not receive the enrichment (the control group) hear about the special program other classes or schools (the experimental group) receive and demand that their children receive the same program or something equal in value. If the demand is met, the research study no longer has a no-treatment condition for comparison. Again, this is a threat to the internal validity of a between-subjects design because it can wipe out the true effects of the treatment (i.e., make it look as if there are no differences between the groups on the dependent variable). Finally, problems can occur when participants in an untreated group change their nor- mal behavior when they learn about a special treatment that is given to another group. One possibility is that the untreated group works extra hard to show that they can perform just as well as the individuals receiving the special treatment. This is referred to as compensa- tory rivalry. In this case, the performance observed by the researcher is much higher than would normally occur. It is also possible that the participants in an untreated group simply give up when they learn that another group is receiving special treatment. This is referred to as resentful demoralization. In this case, the untreated group becomes less productive and less motivated because they resent the expected superiority of the treated group. As a result, the effect of the treatment appears to be much greater than it really is. In each case, internal validity is threatened because the observed difference between groups can be explained by factors other than the effects of the treatment. The best way to minimize each of these threats to internal validity resulting from communication between the groups is to separate the groups of participants as much as possible and keep them from being aware of one another. Notice that these problems are exclusive to between-subjects experimental designs in which different groups of participants are used to compare different treatment conditions.

application and analysis - Chapter 10

The application and analysis of the between-subjects designs presented in this chapter (non- equivalent group designs, including cross-sectional designs) follows exactly the same pat- tern as the application and analysis of between-subjects experiments presented in Chapter 8 (pp. 204-207). Similarly, the application and analysis of within-subjects designs (pre-post and longitudinal) is the same as that presented for within-subjects experiments in Chapter 9 (pp. 233-234). The only exception to this rule is the quasi-experimental pretest-posttest nonequivalent control group design, which includes within-subjects and between-subjects components and is discussed at the end of this section. Two group designs have the advantage of simplicity; they are easy to set up and the results are easy to understand. However, a two-group does not provide the full functional relationship between variables that is available in a multigroup design. When the data con- sist of numerical scores, then the statistical analysis consists of comparing means with either a t test (independent- or repeated-measures) for two means or a single-factor analysis of variance (independent- or repeated-measures) for multiple means. For non-numerical data, the appropriate statistical analysis for a between-subjects design is a chi-square test for independence. These statistical tests are presented in Chapter 15

Summary and recommendations - Chapter 8

The best techniques for minimizing the negative consequences of high variance are to standardize treatments and to minimize individual differences between the participants in the study. Both of these techniques help eliminate factors that can cause differences. inimizing individual differences by holding a variable constant or restricting its range has two advantages: 1. It helps create equivalent groups, which reduces the threat of confounding variables. 2. It helps reduce the variance within groups, which makes treatment effects easier to see. As we noted earlier, however, limiting individual differences has the serious disadvan- tage of limiting external validity. If participation in a study is limited to females between the ages of 18 and 20, for example, then the results cannot be generalized to other ages or to other genders. (An alternative method for reducing individual differences without threatening external validity is presented in Chapter 11, wherein we introduce factorial research designs.)

the Cross-Sectional Developmental research Design - Chapter 10

The cross-sectional developmental research design is a between-subjects design that uses a separate group of participants for each of the ages being compared. A dependent variable is measured for the individuals in each group, and the groups are compared to determine whether there are age differences. For example, a researcher who wants toexamine the relationship between IQ and aging could select three different groups of peo- ple—40-year-olds, 60-year-olds, and 80-year-olds—and could then measure IQ for each group (see Figure 10.7). The cross-sectional developmental research design uses different groups of individuals, each group representing a different age. The different groups are measured at one point in time and then compared.For example, Oppenheimer (2006) used a cross-sectional study to examine changes in people's belief in a just and orderly world as they mature from 12 to 22 years of age. Comparing results from six age groups of students from secondary school through college, the study found that belief in a just world declined as the students aged. A cross-sectional design is an example of a between-subjects nonexperimental design, specifically, a nonequivalent group design. The different groups of participants are not created by manipulating an independent variable; instead, the groups are defined by a preex- isting participant variable (age). Also, the researcher does not randomly assign participants to groups; instead, group assignment is predetermined by each participant's age. Earlier in this chapter, we defined this kind of study as differential research. However, when a study evaluates differences related to age, the design is typically called a cross-sectional study.

Characteristics of Between-Subjects Designs

The defining characteristic of a between-subjects design is that it compares different groups of individuals. In the context of an experiment, a researcher manipulates the inde- pendent variable to create different treatment conditions, and a separate group of par- ticipants is assigned to each of the different conditions. The dependent variable is then measured for each individual, and the researcher examines the data, looking for differ- ences between the groups (Figure 8.1). This chapter focuses on the between-subjects experimental design, that is, the between-subjects design as it is used in experimental research, wherein a researcher manipulates an independent variable. The general goal of a between-subjects experiment is to determine whether differences exist between two or more treatment conditions. For example, a researcher may want to compare two teaching methods (two treatments) to determine whether one is more effective than the other. In this case, two separate groups of individuals would be used, one for each of the two teaching methods. We should note that between-subjects designs are also commonly used for other research strategies, such as nonexperimental and quasi-experimental designs. However, nonexperimental and qua- si-experimental between-subjects designs do not contain a manipulated variable. Nonex- perimental and quasi-experimental strategies are examined in Chapter 10.

Limiting Confounding by Individual Differences - Chapter 8

The first step in conducting a between-subjects experiment is to assign participants to different groups corresponding to the treatment conditions. If the assignment process produces groups of participants with different characteristics, then the study is confounded from individual differences. Specifically, any difference in the scores from one group to another may be caused by individual differences between groups instead of the treatments. Therefore, the initial groups must be as similar as possible. To accomplish this, researchers typically use one of the following three procedures to set up groups for a between-subjects experimental study. The three procedures are the same methods that were identified for controlling potentially confounding variables in an experiment (Chapter 7).

the Longitudinal Developmental research Design - Chapter 10

The longitudinal developmental research design involves measuring a variable in the same group of individuals over a period of time (typically every few months or every few years). The individuals are usually cohorts, roughly the same age, who have grown up in similar circumstances. Several measurements of a particular variable are made in the same individuals at two or more times in their lives to investigate the relationship between age and that variable. For example, to examine IQ and age using the longitudinal approach, a researcher might measure IQ in a group of 40-year-olds and then measure the same indi- viduals again at ages 60 and 80 (Figure 10.8) The longitudinal developmental research design examines development by observing or measuring a group of cohorts over time. A longitudinal study is an example of a within-subjects nonexperimental design, specifically, a one-group pretest-posttest design. In a longitudinal design, however, no treatment is administered; instead, the "treatment" is age. That is, a longitudinal study can be described as a set of observations followed by a period of development or aging, then another set of observations. The differences between the initial observations and the final observations define the effects of development. Thus, longitudinal studies can be viewed as a kind of pretest-posttest study. However, when this type of research is used to evaluate development or the effects of age, the design is typically called a longi- tudinal study. The distinction between a longitudinal design and a time-series design is not always clear. For example, Sun (2001) examined the well-being of a group of adolescents for an extended period before and after their parents' divorces. This can be viewed as a longitudi- nal study because it examined the changes that occur over time for a group of participants. However, it also can be viewed as a pre-post time-series study that compared a series of observations made before an event (the divorce) with a series of observations made after the event.

Controlling time - Chapter 9

The possibility that a research study will be affected by a time-related threat such as history or maturation is directly related to the length of time required to complete the study. For example, if participants go through a series of two or three treatment conditions in a single 45-minute laboratory session, it is very unlikely that time-related threats will have any influence on the results. On the other hand, if the different treatment conditions are scheduled over a period of weeks, the chances greatly increase that an outside event (history), maturation, or change in the measurement instrument will have an influence on the results. By controlling the time from one treatment condition to the next, a researcher has some control over time-related threats to internal validity. Although shortening the time between treatments can reduce the risk of time-related threats, this technique can often increase the likelihood that order effects will influence the results. For example, in situations in which order effects are expected to be temporary, one strategy is to increase the time between treatment conditions so the order effects can dissi- pate. Fatigue, for example, is less likely to be a problem if participants are allowed ample opportunity to rest and recover between treatments. As we have noted, however, increasing the time between treatments increases the risk of time-related threats to internal validity

individual Differences - Chapter 8

The primary disadvantage of a between-subjects design stems from the fact that each score is obtained from a unique individual who has personal characteristics that are different from all of the other participants. Consider the following descriptions of two individuals participating in the same research study Clearly, these two individuals differ on a variety of dimensions. It should also be clear that we have identified only a few of the countless variables that differentiate the two people. Differences between participants on variables such as gender, age, personality, and family background that exist at the beginning of an experiment are called individual differences. The concern with individual differences is that they can cause two different individuals to produce two different scores when a dependent variable is measured in a research study.Occasionally, research is designed with the intention of examining a specific individual difference; for example, a study may be designed to compare behavior or attitudes for people in different age groups (this type of research is discussed in Chapter 10). Most of the time, however, individual differences are simply extraneous variables that are not directly addressed in the research design. For a between-subjects experimental design, individual differences are a particular concern and can create serious problems. The two major concerns are: 1. Individual differences can become confounding variables. Suppose that a researcher finds that participants in treatment A have higher scores than participants in treatment B. The researcher would like to conclude that the higher scores were caused by the treatment; however, individual differences may also provide an explanation for the difference in the scores. If the individuals in one treatment are generally older (or smarter, or stronger) than the individuals in another treatment, then the individual differences between groups may explain why one group has higher scores. 2. Individual differences can produce high variability in the scores, making it difficult to determine whether the treatment has any effect. The unpredictable variability caused by individual differences can obscure patterns in the data and cloud a study's results. One more look at our two hypothetical participants, John and Mary, illustrates the problems that individual differences can cause. 1. Suppose John is assigned to treatment A, where he produces a score of 45, and Mary is assigned to treatment B and has a score of 51. The researcher has found a 6-point difference between the two scores. The researcher must determine what caused the difference. Notice that the difference in scores could be caused by the different treatment conditions. However, the difference could also be explained by the obvious fact that John and Mary are different people with different characteristics. Thus, the 6-point difference in scores could be caused by individual differences. 2. If John and Mary are both assigned to the same treatment condition, then you still expect them to have different scores. In this case, the difference between their scores increases the variance within the treatment, which reduces the likelihood of finding a significant difference between treatments. The problems of confounding variables and high variability are discussed in detail in the following sections. In a between-subjects design, each level of the independent variable (each treatment condi- tion) is represented by a separate group of participants. In this situation, a primary concern is to ensure that the different groups are as similar as possible except for the indepen- dent variable used to differentiate the groups. Any extraneous variable that systematically differentiates the groups is a confounding variable. For example, in a between-subjects experiment comparing two treatments (I and II), one group of participants is assigned to treatment I and a separate group to treatment II. If the participants in one group are gener- ally older (or smarter, or taller, or faster, etc.) than the participants in the other group, then the experiment is confounded by individual differences. Figure 8.2 shows an example of an experiment in which the participant's age is a confounding variable. In the figure, the two groups of participants are differentiated by treatment (I vs. II) and age (one group is older than the other). If the results from this example showed that the scores in one group were consistently higher than scores in the other group, it would be impossible to determine whether treatment or age is responsible for causing the difference between groups. Because the experiment is confounded, it is impossible to draw any clear conclusions. Note that this problem applies exclusively to research designs comparing different groups; that is, between-subjects designs. Whenever the individuals in one group have characteristics that are different from those in another group, the internal validity of the study is threatened

Within-Subjects Nonexperimental and Quasi-Experimental Designs: Pre-Post Designs - Chapter 10

The second general category of nonexperimental and quasi-experimental designs consists of studies in which a series of observations is made over time. Collectively, such studies are known as pre-post designs. In a typical pre-post study, one group of participants is observed (measured) before and after a treatment or event. The goal of the pre-post design is to evaluate the influence of the intervening treatment or event by comparing the observations made before treatment with the observations made after treatment. You may have noticed that a pre-post design is similar to the pretest-posttest nonequivalent control group design discussed earlier. However, a pre-post design has no con- trol group. In addition, the primary focus of a pretest-posttest nonequivalent control group design is to compare the treatment group and the control group, not to compare the pretest scores with the posttest scores. As a result, the pretest-posttest nonequivalent control group design is primarily a nonequivalent group design, and we have classified it in that category A pre-post design is a research study in which a series of observations is made over time for one group of participants

two-treatment Designs - Chapter 9

The simplest application of a within-subjects design is to evaluate the difference between two treatment conditions. The two-treatment within-subjects design has many of the same advantages and disadvantages as the two-group between-subjects design discussed in Chapter 8 (see pp. 204-205). On the positive side, the design is easy to conduct and the results are easy to understand. With only two treatment conditions, a researcher can easily maximize the difference between treatments by selecting two treatment conditions that are clearly different. This usually increases the likelihood of obtaining a significant differ- ence. In addition, with only two treatment conditions, it is very easy to counterbalance the design to minimize the threat of confounding from time-related factors or order effects. On the negative side, a study with only two treatments provides only two data points. In this situation, it is possible to demonstrate a difference between conditions, but the data do not provide any indication of the functional relationship between the independent and dependent variables. That is, we cannot determine how the dependent variable would respond to small, gradual changes of the independent variable. With data measured on an interval or ratio scale, the most common strategy for data analysis is to compute a mean score for each treatment condition. The means are used to describe (summarize) the individual treatments, and the difference between means is used to describe the differential effects of the treatments. With two treatment conditions, a repeated-measures t or a single-factor ANOVA (repeated measures) can be used to eval- uate the statistical significance of the mean difference, that is, to determine whether the obtained mean difference is greater than what would be reasonably expected from sam- pling error (see Chapter 15). If the data do not permit the calculation of treatment means, there are alternative methods for statistically evaluating the difference between treatments. If the data are measured on an ordinal scale (or can be rank ordered), a Wilcoxon Signed- Ranks test can be used to evaluate significant differences. Occasionally, a within-subjects study comparing two treatments produces data that show only the direction of difference between the two treatments. For example, a therapist may be able to classify individual clients as showing improvement or showing decline after treatment. In this situation, the data can be statistically evaluated using a sign test to determine whether the changes are consistently in one direction (enough to satisfy statistical significance).

two-Group Mean Difference - Chapter 8

The simplest version of a between-subjects experimental design involves comparing only two groups of participants: The researcher manipulates one independent variable with only two levels. This design is often referred to as the single-factor two-group design or simply the two-group design. This type of design can be used to compare treatments or to evaluate the effect of one treatment by comparing a treatment group and a control group. When the measurements consist of numerical scores, typically, a mean is computed for each group of participants, and then an independent-measures t-test is used to determine whether there is a significant difference between the means (see Chapter 15). The primary advantage of a two-group design is its simplicity. It is easy to set up a two-group study, and there is no subtlety or complexity when interpreting the results; either the two groups are different or they are not. In addition, a two-group design provides the best opportunity to maximize the difference between the two treatment conditions; that is, you may select opposite extreme values for the independent variable. For example, in a study comparing two types of therapy, the two therapies can be structured to maximize or even exaggerate the differences between them. Or, in a research study comparing a treat- ment and a no-treatment control, the treatment group can be given the full-strength version of the treatment. This technique increases the likelihood of obtaining noticeably different scores from the two groups, thereby demonstrating a significant mean difference. The primary disadvantage of a two-group design is that it provides relatively little information. With only two groups, a researcher obtains only two real data points for com- parison. Although two data points are sufficient to establish a difference, they often are not sufficient to provide a complete or detailed picture of the full relationship between an independent and a dependent variable. Figure 8.5 shows a hypothetical relationship between dosage levels for a drug (independent variable) and activity (dependent variable). Notice that the complete set of five data points, representing five different drug doses, gives a good picture of how drug dosage affects behavior. Now, consider the limited data that would be available if the researcher had used only two different drug doses. If, for example, the researcher had used only a zero-dose and a one-dose group (points A and B in the figure), the data would seem to indicate that increasing the drug dose produces an increase in activity. However, a researcher comparing a two-dose versus a four-dose group (points C and E) would reach exactly the opposite conclusion. Although both of the two- group studies are accurate, neither provides a complete picture. In general, several groups (more than two) are necessary to obtain a good indication of the functional relationship between an independent and a dependent variable. A two-group study also limits the options when a researcher wishes to compare a treatment group and a control group. Often, it is necessary to use several control groups to obtain a complete picture of a treatment's effectiveness. As we noted in Chapter 7, two common controls that often are used together are a no-treatment control and a placebo control. With these two control groups, researchers can separate the real treatment effects. from the placebo effects that occur simply because participants think that they are receiv- ing treatment. However, as we noted in Chapter 4 (p. 109), there is some ethical concern regarding the use of no-treatment or placebo groups in clinical research. Rather than deny- ing treatment to some participants, it is suggested that an established, standard therapy be used for the control comparison (LaVaque & Rossiter, 2001).

a Nonexperimental pre-post Design - The Pretest-Posttest Design - Chapter 10

The simplest version of the pre-post design consists of one observation for each partici- pant made before the treatment or event, and one observation made after it. Schematically, this simple form can be represented as follows: OXO This type of study is called a pretest-posttest design. For example, a political consultant could evaluate the effectiveness of a new political television commercial by assessing voters' attitudes toward a candidate before and after they view the commercial. The results from this study may demonstrate a change in attitude. However, because this design makes no attempt to control the many threats to internal validity, the study cannot conclude that the change was caused by the intervening commercial. Because the pretest- posttest study precludes a cause-and-effect conclusion, this type of research is classified as nonexperimental. In the nonexperimental pretest-posttest design, each individual in a single group of participants is measured once before treatment and once after treatment.

Differential attrition - Chapter 8

The term attrition refers to participant withdrawal from a research study before it is com- pleted. As long as the rate of attrition is fairly consistent from one group to another, it usu- ally is not a threat to internal validity. However, big differences in attrition rates between groups can create problems. The different groups are initially created to be as similar as possible; if large numbers of individuals leave one group, the group may no longer be similar to the others. Again, whenever the groups of participants are noticeably different, the research is confounded. Differential attrition refers to differences in attrition rates from one group to another and can threaten the internal validity of a between-subjects experiment. For example, a researcher may want to test the effectiveness of a dieting program. Using a between-subjects design, the researcher forms two groups of participants with approximately equal characteristics (weight, gender, dieting history). Next, one group of participants begins the 10-week dieting program, and the other group receives no treat- ment (this group, recall from Chapter 7, is the no-treatment control group). At the end of 10 weeks, the weights of the two groups are compared. During the course of 10 weeks, however, it is likely that some participants will drop out of the study. If more participants drop out of one group than the other, there is a risk that the two groups will no longer be similar. For example, some of the individuals in the dieting program may decide that it is too demanding and withdraw from the study. As a result, only the most motivated partici- pants stay in the diet program. Although the study started with two equivalent groups, the individuals who are left in the program at the end have a higher level of motivation than those in the control group. In this case, the difference in dropout rate between the groups could account for the obtained differences in mean weight. Differential attrition is a threat to internal validity because we do not know whether the obtained differences between treatment conditions are caused by the treatments or by differential attrition. Whenever participants drop out of a study, a researcher must be concerned about differential attrition as an alternative explanation for treatment effects.

Single-Case applications of time-Series Designs - Chapter 10

The time-series design was introduced as a research study that involves observing a group of participants at several different times. However, this design is often applied to single individuals or single organizations. For example, a high school could evaluate the effects of an anger-management program by monitoring the number of fights at the school for 3 months before the program is enacted and for 3 months afterward. This is an example of a time-series design, but it involves measurements for one high school, not for individual participants. Similarly, a therapist could monitor instances of compulsive behavior in one client for 3 weeks before therapy and for 3 weeks after. This is an example of a time-series design applied to a single individual. Research designs that focus on a single case, rather than a group of participants, are occasionally called single-case time-series designs but are more often classified as single-case or single-subject designs. Single-case designs are discussed in Chapter 14.

Counterbalancing and the Number of Treatments - Chapter 9

To completely counterbalance a series of treatments, it is necessary to present the treat- ments in every possible sequence. The idea behind complete counterbalancing is that a particular series of treatment conditions may create its own unique order effect. For exam- ple, treatments II and III, in sequence, may produce a unique effect that carries over into the next treatment. Treatments I and III, in sequence, may produce a different order effect. To completely balance these combined effects, the research design should use every possi- ble ordering of treatment conditions. With only two treatment conditions, complete counterbalancing is easy: There are only two possible sequences. However, as the number of treatments increases, complete counterbalancing becomes more complex. If the number of different treatment conditions is identified as n, then the number of different sequences is n! (n factorial). For example, with four treatment conditions, there are 4! 5 4 3 3 3 2 3 1 5 24 dif- ferent sequences. If the four treatments are identified as A, B, C, and D, the 24 sequences can be listed as follows: ABCD BACD CABD ABDC BADC CADB ACBD BCAD CBAD ACDB BCDA CBDA DBCA ADBC BDAC CDAB DCAB ADCB BDCA CDBA DCBA To completely counterbalance a within-subjects experiment with four treatment condi- tions, the researcher must divide the participants into 24 equal-sized groups and assign one group to each of the 24 different sequences. Obviously, this study would require at least 24 participants (one per group), which may be more than the researcher needs or wants. With even more treatments, the demands of complete counterbalancing can become out- rageous. With n 5 6 treatments, for example, there are 6! 5 720 different treatment sequences, which means that the study would require a minimum of 720 participants. One solution to this problem is to use what is known as partial counterbalancing. Instead of every possible sequence, partial counterbalancing simply uses enough different orderings to ensure that each treatment condition occurs first in the sequence for one group of participants, occurs second for another group, third for another group, and so on. With four treatments, for example, this requires only four different sequences, such as: ABCD, CADB, BDAC, DCBA. To conduct a partially counterbalanced study with four treatments, a researcher needs to divide the participants into four equal sized groups and assign one group to each of the four sequences. One group of participants receives treatment A first, one group has A second, one has A third, and one has A fourth. Similarly, each of the other treatments appears once in each ordinal position. Because partial counterbalancing does not use every possible sequence of treatment conditions, one problem is to decide exactly which sequences to select. A simple and unbi- ased procedure for selecting sequences is to construct a Latin square. To create a Latin square for four treatment conditions, start with a 4 3 4 matrix and fill it in with the letters A, B, C, and D, as follows: List the letters ABCD in order in the top row of the matrix. To create the next row, simply move the last letter in line to the beginning. This creates DABC for the second row. Continue moving the last letter to the beginning of the line to create each new row. The result is the following Latin square: List the letters ABCD in order in the top row of the matrix. To create the next row, simply move the last letter in line to the beginning. This creates DABC for the second row. Continue moving the last letter to the beginning of the line to create each new row. The result is the following Latin square: By definition, a Latin square is a matrix of n elements (letters) where each element appears exactly once in each column and in each row. Each row in the square provides a sequence of treatment conditions for one group of participants. For this example, the first group receives the four treatments in the order ABCD. A second group receives the order DABC, and so on. The Latin square in the preceding paragraph is not a particularly good example of partial counterbalancing because it does not balance every possible sequence of treat- ment conditions. For example, the first three groups all receive treatment A followed immediately by treatment B. On the other hand, no one receives treatment B followed by treatment A. Whenever possible, a Latin square should ensure that every possible sequence of treatments is represented. One method for improving the square is to use a random process to rearrange the columns (e.g., a coin toss to decide whether or not each column is moved), then use a random process to rearrange the rows. The resulting rows in the square should provide a better set of sequences for a partially counterbalanced research study.

threats to Internal Validity for pre-post Designs - Chapter 10

Whenever the same group of individuals is observed repeatedly over time, time-related factors can threaten internal validity. As we noted in Chapter 9, the five categories of time-related threats are history, instrumentation, order effects, maturation, and statistical regression. Clearly, pre-post studies are vulnerable to these threats; any dif- ferences found between the pretreatment observations and the posttreatment observations could be explained by history, instrumentation, order effects, maturation, or regression (see Chapter 9, pp. 214-217). You may recognize that a pre-post design is similar to the within-subjects experimental design presented in Chapter 9. However, the experimental design uses counterbalancing to control order effects and other time-related threats to internal validity. In a pre-post design, it is impossible to counterbalance the order of treat- ments. Specifically, the before-treatment observations (pretest) must always precede the after-treatment observations (posttest). In general, the internal validity of a pre-post study is threatened by a variety of fac- tors related to the passage of time. During the time between the first observation and the last observation, any one of these factors could influence the participants and cause a change in their scores. Unless these factors are controlled or minimized by the structure of the research design, a pre-post study cannot approach the internal validity of a true experiment. In this section, we introduce two examples of pre-post studies: the one-group pretest-posttest design and the time-series design. The first of these designs makes no attempt to control the threats to internal validity and, therefore, is classified as nonexper- imental. The second design manages to minimize most threats to internal validity and is classified as quasi-experimental.

Dealing with Time-Related Threats and Order Effects - Chapter 9

Within-subjects designs can control environmental threats to internal validity using the same techniques that are used in between-subjects designs. Specifically, environmental factors such as the room, the experimenter, or the time of day, can be controlled by (1) randomization, (2) holding them constant, or (3) matching across treatment conditions. Time-related factors and order effects, on the other hand, require special attention and new strategies for control. Because order effects and time-related threats to internal validity can be serious problems whenever a within-subjects design is selected, researchers have developed a variety of ways to control these potential threats. In this section, we examine some of the methods for dealing with order effects and time-related threats to gain the full benefit of within-subjects design

Standardize Procedures and Treatment Setting - Chapter 8

a between-subjects design, each group of participants represents a single treatment condi- tion. One obvious way to help minimize the variability within each group is to be sure that all participants within a group are treated exactly the same. Although existing individual differences are not reduced, at least care is taken not to increase them. Thus, researcher hould avoid making any changes in the treatment setting or the procedures used from one individual to another. Whenever two individuals are treated differently, there is a chance that differences between their scores will be increased, thus increasing the variance within the group. In general, if two participants are in the same group (the same treatment condition), a researcher should not do anything that might cause their scores to be different. Standardizing procedures also makes it easier for other researchers to understand exactly how your study was done and makes it possible for them to replicate your study in their own research facility

Developmental research designs - Chapter 10

are another type of nonexperimental research that can be used to study changes in behavior that relate to age. The purpose of developmental research designs is to describe the relationship between age and other variables. For exam- ple, if a researcher is interested in how language ability changes with age, a developmental research design would be appropriate Developmental research designs are used to examine changes in behavior related to age. Two basic types of developmental research designs are the cross-sectional design and the longitudinal design. Each has its strengths and weaknesses.

Chapter Over view - Chapter 8

esearch suggests that it is more effective for students to study text on printed hardcopy than to study text displayed on a computer screen (Ackerman & Goldsmith, 2011). In the study, college students were randomly assigned to one of the two media conditions to study text of 1,000-1,200 words and then were given a multiple-choice test on the mate- rial. When the students controlled their own amount of study time, test performance was significantly worse for students who studied on a computer screen. Because the researcher carefully controlled other variables, they can conclude confidently that the type of media causes a difference in learning performance. You should recognize this study as an example of an experiment. The researchers manipulated the type of media to create two treatment conditions and randomly assigned students to conditions to control extraneous variables. They recorded scores on the multiple-choice test and then compared the two sets of scores. Another characteristic of this study is that the two groups of scores are obtained from two separate groups of participants. An experiment always involves the comparison of different groups of scores. However, each group of scores can be obtained from a separate group of participants (as in this experiment), or they can be obtained from the same group of participants. You may recall, for example, in Section 1.2 we discussed an experiment showing that people are able to tolerate more pain when they are shouting swear words than when they shout neutral words (Stephens, Atkins, & Kingston, 2009). In that experiment, each partici- pant was measured in both the swearing and the neutral word condition so that the two groups of scores came from the same group of participants. The choice between one group and multiple groups of participants is one of the characteristics that differentiate one type of experiment from another, and hence deter- mines for a particular research strategy, the selection of a research design (see Step 6 of the research process). In this chapter, we discuss in detail one type of experimental research design: the between-subjects design. The between-subjects design uses a sepa- rate group of individuals for each of the different treatment conditions. We consider the advantages, disadvantages, and different versions of between-subjects designs.

A differential research design - Chapter 10

is a research study that simply compares preexisting groups. A differential study uses a participant characteristic such as gender, race, or personality to automatically assign participants to groups. The researcher does not randomly assign individuals to groups. A dependent variable is then measured for each participant to obtain a set of scores within each group. The goal of the study is to determine whether the scores for one group are consistently different from the scores of another group. Differential research is classified as a nonexperimental research design.

Comparing Means for More than two Groups - Chapter 8

o evaluate the functional relation between an independent and a dependent variable or to include several different control groups in a single study. In these cases, a single-factor multiple-group design may be used. For example, a researcher may want to compare driving performance under three telephone conditions: while talking on a cell phone, while texting on a cell phone, and without using a phone. Another researcher may want to examine five different dosages of a drug to evaluate the relation between dosage and activ- ity level for laboratory rats. In the first example, the independent variable is the telephone condition with three levels compared. In the second example, the researcher compares five levels of drug dosage. For either study, the mean is computed for each group of partici- pants, and a single-factor analysis of variance (ANOVA) (independent measures) is used to determine whether there are any significant differences among the means (see Chapter 15). When the ANOVA concludes that significant differences exist, some form of post hoc test or posttest is used to determine exactly which groups are significantly different from each other. In addition to revealing the full functional relationship between variables, a multiple-group design also provides stronger evidence for a real cause-and-effect relation- ship than can be obtained from a two-group design. With a multiple-group design, the researcher changes the treatment conditions (independent variable) several times across several groups, demonstrating differences in performance for each different treatment condition. By contrast, a two-group design changes the treatment condition only once and observes only one difference in performance.

Individual differences - Chapter 8

personal characteristics that differ from one participant to another.

A between-subjects experimental design - Chapter 8

requires a separate, independent group of individuals for each treatment condition. As a result, the data for a between-subjects design contain only one score for each participant. To qualify as an experiment, the design must satisfy all other requirements of the experimental research strategy, such as manipulation of an independent variable and control of extraneous variable

Counterbalancing and Variance - Chapter 9

t treatment conditions. However, this process does not eliminate the order effects. In particular, the order effects are still part of the data, and they can still create problems. One is that they can distort the treatment means. In Table 9.2, the order effects are present in both treatments and inflate both of the treatment means. Usually, this kind of distortion is not important because researchers typically are interested in the amount of difference between treatments rather than the absolute magnitude of any specific mean. When counterbalancing works as intended, the differences between means are not changed. However, in situations in which the absolute level of performance (the true mean) is important, the process of counterbalancing can disguise the true value of a treatment mean. A more serious problem is that counterbalancing adds the order effects to some of the individuals within each treatment but not to all of the individuals. In the example shown in Table 9.2, some of the individuals in treatment I receive an extra 5 points and some do not. As a result, the differences between scores are increased within each treat- ment, which adds to the variance within treatments. Recall from Chapter 8 (p. 196) that large variance within treatments can obscure treatment effects. In statistical terms, high variance within treatments decreases the likelihood that a research study will obtain sig- nificant differences between treatments. Thus, in situations in which order effects are relatively large, the process of counterbalancing can undermine the potential for a suc- cessful experiment

restricted random assignment - Chapter 8 def.

the group assignment process is limited to ensure predetermined characteristics (such as equal size) for the separate groups.