Chapter 11 research methods

A researcher previously conducted a one-group pretest-posttest design measuring the effects of a new seatbelt law on the use of seatbelts. In order to improve the internal validity of the study, the researcher conducts a simple interrupted time-series design. Although the additional pretest and posttest observations improve the internal validity of the study, they cannot address the effect of participants A. attrition. B. maturation. C. extreme scores. D. fatigue.

A

A researcher previously conducted a one-group pretest-posttest design measuring the effects of a new seatbelt law on the use of seatbelts. In order to improve the internal validity of the study, the researcher conducts a simple interrupted time-series design. Although the additional pretest and posttest observations improve the internal validity of the study, they cannot address the effect of participants': A. attrition. B. maturation. C. extreme scores. D. fatigue.

A

Events that occur while a study is being conducted and that are NOT part of the experimental manipulation or treatment are referred to as A) history B) maturation C) testing D) Instrumentation

A

A quasi experiment has some features of an experiment but Lacks key aspects of experimental control is higher in internal validity

A

A researcher examining the effects of a new mathematics instructional technique measures the math scores of the students in the intervention classroom 10 times before the intervention and 10 times after the intervention. Which threat to validity still remains? A. history B. regression to the mean C. testing D. maturation

A

Using an interrupted time-series design with a nonequivalent control group:

A. reduces the likelihood that differences on the dependent variable are caused by demand characteristics. B. allows researchers to understand differential attrition by examining pretreatment trend lines. C. ensures that the treatment and control groups receive the treatment at the same time. D. eliminates the possibility of differential maturation and testing.

A researcher administers a treatment to one of the groups in her study and then, after some time has passed, administers the same treatment to the other nonequivalent group in her study. This is an example of a: A. pretest-posttest design with delayed treatment. B. switching replication design. C. wait-list control design. D. switching treatment design.

B

Why would one use a quasi experiment if they used random assignment of 6 groups matched on government division, similar workloads and time pressures, age, education, job ranks and gender composition

Bc there is only 6 groups groups differed on tasks they performed

9. In a diagram of the procedure for the group that receives the treatment first in a pretest-posttest design with a switching replication, the symbol X refers to the: A) beginning of the observation period. B) end of the observation period. C) removal of the treatment. D) introduction of the treatment to the nonequivalent control group.

C

A group of researchers investigates the effects of a hurricane on post traumatic stress disorders. The reserachers compare post traumatic stress of residents of a small town before and after the hurricane. This study qualifies as a quasi experiment primarily because a)there is an adequate sample size b)there is no dependent variable c)there is only one group of participants d)the dependent variable is a sensitive disorder

C

A researcher compares the cigarette smoking rates of two towns, one that had instituted a tax on cigarettes a month earlier and one that had no tax. This is an example of a: A. posttest-only design. B. simple posttest-only design. C. posttest-only design with a nonequivalent control group. D. posttest-only design with a control group.

C

A researcher is interested in whether she can reduce college student drinking using a drinking intervention. She designs a 2-year study. In the first year, she has students fill out a drinking survey every 3 months. At the end of the first year, she conducts the drinking intervention. In the second year, she has students fill out the same drinking survey every 3 months. This is an example of a(n) _____ design. A uninterrupted time-series B. one-group pretest-posttest C. simple interrupted time-series. D. longitudinal pretest-posttest

C

A school district asks a research team to evaluate an antibullying program that the school implemented the year before. Since the program had already been implemented, the researchers inform the district that they would have to use a(n) _____ design, because their research would lack many key aspects of experimental control. A. experimental B. partial-experimental C. quasi-experimental D. simple

C

For some experts, the key feature of a quasi experiment is the inability to a) measure the independent variable b) develop the key hypothesis C) randomly assign procejures d) conduct appropriate statistical analysis

C

In a pretest-posttest design with a switching replication design, each group of participants receives the treatment _________ time(s) A. two B. six C. one D. three

C

In quasi experiments A. the control group does not receive the treatment. B. there is always a control group. C. practical limitations may make random assignment impossible. D. researchers control most extraneous environmental variables.

C

The procedure for the control group appears _____ the dashed line a) above B) to the left of C) below D) to the right of

C

n a simple interrupted time-series design, a dependent variable is repeatedly measured A. once before and once after a treatment. B. at periodic intervals before a treatment and once after treatment. C. at periodic intervals before and after a treatment. D. once before a treatment and at periodic intervals after treatment.

C

Differential

Selection and history; differential history selection interactions involved the interaction of selection with another threat to internal validity When selection interacts with other threats such as history, maturation, testing, instrumentation, regression, and attrition, this is called differential _________

Diagramming Quasi designs

X; treatment O; observation (measurement) temporal order proceeds from left to right O1 X 02 ----------- 01 02 subscripts 1 and two represent the observational periods dashed line represents a non equivalent control group the control groups receives the pre and post test but not the treatment O refers to a distinct measurement period

limitations to a interrupted times series design with a non equivalent control group

adding a control group does not address other control problems like demand characteristics, experimenter expectancy effects, and placebo effects

Advantages to a pretests post test with a non equivalent control group

adding a pretest provides information about the extent to which the treatment and control groups were non equivalent on the dependent variable prior to the treatment the pretest lets us gauge how much the dependent variable changed during the study for each group Pretest scores may give us some clues about possible threats to internally validity, most notable statistical regression and attrition

limitations to follow up observations

adding more post tests does not help rule out basic threats to internal validity

advantages to follow up observations

allows researches to examine whether any immediate post treatment changes persist over time an intervention may have delayed effects that would not be detected by using only one post test

outcome evaluation

assesses a program's effectiveness

A researcher assigns participants to their respective conditions based on what time of day they arrived for the experiment. The first 50 who arrived were placed in the treatment group and the second 50 who arrived were placed in the control group. The researcher is concerned that the two groups of participants may vary on a key variable that affects the dependent variable. The experimenter is worried about a _____ threat to validity. A. testing B. selection C. maturation D. history

b

differential maturation

overall, the two groups have different maturations rates

differential attrition

the rate of attrition, or the reasons for attrition, differ between the two groups

One group post test only design

treatment occurs and the DV is measured after once people receive special skill training then their ability is measured after ex; doctor treats 20 ppl with depression (no pretest measurement they just claim this) w therapy, then measures depression afterwards weak design bc lots of internal validity

program evaluation

use of research methods to asses the need for and the design implementation and effectiveness of a social intervention

Advantages to a interrupted times series design with a non equivalent control group

we can examine how nonequivalent the two groups are prior to the intervention on the dependent variables both groups will be exposed to the same general history tests and general instrumentation changes adding a control group helps to determine whether the treatment produced an effect

efficiency assessment

weighs the program's benefits and effectiveness in relation to its costs to determine whether it is an efficient method for addressing the program

differential regression

when one group is selected on the basis of more extreme scores (pretest measure) than another group, it is likely that the groups average post test scores will reflect different degrees of regression to the mean

program diffusion

implementing and maintaining effective programs in other settings or with other groups

dissemination (program diffusion)

informing target groups and organizations that a certain program exists

In a study of the effects of a peer-to-peer treatment for post-traumatic stress disorder flashbacks, researchers selected two groups of participants from two neighboring towns that had recently experienced an earthquake. After initial observations, the researchers administer the treatment to the first group, take another set of observations, administer the treatment to the second group, and then take a final set of observations. This design is called a A. pretest-posttest design with switching replication. B. switching replication with treatment removal. C. simple interrupted time-series design with switching replication. D. simple interrupted time-series design.

A

In a switching replication design: A) one group receives a treatment, and a nonequivalent group initially does not receive the treatment but is then exposed to it at a later time. B) Both groups receive the treatment at the same time, but one group as pre and post observations. C) a group receives the treatment on and off and a nonequivalent groups is never exposed to the treatment

A

Which type of study is diagrammed as X 01 A) one group posttest only design B) One group pretest-posttest design C) simple interrupted time series design D) complex interrupted time series design

A

Interrupted times series design with a non equivalent control group

A series of pre and post treatment scores are obtained for a treatment group and a non equivalent control group

In a posttest-only design with a nonequivalent control group, participants in one condition: A. and a nonequivalent group are exposed to the treatment, and scores from both groups are obtained before and after the treatment ends. B. are exposed to a treatment, a nonequivalent group is not exposed to the treatment, and scores from both groups are obtained after the treatment ends. C. are exposed to a treatment, a nonequivalent group is not exposed to the treatment, and scores from both groups are obtained before and after the treatment ends. D. and a nonequivalent group are exposed to the treatment, and scores from both groups are obtained after the treatment ends.

B

Using an interrupted time-series design with a nonequivalent control group: A. reduces the likelihood that differences on the dependent variable are caused by demand characteristics. B. allows researchers to understand differential attrition by examining pretreatment trend lines. C. ensures that the treatment and control groups receive the treatment at the same time. D. eliminates the possibility of differential maturation and testing

B

Which research design can be diagrammed as: O1 X O2 -------------- O1 O2 A. a time-series design with a nonequivalent control group B. a pretest-posttest design with a nonequivalent control group C. a simple interrupted time-series design with a nonequivalent control group D. a posttest-only design with a nonequivalent control group

B

Simple interrupted time series design

DV is repeatedly measured at periodic intervals before and after a treatment key is to have enough observations for pretreatment and post treatment trends in IV researchers suggest including about 100 observations

8. A researcher conducts a study with a pretest-posttest design with a nonequivalent control group. The researcher can be reasonably confident that a threat to internal validity is NOT: A) demand characteristics. B) experimenter expectancy effects. C) placebo effects. D) differential history.

D

A national team of researchers examines the effect of a new antidepressant on depressive symptoms using a depression interview that they created. They use inpatients from research hospitals around the country and statistically control for extraneous factors such as depression severity. This study is an example of a quasi-experiment because the researchers are: A. unable to determine the active ingredient in the antidepressant. B. using interviewers who they hired. C. using a dependent variable that they created. D. only using a treatment group.

D

A school psychologist compares the effectiveness of a hall monitor on children's disruptive behavior between class periods. The school psychologist compares two different schools, one that has the monitor and one that does not have a monitor. He compares the schools several times during the year, both before and after the hall monitor was introduced in one of the schools. This is an example of a(n): A. interrupted time-series design. B. pretest-posttest design with a nonequivalent control group. C. posttest-only design with a nonequivalent control group. D. interrupted time-series design with a nonequivalent control group.

D

A university considers opening a hotline for students experiencing stress from balancing their school, work, and family responsibilities. Before they create the hotline, the university decides to call in a researcher to determine whether students are experiencing stress and if they would use a hotline. The researcher will MOST likely conduct a(n) _____ to determine whether the university should create the hotline A. design assessment B. outcome evaluation C. process evaluation D. needs assessment

D

An advantage of a pretest-posttest design with a nonequivalent control group compared to a posttest design with a nonequivalent control group is the ability to rule out _____ threats to internal validity. A. regression to the mean and selection B. maturation and history C. attrition and maturation D. attrition and regression to the mean

D

In order to assess for selection effects in an interrupted design with a nonequivalent control group, the researcher needs to look at the: A. differences in the pretest-posttest scores. B. differences in the trends between the pretest and posttest observations. C. trends in the pretest observations. D. trends in the posttest observations.

D

One of the things that a switching replication design with treatment removal can do that a basic switching replication design cannot is: A) determine if the control group changed as a result of the treatment. B) rule out differential history effects. C) control for confounding variables such as placebo effects and demand characteristics. D) determine if the initial treatment group maintains their improvement.

D

When participants in one group experience outside events that the other group does not, what has occurred? A. differential maturation B. differential testing C. differential regression D. differential history

D

Which type of study is diagrammed as: O1 X O2? A. one-group posttest-only design B. simple interrupted time-series design C. complex interrupted time-series design D. one-group pretest-posttest design

D

one group pretest-post test design

DV is measured before and after a treatment occurs

threats to internal validity

history maturation (spontaneous remission) testing instrumentation regression to the mean attrition selection

limitations to simple interrupted time series designs

History is often the main threat to potential internal validity

Designs with a non equivalent control group

basic threats to internal validity selection emerges as a major threat to internal validity selection may interact with other threats in such case ???? DIFFERWNTIAL

Advantages for Quasi experiments

best way to evaluate a social program when randomized experimental cannot be used

differential instrumentation

changes occurs in a measuring instrument that affects how one group's behavior is recorded but do not affect, or have a different effect on recording in the other group

sustainability (program diffusion)

concerns maintaining the program over time

process evaluation

determined whether a program is being implemented as intended. Key issues include whether the program services are being delivered 1) in the form that they were intended 2) by the people who are supposed to deliver them 3) to members of the target population 4)to the number of recipients that had been projected

needs assessment

determines whether there is a need for a social program and the general steps required to meet that need

program theory and design assessment

evaluates the rationale for why a program has been or will be designed in a particular way what components or services should the program include? How should services be delivered? whats the logic justifying these choices?

benefits to program evaluations

help organizations, governments, other agencies, and local communities determine whether there is a need for a social program. to identify whether a program is effective and should be continued expanded reduced or terminated to help w current programs to add scientific knowledge

switching replication with treatment removal

initial treatment group continues to receive the treatment while the control group receives delayed treatment O1 X O2 X(line through it) O3 ---------------------------------------------- O1 X O2 X O3 participants are not randomly assigned the initial control group receives treatment but on a delayed basis when the initial group begins treatment, treatment for the other group may either continue or be terminated threats to internal validity are in general better controlled that in non equivalent control group designs that don't include switching replications

The major type of validity that is threatened in a quasi-experiment with only one group is _____ validity.

internal

Selection interactions

involve the interaction of selection with another threat to internal validity

implementation (program diffusion)

involves initiating the program in the new setting and executing it properly during a trial period

Treatment and control groups are considered non equivalent when

it isn't possible to randomly assign participants to conditions or random assignment is used in a way that cannot be assumed confidently to create equivalent groups at the start of a study

Follow up observations

measurement of DV after long period of time

limitations

more vulnerable to issues w internal validity hard to establish causal effects

adoption (program diffusion)

occurs when a new group or organization agrees to implement the program

contamination

occurs when knowledge, services, or other experiences intended for one group are unintentionally received by another group

Switching Replication design (pre post test design)

occurs when one group receives a treatment while a nonequivalent group does not receive a treatment but is then exposed to a treatment down the road can be used with both pretest-post test and time series designs there is no random assignment of participants to non equivalent groups

constraints to program evaluations

one challenge fore program evaluation researchers is the need to carry out their work in a context that can be politically charged there may be conflicting interests between sponsors on the one hand and stakeholders in others researchers may not get to choose the objectives of the evaluation ethical issues

differential testing

one group experiences testing effects that the other group does not

post test only design with a non equivalent control group

participants in one condition are exposed to a treatment participants in the other nonequivalent condition are not exposure to the treatment scores from both groups are obtained after the treatment ends addition of the control group may enhance the studies validity, but it may nothing to reduce potential confounding variables (all depends on the similarity between the treatment and control groups) no random assignment confounds weakest of the quasi experimental designs with a control group, the researcher may not have been able to gather pretest data

Differential history

participants in one group experience outside events that those in the other group do not

Pretests post test with a non equivalent control group

pre ad post treatment scores are obtained for a treatment group and a non equivalent control group. This design is diagrammed as follows O1 X O2 -------------- O1 X O2

Quasi Experimental Designs

resemble experiments, lack experimental control might not have control over the design of the treatment (IV), when it occurs, who it effects. These factors may be determined by other ppl, school, business, gov officials, or natural events. (earthquakes) Might nor have control over the choice of the dependent variables, or how, when and for whom they are measured little control over extraneous environmental events during the study Participant characteristics inability to randomly assign participants to conditions frequently used to measure social programs

limitations to a pretests post test with a non equivalent control group

selection and history are the primary confounding variables to consider

advantages to simple interrupted time series design

strengthens the argument that confounding variables are not responsible time series designs help us determine whether maturation, testing, and regression are potential confounding variables