Research Methods Chapter 13

Nesting a quasi-experimental design within a randomized experiment refers to:

a way of backing up a true experiment

Randomization may not be appropriate for assignment of people to treatment or programs for all of the following reasons except:

agency support

In order to measure a program's success in meeting goals, the researcher should be able to do which of the following in advance of the inception of the research?

all of the above

A scouting report is a product of:

an evaluability assessment

Process evaluations:

assess the extent to which policies are being implemented as intended

True or "classical" experiments differ from quasi-experiments with respect to the fact that

classical experiments use random assignment to control and experimental groups while quasi-experimental designs do not

What is the fundamental first step in conducting evaluation studies?

clearly specifying program goals

Many law enforcement agencies are using a program to detect emerging patterns in crime, this program is:

computerized crime maps

Specifying program goals and objectives in operational terms produces:

conceptual definitions

Which of the following represents potential problems in doing evaluation research?

conflicting goals

With regard to evaluation designs, which of the following statements is not accurate?

controlled variation in treatment represents a threat to generalizability

When a researcher prepares their understanding of a program's goals, elements, and operations, she is preparing a program:

description

Program goals represent:

desired outcomes

What is the relationship between desired outcomes and empirical indicators?

desired outcomes represent the program goals while empirical indicators determine whether the desired outcomes are achieved

Evaluation research is designed to:

determine if a program is effective in its implementation

Which of the following would not be an example of problem analysis?

determine the average sentence given for those convicted of armed robbery within the state of Georgia

Which of the following is not something of concern in the evaluability assessment?

determine the degree of support for your project

In its most elementary form, evaluation research is a process of:

determining whether the intended result, of a given policy intervention was produced

According to your text, "theory driven" evaluations that consider how each element of a program should affect its targets should:

develop a micro model of the program production process

An evaluation to determine whether a program was implemented as intended is

A process evaluation.

An evaluation to determine whether a program achieved its intended result is

An impact assessment.

What type of research is appropriate whenever some policy intervention occurs or is planned?

Evaluation research.

The linking of the actions of justice agencies to evidence used for planning and evaluation is

Evidence-based policy.

Random assignment of people to receive some especially desirable or punitive treatment may not be possible

For ethical reasons.

The program intervention is the

Independent variable.

An analytic method to help officials plan and select alternative actions is

Problem analysis.

Case flow must be adequate to

Produce enough subjects in each group.

Whether an experimental intervention is delivered as intended describes

Treatment fidelity.

Policy outputs refer to

What is actually produced.

All of the following would be examples of goal statements that must be clarified before empirical testing can follow except for:

encouraging participants to obtain their GED upon completion of the program

This is considered sort of a "preevaluation"?

evaluability assessment

A community policing program began in an entertainment area in a large metropolitan area. City leaders are interested to see if the change in policing strategy has reduced the number of crimes and has asked faculty from a local university to complete a study. What type of evaluation would be best?

ex-post evaluation

Which of the following would a researcher be least likely to use randomized designs?

ex-post evaluations

Which of the following is accurate concerning the use of evaluation research?

federal requirements for program evaluations often accompany the implementation of new programs

Which of the following does not represent steps in the policy analysis process?

formulate a hypothesis that will allow for exploration

When dealing with stakeholders a researcher should be aware of all of the following except:

get particularly friendly with the stakeholders so they will not interfere with your research

Which of the following is a type of program evaluation?

impact assessment

The relationship between process evaluations and impact assessments can be stated as:

impact assessments answer questions about program effects while process evaluations monitor a program's implementation

Which of the following is accurate with respect to the relationship between impacts and outputs?

impacts are related to what the policy goals attempt to achieve while outputs are related to the means of achieving the goals

It is often said that in evaluation research, the research questions may already be formulated. Which of the following is an example of that?

in impact assessments the evaluation is on whether specified goals are attained

Which of the following is not correct with respect to process evaluations?

information about program implementation can be linked to outcome measures, even when accompanied by a process evaluation

Any public policy discussion involves politics. Which of the following is not accurate with respect to ethics and the political aspects of applied research?

just as there are codes for ethical conduct for researchers, there are formal codes of accepted political conduct

Which of the following best describes the example of the use of evidence-based policy?

justice agencies are linked to evidence used for planning and evaluation

Which of the following would a researcher completing an impact assessment be concerned with?

measuring the dependent variable

Dr. Smith is interested in implementing a program that is designed to reduce the recidivism rates of property offenders. He selects two state prisons and with the blessing of the Department of Corrections begins the programs in the main state prison for males and a female institution that is nearby. Which design would be best for Dr. Smith to use?

non-equivalent groups

There are two jails in your city, one houses state detainees waiting trial while the other houses federal prisoners awaiting trial. You have been asked to assess why there are differences in the institutional behavior between these two institutions. Which design would be appropriate?

non-equivalent groups

Which of the following embody the means to achieve desired policy goals?

outputs

At what stage does problem analysis take place?

policy making

Which of the following can color research in many ways?

politics

This type of analysis can help public officials plan and select an alternative action:

problem analysis

Which of the following is in contrast to incident-oriented policing?

problem-oriented policing

What can be said about the relationship between process evaluations and impact assessments?

process evaluations can be used in interpreting results from an impact assessment

Which of the following statements is accurate concerning program evaluations?

program evaluation is designed to link the intended actions and goals of policy to empirical evidence that supports them having the desired effects

The relationship between policy analysis and program evaluation is best characterized as:

program evaluation is used to test the achievement of goals and objectives worked out through policy analysis

When evaluating full-coverage programs, it is typically best to use:

quasi-experimental design

Your state prison system is determined to reduce the recidivism rates of adult offenders leaving the system. To accomplish this goal, a pre-release program is implemented. The Director of the department of corrections suggested an ideal design involving:

randomization

In the home detention studies used in your text as examples of randomized studies, the authors concluded that:

randomization does not control for variation in treatment integrity and program delivery

Relating to randomization, which of the following statements is not accurate?

randomization in experiments for evaluation purposes are best suited for programs where there are many exceptions

Treatment integrity is roughly equivalent to:

reliability

Which of the following is not a concern when a researcher uses an interrupted time-series design?

reliability of experimental treatment

Policy analysis is used to:

select an alternative course of action

Scientific realism is rooted in the principle that:

similar interventions can naturally be expected to have different outcomes in different contexts

If the number of subjects in a program is small, statistical tests can detect only very large program effects of differences in outcome measures between the two groups. This illustrates the problem of:

statistical conclusion validity

Your text discusses a concept called problem guides. Which of the following would be an example of this concept?

the analysis of specific problems such as crimes against the elderly

Policy analysis searches for:

the consequences that would follow the implementation of various courses of action

Evaluation in the criminal justice system seeks to link what?

the intended actions and goals of the policy to evidence that the policies are carried out

In process evaluation, the focus is on:

the program outputs

Which of the following would not be an example of an evidence-based policy?

the use of CCTV in public entertainment districts to reduce property crime

Which of the following would not be an example of a program's goal statement?

to study why some offenders were allowed in the program and others were not