CRJ 510
mediator analysis investigated differential effects for different participant subgroups
false: m.a. investigates the causal pathways from proximal to distal outcomes; moderator analysis assesses program effects for different subgroups
what has been touted as the main cause of the loss of manufacturing jobs in the united states as companies have moved their factories back to mexico, china, india, and other countries where cheaper labor and lower taxes could be found
globalization
intent to treat estimates
include people assigned to the treatment group only if they do not experience the full treatment
the effect size most commonly used to represent program effects that vary numerically is
standardized mean difference
an impact evaluation
tests the extent to which a program achieves its intended effect
implementation fidelity refers to
the extent to which a program adheres to program theory and design
which of the following is true about unintended effects
they may be negative they may be positive they are difficult to anticipate
if program process theory is poorly specified, it may not be possible to adequately describe the nature of the program that produced, or failed to produce, the outcomes of interest. what kind of evaluations are made without much insight into what is causing the outcomes
black-box evaluation
what methodology may be used to estimate the size of hard-to-reach populations such as persons involved in criminal activity, drug use, adn high risk health behaviors
capture-recapture methods
what term is used to describe the tendency to see things in ways favoring preexisting benefits
confirmation bias
the reliability of a measure refers to its
consistency
in a matched comparison group design, evaluations
construct a comparison group that matches the treatment group
in a matched comparison group design, evaluators
construct a comparison group that matches the treatment group
in the context of a process evaluation, coverage refers to
the proportion of eligible clients who receive the program's services
which of the following is not one of the three basic components of program theory
the supply chain three that ARE the program's organizational plan the service utilization plan the program impact theory
regression to the mean refers to
the tendency for extreme outcomes to become less extreme over time
regarding the evaluator's relationship to stakeholders, what is the term using to describe when the evaluator has the primary responsibility for developing the evaluation questions for in collaboration with key stakeholders, conducting the evaluation, and disseminating the results
independent evaluation
the accounting perspective that often produces higher benefit to cost results is
individual target population
which of the following is a potential category that evaluations can influence
individuals collective actions interpersonal behaviors
the sensitivity of a measure refers to
its ability to detect program effects
the synthesis of numerous impact assessment findings is referred to as
meta analysis
an additional strain places on evaluators is to assess program impact within a limited amount of time
true
a covariate is
a variable included in a statistical model when estimating program effects
a review of the logic and plausibility of program theory will necessarily be a relatively unstructured and open-ended process. nonetheless, there are some general issues such reviews should address. which of the following questions should reviewers ask
all of the above are the program goals and objectives well defined is the change process assumed in the program theory plausible are the program goals feasible
chapter 3 of the textbook describes assessing program theory and design. evaluability assessment involves three primary activities. which of the following is NOT one of these activities
all of the above are evaluability assessment activities ex. description of the program model assessment of how well defined and evaluable the model is identification of stakeholder interest in evaluation and the likely use of the findings
when a program's theory is spelled out in program documents and well understood by staff and stakeholders, the program is said to be based on what
an articulated program theory
outcome monitoring is
an ongoing effort to monitor a program's outcomes
process monitoring is
an ongoing effort to monitor a program's processes
what term describes the organizations, groups, and individuals in the environment of a program with interests in what the program is doing of what happens to it? (these may include other agencies or programs, journalists, public officials, advocacy organizations, and citizen groups in the jurisdiction in which the program operates
contextual stakeholders
social intervention programs designed and implemented explicitly to test the value of an innovative program concept may be referred to as what
demonstration program
the technique of converting future costs and benefits to a common monetary base by adjusting them to their present values is known as
discounting
in what type of evaluation is the relationship between the evaluator and stakeholder participatory and collaborative
empowerment evaluation
evaluation consequences that could plausibly lead toward or away from social betterment is a description of
evaluation influence
the evaluation plan is generally organized around questions posed about the program by who? this person also commissions the evaluation, and other pertinent stakeholders, individuals, groups, or organizations with a significant interest in how well a program is working
evaluation sponsor
it is often necessary to forecast or estimate the magnitude of what a social problem is likely to be in the future. what precautions should evaluators take
evaluators should examine forecasts and how they were produced
the type of efficiency analysis that is conducted using a program's anticipated cost and outcomes is referred to as
ex ante efficiency analysis
efficiency analyses most commonly take place before an impact evaluation has been completed
false
gender is an example of a mediator variable that can be used when considering whether a program effect was different for males and females
false
it is possible to conduct a cost-benefit analysis without knowing the estimates of the costs involved in implementing the program
false
snowball sampling is a good method to use when your population is known, and questionnaires can be mailed to each respondent
false
the effectiveness of a social program is not measured by the improvements in the social conditions it addresses, but by the cost (dollars) of the program being offered
false
what type of assessment is most appropriate for mature, stable programs with well-defined program models and a clear intention to use the results to justify the effort required
impact assessment
what is the term used to describe the assumptions about the change process actuated by the program and the outcomes that are expected to be effected as a result? this is also referred to as a "theory of change"
impact theory
what is it called when the underlying assumptions about how program services and practices are presumed to accomplish their purposes have not been fully articulated and recorded
implicit program theory
treatment-on-the-treated effect
include people assigned to the treatment group only if they experience the treatment
imagine that i have 60 months of data on gun crime for a city, in month 30, the police department in the city implemented a special unit intended to reduce gun crime. if i compare the mean number of gun crimes before and after the special unit was implemented, which design am i using
interrupted time series design
what term is used to describe persons whose position or experience gives them some knowledge of the nature, magnitude, and distribution of he problem at issue
key informants
which of the following is a common way of expressing monetary benefits of a program
money measurement observing funding allocations market evaluation econometric estimation
secular trends are
naturally occurring trends that may alter the estimate of program effects
One important form of evaluation assesses the nature, magnitude, and distribution of a social problem; the extent to which there is a need for intervention; and the implications of these circumstances for the design of the intervention. what are these diagnostic activities called
needs assessment
program effects may be
negative null positive
when doing an impact assessment, what term is used to describe the changes brought about by the intervention above and beyond those resulting form other processes and events effecting the targeted social conditions
net effects
what kind of analysis determines whether a program produces sufficient benefits in relation to its costs
none of the above NOT bias analysis, go-no go analysis, or yes/no analysis
what type of evaluation is intended to furnish information for guiding program improvement
none of the above NOT cross-sectional evaluation, summative evaluation, or cohort evaluation
a program must be organized in such a way that it can actually provide the intended services. what is the component of program theory that relates to program resources, personnel, administration, and general organization
none of the above NOT program set up, program access, program monitoring
evaluation is said to have "two arms" what are the "two arms"
one is data gathering and the other is collecting, clarifying, adn verifying relevant values and standards
the term selection bias refers to
pre treatment differences between the treatment adn control group
what are the three phases of needs assessments
preassessment, assessment, postassessment
when describing the conditions a program aims to improve, what term describes the total number of existing cases in that area at a specific time
prevalence
the type of dissemination that refers to the communication of results to meet the needs of primary sponsors and technical audiences is
primary dissemination
what type of sampling is a commonly used method in the social sciences to ensure that the characteristics of a sample of the population can be used to estimate the characteristics of the full population from which the sample was drawn
probability sampling
when the process evaluation is an ongoing function that occurs regularly, it is usually referred to as what
program monitoring
a program's organization and the service delivery system that organization supports are the parts of the program most directly under the control of program administrators and staff. these two aspects together are often referred to as __
program process
in chapter 1, the textbook describes the five domains of evaluation questions and methods. which of the following is NOT one of the domains
program timeline
which of the following can serve as the source of an outcome measure
records observations interviews
what a training program has indirect spillover effects to others not directly targeted, this is often referred to as externalities or
secondary effecrs
evaluators can use existing data sources to assess certain aspects of a social problem. what term is used to describe these regularly occurring measures
social indicators
what type of evaluation is conducted to render a judgement on the program's performance
summative evaluation
peter h rossi was a program evaluator whose work is highlighted in chapter 1 of the textbook. what approach to evaluation did he use when assessing the needs of the homeless in chicago
systematic approach to evaluation
what term describes the persons, households, or other units that are intended to receive the intervention or services being evaluated
target participants
a program effect refers to
the change in outcomes experienced by program participants due to the
in the context of a process evaluation, accessibility refers to
the ease with which eligible participants can access program services
which of the following are the four factors that determine statistical power
the effect size to be detected the alpha level for statistical significance the sample size the statistical significance text used
fidelity of implementation refers to
the extent to which a program is implemented as intended
in the context of a process evaluation, dosage refers to
the extent to which a program's participants receive the intended services
the validity of a measure refers to
the extent to which it measures what it is intended to measure
in the context of a process evaluation, bias refers to
the extent to which some subgroups receive more services than others
external validity
the generalizability of an estimate of a program's effect
in evaluation research, maturation refers to
the natural developmental changes that study samples experience with aging
if you were asked to conduct a process evaluation of an alcohol treatment program, which of the following should the evaluation focus on
the number of eligible participants who receive the program's services
if you were asked to conduct an outcome evaluation of a teen pregnancy prevention program, which of the following should the evaluation focus on
the number of program participants who become pregnant
the term counterfactual refers to
the outcomes that would have occurred in the absence of a program
evaluations are initiated for many reasons. which of the following is NOT one of those reasons
to determine new product lines reasons ARE: to help management improve a program to support advocacy by proponents or critics to gain knowledge about the programs effects
although the .05 alpha level has become the most widely used in determining statistical significance, there are instances where an alpha of .10 might be acceptable
true
cost-benefit analyses and cost-effectiveness analyses are two ways of determining the economic efficiency of programs
true
cost-effectiveness analysis requires the monetization of a programs costs
true
in describing an evaluation's purpose, there are generally 5 domains of evaluation questions and methods
true
meta-analysis is most useful for examining effect sizes of outcomes that have already been examined in previous evaluations
true
social problems are not objective phenomena. rather, they are social constructs involving assertions that certain conditions constitute problems that require public attention and deliberate, organized intervention
true
the most direct way to characterize the magnitude of program effect is the numerical difference between the means of the two sets of outcome values
true
the simplest program impact theory is the basic "two step" in which services affect some intermediate condition that, in turn, improves the social conditions of concern
true
to conduct evaluation work effectively, evaluators must understand their relationships to stakeholders involved as well as the relationships among stakeholders
true
to maximize the use of the findings of an evaluation, considerations should be made as to how the program addresses a national or state policy agenda and generalizability in meeting the needs for the program
true
what an effect - statistical significance - is found when there is no actual effect, this is referred to as
type I error
when an effect - statistical significance - is not found when there is a real effect, this is referred to as
type II error
regression discontinuity designs
use cut point variable to assign units to treatment and control conditions
randomized control design
use random assignment to assign units to treatment and control conditions
in propensity score matching
use statistical matching procedures to construct a control group
randomization refers to
using random assignment to assign units to treatment and control groups
a process evaluation evaluates
whether a program performs its intended functions