Program Evaluation Practice Quizzes

When disseminating the results of the needs assessment, you should present preliminary results to the stakeholders before finalizing and presenting the information to the community.

(T/F) True

Which of the following is NOT on of the four A's to guide planning?

*assessment Four A's include - awareness, accessible, ability of services study, acceptable

According to the APA's Clinical Psychology Task Force on Promotion and Dissemination of Psychological Procedures, in order for a psychosocial intervention to be considered for inclusion in the list of empirically supported treatments, it has to meet which of the following criteria when there have been at least two good between-group design experiments?

1) Superior to pill or psychological placebo or to another treatment 2) equivalent to an already established treatment in experiments with adequate statistical power

When is it possible that a program evaluation study may not need to be submitted to the IRB?

?

Research conducted in an established educational setting involving normal educational practices such as comparing instructional techniques or classroom management styles and studies involving the study of existing data fall under what category according to the Department of Health and Human Services issued Federal Ruling 45 CFR 46?

? full review, exempt from review, risk vs. benefit review ?

According to the National Commission for the Protection of Human Subjects in Biomedical and Behavioral Research (The Belmont Report, 1979), which of the following are the three ethical principles that must guide research involving humans?

Beneficence, respect (for persons), justice

If there have been a large series of single case designs (more than 9) demonstrating efficacy, which criteria must be met to be considered an empirically supported treatment?

Compared intervention to another treatment, use of good experimental design, demonstrated effects by at least two different investigators or practitioners, use of treatment manual

What are some ways to figure out the best solution when dealing with an ethical dilemma in research?

Consult applicable codes of ethics and apply those principles to the situation, consult with supervisors, colleagues and other professionals, .... ?

A researcher utilizes a survey as well as focus groups to determine the outcomes of a particular intervention. When analyzing the data, the researcher is careful to analyze how the different focus groups as well as the survey yield similar findings (or not). This is called what type of analysis?

Convergent

The _____________________________ technique is utilized when a group of experts is asked to comment on a questionnaire designed for a study, provide input, and assist the researcher in refining the measure.

Delphi

Historically speaking, evaluation research has been primarily qualitative in nature. True or false?

False

One of the main goals of summative evaluation is program description. True or false?

False

Program monitoring usually requires an experimental research design. True or false?

False

The purpose of formative evaluation is to show that participants of the program are improving as a result of the intervention in which they are taking part. True or false?

False

When conducting an evaluation, it is not good practice to provide on-site feedback as you uncover information that could benefit the agencies' work during the study. True or false?

False

An expressed need is when a need of one group is determined to be the need of a group with similar characteristics, but you are unable to talk to the other group directly.

False (T/F)

Deception of research participants should not occur in any study.

False (T/F)

If a research participant is cognitively impaired, it is not necessary to seek their assent to participate in a research study.

False (T/F)

In order to meet IRB guidelines, respondents must always be allowed to remain anonymous

False (T/F)

Minors (under age 18) can give informed consent to participate in research studies.

False (T/F)

Previously collected data for another purpose cannot be used by researchers for program evaluation studies.

False (T/F)

_____________________________ indicators are data items usually collected from participants of a program after their completion to determine if it works, while _____________________________ indicators are collected by the program routinely to determine if it is being implemented as planned.

Outcome; process

________________________ are the countable products resulting from a program, while ________________________ are the changes in clients that are achieved as a result of program participation.

Outputs; outcomes

What type of process evaluation is mainly focused on checking the progress of a program in terms of meeting their objectives; particularly exploring the inputs, process, and outputs?

Program monitoring

_____________ methods are typically used to study process evaluation, while ________________methods are typically used to determine outcomes.

Qualitative; quantitative

Before conducting the needs assessment, it may be helpful to consult census data available in geographical units to learn more about the population being studied. This is an example of use of ____________________________________

Secondary Data

Research participants must be fully informed about the processes of the research and must be allowed to end their participation in a study, if they choose, at any time with a penalty. This is an example of honoring what principle of social work?

Self-determination

Within an agency, two programs may service philosophies that differ

T/F? - True

Program evaluation and basic research have some similarities. What makes program evaluation and basic research different?

The expected use or utility of the data

What are some required sections in an informed consent document?

The purpose of the study, how participants' confidentiality will be safeguarded, and the risks vs benefits of the study

It is generally considered best practice to use qualitative methods to evaluate community level practice, as opposed to single-system designs. True or false?

True

Logic models can be conceptual, as a program is in development, or actual after it has been operating for a period of time. True or false?

True

One goal of process evaluation is to describe the problem. True or false?

True

If a research wants to conduct a study using incarcerated prisoners as research participants, the study must undergo full review including the presence of a prisoner or prisoner representative at the meeting

True (T/F)

Program evaluation involves making comparisons to other agencies, past or prior efforts or against a stated objective.

True (T/F)

There is a section of the NASW Code of Ethics that addresses research and evaluation

True (T/F)

What type of survey is utilized to determine the extent of problems in a certain area and then express the findings as rates affected within the population?

a) environmental b) empirical c) epidemiologic * (I think this is the answer) d) evidence-based

_______________________________ involve(s) a small group of individuals gathered to be asked questions by a facilitator related to a certain topic, with the goal being to gather relevant information about a problem or issue about which all the members are aware or have direct knowledge.

a. a survey key informant panel b. the delphi technique c. focus groups * need to confirm answer d. nominal groups

Which of the following steps comes first when planning a needs assessment?

a. establish the research questions. * need to confirm answer b. determine the information needed and what you already have. c. petition your board of directors for funding for the study d. hire a consultant with experience in your program area.

People who have knowledge of the problems you are assessing, through work or educational experience with a program or agency, or with a certain population with which you should consult before conducting your needs assessment are known as which of the following terms?

a. key informants * - need to confirm answer b. stakeholders c. secondary data d. informed consent

A ________________________ explains the work of the agency, providing clarity to those within the organization and is used to gain the support of those outside, and is very important to know in program monitoring.

a. mission statement b. quality assurance plan c. program goal statement d. list of program objectives

Which of the following is/are a form of data collection in qualitative research?

a. taking notes when observing services provided b. transcripts from interviews conducted c. documents and other existing data acquired d. all of the above*

The definition of a program includes

an organized collection of activities designed to reach certain objectives

What is important to include in a qualitative report, because it is often the only section closely read by those to whom the report is disseminated?

conclusions and recommendations

Which of the following statements is true regarding the protection of sensitive information in research?

confidentiality means that the researcher will protect and not reveal the participant's identity, while anonymity means that the research participant cannot be identified by anyone, including the researcher.

When planning a qualitative research study, what areas typically require the most work, and therefore must be well-planned in order to meet your budgets and timeline?

data collection and analysis

The type of sampling utilized when needing to explore people who have had unusual successes or failures as a result of participation in a program is called _________________________.

deviant case sampling

the main purposes of program evaluation are to attempt to build theory and add to the knowledge base of the profession.

false (t/f)

According to Bradshaw (1977), a(n) __________________________ need is one that is expressed by actual clients and those directly affected by the issue

felt

What type of evaluation focuses on gathering information to aid in program refinement and improvement?

formative or process

evaluative research helps those in the social sciences to

help determine whether the needs of the clients are being met

In qualitative analysis, what is usually the unit of analysis?

individuals

Why should you base a program on a sound theoretical model?

it helps ensure the consistency of the approach, it aids in problem conceptualization and proposed interventions, it helps to organize the program

Which of the following is a tool used for diagramming theoretically how a program will work to meet its outcomes?

logic model

What is the order for listing program components in a logic model?

model, inputs, activities, outputs, client outcomes

A consultant is hired by your agency to conduct a gaps analysis to determine groups of people not being served by the current services provided to the homeless. The consultant concludes that adolescents ages 17-18 who have formerly been in foster care comprise a large percentage of the homeless and you do not have any programs that target this group. This is an example of what type of need?

normative

A(n) ___________________________________ definition is the way a variable is defined and measured for the purposes of the evaluation or study.

operational

A study design that focuses on empowering the least powerful stakeholders in the evaluation, and valuing their input from the beginning to the end of the study is known as _____________________.

participatory evaluation

The belief that valid knowledge about the objective world can be determined through scientific research is which philosophical foundation of program evaluation is

positivism

What are the three overarching purposes of process or formative evaluation?

program description program monitoring, quality assurance

Which of the following is a viable alternative for a researcher when needing an experimental group, but you do not want to deny clients access to a needed service?

provide services to clients that are an alternative form of those being evaluated

Qualitative methods:

provide vivid descriptions and an understanding of how results came to pass

A focus on naturalistic inquiry, use of the researcher as an instrument in data collection, and reports that focus on narrative are common ingredients of what type of studies?

qualitative

What type of evaluation is mainly concerned with determining if an agency or program is in compliance with a certain set of standards, such as the Joint Commission of Accreditation of Health Organizations?

quality assurance

What is one of the differences between quality assurance and "program evaluation?"

quality assurance relies more on peer review and sampling records to see that expected standards were met

Agency A is known in their large urban community for their annual fund-raiser held every year in the summer, which is regionally televised event. This is an example of which of the following characteristics of a "good" program?

recognized identity

When collecting data in a qualitative study, and you begin to notice repetition in the data and confirmation of existing findings, not anything new revealed, this point is termed what?

saturation

Common issues that can affect your needs assessment approach include:

time to finish the project, the importance of the effort, budget and available resources

Institutional Review Boards (IRBs) were mandated to be convened for all of the following reasons:

to oversee research in organizations receiving federal funds, to review research protocols especially related to recruiting participants, to determine if ethical guidelines are being followed.

Using more than one type of data collection to corroborate findings and verify the interpretations from each is known as ________________________________.

triangulation of data

programs have their own identity

true (T/F)

Program evaluation studies are considered an inadequate way to test the validity of theories

true/false ?

The Campbell Collaboration and the Cochrane Collaboration are

websites that contain systematic reviews of research projects