M4: MONITORING AND EVALUATING COMMUNITY HEALTH PROGRAMS IMPLEMENTED
demonstrate the specific skills
If SKILLS are the focus of the evaluation, the client can be asked to ------ that she/he learned or observed for specific health practices or behaviors
observation checklist
If a post-CVA patient, an ---- can be used to determine his/her response to nursing intervention
relevant questions
If the evaluator would like to measure the knowledge of the client, ---- should be asked
implementation phase
Monitoring which is done at the ---- compares the actual progress against what was planned
Purposes of giving feedback:
Motivates and reinforces positive behaviors Enhances client's self-image and increases client's awareness of the need to improve their repertoire of coping behaviors. Provide opportunity for clients to articulate their thoughts regarding the task on hand The result of the feedback sessions should be properly documented The family health record and other pertinent records should be updated regularly
Key for a better presentation:
Prepared a good visual aids Rehearse and prepare for the questions that may be asked A good written report and an impressive oral presentation can influence decision makers positively
B. Design the Evaluation Plan
Specifying data collection methods and tools and sources of data Records and reports should be analyzed Surveys can be conducted Local officials, community leaders and program implementers can also be interviewed Data collection tools Qualitative interviews or focus group discussions can be conducted
C. Collect Relevant Data
The evaluator's primary aim is the generation of accurate and reliable data Prior to actual data collection, data collection method and tools should be field-tested and data collectors should trained
output, effect and impact
The program results-----, ----, ----correspond to the three levels of program objectives: short-term, intermediate or medium-term and long-term.
F. Report/Give Feedback
The result of the program evaluation should be submitted to the local authorities An executive summary should be prepared nurse and other health workers must be prepared to make a presentation to the SB or Local Health Board
Example of Different Tool or Instrument:
Thermometer BP weighing scale tape measure checklist or interview guide.
informal talks
Through ----, it is easier to assess our clients' attitude.
2 Purposes of Terminal and Ex post Evaluation
To assess the achievement of overall results of the program, in terms of efficiency, outputs, effects and impact To learn lessons for future planning
A. Decide what to Evaluate
WHO suggested five dimensions of program performance that could be then evaluated: (REPIE) the evaluator should review the program context and objectives
The following are questions that the evaluator should reflect:
What do the figures/statistics mean? What do the qualitative data reveal? Is the program relevant? Is it progressing in accordance with the program plan? Is it effective? Is it efficient? Did it make a significant impact on the beneficiaries and the community? Do the benefits outweigh the problems created? What are the lessons that could be learned from the program?
A. Decide what to Evaluate: questions that need to be answered at this point are:
What should be evaluated? What indicators should be used?
Qualitative interviews or focus group discussions
can be conducted among a smaller number of participants to have more in-depth understanding of he program outcomes from the perspective of the beneficiaries.
Surveys
can be conducted to collect information on clients' knowledge, attitudes and practices
internal
formative evaluation (sometimes referred to as ----
Checklist
is a good evaluation tool if there are a number of criteria for an objective.
formative evaluation
is a method for judging the worth of a program while the program activities are forming (in progress). They can be conducted during any phase of the Analysis Design Development Implementation and Evaluation (ADDIE) process.
summative evaluation
is a method of judging the worth of a program at the end of the program activities (summation).
Ongoing Evaluation
is the analysis during the implementation of the activity, of its continuing relevance, effectiveness, efficiency and present and likely future outputs, effects and impact. focuses on the appropriateness, adequacy and timeliness of processes or activities
Evaluation
is the analysis of the effectiveness, quality, scope and timeliness of services given. - Define as the process for determining systematically and objectively the relevance, efficiency and effectiveness and impact of activities in the light of their objectives
B. Terminal Evaluation
is undertaken from 6-12 months after the project completion it is a substitute for an Ex post evaluation of projects with short duration
C. Ex post Evaluation
is undertaken some years after the project completion when full program/project benefits and impact are expected to have been realized
mayors, chair of the SB committee on health, and the Local Health Board
local authorities such as ---- - are the key decision makers in the Local Health System
Valid
measures what is supposed to measure
WHO suggested five dimensions of program performance that could be then evaluated:
relevance progress effectiveness impact and efficiency
executive summary
should be prepared for local authorities and should contain: a brief description of the focus and procedures of the evaluation, summary and interpretation of results, conclusions & recommendations
Public Health Nurse
should specify the criteria and corresponding evaluation tool for each objective.
According to the UN, evaluation
the collection and analysis of information to determine program performance.
If the program is not relevant,
the evaluator should recommend its modification or termination.
If there is still another phase of the program,
then positive evaluation results should serve as a go-signal to start the next phase
If the intervention or program was effective and efficient,
this could be continued and/or applied to another client or group, given similar circumstances.
purpose of monitoring is:
to identify deviations or problems so that corrective actions or interventions can be instituted immediately. This requires reporting to appropriate persons or offices at regular intervals
Indicators should be
valid, reliable, objective, sensitive, specific, cost-effective and timely
UN defined monitoring as the:
"continuous or periodic review and surveillance by management at every level of the hierarchy of the implementation of an activity to ensure that input deliveries, work schedules, targeted outputs and other required actions are proceeding according to plan.
process
(formative evaluation) This part of the evaluation focuses on the ----.
outcome
(summative evaluation) The focus is on the ----.
Objective
- It it is not influenced by personal biases or if he answers are the same if measured by different people in similar circumstances
Sensitive
- if changes in the indicator in fact reflect changes in the situation or phenomenon
Timely
- if it is possible to collect data reasonably quickly
Specific
- if it is sensitive to the given situation or phenomenon only
Reliable
- if it lends itself to measurement with minimum error
Cost-effective
- if the results are worth the time and money it costs to apply them
Aims of Evaluation:
Discover how well the objectives are fulfilled Determine the reasons for specific successes and failures To uncover principles underlying a successful program Prevent costly mistakes and improve program planning and implementation in the future
D. Analyze Data
Evaluators should assess the quality of the data before they start their analysis
E. Make Decisions:
Evaluators should assess the quality of the data before they start their analysis
feedback
After collection and analysis of data/information, the nurse should give his/her clients ------. on the results of evaluation
external
A summative evaluation (sometimes referred to as
Steps of Program Evaluation
A. Decide what to Evaluate B. Design the Evaluation Plan C. Collect Relevant Data D. Analyze Data E. Make Decisions: F. Report/Give Feedback E. Make Decisions:
Types of Evaluation Planning
A. Ongoing Evaluation B. Terminal Evaluation C. Ex post Evaluation
interviews
ATTITUDE can be assessed through qualitative, semi-structured or unstructured ----.
E. Make Decisions:
If the intervention or program was effective and efficient, this could be continued and/or applied to another client or group, given similar circumstances. If there is still another phase of the program, then positive evaluation results should serve as a go-signal to start the next phase Based on the lessons learned from the earlier phase, the implementation of the next phase will have to be guided, modified or improved If the program is not relevant, the evaluator should recommend its modification or termination.
immediate members of the family, interviewed
If the level of consciousness or orientation is altered, the ----- can be ---.
clear, easy to understand
In preparing a list of questions to be asked, the evaluator should make sure that the questions are ---- and ---.
FOCUS of EVALUATION: Three Major Foci of Program Evaluation
Inputs Process Outcome/Result
Indicators
It is a specific and objectively verifiable measure of changes or results brought about by an activity Used as marker of progress towards the attainment of program objectives
evaluation
Monitoring is closely related to ----.