M4: MONITORING AND EVALUATING COMMUNITY HEALTH PROGRAMS IMPLEMENTED

demonstrate the specific skills

If SKILLS are the focus of the evaluation, the client can be asked to ------ that she/he learned or observed for specific health practices or behaviors

observation checklist

If a post-CVA patient, an ---- can be used to determine his/her response to nursing intervention

relevant questions

If the evaluator would like to measure the knowledge of the client, ---- should be asked

implementation phase

Monitoring which is done at the ---- compares the actual progress against what was planned

Purposes of giving feedback:

Motivates and reinforces positive behaviors Enhances client's self-image and increases client's awareness of the need to improve their repertoire of coping behaviors. Provide opportunity for clients to articulate their thoughts regarding the task on hand The result of the feedback sessions should be properly documented The family health record and other pertinent records should be updated regularly

Key for a better presentation:

Prepared a good visual aids Rehearse and prepare for the questions that may be asked A good written report and an impressive oral presentation can influence decision makers positively

B. Design the Evaluation Plan

Specifying data collection methods and tools and sources of data Records and reports should be analyzed Surveys can be conducted Local officials, community leaders and program implementers can also be interviewed Data collection tools Qualitative interviews or focus group discussions can be conducted

C. Collect Relevant Data

The evaluator's primary aim is the generation of accurate and reliable data Prior to actual data collection, data collection method and tools should be field-tested and data collectors should trained

output, effect and impact

The program results-----, ----, ----correspond to the three levels of program objectives: short-term, intermediate or medium-term and long-term.

F. Report/Give Feedback

The result of the program evaluation should be submitted to the local authorities An executive summary should be prepared nurse and other health workers must be prepared to make a presentation to the SB or Local Health Board

Example of Different Tool or Instrument:

Thermometer BP weighing scale tape measure checklist or interview guide.

informal talks

Through ----, it is easier to assess our clients' attitude.

2 Purposes of Terminal and Ex post Evaluation

To assess the achievement of overall results of the program, in terms of efficiency, outputs, effects and impact To learn lessons for future planning

A. Decide what to Evaluate

WHO suggested five dimensions of program performance that could be then evaluated: (REPIE) the evaluator should review the program context and objectives

The following are questions that the evaluator should reflect:

What do the figures/statistics mean? What do the qualitative data reveal? Is the program relevant? Is it progressing in accordance with the program plan? Is it effective? Is it efficient? Did it make a significant impact on the beneficiaries and the community? Do the benefits outweigh the problems created? What are the lessons that could be learned from the program?

A. Decide what to Evaluate: questions that need to be answered at this point are:

What should be evaluated? What indicators should be used?

Qualitative interviews or focus group discussions

can be conducted among a smaller number of participants to have more in-depth understanding of he program outcomes from the perspective of the beneficiaries.

Surveys

can be conducted to collect information on clients' knowledge, attitudes and practices

internal

formative evaluation (sometimes referred to as ----

Checklist

is a good evaluation tool if there are a number of criteria for an objective.

formative evaluation

is a method for judging the worth of a program while the program activities are forming (in progress). They can be conducted during any phase of the Analysis Design Development Implementation and Evaluation (ADDIE) process.

summative evaluation

is a method of judging the worth of a program at the end of the program activities (summation).

Ongoing Evaluation

is the analysis during the implementation of the activity, of its continuing relevance, effectiveness, efficiency and present and likely future outputs, effects and impact. focuses on the appropriateness, adequacy and timeliness of processes or activities

Evaluation

is the analysis of the effectiveness, quality, scope and timeliness of services given. - Define as the process for determining systematically and objectively the relevance, efficiency and effectiveness and impact of activities in the light of their objectives

B. Terminal Evaluation

is undertaken from 6-12 months after the project completion it is a substitute for an Ex post evaluation of projects with short duration

C. Ex post Evaluation

is undertaken some years after the project completion when full program/project benefits and impact are expected to have been realized

mayors, chair of the SB committee on health, and the Local Health Board

local authorities such as ---- - are the key decision makers in the Local Health System

Valid

measures what is supposed to measure

WHO suggested five dimensions of program performance that could be then evaluated:

relevance progress effectiveness impact and efficiency

executive summary

should be prepared for local authorities and should contain: a brief description of the focus and procedures of the evaluation, summary and interpretation of results, conclusions & recommendations

Public Health Nurse

should specify the criteria and corresponding evaluation tool for each objective.

According to the UN, evaluation

the collection and analysis of information to determine program performance.

If the program is not relevant,

the evaluator should recommend its modification or termination.

If there is still another phase of the program,

then positive evaluation results should serve as a go-signal to start the next phase

If the intervention or program was effective and efficient,

this could be continued and/or applied to another client or group, given similar circumstances.

purpose of monitoring is:

to identify deviations or problems so that corrective actions or interventions can be instituted immediately. This requires reporting to appropriate persons or offices at regular intervals

Indicators should be

valid, reliable, objective, sensitive, specific, cost-effective and timely

UN defined monitoring as the:

"continuous or periodic review and surveillance by management at every level of the hierarchy of the implementation of an activity to ensure that input deliveries, work schedules, targeted outputs and other required actions are proceeding according to plan.

process

(formative evaluation) This part of the evaluation focuses on the ----.

outcome

(summative evaluation) The focus is on the ----.

Objective

- It it is not influenced by personal biases or if he answers are the same if measured by different people in similar circumstances

Sensitive

- if changes in the indicator in fact reflect changes in the situation or phenomenon

Timely

- if it is possible to collect data reasonably quickly

Specific

- if it is sensitive to the given situation or phenomenon only

Reliable

- if it lends itself to measurement with minimum error

Cost-effective

- if the results are worth the time and money it costs to apply them

Aims of Evaluation:

Discover how well the objectives are fulfilled Determine the reasons for specific successes and failures To uncover principles underlying a successful program Prevent costly mistakes and improve program planning and implementation in the future

D. Analyze Data

Evaluators should assess the quality of the data before they start their analysis

E. Make Decisions:

Evaluators should assess the quality of the data before they start their analysis

feedback

After collection and analysis of data/information, the nurse should give his/her clients ------. on the results of evaluation

external

A summative evaluation (sometimes referred to as

Steps of Program Evaluation

A. Decide what to Evaluate B. Design the Evaluation Plan C. Collect Relevant Data D. Analyze Data E. Make Decisions: F. Report/Give Feedback E. Make Decisions:

Types of Evaluation Planning

A. Ongoing Evaluation B. Terminal Evaluation C. Ex post Evaluation

interviews

ATTITUDE can be assessed through qualitative, semi-structured or unstructured ----.

E. Make Decisions:

If the intervention or program was effective and efficient, this could be continued and/or applied to another client or group, given similar circumstances. If there is still another phase of the program, then positive evaluation results should serve as a go-signal to start the next phase Based on the lessons learned from the earlier phase, the implementation of the next phase will have to be guided, modified or improved If the program is not relevant, the evaluator should recommend its modification or termination.

immediate members of the family, interviewed

If the level of consciousness or orientation is altered, the ----- can be ---.

clear, easy to understand

In preparing a list of questions to be asked, the evaluator should make sure that the questions are ---- and ---.

FOCUS of EVALUATION: Three Major Foci of Program Evaluation

Inputs Process Outcome/Result

Indicators

It is a specific and objectively verifiable measure of changes or results brought about by an activity Used as marker of progress towards the attainment of program objectives

evaluation

Monitoring is closely related to ----.