Outcome Mapping
Learning community
Independent Evaluation of the Tertiary Health Pacific Islands Program (PIP) and Strengthening Specialised Clinical Services in the Pacific (SSCSiP)
Plowman, Beth
Summary: This document is evaluation report for the program, Strengthening Specialised Clinical Services in the Pacific. It discusses the appropriateness and effectiveness of the M&E approaches used by SSCSiP. These were developed by integrating concepts and approaches from Outcome Mapping with the originally envisioned logframe approach. The report concludes:
"SSCSiP has faced challenges in its M&E function. The post of M&E Officer went unfilled for most of the project’s life cycle. With an unsuccessful search for an M&E Officer candidate, the project took several approaches to fill that gap. One was to have external support provided through short-term technical assistance to develop the project M&E Plan. In addition, the project provided additional training for the Health Planning Officer so that he could assume some of the responsibilities of the M&E Officer. This was a diversion from the original design, in which the Health
Planner was meant to work with countries on health and HR planning, and the M&E Officer was intended to work at the level of the project.
Despite these challenges, early in the project, there were well-conceived efforts to develop information on the
basic specialised clinical situation through country-specific specialist clinical situation analyses. Some follow-up has been published in the SSCSiP 2013 Report which included country-specific “clinical services at a glance” summaries.
The project should also be credited with developing what is, on paper, a solid M&E Plan. The plan is thoughtful and well-structured and includes a program theory of change as well as a results framework. Annexes include tabulation of progress markers for each outcome area with markers identified at the level of activities, outputs, intermediate outcomes and end-of-program outcomes. Also annexed to the document are reporting templates in order to collect the underlying data required to generate performance measures.
To date, monitoring data are reported in the annual reports. This primarily comprises activity-level reporting against the outcomes and sub-outcomes as described in the M&E Plan. The reports use a traffic light system for project self-assessment of progress during the reporting period. The project reports difficulty in acquiring requested data from the countries. As a result, many of the monitoring tools included in the M&E Plan have not been made fully operational. In sum, elements of the M&E Plan are conceptually sound but in practical terms cannot be implemented. This presents a problem for the project in its ability to link activities to outputs and higher level outcomes. As a further consequence, SSCSiP does not have the data required to provide a valid estimate of cost-effectiveness."
Type: Evaluation Reports
Theme: OM Resources: Other
Contibuted by: Ann Braun, on: 9 Jan 2017
Download: http://dfat.gov.au/about-us/publications/Documents/pacific-sscip-pi p-joint-evaluation-report.pdf