Evaluation report 2012:8 Meta-Evaluation of Decentralised Evaluations in 2010 and 2011
Svend Erik Sørensen
ISBN 978-952-281-079-3 (printed)
ISBN 978-952-281-080-9 (pdf)
Meta-Evaluation of Decentralised Evaluations in 2010 and 2011
The purpose of the meta-evaluation was to draw lessons from the project evaluations to benefit development cooperation.
The meta-evaluation was a desk study only, including 41 project evaluations from 2010 and 2011. It analysed the quality and contents of the evaluation reports and their terms of Reference (TORs). It studied e.g. how poverty reduction and cross-cutting objectives had been addressed in the reports and TORs, and promoted in the projects. Ten projects were studied based on more comprehensive sample of documents, for example on how the evaluation findings were used in the projects.
Findings showed an improvement in the quality of reports and TORs compared to earlier meta-analyses. Most reports were well written, included clear analysis and reference to TOR. TORs were well designed and included relevant evaluation criteria. There was a significant improvement in addressing the cross-cutting objectives in TORs and reports. Poverty reduction issues were addressed to some extent in the reports and TORs. Correlation between high quality TORs and high quality reporting was not found.
Quality of development cooperation described by evaluation reports was overall poor showing little effect towards sustained outcomes. Project design and results-based management were inadequate and generally unable to monitor progress, measure achievements and analyse risks. Full involvement of key beneficiary groups lacked, including the poorest and most vulnerable. Results from ten evaluation reports were generally used in the preparation of project proposals or extensions, though not systematically.
This meta-evaluation suggests improvements to the above shortcomings. They include, among other things, applying a coherent framework for quality assurance with particular focus on project design, results-based approach, risks and reporting. Also, tools for identifying real needs of beneficiaries in projects should be applied and budgets for cross-cutting activities included in projects.
Keywords: meta-evaluation, project-based evaluation, results-based management, quality assurance, Finland’s development cooperation.