Evaluation Capacity Development Project in Uganda
Uganda Evaluation Association & Evaluation Capacity Development Project in Kampala discussed key issues in regional development programs, focusing on the Nile Basin Initiative and the challenges faced in program implementation and coordination among participating countries.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Uganda Evaluation Association & Evaluation Capacity Development Project (ECD) KAMPALA EVALUATION TALK Kampala, Uganda 22ndJan., 2015
Inside the presentation (asking questions) 1. Context and Background (The Case of the Nile Basin Initiative) 2. Typical Operational set op of Regional Development Programs 3. Key Evaluation Design and Operation Issues? 4. Working within existing situations to achieve developmental goals
Context and Background Regional Programs: Offer greater benefits, efficiency & effectiveness in channeling and receiving development assistance. But: Take time to set up & implementation pace is different in each country Require flexible program designs for responsiveness and relevance. Participating countries have varied landscapes and Risks (Political, Policy, Financial, Regulatory). Inconsistencies between regional & national programs - through overlaps, contradictions, differential relevance and applicability.
Regional Development Programs will typically have 3 tier operational levels Regional level with an Executive Organ Coordinates, makes polices & plans,& commissions evaluations National level sub- office for coordinating in- country implementation: Receivespolicies, work plans, implementation guidelines and adapts to country context for implementation. All participating countries treated as equals in one whole Ultimately impacted by the development program will be organized to facilitate receipt of program support as designed at regional level Program perception and expectations varied and divergent from the objectives of the regional program Community level (development design and evaluation subjects)
Nile Basin Initiative (NBI) is a Regional 10 countries River Basin Organization in Africa ATP = Applied Training Project NTEAP = Nile Trans-boundary Environment Action Project CBSI = Confidence Building & Stakeholder Involvement SVPC = Shared Vision Coordination Project SDBS = Socio-Economic Development & Benefit Sharing EWUAP = Efficient Water Use Project RPT = Regional Power Trade WRPM = Water Resources Planning and Management ATP Egypt NTEAP Eritrea Sudan Nile-SEC CBSI WRPM S. Sudan SVPC Ethiopia ENTRO SDBS Kenya EWUAP D.R. Congo Uganda NELCU Rwanda RPT Burundi Tanzania 5
NBI STRATEGIC PROGRAMMING 1999-2016 NBI Secretariat Services Nile-SEC Shared Vision Program Basin Cooperation and IWRM building blocks Cooperation & IWRM Dialogue, knowledge & planning NELSPAP & Subsidiary Action Program Expanding Nile Basin Investment Projects ENSAP Subsidiary Action Program Action on the ground NBI Sub-basin & Countries 2008 2012 2016 + 1999 Establishment and Confidence Building Consolidation and Delivering Institutional Strengthening
NBI - Shared Vision To achieve sustainable socio-economic development through the equitable utilization of, and benefit from, the common Nile basin water resources. Strategic Objectives (2012-2016) NBI Portfolio of Investment Projects 2012- 2016 NBI Programs 2012 - 2016 Facilitating Basin Cooperation Program Water Resources Development Program Power Water Resources Management Program Equatorial Lakes Eastern Nile River basin management Basin Monitoring Power Platform for Cooperation Agriculture Knowledge base River basin management Analysis Climate change Agriculture Secretariat Water Policies Climate change Environment Mgnt. Institutional development Lead: NILE - SEC for basin-wide Lead: Member States Lead: NILE - SEC For basin-wide Lead: SAPs
Approaches, designs, practices and methods for evaluating regional programs should recognize complexity & country variations To achieve greater responsiveness of Trans- boundary/ Regional Evaluations to different contexts, policies, politics, local aspirations and communities ways of doing things
Regional Block Countries have variations in: Levels of rigor in application & use of M&E; 1. 2. Systems & procedures (Policies, Guidelines, Institutional Arrangements) approaches & adequacy; 3. Local capacities, skill levels & expectations at all three key levels of operation; 4. Political appreciation and championed support for M&E versus being perceived as threat to the status quo; 5. Openness to criticism (threats to conclusion validity, statistical power analysis, peer review etc); 5. Ability of local communities to demand accountability; 6. Times / paces at which costs are incurred and benefits accrued at different levels (complexity of results chain).
Achieving goals within existing situations requires multiple approaches to evaluation designs & implementation. Regionally designed & nationally implemented programs to recognize and read at the same page with national systems and policies to build national capacities and increase the efficacy of national systems, & build on synergies and support national governments achieve their development goals . Mixed open ended evaluation approaches, designs and methods that measure essentially contribution of the development intervention need to be applied at different levels of the interventions at different times, and with staggered levels of rigor - in accordance with program stage of implementation in a given country. Consider different stakeholder perspectives, knowledge, skills and capacities, governance issues,
Achieving goals within existing situations requires multiple approaches to evaluation designs & implementation. Encourage local partnerships in addition to deliberate local capacity building, Addressing data constraints - (reconstructing baseline data & control groups, working with non-equivalent control groups, collecting data on sensitive topics & difficult to reach groups, multi-method approaches) Need for multiple solutions - outcome mapping and contribution analysis adapted to multi-country evaluations, with flexibility to take care of socio-economic, cultural, and political peculiarities and differences in countries' levels of "maturity", can greatly increase the relevance and objectivity of present day evaluation theories.
Achieving goals within existing situations requires multiple approaches to evaluation designs & implementation. Emphasizing process analysis, to include; one-whole analysis of complex characteristics of the evaluands, contexts at different geographic levels. Methodologies that seek alternative approaches to the counterfactual since external events and influence of other programs are critical in assessing effect of a regional program on national processes and outcomes. Helping clients use the evaluation (ensuring active participation of clients in the scoping phase, formative evaluation strategies, constant communication with all stakeholders, evaluation capacity building, appropriate strategies for communicating findings, developing and monitoring the follow-up action plans)
Useful Questions to ask whose policies and strategies are applied when a regional evaluation is undertaken in a country with operational national M&E policies and strategies,? Would it be possible to design multiple country-focused localized evaluations that seek to fulfill ToRs for evaluation of one regional development program? Can an aggregate generalization on outcomes or impact of a regional program be made by summing findings from different targeted communities obtained through undifferentiated assessment? In the sample case of NBI, are any water resources management changes and peoples' poverty reduction changes observed at communities and national level as a result of NBI or as a result of own efforts of the National Governments with some level of contribution by NBI's? What is this contribution? What are the perceptions of the local communities in each of the countries' Nile Basin areas? Can these communities be identified with the evaluation results at Regional level?
Concluding with Careful Attribution of Regional Interventions to National Efforts The Missing Casual Link Contribution analysis; assesses the contribution of a particular development agency (and the particular intervention supported) to the achievement of the overall changes resulting from the collaborative financial and technical interventions of a number of different development agencies. Attribution analysis; is carried out when the treatment / project group is matched to a comparison / control group so that alternative explanations of the observed changes can be controlled for elimination of other causes of changes manifested. There is then a prima facie case for claiming that the difference between the treatment / project group and the comparison / control group indicates the contribution of the intervention / project to these changes. THANKYOU FOR YOUR TIME
