Evaluation of LEADER/CLLD for CAP 2024

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Evaluation of LEADER/CLLD European Evaluation Helpdesk for the CAP 2024

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Disclaimer The content of this training material has been prepared by the EU CAP Network, supported by the European Evaluation Helpdesk for the CAP. The material aims to support capacity building activities in relation to the monitoring and evaluation of CAP Strategic Plans. For any use outside of its original intent, or any modifications made to the content of the presentation, prior written permission must be obtained from the European Evaluation Helpdesk for the CAP. Commercial use of this material is strictly prohibited. Please contact: evaluation@eucapnetwork.eu for any questions in this respect. The European Evaluation Helpdesk for the CAP makes no claims, promises, or guarantees about the accuracy, completeness, or adequacy of the content of the presentation and expressly disclaims liability for any errors or omissions in such content.

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Objectives of the module Increase awareness of the evaluation requirements related to the evaluation of LEADER/CLLD and its added value for the 2023-2027 programming period Achieve a common understanding of the evaluation framework for the LEADER added value in the respective Member State Discuss possible challenges and seek solutions related to the evaluation of the LEADER and its added value in the Member State

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Why is LEADER evaluated? Evaluation ensures accountability, transparency and collective learning Assess relevance, effectiveness, efficiency, results and impact Demonstrate the contributions of to national/regional and EU objectives Show the added value of LEADER Learn what works and what does not work Improve the design and implementation of LEADER Build awareness on LEADER method and added value of LEADER

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N LEADER in 2023-2027 The CAP Strategic Plans (CSP) regulation (EU) 2021/2115 defines LEADER as community-led local development (CLLD) referred to in Article 31 of Regulation (EU) 2021/1060 and part of the `Cooperation` intervention defined under Article 77 At the CSP level, Member States have programmed LEADER mainly under Specific Objective 8. According to the intervention logic of the CSP, LEADER can also contribute to the achievement of other SOs

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N LEADER evaluation LEADER/CLLD operates within a multilevel framework and therefore requires a multilevel evaluation: CSP level: Evaluations of SOs to which LEADER has been designed to contribute. Local level: Evaluations of Local Development Strategies (LDS) developed by Local Action Groups (LAGs). LDS are linked to a specific territory and contribute to the achievement of local, national and EU level policy objectives. In both cases, the evaluation can take on multidimensional nature, focusing on the assessment of effectiveness, efficiency, relevance, coherence and Union added value of the interventions at each level, as well as the links between them. Besides the assessment of LDS contribution to the local and relevant CSP objectives, an important aspect is the assessment of the LEADER added value (specific evaluation topic)

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Evaluation of LEADER Evaluation focus Contributions of LEADER to the SO8 and other SOs At CSP level Achievements of local development strategy (LDS) objectives and results (including the contributions of LDS outputs and results to the CSP and CAP objectives) LEADER added value At the local level (self-assessment and or evaluation)

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Legal framework Implementing Regulation (EU) 2022/1475: Article 2: During the implementation period of the CAP SP , (a): MS shall plan the evaluation of the Specific Objectives. (SO 8 To promote employment, growth, gender equality, including the participation of women in farminig, social inclusion and local development in rural areas, including circular bio-economy and sustainable forestry) (d): ... based on the evaluation needs of the MS and taking into account the intervention logic and implementation of the CAP SP, the MS shall also evaluate specific topics, such as the added value of LEADER.

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Legal framework Implementing Regulation (EU) 2022/1475: Article 1(1): define evaluation questions, factors of success to assess effectiveness, efficiency, relevance, coherence and Union added value. Article 1(2): When assessing the effectiveness use the key evaluation elements set out in Annex I. SO 8: Local development => FoS: Local services and infrastructures are improving Rural sustainable economy => FoS Gender equality and social inclusion => FoS

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Legal framework Regulation (EU) 2021/2115: Result indicators (SO 8): R.38 LEADER coverage For simplification at the initial planning stage, the LEADER intervention could be attributed to only this indicator. LEADER intervention (Article 77) may be concerned in a number of other resulted indicators, which will be only indicated based on the approved strategies through a later modification of the Plans and through the annual reporting Implementing Regulation (EU) 2022/1475: Article 8(e): MS shall report to the Commission data on LAGs and their activities(Annex VII) . necessary to enable it to perform the monitoring and evaluation of the CAP.

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Legal framework for LEADER evaluation at LAG level Regulation (EU) 2021/1060: Article 33.3: LAG tasks: (e) ... to monitor progress towards the achievement of objectives of the strategy. (f) ... to evaluate the implementation of the strategy. Article 32.1: Local development strategies set out: (d) the objectives of that strategy, including measurable targets for results, and related planned actions (e) the management, monitoring and evaluation arrangements, demonstrating the capacity of the LAG to implement that strategy

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N THE CONTEXT How is LEADER and its added value going to be evaluated in your Member State? What evaluation related support for LAGs is expected in your Member State?

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Evaluation of LEADER at local level

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Evaluation of LEADER at the local level Responsibility of the LAG LAG can carry out the evaluation activities as: Self-assessment (done by the LAG); Evaluation (done by independent body); A combination of self-assessment and evaluation.

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Evaluation Self-assessment Demonstrate achievements of the LDS and outcomes of other LAG activities, e.g. those ensuring the implementation of the LEADER method and show the added value of the LEADER method at the local level Foster the learning process to design and implement LEADER/CLLD at the local level more effectively and efficiently Why? LAG bodies (assembly, decisions making body) which are involved/responsible for the design and implementation of the LDS and other LAG activities (e.g. cooperation and animation) Independent bodies (e.g. evaluator or hired evaluation expert) not involved/responsible for the design and implementation of LDS, other LAG activities By whom? Assess the achievements of the LDS and other LAG activities, the delivery of the LEADER method and the added value which LEADER/CLLD is generating What? Monitoring and other quantitative data (statistics) and information form qualitative research including participatory approach On what basis? Primarily monitoring data and findings from participatory qualitative assessment Self - reflection on the LAG s own performance, and achievements of the LDS and other LAG activities (cooperation and animation etc.) Independent findings on achievements of the LDS and other LAG activities (cooperation and animation etc.) Outcome?

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Focus of LEADER evaluation at local level Mandatory Recommended Legal requirements: Assessment of: To provide a description of the monitoring and evaluation arrangements in the LDS LAG animation The LEADER delivery mechanism in ensuring the LEADER method To monitor progress towards the achievement of LDS objectives The added value of LEADER To evaluate the implementation of the LDS

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Focus of LEADER evaluation at local level As a recommneded practice Assessment of the local development strategy s (LDS): coherence, contributions of operations to the achievement of LDS objectives and in addressing the LAG area needs contributions of the LDS s operations to the higher level objectives CSP objectives results and impacts success and failure factors in achieving objectives, and generating results and impacts Assessment of: LAG s administrative arrangements to ensure the proper application of LEADER method The added value generated through the LEADER method

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N How to assess the LEADER added value European Evaluation Helpdesk for the CAP

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N LEADER: What has changed? The ECA assessed 'whether the LEADER approach delivered benefits justifying the additional costs and risks compared to the most common (top-down) EU spending programmes' ... and found that ' improvements have been made in some areas, but there is still little evidence that the benefits outweigh the associated costs and risks'. We have to show and assess the LEADER added value!

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Evaluation Helpdesk support Evaluation Support Study on the costs and benefits of implementing LEADER Carried out on behalf of the Commission. Covers all Member States. (2014-2020) Estimation of the costs of implementing local strategies Estimate of the additional costs of implementing LEADER compared to centrally managed actions Assessment of the added value of the LEADER approach Good Practice Workshop 'How to assess the added value of LEADER How to assess LEADER added value | European CAP Network (europa.eu)

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Thematic Working Group - Guidelines Assessing the added value of LEADER Operationalised concept of the LEADER added value Examples of evaluation frameworks of the LEADER added value (2023-2027) Insights on how to interpret the results of LEADER added value evaluations Fiches of LEADER added value indicators Non-binding guidance offers options for evaluating the LEADER added value, according to the context and evaluation needs of MS Published on: EU CAP Network website: https://eu-cap- network.ec.europa.eu/publications/ass essing-added-value-leader_en (Main body translated in all EU languages)

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N The CONCEPT of LEADER added value

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Concept behind the evaluation of LEADER added value Developed by the Evaluation Helpdesk (2017) to capture LEADER benefits (including intangible benefits) Economic, social and environmental dimensions of local development

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N LEADER added value: benefits that are obtained through the proper application of the LEADER method, compared to those benefits, which would have been obtained without applying this method. 7 principles of the LEADER method: bottom-up approach; public-private partnerships (LAGs); area-based local development strategies; integrated and multi-sectoral strategy; innovation; networking; territorial cooperation

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Focus of the capacity building event: How to identify LEADER added value (LAV)? What is the expected change (on local level)? Operationalisation of LEADER added value concept Structural changes in the LAG s areas How to measure the LAV and the changes? Evaluation framework

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Operationalisation How LEADER principles and added value components manifest in action through the implementation of the local development strategy (LDS), delivery mechanism and animation?

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Improved social capital in LEADER includes networks, mutual trust, shared mental models and beliefs that facilitate the quality of collaboration and cooperation within and among LEADER areas. Shared mental models, values and beliefs Networks Mutual support and trust Groups of interdependent actors and the relationships between them. Generalised trust, i.e. perceptions that most people can be trusted or that one needs to be very careful in dealing with people. Well established networking and cooperation among involved stakeholders, including interregional and transnational cooperation projects. Thought processes on how something works, e.g.: sense of belonging in the area, attitudes of solidarity, inclusiveness by welcoming disadvantaged groups, including women and young, recognition of social problems. Level of trust in the LAG thanks to the activities performed by the organisation and its network. Quality of interactions within and outside the network, e.g., being heard, common understanding reached, conflicts resolved, useful exchanges. Working definitions

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Governance in LEADER: Local and multi-level processes and mechanisms that ensure effective and transparent implementation of the LEADER method, contributing to bringing the EU closer to citizens. Local governance Multi-level governance Processes and mechanisms established collaboratively between Managing Authority / Paying Agency, regional authorities, and LAGs, based on EU-level standards, to empower LAGs, enhance their decision-making, management, and accountability capacities, and promote responsive, innovative, and tailored local development strategies. Processes and mechanisms established, coordinated, and animated by the LAG to ensure participatory, transparent, and inclusive decision-making and strong community engagement in strategy development and implementation. Working definitions

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Enhanced results and impacts in LEADER: Mobilizing endogenous potential, valorising territorial assets, and strengthening links between local actors to achieve more with available resources and foster innovative, sustainable, and integrated, community-driven projects that drive lasting structural changes in the LAG area. More innovative projects Increased leverage Projects well-tailored to the needs of the territory Contribution to local development as a result of stimulation of additional funding going into the territory. The proximity of the LAG contributes to the selection of projects that are focused on the most pressing needs of the territory. Projects that generate new ideas, products or processes in the LAG territory that are innovative in the local context (compared to mainstream projects). More project ideas developed and presented for funding under local development strategies. They also encompass several fields/sectors to address a range of issues/needs facing the territory. The proximity of the LAG to the territory encourages more people to present projects, i.e., beneficiaries that would not have applied without LEADER. In some cases, the links between projects, also enable a focus on the needs of the territory. Working definitions

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Enhanced results and impacts in LEADER: Sustainable projects Projects that promote links between actors Examples of links include, e.g. smart specialisation, projects with several partners, integrated projects, umbrella projects, etc. Projects that produce sustainable results, such as evolving into other projects or networks. Partnerships, networks and collaborations that would not have existed without LEADER. Sustainability is also influenced by the knowledge of local conditions. Working definitions

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N These LEADER added value elements are achieved by applying LEADER method with the combination of the three enabling components (the implementation of the local development strategy, the delivery mechanism and the animation/capacity building support). How does it work in your Member State? Exercise in groups

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N LEADER added value elements Networks Mutual support and trust bottom-up approach Shared mental models, values and beliefs Improved social area capital local partnership Local governance Multi-level governance Improved governance innovation Increased leverage networking Projects well-tailored to the needs of the territory Innovative projects integrated and multi- sectoral strategy Enhanced results and impacts Sustainable projects Projects that promote links between actors cooperation

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Operationalisation: How LEADER principles and AV components manifest in action through LDS implementation, delivery mechanism and animation in your MS? LEADER added value components / elements: Improved social capital Improved governance Enhanced results and impacts - Networks - Mutual support and trust - Shared mental models, values and beliefs - Local governance - Multi-level governance - Increased leverage - Projects well-tailored to the needs of the territory - Innovative elements - Sustainable projects - Projects promote links between actors LEADER principles: bottom-up approach public-private partnerships (LAGs) List relevant activities in your LAG area-based LDS integrated and multi-sectoral LDS innovation networking territorial cooperation Optionally stakeholders can discuss these activities from their own perspective: From LAG perspective; From other LEADER stakeholder perspective (MA, PA, NN, other) Discuss in groups

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Operationalisation of the LEADER added value concept Some insights: LEADER added value components => related elements => examples how to operationalise. (Conceptual basis) Elements Examples on how to operationalise Analyse inclusiveness by looking at the composition and membership of the LAG decision making body and how it influences the decision-making process. Look for evidence of improvements in the technical and administrative capacities / competences of the LAG and its staff. Analyse local mobilisation and encouragement of non-beneficiaries to participate in local development. Identify if LAGs have a detailed communication strategy. Local governance

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Structural changes What is the expected change (on LAG level) and how this change is expected to happen?

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Structural changes Significant shifts in a LAG area brought about by the implementation of LDS, related animation activities and delivery mechanisms, which respond to the needs of the population and territory, and consider the global aspects of local development.

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Possible dimensions / areas of structural change: (non-exhaustive list) Environmental and spatial dimensione.g. fighting climate change and/or adapting the local territory; promoting the circular economy; valorising spaces that have been neglected or unused. Economic dimensione.g. creating additional sources of income for the local population; reducing unemployment and creating jobs; digitalisation in rural areas; finding new markets; strengthening value chains; opening up new types of economic activities / diversification. Social dimensione.g. preventing depopulation through access to basic services; keeping young people in the area; reducing poverty and social exclusion; changes in mental and behavioural patterns (e.g. sense of identity, willingness to collaborate). Cross-cuttinge.g. increased community resilience.

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N What are the expected changes (on LAG level) brought about by LEADER? Participants in groups discuss and identify dimensions / areas in which the LDS, related animation activities and delivery mechanisms intervene and where it is expected to make changes according to the identified needs. Discuss in groups

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Evaluation framework How to measure the added value of LEADER and structural changes it brings in the LAG area?

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Evaluation framework What would be considered as success? What is the purpose of evaluation? How can success be measured? Indicators and data sources Evaluation questions Taking into account key elements to assess Factors of success Evaluation methods for calculating indicator values and the contribution of LEADER

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Evaluation framework CAP SP impact evaluations LEADER added value - evaluation topic in MS LEADER added value Evaluation of Enhanced results and impacts Social capital Governance Pertinent (context, evaluation needs) key elements, FoS and indicators can be applied / further adapted by MS Key element / Evaluation question Factor of success Factor of success Outputs Results Impacts

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Improved social capital Number of LAG members Diversity of LAG members Quality of cooperation Size, diversity. Quality of interactions within the LAG Number of joint projects Number of participants in joint projects Capacity to organise Improved social interactions Social interactions in the LAG area Networks Number of cooperation projects Quality of interactions with external actors Networking outside the LAG area Trust in the LAG (internal) Trust towards the LAG (community) Change in trust (community) Mutual support and trust between LEADER stakeholders Social awareness Values in cooperation / joint projects Local identity / image of LAG area Shared mental models, values and beliefs vision for the LAG area, territorial identities, common objectives, values Sources mainly the DME data (Annex VII, Reg. (EU) No 2022/1475) Additional data collection (surveys, focus groups)

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Improved governance Number of LAG members in decision making LAG processes for inclusiveness LAG brings togeather organisations and people Inclusiveness in decision making Number of trainings improving skills Evidence of the improvement of administrative and technical skills Skills of LAGs (administrative and technical) Local governance Number of animation activities Evidence of support to (potential) beneficiaries, including hard-to-reach groups Number of new applicants Animation and support to (potential) beneficiaries Communication tools/activities Local awareness (community) EU closer to citizens (community) Communication capacity Number and type of interactions LAGs/MA/PA Improved quality of interactions LAGs/MA/PA/CAP NN Evidence on improvements in governance / strengthened links Coordination and communication between LAGs and other levels Multi-level governance

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Enhanced results and impacts Mobilisation of additional resources: - Expenditure on other investments - Voluntary work - Generation of new projects (outside EAFRD) Projects that: Stimulation of investment /activities Increase leverage Number of new project promoters Capacity to identify and empover new project promoters New project promoters Meet local needs and objectives by a variety of project promoters that valorise local assets Different types of promoters Projects that contribute to local needs Valorisation of unique territorial assets Capacity to produce community benefits and reinforce community identity Tailored to needs of the LAG area No, % expenditure, types of innovative projects Innovative elements at local level Innovation No of projects that contribute to econ., env., soc. sustainability Knowledge of local conditions LEADER projects produce sustainable results Sustainability Promote links between actors No of projects working in synergy % of projects that produce cooperation Collaborative projects

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Structural changes The implementation of the LDS with the application of the LEADER method produces structural changes in the dimensions covered by the LDS The capacity of LEADER to produce structural changes in the dimensions on which the LDS intervenes Structural changes Sources: surveys, focus groups

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Evaluation framework Which key elements to assess, factors of success and indicators would be most helpful to measure the LEADER added value in your Member State? What challenges do you see when applying this evaluation framework in practice? Discuss in groups

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N What are the next steps? ... Further needs for capacity building Arrangements to be done ...

E U C A P N E T W O R K E U C A P N E T W O R K P R E S E N T A T I O N Get in touch European Evaluation Helpdesk for the CAP evaluation@eucapnetwork.eu Rue Belliard 12 Brussels, Belgium Tel. +32 2 808 10 24 https://eu-cap- network.ec.europa.eu/support/evaluation_en