Aligning National Indicators to Policy Priorities in Statistical Literacy Workshop

Sub-regional Workshop on the Pacific Statistical Literacy and Capacity Development, 22-26 May, 2023 ALIGNING NATIONAL INDICATORS TO POLICY PRIORITIES CHRIS RYAN STATISTICIAN ESCAP PACIFIC OFFICE

OUTLINE OF SESSION Phase 1: What role do statistical indicators play in the planning process Phase 2: What constitutes a good statistical indicator Phase 3: Steps for developing a national indicator framework Phase 4: Exercise on identifying indicators Phase 5: How can countries seek support with indicator development

Phase 1 What role do statistical indicators play in the planning process?

WHAT IS A STATISTICAL INDICATOR? In the National Context A Statistical Indicator Statistical Indicator is a measure which provides meaningful evidence to help monitor progress against key national issues for which desired outcomes are sought Breaking down this definition Measure: The measure can take many forms (eg, number, proportion, rate), so it s extremely important it is clearly defined Meaningful evidence: It s crucial the indicator is of relevance to the issue Help monitor: The indicator doesn t need to monitor progress against all aspects of the issue it is trying to monitor Key national issues: Understanding what the key issue is about is a crucial first step to developing good indicators Desired outcomes: The indicator should be providing guidance on whether a goal is being achieved

HOW DO STATISTICAL INDICATORS FIT IN TO THE SCHEME OF PLANNING/ POLICY MAKING? One way of looking at the role of indicators in a national plan/policy is that they measure the degree of progress being made against a desired outcome, when strategies or activities have been implemented to address an issue which requires action. In other words, the indicators can be considered the last step of a 4-step process. What are the priority issues requiring action ( Issues requiring Action Issues requiring Action ) identified in the plan/policy Step 1 What Strategies/Activities Strategies/Activities are presented in the plan/policy to address these Issues requiring Action Step 2 What Outcomes Outcomes do you want to see achieved with respect to these Issues requiring Action , often as a result of implementing Strategies/Activities Step 3 What Statistical Statistical Indicators Issues requiring Action , to assess if progress towards the desirable Outcomes is being achieved Indicators can be generated to help assess the status (and monitor progress), against these Step 4

UNDERSTANDING THE CONCEPTS OF EACH STEP IN THE PROCESS Issues requiring Action v Strategies/Activities Issues requiring Action v Strategies/Activities Planning and Policy documents often intermix the issues and strategies in their narrative, so there s not always a clear distinction between the two in the document text Issues requiring Action: Are normally problems which need addressing Strategies/Activities: Are normally actions taken by government (or another body) to address these problems

ISSUES REQUIRING ACTION V STRATEGIES/ACTIVITIES: EXAMPLE 1 Example of some text in a National Energy Sector Plan Example of some text in a National Energy Sector Plan A centralized database for renewable energy projects needs to be developed to ensure information is readily made available and accessible across the sector. One of the ongoing challenges faced by the sector is the accuracy and reliability of the available data. Also, there are significant delays in data collection from relevant stakeholders. Therefore, an integrated information system advocates the consistency of information and quality data and reports are produced on time. This information is also useful in sector planning including accessing financial and technical assistance from potential development partners. Information needs to be readily available and accessible across the sector The accuracy and reliability of the available data is poor Significant delays in data collection from relevant stakeholders Issues requiring Action Development of a centralized database (integrated information system) for renewable energy projects Strategies/ Activities

UNDERSTANDING THE CONCEPTS OF EACH STEP IN THE PROCESS Outcomes v Statistical Indicators Outcomes v Statistical Indicators Planning and Policy documents sometimes make the mistake of listing outcomes as indicators, but there is a clear distinction between these two concepts Outcomes: Are a description of what you are trying to achieve (a target) and often uses terms like increase , improve , reduce etc. They may also include a measure of achievement, such as reduce by 50% , and ideally should have a date of desired achievement Statistical Indicators: Are simply the measurement/s suggested to determine if the outcome is being met or likely to be met

OUTCOMES V STATISTICAL INDICATORS: EXAMPLE 1 Outcome Outcome SDG Target 3.1 SDG Target 3.1 By 2030, reduce the global maternal mortality ratio to less than 70 per 100,000 live births Statistical Indicators Statistical Indicators SDG Indicators 3.1.1 & 3.1.2 SDG Indicators 3.1.1 & 3.1.2 Maternal mortality ratio Proportion of births attended by skilled health personnel

Phase 2 What constitutes a good statistical indicator?

CHARACTERISTICS OF A GOOD INDICATOR THE SMURD REVIEW 5 Questions to ask yourself when assessing the usefulness of an indicator Q1) Is the information in the indicator description S Specific enough to make it clear as to what is being measured? Q2) Is the indicator M Measurable given what data is likely to be available or feasible? Q3) Have the U Units of measurement been clearly spelt out? Q4) Is the indicator significantly R Relevant to the issue it is meant to be monitoring? Q5) Have the levels of D Disaggregation been carefully thought through and included in the indicator description?

SMURD INDICATORS FOR SMARTTARGETS S Specific S Specific Similar concepts M Measurable M Measurable U Units Clarified A Achievable Focus on how achievable the target is within a time frame Further focus on clarity of indicator R Relevant R Relevant D Disaggregated T Timely

Q4) IS THE INDICATOR SIGNIFICANTLY R RELEVANT TO THE ISSUE IT IS MEANT TO BE MONITORING? Example of relevance missing the mark Issue requiring Action In the Education Sector a common problem needing to be addressed in some countries is truancy (ie, school aged children skipping school without permission) Proposed Indicator School enrolment rates by age, sex Assessment via Q4: Whilst this is a useful indicator to help gauge how many school aged children may be missing out on an education, it doesn t capture the truancy problem Proportion of school aged children who were enrolled in school yet reported missing a day of school for no reason last week The indicator needs to be more focused on the truancy issue

Q5) HAVE THE LEVELS OF D DISAGGREGATION BEEN CAREFULLY THOUGHT THROUGH AND INCLUDED IN THE INDICATOR DESCRIPTION? Example 1 Leaving no-one behind 1.1.1 Proportion of population below the international poverty line, by sex, age, employment status and geographical location (urban/rural) Other levels of disaggregation which could be adopted for 1.1.1 Race Ethnicity Migratory Status Disability Status Important to combine levels of disaggregation to better understand full picture People most impacted by poverty could be those who meet a number of characteristics. For example: Women, aged 65+

Q5) HAVE THE LEVELS OF D DISAGGREGATION BEEN CAREFULLY THOUGHT THROUGH AND INCLUDED IN THE INDICATOR DESCRIPTION? Example 2 Other benefits of disaggregation (tourism) Indicator: Number of tourists each year Levels of disaggregation which can add value to understanding different types of tourists visiting a country Country of origin Sex Age Length of Stay Total spent on visit

Phase 3 Steps for developing a national indicator framework

BROAD STEPS IN DEVELOPING INDICATORS FOR A POLICY/PLAN Step 1: Understanding the Policy Priorities of the Policy/Plan Broad Issues being addressed Step 2: Understanding for whom are these issues relevant? Specific Groups in particular? Step 3: Mapping all relevant existing indicators to these Policy Priorities Global/Regional/National Step 4: Refine the final indicator selections for each Policy Priority What s most relevant? What s measurable? Ensure clarity with terms and units Step 5: Ensure all relevant levels of disaggregation are included based on Step 2

STEP 1 UNDERSTANDING THE POLICY PRIORITIES Two obvious places to look within the Plan/Policy 1. Narrative 2. Logframe

STEP 2 UNDERSTANDING FOR WHOM ARE THE ISSUES RELEVANT Which population does the issue relate to? Which sub-populations in particular may the issue be more problematic? (NB: These questions may not be relevant for all issues in the policy/plan) Example Issue: Limited access to Early Childhood Education (ECE) due to geographical and financial barriers Population of relevance: Children in the ECE age bracket Sub-populations of importance (in document): Geographical locations (remote) and income status of households (low-income) Sub-populations of importance (additional): Sex

STEP 3 MAP INDICATORS OF RELEVANCE TO THE ISSUES Two main places to look National Indicator Frameworks Global/Regional Indicator Frameworks EG Education Sector Global National SDGs National Develop Plan Incheon Declaration Past Education Sector Plans Regional Pacific Roadmap for SD

STEP 3 MAP INDICATORS OF RELEVANCE TO THE ISSUES (CONT) Example Issue: Limited access to ECE due to geographical and financial barriers Potential indicators could be: SDGs: 4.2.1 Proportion of children aged 24 59 months who are developmentally on track in health, learning and psychosocial well-being, by sex SDGs: 4.2.2 Participation rate in organized learning (one year before the official primary entry age), by sex Incheon Declaration: Gross pre-primary enrolment rate National Plan: Proportion of 3-5 year olds enrolled in ECE learning

STEP 4 REFINE FINAL INDICATOR SELECTIONS This step of the process involves selecting the most appropriate indicator Could be more than one indicator Can modify the indicator to make more appropriate to national Criteria What s most relevant? What s measurable? Example Issue: Limited access to ECE due to geographical and financial barriers Selected indicator could be: SDGs: 4.2.1 Proportion of children aged 24 59 months who are developmentally on track in health, learning and psychosocial well-being, by sex Ensure clarity with terms and units SDGs: 4.2.2 Participation rate in organized learning (one year before the official primary entry age), by sex Incheon Declaration: Gross pre-primary enrolment rate National Plan: Proportion of 3-5 year olds enrolled in ECE learning

STEP 5 INCLUDE ALL RELEVANT LEVELS OF DISAGGREGATION This step involves ensuring all relevant levels of disaggregation are properly addressed Example Issue: Limited access to ECE due to geographical and financial barriers Target Groups Final indicator should thus look something like: SDGs: 4.2.2 Participation rate in organized learning (one year before the official primary entry age), by sex, by urban/rural, by income status by urban/rural, by income status

Phase 4 Exercise on identifying indicators

Phase 5 How can countries seek support with indicator development?

HOW TO SEEK HELP WITH DEVELOPING INDICATOR FRAMEWORKS Reach out to ESCAP and seek support Once a document (or part of document has been identified) roles would be: Role Role Step Step Country Country ESCAP ESCAP 1. Policy Priorities 1. Policy Priorities Lead Support & Review 2. Populations of interest 2. Populations of interest Review Lead 4. Mapping Indicators 4. Mapping Indicators Review Lead 5. Finalize Indicators 5. Finalize Indicators Finalize Propose 6. Disaggregation review 6. Disaggregation review Finalize Propose