Evidence Reliability Criteria in IPC

Protocol 2.4 Evidence Reliability Criteria Session 9 IPC Acute Malnutrition Level 1 Training IPC Global Partners IPC Global Partners With the support of With the support of

Protocol 2.4: Meet Minimum Evidence and Analysis Requirements What is the objective of Protocol 2.4? To provide guidance on assessment of soundness of method and time relevance of evidence for use in IPC How does Protocol 2.4 work? It provides general statements for three reliability scores (R1-, R1+ and R2) made up of soundness of method (M) and time relevance (T) of evidence It provides detailed parameters for M and T for methods included in the Reference Table Every piece of evidence used in IPC should be assigned a Reliability Score. Evidence scoring less that R1- (i.e. R0) may only be used in areas with limited humanitarian assistance Disclaimer: Reliability Scores are a general statement not based on a rigorous statistical process. Reliability Scores are assigned for IPC convergence of evidence analytical purposes only and should not be used as guidance for primary data collection. 2

Evidence Reliability Criteria Reliability Score Reliability in IPC refers to trustworthiness of the information IPC assess the reliability of each piece of evidence used in IPC and assign a Reliability Score Reliability Score in IPC AMN (a) R2 = Reliable (b) R1 = Somewhat Reliable (R1 is further divided into R1+ and R1-) Reliability Scores are applied to both outcome indicators AND contributing factors 3

Evidence Reliability Criteria Reliability Score Data on different indicators from different methods (M) and seasons/time (T) are used in IPC AMN analysis Some of these information is considered to be relatively more reliable than the others The reliability Score is a function of M and T 4

Evidence Reliability Criteria Reliability Score General Guidance for Evaluation of M GAM based on WHZ from surveys representative at the unit of analysis with adequate precision GAM based on WHZ from surveys representative at the unit of analysis with adequate precision and validated by an in country authority and validated by an in country authority Cluster surveys with 25 clusters Simple or systematic surveys with 150 observations Good GAM based on WHZ that partially meets representativeness and quality standards or GAM based GAM based on WHZ that partially meets representativeness and quality standards or GAM based on on MUAC from minimally acceptable methods MUAC from minimally acceptable methods o Surveys representative at the unit of analysis Surveys representative at the unit of analysis Estimates validated with caution (for GAM based on WHZ only) Estimates based on GAM based on MUAC from surveys of Good Method o GAM based on WHZ from disaggregated surveys representative at a higher administrative unit GAM based on WHZ from disaggregated surveys representative at a higher administrative unit 5 clusters and 100 observations o GAM based on WHZ/MUAC from Sentinel sites GAM based on WHZ/MUAC from Sentinel sites 5 sites per unit of analysis with 200 total observations (if the area is pastoral, 5 sites with 100 observations is acceptable) o GAM based on MUAC from Screening GAM based on MUAC from Screening Exhaustive screening (door-to-door) carried out at the unit of analysis (>80% coverage) or Screening from 3 sites (selected either randomly or purposively, for variability reasons) from the unit of analysis and 200 observations selected randomly or exhaustively (>80% coverage) from each site and with convergence of estimates from these sites o Surveys from similar areas Surveys from similar areas GAM based on WHZ from a survey with good method from a similar area Soundness of Method (M) Limited 5

Write the meaning of each score in 1 minute with a partner Exercise 9.1 1. R2 means ___, and includes ____method and ____ time relevance 2. R1+ means ___, and includes ____method and ____ time relevance 3. R1- means ____, and includes ____method and ___ time relevance 6

Evidence Reliability Criteria Reliability Score General Guidance for Evaluation of M Evidence reflecting current conditions. Evidence reflecting current conditions. o Evidence collected during the same season of analysis is seasonality o Evidence collected anytime during the previous 12 months when there is no seasonality no seasonality nor significant shock to acute malnutrition nor significant shock to acute malnutrition contributing factors contributing factors Evidence inferred to reflect current condition Evidence inferred to reflect current condition o Inferred estimates of evidence collected within the last 6 months but not from the same acute malnutrition season (12 months for areas with no seasonality). o Historical evidence collected during the same acute malnutrition season from at least two similar years in the last 5 years only to be used in the absence of any unusual shocks same season of analysis, when there Good Time Relevance (T) Limited 7

Assign reliability for the following Exercise 9.2 a) GAM based on WHZ from a SMART survey from the same season of a current analysis that meet the IPC AMN criteria for outcome indicators b) GAM based on MUAC from a food security and nutrition survey that was conducted during the same season of a current analysis and meet the IPC AMN criteria for outcome indicators c) GAM based on WHZ from a SMART survey conducted during the same season of a current analysis in a district that is comparable to that of the unit of analysis d) Exhaustive MUAC screening that meet the IPC AMN criteria for outcome indicators carried out during the previous season of the current analysis e) GAM based on MUAC from a SMART survey conducted 2 months prior to the current season of analysis 8

www.IPCinfo.org The End IPC Global Partners IPC Global Partners With the support of With the support of