Tuvalu Long-Form Census Experiment Report: Findings, Rationale, and Outputs

Tuvalu Long-Form Census experiment report (PacStat IE1) 14thMeeting of the PSMB, October 2024, Port Vila, Vanuatu Lae Peleti, Michael Sharp, Benuel Lenge

Objectives Presentation objective Experiment objectives Present the draft findings on the innovative experiment relating to the experiment objective: Comparing the quality of data of the previous HIES and census with that of the LFC. Comparing the budget of the previous HIES and census with that of the LFC. 1. Determine if cost-efficiency can be achieved by conducting an integrated survey, 2. Evaluate the quality of the obtained datasets when compared to the benchmark of two stand-alone surveys 3. Identify any other identifiable costs or benefits resulting from the integration, particularly in the context of SIDS

Experiment rationale Integration potential benefits High return on investment: more data can be produced for the same cost; Lower per-unit cost: fixed costs are spread across two collections; Production of a more comprehensive dataset additional or more disaggregated analysis; Reduced burden on respondents and the NSO due to fewer surveys (1-visit approach: 1 questionnaire, 1 training, 1 field work, etc.). Underlying issues Pacific remoteness Logistically challenging Long to implement Expensive to implement Resource-intensive on NSOs (small number of staff) Census and surveys are huge undertakings Facing increasing data requirements to measure and report against development indicators Result: frequency and timeliness of household survey data production are not always optimal Heavy burden on respondents Constraints Length of the interview; Wider range of skills required to the enumerators.

Experiment Outputs The Tuvalu Long Form Census project has delivered a dataset which has been used to: 1. 2022 Census Analytical report - update current population estimates and structure in Kiribati (report forthcoming) 2. 2022 HIES report - update current estimates of household income and expenditure 3. Complete a food security assessment 4. Complete a poverty assessment (report forthcoming) 5. Experiment report (report forthcoming) Available to support initiatives, such as population projections and gender, disability and social inclusion analysis, and, notwithstanding the caveat with respect to the data not being representative of an entire year, the rebase of the CPI and household component in GDP.

Experiment results: Experiment results: Interview duration Interview duration Distribution by interview type and household size. On average, the PHC took 2.21 hours to complete the interview, while the LFC (PHC+HIES) took 3.85 hours The length of interview does not appear to be vastly different for urban or rural locations, Length of interview increases as household size increases,

Experiment results: Experiment results: Data Processing and Data Quality Data Processing and Data Quality Census Data: Integrated census and HIES. Different household definitions cause discrepancies. Large households often excluded due to reporting difficulties. Data Quality Issues: Longer interviews led to respondent burden and interview fatigue. Coverage and content errors noted. Conceptual issues on type of count (de facto vs de Jure)

Experiment results: Experiment results: Data Processing and Data Quality Data Processing and Data Quality Food Consumption Data: Collected in-house and away-from- home consumption. Issues with under- and over- reporting. Final valid sample: 492 households. Data useful for consumption patterns despite limitations. Food Insecurity Experience Scale (FIES): Measures moderate and severe food insecurity. Uses 8 yes/no questions over 12 months. High percentage of extreme cases; dataset not suitable for FIES analysis.

Experiment results: Comparative analysis of costing Limitations: Different methodologies and definitions. Variations in time and costs. 2015 HIES: Paper-based questionnaire. Emphasis on post-enumeration data entry and processing. 12 months of fieldwork with 7 household visits. 2022 Integrated Long Form Census: Used CAPI Centralized fieldwork in Tuvalu for quality control. Comprehensive training, dissemination, and post-enumeration activities by ADB and SPC. 2017 Mini Census: Used CAPI All households canvassed and interviewed like a full census.

Experiment results: Comparative analysis of costing Proportion of budget by activity component Activity Budget ($) Popn HH HH size Av. cost ($) pp 60% 2015 HIES 239,828 10,782 1,872 5.8 22 2017 Census 2022 LFC 90,869 10,645 1,688 6.3 9 50% 508,915 10,876 1,799 6.1 47 40% 30% LFC is three times more than the average cost. LFC is 54% more than the 2 independent operations combined. 20% 10% 0% The regional average is USD 12 per person, while surveys (incl. HIES) cost around USD 6 per person . 2015 HIES 2017 Census 2022 LFC

Experiment summary Fieldwork: Covid pandemic and Christmas festive season contribute to delay in fieldwork Conceptual issues and questionnaire translations and mixed methodological approaches contribute to data quality issues High Turnover: High enumerator turnover extended data collection from 6 weeks to 6 months. Monitoring: Challenges in fieldwork monitoring and backlogs on interview approvals. Data Quality: No Control Sample: Lack of a 12-month HIES control sample; results should be interpreted cautiously. Financial Implications: Cost Efficiency: 2022 experiment costs were 54% higher than standalone HIES and Census combined. Experiment Replicability: Success: Produced usable data for census and HIES. Challenges: More related to implementation than design. Cost Efficiency: Not met, raising doubts about replicability to other countries.

Recommendations Considering the challenges and lesson learnt, the following is noted for recommendations Avoid Combining HIES and Census: Based on the experiment, it is not advisable to combine a Household Income and Expenditure Survey (HIES) with a Census. Acknowledge Experiment Limitations: Recognize that this conclusion is based on a single experiment under challenging conditions and may not be definitive. Integrated Survey Potential: Note that an integrated survey can work in principle and produce the required data, except for the lack of 12-month data collection in HIES. Address Seasonal Data Gaps: Consider the limitation that HIES does not capture seasonal changes in income and expenditure. Improve Field Work Planning: Avoid extending field work significantly beyond the planned duration to prevent implementation challenges. Enhance Response Rates: Work on strategies to improve response rates to the HIES instrument. Consider Timing of Implementation: Avoid implementing surveys during periods of significant disruption, such as post-Covid-19 outbreaks or major holidays. Ensure Effective Training: Provide effective training for field staff, ideally in-person or with reliable internet connectivity to overcome remote training challenges

PSMB is invited to 1. Discuss the results of the innovative experiment, 2. Support the recommendations based in the experiment finding, and 3. Provide feedback on report template and contents they see necessary to adjust to be considered as final.