Ensuring Data Quality in 2021 Census Amidst COVID-19 Challenges
The 2021 Census Quality Assurance process addresses potential impacts of COVID-19 on data collection and processing. It includes measures like Post-Enumeration Survey and Independent Assurance Panel to maintain data quality. Strategies involve near-real-time field data monitoring, forums for data review, and using external datasets for validation. Quality assurance efforts adapt to changing circumstances and prioritize accuracy in census data collection.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
2021 Census Quality Assurance UNSD EGM, November 2021
Overview Understanding the potential impacts from COVID-19. Quality assurance in data collection. Quality assurance in data processing. The Post Enumeration Survey. The Independent Assurance Panel. Summary of key data quality measures. 2
Understanding potential COVID-19 impacts Impacts in data collection: Potential impacts in interpretation of Census questions (e.g. business location when working from home). Impacts in the field (e.g. lockdown, COVID-19 restrictions). Some population groups may be more affected than others (e.g. homelessness). Impacts on the data and data processing: Real world changes (e.g. journey to work, employment, population). How does this affect interpretation of timeseries and data comparison? Census is a point in-time collection. What is the longevity of Census data that is conducted at an atypical time? Changes to how the processing workforce operates. 3
Quality assurance in data collection Near-real time field data in Operational Insights tool. Reports at three levels: strategic, tactical and operational for relevant staff/teams. Breadth of data digital service, contact centre, field data. Focus on small area results against expectations and ranking. Reports that bring disparate data together to understand full picture for small areas. STRATEGIC Taking action when response targets threatened TACTICAL Heeding warning signs that we need to adjust course for underperforming areas OPERATIONAL Pro-actively fixing the problems on the ground before they affect response 4
Quality assurance in data collection Field data quality review forum that met several times per week during field enumeration. Complementary data operations forum weekly to consider implications of field outcomes and early form data. Framework guiding what we needed to consider across time. COVID impacts on response and mitigations monitored in field data. Forecasts and Targets What are we expecting? How do we prioritise? Recipe Book What do we need to achieve in these phases? What OI Reports will inform us? Decision Points When do we need to make key decisions? Diary What are the phases and what questions will we be asking? 5
Quality assurance in data collection Example: Quality assurance for homelessness data. Homelessness is not asked directly on the Census form, but is derived based on a number of characteristics. COVID-19 has meant that we are using a range of de-identified datasets provided by external organisations to support enumeration planning, for data validation and/or for data substitution, for example: by-name lists maintained by local homelessness services. rough sleeper survey results collected by state governments. Supported accommodation administrative data. Additional assurance provided through our Homelessness Statistics Reference Group (made up of external stakeholders from Government, not for profit and academia) 6
Quality assurance in data processing The key objectives of quality assurance throughout data processing are to ensure that: coverage is accurate (noting that final assessment of Census Coverage is determined by a separate coverage survey) systemic errors and significant anomalies are detected and resolved. results are accurate, relevant, and coherent (explainable and reflect real-world). 7
Quality assurance in data processing Approaches to quality assurance in data processing: Micro edit and macro edit data, for input and output data items. Monitor/review changes from content design, enumeration, processing, and coding. Analyse data against expectations and through multiple lens for insights. Analyse/compare results over time with 2016 Census and 2011 Census data sets as benchmarks Confront results with alternate administrative and survey data. COVID-19 changes how we interpret differences between the Census results and comparison datasets (e.g. previous censuses, administrative data). 8
Post Enumeration Survey (PES) The PES is a large household survey run shortly after the Census (approx. 0.5% of households). The PES is run independently from the Census. The PES has faced similar challenges as Census due to COVID-19. Strong Census results so far give us more confidence for the PES. A new telephone first approach in 2021 has prepared us well to collect data during COVID- 19. 9
Post Enumeration Survey (PES) PES information is collected via interview, which may be more likely to be accurate than self-reported Census information. The PES produces estimates of the coverage of the Census, including undercount and overcount. Net undercount in 2016 was 1.0%. New emphasis on gross measures of under and overcount in 2021 to better describe Census coverage. The PES also provides measures of misclassification in the Census for some variables (e.g. age, sex, Indigenous status). New in 2021: Exploring potential non-response bias through use of administrative data (e.g. MADIP). 10
Independent Assurance Panel First established following the 2016 Census, with a report published on the same day as data release. The 2016 report found the data was fit for purpose . The 2016 Panel focused on time series consistency (using 2006 and 2011 censuses as benchmarks) and the results of the Post Enumeration Survey to assess quality. A subsequent review found that the report provided assurance, integrity and transparency. The 2021 Panel comprises 6 members of with a wide range of expertise, including academics, international and national Census experts, and senior members of state and territory government departments, including Aboriginal and/or Torres Strait Islander representation. The 2021 Panel will deliver a report to coincide with data release in mid 2022. 11
Summary of key data quality measures Census: Response rate. Online form uptake, and self response rate. Consistency with timeseries and/or expectations. Item non-response. Post Enumeration Survey: Undercount, overcount and misclassification. Assessment by the Independent Assurance Panel. Feedback and use by data users. 12
