Addressing Non-Response Challenges in Data Collection Operations

Collection and Research Initiatives for Non-Response Cindy Ubartas Chief, Research and Innovation Section, Collection Planning and Research Division Statistics Canada

Outline Data Collection Operations at Statistics Canada Context Factors that contributed to declining response rates in social surveys Current state of collection Initiatives to better understand non-response and improve response rates

Data Collection Operations at Statistics Canada Statistics Canada conducts all national social and economic surveys across the country More centralized than US statistical organizations 5 Computer-Assisted Telephone Interview (CATI) call centres National sample for CATI as opposed to sample assigned to a specific call centre Most interviewers are still working from home 5 Computer-Assisted Personal Interview (CAPI) offices Vast majority of surveys are multi-mode (e.g. Web/CATI/CAPI) Web collection: Electronic Questionnaire (EQ) One single questionnaire for all modes

Context Downward trend in response rates in social surveys Example: Labour Force Survey (LFS) is mandatory in Canada Stable response rate observed in 2015-2016 ( 90%) LFS Response Rates Over Time 95.0% 90.0% 85.0% Impact of the Pandemic 80.0% 75.0% 70.0% 65.0% 60.0% 55.0% Slight increase since the beginning of 2022 50.0% 201801 201804 201807 201810 201901 201904 201907 201910 202001 202004 202007 202010 202101 202104 202107 202110 202201 202204 202207 202210 202301

Factors/challenges that contributed to declining response rates in social surveys Changes in respondent behaviour and preference in terms of communication channels Requirements for timely data increased with the pandemic New data requirements identified by the Disaggregated Data Initiative Launched in fall 2021 Objective: obtain statistical information that highlights the experiences of specific population groups (e.g. Indigenous peoples, racialized populations, people living with disabilities) Need to maximize responses in short collection periods with large sample sizes Large number of interviewers hired during the pandemic Large employee turnover throughout the pandemic

Current state of collection EQ works really well in general Between 15% to 35% response rates in EQ Generic electronic questionnaire (EQ) platform developed many years ago In addition to traditional paper letters, the ability to send mass emails and SMS was developed Modes of contacts added to the collection strategies Back to regular CAPI since the fall 2022 Results mitigated Requirements now different for conducting in-person interviews CATI collection Large variations between surveys Improvement observed for some of them in the last few months

What are the short/mid-term plans? Action items to improve CAPI results Hiring still ongoing Focus on training and monitoring (including recordings of parts of interviews eventually) Improved reports to help the regional offices keep track of performance of their interviewers Potentially offer virtual interviewing (e.g., videoconferences) as an option Research project this year: explore people s interest according to their characteristics & determine how it could work in practice Reintroduce the use of CATI Adaptive Collection Design Currently testing new functionalities in our collection system to facilitate this process LFS: units are in collection for 6 months, so could take advantage of collection characteristics from the previous cycles Pursue the use of a mobile application to collect data Convenient way for respondents to complete some of the most burdensome surveys Example: Diary used for the Survey of Household Spending Pilot project done in the fall 2021 with subjective well-being in Canada (Vitali-T-Stat) Requires strong communication between a wide range of internal partners and App Store Need experienced developers with mobile-phone applications and extensive testing at each phase of the project

Research projects on non-response Segmentation Used with success for the 2016 and the 2021 Censuses Main ideas: Divide population into homogeneous groups in terms of response/contact rates Target more difficult groups using public relations communication campaign Examples for surveys: Short video clips highlighting our survey data, on topics selected to attract the attention of these segments Short articles made available to the public through social media and community-level media Special landing page on our website where all the tools for this campaign are available Develop an insert to mail to selected households at the same time than the introduction letters Develop partnerships with organisations that could make these groups more aware of our surveys and their benefits Objective: apply this campaign for the LFS Currently looking at characteristics of non-respondents to identify targeted subgroups

Research projects on non-response Analysis of feedback from the interviewers Notes provided during the attempts and debriefing questionnaires Could potentially add observations on the field (presence of children, neighbourhood, etc.) Need parsing/use of artificial intelligence methods to better analyze results Can help understand: Characteristics of non-respondents Reasons for non-response, including understanding/interpretation of the questions Advantages Short-term: can improve the overall quality of the next iteration of a questionnaire for a survey Long-term: could modify our communication material (letters, introduction messages in CATI/CAPI) based on the respondent s profile

Research projects on non-response Non-response Bias Study Non-response follow-up (NRFU) survey to get more insights on the characteristics of non-respondents Short (15-minutes) EQ developed in our collection system with accelerated timelines Could potentially help detecting the presence of bias among respondents First candidate: International Study of Adults (ISA) in May 2023 Will also serve as an example to establish a framework (standard collection vehicle) for future NRFU evaluations

Thank you! Questions? cindy.ubartas@statcan.gc.ca