Unpaid Time Statistics Study in Canada Post-Pandemic

Survey on Unpaid Time Statistics Canada s first non-response bias study (post-pandemic) Cindy Ubartas Marie-H l ne Miville Genevi ve V zina Statistics Canada April 2024

Outline Context Description of the survey selected Objective of the non-response (NR) bias study General Social Survey (GSS) Time Use vs Survey on Unpaid Time (SUT) collection strategies Collection results Analyses and basic results Lessons learned / future plans

Context Downward trend in response rates in social surveys Variance was the main focus in the past but questions about potential bias are now raised more often Options to evaluate potential bias are limited: Link to other sources of information whenever possible 2021 Census can be an option in some cases NR follow-up studies become the best option for certain surveys Especially if no other sources have information comparable to the survey s key variables

Description of the Survey Selected General Social Survey (GSS) on Time Use Held every 5 to 7 years (last iteration was in 2022) Gathers data on how Canadians use their time in a day Monitors changes over time Particularly around paid and unpaid work (including caregiving), transportation, and personal care Each unit has a reference day for which they need to report their activities through a 24-hour diary Historically, a recall period of 48 hours was allowed to respond to the survey

Objective of the NR Bias Study Get insights on non-respondents Evaluate their socio-demographic characteristics Evaluate their responses to key variables Comparison with respondents Small sample size: 400 non-respondents from GSS Time Use Seen as a pilot Will be used to get an idea of potential bias, not to correct bias Costs-benefit analysis to assess future requirements Could we start with a smaller total sample size for the main survey but plan for a NR Bias Study to be launched if needed?

Collection Strategies Comparison Item / Survey Objective GSS Time Use Survey on Unpaid Time (SUT) Strategy thought to maximize chances to get a response, if not at least the dwelling status Need good representation of all days during the year Strategy thought to maximize chances that a person completes the survey on the specified date chosen Not necessarily the best strategy for collection purposes but required for the topic July 16th, 2022 to July 15th, 2023 (12 waves of about a month) Complex survey: diary with 24- hour schedule, 45 minutes on average Collection Dates 8 weeks between November 6 to February 3, 2024 Less than 15 minutes to complete, 34 questions on basic household demographics, unpaid work, care and health & main activity + education Complexity

Collection Strategies Comparison (continued) Item / Survey GSS Time Use Initial Collection Mode questionnaire online application) Email invitation when email available (about 69% of sample), 2 or 3 email reminders Letter invitation when mailable address but no phone number (about 8% of sample) 2) By phone (interview-assisted) otherwise Introduction letter sent to let household know calls will follow (about 23% of the sample) Non-response follow-up Interviews (CATI) Between 6 to 14 days depending on the wave Survey on Unpaid Time (SUT) 1) Introduction letter by mail with instructions to answer online whenever possible About 89% of the sample, 2 letter reminders, 2 email and one SMS reminders 2) Computer-assisted personal interviews (CAPI) otherwise 1) Self-response (EQ - electronic Computer-Assisted Telephone CAPI With phone calls first when a number is available Personal visits coordinated with other surveys in the field

SUT Collection Results Final response rate: 69.3% (target was 85%) EQ completion: 34.7% CAPI completion (phone calls or visits): 65.3% Refusals: at least 59 cases with one refusal (14.8% of the sample) Converted 14 of them (23.7%) Conclusion: There will always be a certain proportion of the population who refuses to answer Statistics Canada s surveys but When a different collection strategy and a shorter questionnaire are used, good results can be obtained with a NR Bias Study

Analysis and Results Non-respondents have different characteristics Younger people reached Higher proportion of men Lower level of education CAPI allows to better define an in-scope or out-scope status for the sampling units Sample size was too low to measure the bias

Lessons learned/Future Plans Proved the feasibility of NR bias studies Possible to develop and lauch a new NR Survey very quickly We need to define a framework for them When should we implement these type of studies? How can it be incorporated in our regular survey process? Need to define the parameters of such studies (for example, sample size) to measure and correct bias in key estimates of a survey Early planning in the process of a survey to define the key indicators and have specific questions to measure them in both surveys CAPI is crucial

Thank you! / Merci! Questions? Cindy Ubartas Chief, Research and Innovation Section in Collection cindy.Ubartas@statcan.gc.ca Marie-H l ne Miville Senior Methodologist on the General Social Survey marie-helene.miville@statcan.gc.ca Genevi ve V zina Senior Methodologist on the General Social Survey genevieve.vezina@statcan.gc.ca Delivering insight through data for a better Canada