Insights on Response Rates and Strategies in Labour Force Surveys
Explore an in-depth analysis of response rates and interventions in the Danish Labour Force Survey, including the impact of introducing a lottery for cash prizes and extending data collection periods. Discover key findings on response rates, refusal rates, and the effectiveness of different strategies in enhancing survey participation.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
R-indicators in the Danish Labour Force Survey 17th Workshop on LFS Methodology Neuch tel, 25-26 April 2024 Anna Skovb k Mortensen, aom@dst.dk
Framework Mixed-mode data collection CAWI invitations to entire sample, followed by CATI for non-respondents Four waves Wave 1+2 break for two quarters wave 3+4 Household interviewed in 4th wave Sample size approx. 35,000 Response rate: 46 percent (2023, Q4) Challenge with declining response rate
Weekly response rates 70 60 50 40 30 Total CAWI CATI 20 10 0 2018 2019 2020 2021 2022 2023 2024
Introducing lottery for cash prizes 4th quarter of 2022 11 prizes, with a total cost at 2,700 EUR ( 0.20 EUR pr. interview) 20 percent of the sample as control group Expected outcome + Overall increase in response rate Increase in refusal rate
Lottery for cash prizes response rate Pct. 2.1 percentage points higher response rate overall Prize group Control group 80 70 69.2 6.6 percentage points higher response rate 70 among descendants 60 51.3 48.9 50 45.5 43.4 Approximately the same effect across waves 38.5 37.5 37.2 40 34.6 Greater effect among unmarried and divorced 30 individuals, than among married individuals. 20 10 Relatively uniform effect across educational 0 levels. 15-29 years 30-49 years 50-64 years 65+ years Total
Lottery for cash prizes refusal rate Prize group Control group Pct. 10 9.5 9.2 9 9 8.1 7.9 0.5 percentage points higher refusal rate overall 8 7.4 7 6.3 6.2 6.1 6.1 Largest effect in age groups 30-64 6 5 The result emphasizes the significance of 4 trustworthy communication 3 2 1 0 15-29 years 30-49 years 50-64 years 65+ years Total
Extended data collection period 4th quarter of 2023 Extended from two to four weeks No control group Expected outcome + Increase in response rate
Extended data collection period 60 50 4.3 percent points increase in response rate Changes in CAWI / CATI distribution 40 30 20 Experimenting with reminder procedures 10 0 2023 2024 Total CAWI CATI
How did it effect the representativeness?
Representativeness (strong) Strong definition; A response subset is representative with respect to the sample if - the response propensities are the same for all units in the population - the response of a unit is independent of the response of all other units. Reference Schouten, B., Cobben, F. & Bethlehem, J., 2008. Indicators for the Representativeness of Survey Response. Survey Methodology 35(1), November.
Measuring representativeness 1. Assume know individual response propensities ??, response propensity for sampled unit k ??, design weight for sampled unit k Mean response propensity is then ? ? =1 ? ?=1 ???, ? = ?? ?
Measuring representativeness 2. Standard deviation of the response propensities ? 1 ?? ?2?? ? ? = ? 1 [0,0.5] ?=1 3. R-indicator defined as ? ? = 1 2? ? [0,1]
Representativeness (weak) Weak definition; A response subset is representative of a categorical variable if the average response propensity over the categories is constant. Reference Schouten, B., Cobben, F. & Bethlehem, J., 2008. Indicators for the Representativeness of Survey Response. Survey Methodology 35(1), November.
Measuring representativeness Estimate the response propensities ??using a logit model. The R-indicator is then estimated by ? 1 ?? ?2?? ? ? = 1 2 ? 1 ?=1 Note that the indicator measures representativeness with respect to the chosen variables in the logit model.
Estimating response propensities Logit model based on the variables used to calibrate weights, hence sex * age (12 age groups, 5 year intervals) region * age (two age groups: <75 and >75) labour market status (register based, 10 levels) education level * age (7 age groups, 5 year intervals, 45-74, 75+)
Lottery for cash prizes, R-indicator Prize group Control group Total 0.9 0.8 0.7 Sample size 24,358 Response R-indicator 0.6 rate 0.441 0.764 Prize group 0.5 6,099 0.419 0.776 Control group 0.4 30,457 0.437 0.766 Total 0.3 0.2 0.1 0 Response rate R-indicator
R-indicator, 2021-2023 R-indicator Response rate 0.90 0.83 0.82 0.82 0.81 0.80 0.80 0.79 0.79 0.78 0.78 0.78 0.80 0.75 0.70 0.57 0.60 0.53 0.50 0.49 0.47 0.46 0.50 0.44 0.44 0.43 0.43 0.42 0.42 0.40 0.30 0.20 0.10 0.00 21Q1 21Q2 21Q3 21Q4 22Q1 22Q2 22Q3 22Q4 23Q1 23Q2 23Q3 23Q4
Whats next? Estimate confidence intervals for the R-indicator. Measure representativeness with respect to other variables. Is there an unforeseen challenge associated with the shift in the distribution between CAWI / CATI responses? Who are the non-respondents, and how do we reach them?
Thank you Anna Skovb k Mortensen aom@dst.dk
