Taking Part Survey: Longitudinal Web Panel Insights
Explore the Taking Part survey, a flagship initiative by DCMS and its partners to estimate participation in cultural and sporting activities in England. Learn about the survey's methodology, objectives, and significance in measuring social action. Discover how the web panel was established to address challenges and ensure unbiased results over time.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
TAKING PART SURVEY: LONGITUDINAL WEB PANEL Alex Bj rkegren, Olivia Christophersen; DCMS
The Taking Part Survey: longitudinal web panel 1. What is the Taking Part survey? 2. Why did we set up the web panel? 3. How does the web panel work? 4. Challenges and future work
What is the Taking Part survey? DCMS s flagship survey along with its three funding partners: Sport England, Arts Council England, and Historic England Aim: estimate the number of people taking part in leisure, cultural and sporting activities in England. ...also covers vast array of other subjects. 2017/18 survey topics included: libraries, museums, heritage sites, archives, sports, first world war commemorations, volunteering, charitable giving, newspapers, wellbeing, employment....
What is the Taking Part survey? Face to face survey (Ipsos MORI & NatCen). Adults (16+), youths (11-15) and children (5- 10). Households randomly selected from Post Office address list within ~700 areas. Nationally representative (England) Now in its 14th year (started in 2005/06). Survey year runs from April to March (financial year). National Statistic
What is it used for? One of DCMS s five objectives is to maximise cultural and sporting participation, and social action This is measured using the Taking Part Survey Only robust internal government source for social media use. Evaluate policies e.g. First World War centenary commemorations. ONS (child wellbeing), UK Sport (attendance at sporting events), The Football Association, the Crafts Council, The Royal Horticultural Society, charities, small businesses, other government departments, academics, journalists
Why did we set up the web panel? Objective: identify reasons for changes in participation in leisure, cultural and sporting activities in England over time. Longitudinal panel: interviews from 2011/12 to 2016/17 Face-to-face interviews are expensive! Longitudinal component of the survey was becoming increasingly biased potentially affecting headline results.
Why did we set up the web panel survey structure Current design 2018/19 ??? Mixed mode 2011/12 2016/17 Legacy sample Web panel Fresh sample Fresh sample Face to face survey. Source for headline measures
How does the web panel work? Asked at interview if would like to join the web panel. Active web panelist: those completing at least one quarterly questionnaire per annual cycle If email address valid, sent a link to a registration questionnaire ( 5 incentive). Inactive panelists removed at the end of the year. Invited to complete a short (10 min) questionnaire each quarter ( 2.50 incentive). Must have 5 banked before they can cash it out. Questionnaires change each quarter. 2 further email reminders, then a postal reminder.
How does the web panel work? Some questionnaire topics change each quarter Others are core : Subjective well-being; Free time; Life events; Sport screener Questionnaires repeat annually Quarter Topic per instrument Timing (months since interview) 3 months 6 months Core, Barriers to participation Core, Arts (activities and events), access to news Core, Libraries, Archives, Museums, Heritage Core, Digital, Voluntary work, Charitable giving Core, TV/Free time, Community engagement, Sport motivators Repeat Q1 0 1 9 months 12 months 2 3 15 months 4 18 months 5
Results and Challenges Recruitment Retention Data structure Development burden
Challenges: recruitment % of all face-to-face respondents saying they would be willing to join the web panel 2016/17 Target 55.9% 64.5% 55.9% 64.5% Recruited 56.2% 70.5% 69.5% 86.0% Fresh sample Legacy sample Fresh sample Legacy sample ADULTS YOUTHS
Recruitment: results so far Profile of adult population and those willing to join the web panel Fresh respondents willing to join web Difference Pop- willing to join web panel (pp) Legacy Fresh 2016/17 Population respondents respondents panel Disability No disability 17% 83% 30% 70% 25% 75% 20% 80% 3 3 White Black Asian Other 85% 4% 8% 3% 93% 2% 3% 2% 89% 3% 6% 2% 89% 3% 5% 3% 4 -1 -3 -0 Upper NS-SEC Lower NS-SEC 48% 34% 60% 37% 54% 41% 62% 33% 13 -1 16-24 25-64 65+ 14% 64% 22% 5% 59% 36% 8% 63% 28% 10% 73% 17% -4 9 -5 Visiting a museum 54% 53% 51% 61% 7
Recruitment 1. Refusal counter points 32.5% - no time * 19.4% - lacks internet skills 18.3% - has done enough already 10.8% - doesn t complete questionnaires on the internet 10.4% - other reasons 2. Introduce targeted &/or unconditional incentives Raise incentives for all Just for target groups (e.g. youth, BAME) Just for initial refusals 3. Review/improve interview briefing content 4. Review/improve web panel leaflet
Challenges: Retention 2016/17 recruits 4,000 100% Invited, not completed 90% 3,500 Completed on: 80% 3,000 invitation 70% 2,500 1st (email) reminder 2nd (email) reminder 3rd (postal) reminder 60% 2,000 50% 40% 1,500 30% 1,000 20% 500 10% Still open 0 0% Q1 Q2 Q3 Q4 Q5
Retention Under investigation! Aim to address overall numbers and representation of general population Planned analysis includes: experiment carried out with SMS reminders link between retention and reported interest in the questions
Challenges: Data structure A challenging dataset Stakeholders with very different analytical needs and capabilities Interested in: Data from all quarters vs. one quarter per year Different comparisons: within year comparisons, e.g. Year 1, 1st cohort, Q1 vs Q2 between year comparisons, e.g. 1st cohort, Q1 Year 2 vs Q1 Year 3 between cohort comparisons, e.g. Year 1, 1st cohort, Q1 vs Year 2, 2nd cohort Q1 ...many other permutations! No dedicated analysts vs a full analytical team Different software
Challenge: Development burden Year 3 3rd cohort (2018/19) Q1 Q0 Q3 Q4 Reg Q2 Q1 Q1 Q0 Q3 Q4 Q0 Q3 Q4 Reg Q2 Reg Q2 Q5 Q7 Q8 Q6 Q5 Q5 Q7 Q8 Q7 Q8 Q6 Q6 2nd cohort (2017/18) Q10 Q9 Q12 Q11 Q12 Q11 Q10 Q9 Q9 Q11 Q12 Q10 1st cohort (2016/17) Year 2 2nd cohort (2017/18) Q1 Q0 Q3 Q4 Reg Q2 Q1 Q1 Q0 Q3 Q4 Q0 Q3 Q4 Reg Q2 Reg Q2 Q5 Q7 Q8 Q6 Q5 Q5 Q7 Q8 Q7 Q8 Q6 Q6 1st cohort (2016/17) Year 1 1st cohort (2016/17) Q1 Q0 Q3 Q4 Reg Q2 Q1 Q1 Q0 Q3 Q4 Q0 Q3 Q4 Reg Q2 Reg Q2 Content: Ipsos Mori, November 2017
Development burden: solutions Agree a data structure that minimises need for repeat processing (i.e., data sets should be archivable) Automate/structure checking as much as possible e.g., guided QA logs, RAP Develop quarterly questionnaires which apply to all cohorts (not as simple as it sounds!)
What next? Consider/implement strategies to improve recruitment and retention Harmonise questionnaires Get the datasets Produce statistics! Annual reports Impact of DCMS sector engagement on wellbeing Impact of survey mode face to face vs. phone vs. online
Any questions? Alex.Bjorkegren@culture.gov.uk
