Understanding Evaluation in Public Health Practices

Hamira Sultan, Specialty Registrar (Public Health) Public Health Practitioner Course 24.07.2012

Objectives Have a sound understanding of the principles of evaluation Be able to describe different types of evaluation methodology Be able to design your own evaluations for services you implement

Outline Group work what is evaluation? Why do we do it? What to measure? Some examples from my own work A chance to think through evaluations yourself

Group work What do we mean by evaluation? WHY do we do it?

Why do we measure? The systematic collection of information about the activities, characteristics, and outcomes of programs/ services/ activities to make judgements about the program, improve program effectiveness, and/or inform decisions about future programming (MQ Patton, 1997) We spend tax payers money on health and social care Justify why we are investing resource Need to know what kind of difference we are making Not enough to just say we are doing a good job

What do we measure? Structure Process Output Outcomes EVALUATION

When do we measure? Remember to measure improvement, must have a baseline Throughout a programme .FORMATIVE evaluation At the end of a programme ..SUMMATIVE evaluation Retrospective vs Prospective BUT whichever type you do, you must consider evaluation AT THE START of a programme If a programme has already started/ finished and you realise you wanted to measure something, you ve missed the boat!

What sort of data? First question what do you need to measure to make a judgement about the value of a service/ intervention? Some data is already readily available at PCT/ LA level Mortality from specific conditions Alcohol hospital admission data Drug treatment completion Long term conditions QOF data from GPs Demographic data (from Census) Some data you will need to collect yourself ..if so, how? Numerical? Qualitative? Costs (inc resources)? Mixture? Is there a cost incurred for doing the evaluation? Be realistic about your end point What about confounders?

Study designs for evaluation Ecological Before and after Cohort Controlled PICO Population Intervention/ service Control (not always available) Outcome (randomised or non randomised) Case studies

Examples of evaluations Qualitative evaluation of experience of telehealth pilot (patients with COPD and staff) Willingness to Pay study carers of people with dementia Measuring quality of life and wellbeing in carers across Sandwell

How would you evaluate these interventions/ services? You are putting in place a school based programme to tackle childhood obesity. How would you evaluate the success of cooking workshops for children and parents? You are the prevention lead for alcohol in your borough. How would you evaluate the success of a media campaign? You are trying to increase seasonal flu uptake in frontline health care staff. How would you evaluate the success of using vaccine champions ?

Key messages Evaluation is critical in any public health work Evaluation must be considered at the START of a project Resources (cost and staff) for evaluation must be included Make use of information analysts Dept for Education, Office of National Statistics, Public Health observatories, HPA Epidemiological Units