Using Data for Effective Program Management and Improvement

Measurement and Data 101 Series Using Data for Program Management Wednesday November 8, 2023 | 5.30 PM 7.30 PM

Agenda Introduction to AAHI Mission, Goals and Services (Samila Aryal Bhattarai) Data for Program Management 101 Reviewing your Program Plan Bringing Visuals into your Program Management Tracking data against goals Using data for quality monitoring and improvement Using Qualitative data to monitor program quality and impact

Asian American Health Initiative (AAHI) Mission To improve the health and wellness of Asian American communities in Montgomery County by applying EQUITY, COMMUNITY ENGAGEMENT, and DATA-DRIVEN approaches.

AAHIs Asian American Center of Excellence (AACE) The goal of the AACE is to support Asian American-serving CBOs and FBOs in developing and increasing their capacity to successfully provide health, wellness, and social support programs and services. AACE services include: Micro-grants for AAPI-serving organizations to build organizational capacity Curated email digest for AAPI leaders Networking opportunities for leaders of AAPI-serving organizations Data roundtables for AAPI leaders Data collection and analysis workshops One-on-One data strategy consultations

Join the AACE Email Digest! Link to join: https://lp.constantcontactpages. com/su/1J8XPmW/AACE Or scan the QR Code!

Workshop Evaluation Please be sure to complete the program evaluation! Link to evaluation: Or scan the QR Code!

Overview and Learning Objectives Overview How to use simple visuals to show progress towards numeric goals How to visually display data to monitor program quality and data completeness Using satisfaction surveys and pre-post questionnaires for client feedback and measuring change.

Reviewing your Program Plan or Scope of Work Using your data for program management involves measuring your program against specific goals and planned activities. This requires your goals and objectives to be SMART. Specific S Who and how M Measurable Requires a numeric goal Achievable A Should be realistic given your capacity R Relevant Needs to be aligned with the work you are trying to accomplish T Timebound Needs a deadline

Reviewing your Program Plan or Scope of Work Silver Cloud Senior Center serves adults over 50 years old, providing a variety of social activities, exercise classes, a weekly shared meal, and education classes. They have about 250 people who regularly attend the center from a variety of backgrounds. They have recently allocated some fund to expand the number of adults that they serve and plan to add a second shared meal each week, as well as starting a weekly coffee and chat session. GOAL: Serve 50 new seniors over 50 years old at the senior center in 2024 GOAL: Provide 2,500 additional meals for seniors over 50 years old in 2024 GOAL: Host 52 coffee and chat sessions in 2024

Other potential measures Budget compliance Comparing success Data completeness Tracking improvement activities Client surveys What else would you measure?

Bringing visuals into your program management Why use visuals Quickly see program status Makes the data more accessible to a wider audience

Using visuals well Why use visuals Quickly see program status Makes the data more accessible to a wider audience If all you want to present is end of year goal accomplishment, the second chart does all you need it to.

Tracking linear data against a goal run charts Any numeric data can be tracked using a run chart A simple run chart presents data over time More complex run charts can present data versus a goal or median. They can include trend lines or can be presented in terms of % goal completed. Goals can be presented as a monthly or annual goal.

Simple Run Charts Simple run charts plot totals against time. Totals can be by unit of time, or as in this case presented cumulatively Cumulative totals are useful when progress is not evenly distributed by month for example when the majority of your clients are seen on a monthly basis

Plotting against goals Simple run charts plot totals against time. Totals can be by unit of time, or as in this case presented cumulatively

Plotting against a moving numeric goal Using either bar or line graphs, measures with goals that move each month are not best suited to this type of chart.

Plotting against a moving numeric goal Separating your data into 2 charts can create more effective visuals to help you understand what is going on in your program. In this case, if you wanted to, you could plot a goal line representing the % of goal you want to achieve.

Showing your data as under or over your goal Works well for financial tracking or measures that can both exceed or not reach a target on a regular basis Is not usually a good choice of visual for measures that cannot exceed the goal unless you need to particularly draw attention to falling below the goal.

Using data and visuals for program quality Using visuals to monitor the quality of your programs enables you to identify potential challenges as well as opportunities for improvement. What do you want to focus on What constitutes success Start with limited number of elements you want to measure What constitutes success

Comparing success of different activities Simple charts are a good way of identifying variation between similar activities. For example Comparing popularity of events Comparing attendance rates at classes/activities by location or subject matter Comparing staff effectiveness eg number of intakes per day

Tracking data completeness You can use charts to monitor the completeness of your data. Ensure you have the data you need to track your goals Follow up on unmet goals is the goal not met, or the data not available?

Visuals for quality improvement Run charts are an effective way to monitor any quality improvement efforts. You can use a goal line if set a target Plotting the median on your run chart enables you to identify sustained improvement.

More complex chart types Radar Charts Treemap

Combining visuals Using tools such as Microsoft word and google documents, alongside excel or google forms, you can create combination reports with your visuals to pull your whole data strategy together In a future workshop we will be looking at this in more detail and looking at ways to use your data to tell your story.

Using qualitative data to monitor program quality and impact Focused surveys to get additional feedback on services Things to consider Keep them short Use simple language Anonymous versus Identified Paper, Electronic, mass survey or targeted

Satisfaction Surveys Great way to get general feedback on your services. Considerations: One time input or tracking satisfaction over time Scorable or free-text responses Likert scale: (1) Strongly Disagree, (2) Disagree, (3) Neither Agree or Disagree, (4) Agree, (5) Strongly Agree. Free-text questions should be open ended How many surveys do you need? Do you want to be able to compare results between different client groups, or will you aggregate at the organization level?

Example Satisfaction Survey Questions How easy is it to access our services. (1) Very difficult, (2) Difficult, (3) Neither difficult or easy, (4) Easy, (5) Very easy Staff have the knowledge and resources to respond to my needs (1) Strongly disagree, (2) Disagree, (3) Neither agree or disagree, (4) Agree, (5) Strongly agree How likely are you to recommend our organization to family or friends (1) Very unlikely, (2) Unlikely, (3) Neither likely or unlikely, (4) Likely, (5) Very likely How satisfied are you with our range of activities (1) Very unsatisfied, (2) Unsatisfied, (3) Neither unsatisfied or satisfied, (4) Satisfied, (5) Very Satisfied This question invites a follow up open-ended question what other activities would you like to see?

Pre-Post Surveys Great tool for measuring change in response to services and especially education and training. Best used for distinct units eg a 12 session program, or something that has a clear start and finish, but can also be used to measure the impact of a program over a longer period. Understand your intended impact eg growth of knowledge, reduced loneliness, increased activity levels. Ensure questions are specific to your intended impact Scoring your surveys Likert scale, quiz, stages of change Stages of change: Pre-contemplation I am not considering changing my behavior Contemplation I am considering making changes Preparation I am planning to make changes Action I have made changes Maintenance I am maintaining changes Relapse I have gone back to my old behaviors

Example Pre-Post Survey questions I have increased my knowledge about healthy foods (1) Strongly disagree , (2) Disagree, (3) Neither agree or disagree, (4) Agree, (5) Strongly agree I have food when I am hungry Never, sometimes (1-2 days per week), mostly (3-4 days per week), often (5-6 days per week), Always I include exercise in my daily life I am not considering increasing my activity level, I am considering increasing my activity level, I am making changes to increase my activity level, I have increased my activity levels , I am maintaining an increased activity level, I have returned to my previous activity levels. Subject matter quiz

Resources Simple run charts: https://www.youtube.com/watch?v=WIDSUXTLVE4

Workshop Evaluation Link to evaluation: Or scan the QR Code!

Thank you! Bethany Sanders Sanjana Quasem Consultant for Primary Care Coalition (PCC) bethanysanders@haii.co.uk Asian American Health Initiative (AAHI), MCDHHS Sanjana.Quasem@montgomerycountymd.gov Lisa Han Samila Aryal Bhattarai Asian American Health Initiative (AAHI), MCDHHS Samila.AryalBhattarai@montgomerycountymd.gov Primary Care Coalition (PCC) Lisa_Han@primarycarecoalition.org