Improving Performance in Behavioral Health Projects: Tips and Mistake Avoidance

Performance Improvement Projects Tips and Common Mistakes Behavioral Health Concepts CAL-EQRO for MHP and DMC-ODS By Amy McCurry Schwartz , Saumitra SenGupta and Rama Khalsa

Common PIP Mistakes Lack of facts to support the problem Lack of data Support your position Lack of analysis of the problem Analyze existing data to identify targets Indicators/Interventions do not line up with the identified problem

Avoiding Common PIP Mistakes Identify a problem area or opportunity for improvement in treatment or process, but do not take it to the bank without first reviewing your facts and data Include information from clients, families, staff and community partners Client, families, community partner and staff input should be obtained on problem What do they see as the barriers to better care? Why do they think the problem is happening? What ideas do they have to fix it?

Avoiding Common PIP Mistakes Do a thorough data review before beginning the PIP County specific data Review local data linked to your challenge/problem Review system procedures and process linked to your challenge/problem National data Helpful background, benchmarks and possible interventions SAMHSA, NIDA, NIMH, NIATX, AHRQ, NCQA, ASAM, NSDUH, etc

Avoiding Common PIP Mistakes Set up a solid foundation to identify the issues leading to the problem A special survey or focus group may be necessary to understand the full extent of the issues surrounding the problem area Key indicators will flow from a solid analysis Collect baseline data for your indicators Identify effective and meaningful interventions/actions to mitigate the problem.

The BEST PIP Concepts have. Knowledge of what might have changed in your environment to contribute to or improve the problem (examples, opening or closing problems due to budget changes or new medications to help improve recovery for specific SUDs or diagnoses) A clear study question based on thorough data collection and analysis of the problem and potential causes

CMS Protocol: Validating Performance Improvement Projects OMB Approval No. 0938-0786 EQR PROTOCOL 3: VALIDATING PERFORMANCE IMPROVEMENT PROJECTS (PIPs) A Mandatory Protocol for External Quality Reviews (EQR)

CMS Protocol: Validating Performance Improvement Projects 10 Protocol Steps: 1) Select the study topic(s) 2) Define the study question(s) 3) Use a Representative and Generalizable study population 4) Select the study variable(s) 5) Use sound sampling techniques

CMS Protocol: Validating Performance Improvement Projects 10 Protocol Steps Continued: 6) Reliably collect data 7) Analyze data and interpret study results 8) Implement intervention/improvement strategies 9) Plan for real improvement 10) Achieve sustained improvement

Key Issues to Remember in PIP Design *Preparation and ground work are important for PIP success train staff on why the PIP is important and what their role is *Training is important but is not an intervention. The activity you are training for is the intervention, such as a new treatment program or new process to improve access. *Create a baseline data set to measure against for improvement

Key Issues to Remember in PIP Design *Measure similar time periods (like quarterly) to understand the changes occurring because of your interventions. *A PIP IS NOT ACTIVE UNTIL ONE INTERVENTION has started.

PIP Problems and Solutions Problem Solution No local data to show that the issue is a local problem not just a national one. Develop a local data source to confirm it is a local as well as national problem using claims, CalOMS, CSI, MMEF, Satisfaction Surveys, focus group, CHIS, etc Need help understanding if this is good concept for PIP and how to evaluate it Call or email your assigned review leader who can help you with the process 855- 385-3776

PIP Problems and Solutions Problem Solutions Sampling approach is not statistically sound and creates bias. Check with review staff or professional with statistical background to get consultation before using sampling Data plan is not clear to staff Training and buy-in from staff very important to PIP success. Importance of PIP activities not understood by stakeholders so poor cooperation with data access Incomplete or inaccurate data can compromise PIP so this is important to clarify up from with partners and have back up plan for problems. Monitoring data quality is so important to find problems early.

PIP Problems and Solutions Problem Solutions Interventions not working, can I add a new intervention and extend the PIP Data displayed in confusing fashion and not shared with stakeholders Yes an additional or refined intervention based on learnings can extend the PIP Display data in way that all stakeholders can participate in learnings from PIP and possible changes in strategies for improvement.

HELPFUL TIPS MHPs & DMC-ODS teamwork Use the PIP library at the CalEQRO.com web site to see if other counties have similar problems and have PIPs to evaluate Consult with county and BHC staff to adapt to your local challenges; Consider getting outside help with using statistical significance tests if no internal resources Give concept only draft PIP to BHC reviewer early to make sure there are no major problems.

HELPFUL TIPS Resources to Consider There are many PIPs on timeliness, access, no-shows, engagement of special populations, improvements in care using quality metrics. Consult with BHC staff, and other quality resources for Behavioral Health for ideas. Make sure you have data support to accomplish your planned PIP analysis and activities; Picking a problem program staff care about enhances cooperation and completeness of data collection; Do updates to staff to keep up morale and buy-in to improving the PIP problem, process or removing a barrier;

Framework Create an objective, empirical foundation for your PIP - start the process with an exploratory analysis verify and quantify the problems (when does it occur, how often, how severe is it, what does the problem impact, etc. Indicators are the data you measure success by Interventions are actions to impact the problem or improvement you want to make. Understand how they link together! AND how they link to the study question. Don t make the PIP overly complex, start small and if successful bring to scale. Make the process manageable so it can be done well.

Continuous Quality Improvement How not to Assess: I think we have an issue with X . Oh, ok, let s make that a PIP. How to begin to Assess: I think we have an issue with X. Why do you think there is an issue? Well, Dr. Jones says people are never here for their scheduled appointment. OR I keep seeing the same patients back in 10 days after discharge. OR Clinic X s patients ANSA scores don t seem to be improving . Not yet - A PIP doesn t start here.

Assess Ask: Do we have a way to measure that? What data do we have that would show us if there is in fact an issue? Do we know if our issue is out of whack with other similarly sized counties? Do we know if this is problem nationally? What are the national averages for this problem/issue/condition? Is there a norm we are not meeting? Is this a problem for our county? After looking at these questions and answering them, if the MHP or DMC-ODS feels there is an issue, a baseline measurement must be established.

Data There is a problem! FY 10/11 FY 11/12 FY 12/13 FY 13/14 Average Quarterly Percent Meeting Benchmark (Adult Only) % Avg Days % Avg Days % Avg Days % Avg Days Target Meet Meet Meet Meet 20 33 27 36 Benchmark 1 Admit to 1st Face to Face 14 days 57.2% 24.5% 38.4% 23.2% 42 52 60 64 Benchmark 2 Admit to 1st Psychiatric/Med Appt 28 days 44.2% 27.8% 25.2% 20.8% 18 19 20 22 Benchmark 3 - Intake to 2nd Face to Face 30 days 82.0% 82.3% 64.4% 76.3% Benchmark 4 Hospital discharge to 1st Psychiatric/Med Appt 30 days 72.1% 67.5% 65.6% 71.6% 29 34 36 26 13 17 28 19 Benchmark 5 Hospital discharge to 1st Face to Face 7 days 65.2% 58.4% 44.2% 48.3%

Formulate a Study Question Does creating a Care Coordination Team with strategies to engage and provide timely access to outpatient services increase engagement and mental health treatment and increase overall client satisfaction?

Plan The MHP or DMC- ODS should gather input from stakeholders to explore solutions to the problem. What barriers exist? Can the MHP effect those barriers? How will the MHP/ DMC-ODS measure their effect? Not yet - A PIP hasn t started yet.

Indicators Describe Performance Indicator Baseline for Performance Indicator Numerator Denominator Goal Timeliness to first face to face appointment at RST Number of episodes where clients had their initial face-to-face service at the RST s within 14 days of referral to RST Total number of referrals to the RST s where the client received a service 1 24.8% 50% Timeliness to first outpatient psychiatric service Number of episodes where clients had their first outpatient psychiatric service at the RST s within 28 days of referral to RST Total number of referrals to the RST s where the client received a service 2 21.5% 50% Number of acute discharges with a face-to-face service at the RST s within 7 days of acute discharge Timeliness from Acute care discharge to first face- to-face outpatient appointment Number of acute discharges for RST clients 3 38.5% 60% Timeliness from acute care discharge to first psychiatric medication appointment Number of acute discharges with an outpatient psychiatric service at the RST s within 30 days of acute discharge Number of acute discharges for RST clients 4 62.9% 80% Number of episodes where clients had a no-show or cancellation prior to first outpatient appointment No show and cancellations rates prior to first appointments Total number of referrals to the RST s 5 3.7% TBD

Indicators Baseline for Performance Indicator Describe Performance Indicator Numerator Denominator Goal 6 % of referred clients that engage in RST services Number of clients referred who receive a service from the RST s Total number of referrals to the RST s 62.8% 80% % of clients that are hospitalized while waiting for 1st appointment (includes clients that eventually received outpatient services and clients that never showed for services) Number of RST clients referred who had an inpatient hospitalization in between being authorized for services and their first face to face appointment Number of clients referred to the RSTs 7 3.4% 2% Scores on the General Satisfaction domain of the CPS for RST clients meeting benchmark #1 and those not meeting benchmark #1 Total number of RST clients who completed the CPS and meet benchmark #1 and those not meeting benchmark #1 Timely Access=4.51 Not timely= 4.03 8 Level of satisfaction TBD Number of episodes where clients had an engagement service prior to their first appointment Total number of referrals to the RST s 9 10.4% 95% % of referred clients that receive engagement services prior to first face to face appointment Number of clients discharged from the RSTs to a health provider/lower level of care Number of clients discharge from the RSTs in the FY 10 TBD TBD % discharged to health provider/lower level of care for behavioral health treatment

Improve Identify interventions (Do not start a PIP with the intervention and then try to make it work) New treatment we want to try New training we want to try The MHP or DMC- ODS should decide what interventions they want to use to try to improve the issue after reviewing the causes and barriers. Then put those interventions in to place. A PIP has started.

Interventions Specific Intervention Barriers/Causes Addressed Number of Intervention Correspondin g Indicator Date Applied Hire additional staff to create a Community Care Team (CCT) Staff Resources/Capacity/Engag ement 1 1-10 RSTs to implement orientation groups to complete intake appointments to facilitate more timely access to services Staff Resources/Engagement 2 1/8 9/1/2015 Engagement 3 1/6/8/9 7/1/2015 CCT will outreach and provide engagement services to clients prior to their first appointment Engagement 4 1/6/8/9 9/1/2015 CCT staff to call the client within a 14 days of access opening the episode Increase documentation of engagement services provided prior to first appointment Measurement/Engagement 5 1/6/9 7/1/2015

Interventions Specific Intervention Barriers/Causes Addressed Number of Intervention Corresponding Indicator Date Applied 6 Increasing medical staff hours at each RST Staff Resources/Engagement 2/4/8 9/1/2015 When the RST is notified that a client is hospitalized, the CCT/RST will engage with the client to get them into services. Engagement 7 3/4/7/8 9/1/2015 8 Increase documentation of no show and cancellations Measurement 5 7/1/2015 CCT will assist stable clients to transition to a PCP or lower level of care Capacity 9 8/10 7/1/2015 CCT will serve as the lead contact for clients who are touched by the Navigator program in the emergency rooms and jails Engagement 10 1/6/8/9 10/1/2015 Discharge codes will be added to the EHR to track clients discharging to lower levels of care (i.e. Primary Care and GMC providers). Measurement/Capacity 11 10

Contact Info Amy McCurry Schwartz, MHSA MO Project Director EQRO Consultant CA 855-385-3776 x103 Amy.mccurry@bhceqro.com Behavioral Health Concepts, Inc Phone: (855) 385-3776 Email: info@bhceqro.com Website: www.caleqro.com YouTube channel: Behavioral Health Concepts Rama Khalsa, PhD Director, Drug Medi-Cal EQRO 855-385-3776 x136 rama.khalsa@bhceqro.com