Innovative Strategies for Evaluating Healthcare Implementation Processes

Mapping Dynamic Implementation Strategies In Complex Settings: An Analytic Protocol for Process Evaluation 4thBiennial Society for Implementation Research Collaboration (SIRC 2017) September 8, 2017 Alexis K Huynh, PhD, MPH

Acknowledgement Co-authors: Erin Finley, Melissa Farmer, Bevanne Bean-Mayberry, Alison Hamilton Funding provided by VA QUERI (QUE 15-272) Disclaimer: The views expressed in this presentation are those of the authors and do not reflect the position or policy of the Department of Veterans Affairs or the United States Government

Background Implementation Strategies Methods or techniques used to enhance adoption, implementation, and sustainability of a program or practice1 Constitute the how to component of changing health care practice1 Importance of Implementation Strategies Overcome barriers to realize the benefits of evidence-based care in routine practice Assessment of approaches to implementation that are more or less effective in different settings

Background Greater precision in reporting implementation strategies moves the science forward Persistent challenges in describing implementation strategies Varying terminology Multi-component strategies Variable utilization (dose, frequency) Measurement and replication Need innovative approaches to evaluate multiple strategies across project phases and settings

EMPOWER QUERI Strengthen VA women s health organizational capacity for innovation in patient-centered care Three implementation projects addressing women s health needs Replicating Effective Program (REP) guides evaluation in project phases

Replicating Effective Programs (REP) 2 3

Cardiovascular (CV) Toolkit Multi-site implementation of a patient- and provider-facing toolkit: Increase CV risk identification & documentation Enhance patient-provider communication Engage women Veteran patients in appropriate services CV Toolkit components: CV Patient Screener CPRS Template Referral to services including the Gateway to Healthy Living Group & follow-up phone calls Provider and patient education, shared decision making

Methods: CV Toolkit Patient Flow

Objective Describe a protocol for mapping REP implementation strategies to outcomes facilitating Prospective simulation-based implementation plan refinement Longitudinal evaluation

Methods: Strategy Matrix Apply Proctor s recommendations to characterize and map strategies by specifying a matrix: Actions taken Discrete strategies Strategy conceptual categories REP phase(s)

Methods: Matrix Linking REP Phase Activities, Implementation Strategies &Components REP Pre-implementation a. Develop package b. Distribute & discuss package c. Identify champion ERIC Implementation Strategy Compilation e.g. Build a coalition Provide local technical assistance Conduct educational meetings Proctor s Rubric Actor Action Action Target Temporality Dose Outcome Justification

Methods: Strategy Matrix Application of Proctor's Rubric Recommendations In Constructing Implementation Strategy Matrix Number Strategy Strategy Conceptual Cluster Pre-Condition Pre-Implementation Implementation Maintenance & Evolution S1 Involve executive boards Develop stakeholder interrelationships Adapt pilot package for WV Review potential sites with partners Send trainers for Gateway training Re-tailor as needed with spread S2 Build a coalition Develop stakeholder interrelationships Engage selected site Identify local champions Review business case S9 Develop formal implementation blueprint Use evaluative and iterative strategies Local tailoring with training & technical assistance of toolkit package Provide and elicit feedback to make modifications to implementation process S15 Provide local technical assistance Provide interactive assistance Discuss educational needs of WH teams at sites Launch large meeting at site; Additional training for each provider on the CPRS template; Assess additional need for provider training and technical assistance

Methods: Matrix Linking REP Phase Activities, Implementation Strategies &Components Outcome = CV Toolkit Adoption Strategies Provide local technical assistance Launch CV toolkit at site; Train PCPs on CPRS template Temporality Pre-Implementation

Methods: CV Toolkit Patient Flow

Results-Mapping Strategies Strategy Initiated in REP Phase Used once Use recurringly 1. Conduct local needs assessment 2. Inform local opinion leaders 3. Develop educational materials 4. Promote adaptability 5. Build a coalition 6. Conduct educational meetings 7. Tailor strategies 8. Provide local technical assistance 9. Involve executive boards 10. Identify and prepare champions Pre-condition Pre-condition Pre-condition Pre-condition Pre-condition Pre-condition Pre-condition Pre-condition Pre-condition Pre-implementation 11. Assess for readiness and identify barriers and facilitators (local resources) 12. Develop formal implementation blueprint 13. Audit and provide feedback 14. Purposely reexamine the implementation 15. Conduct cyclical small tests of change 16. Develop an implementation glossary Pre-implementation Implementation Implementation Implementation Implementation Maintenance & Evolution

Results-Mapping Strategies

Results-Simulated Implementation Scenarios

Discussion Mapping & delineating implementation strategies, activities & outcomes Assess strategies deployed over time (once vs recurringly) Iterative and interactive process that refine implementation plans/activities Identify key strategies to improve outcomes

Practice Implications Operationalizing and specifying implementation strategy contexts Advance understanding of how simulation can be used to refine implementation blueprints during early stages Exciting developments include using participatory systems dynamics modeling to improve implementation planning

Questions & Comments Thank you for your attention

Selected References 1. Proctor, E.K., Powell, B.J. and McMillen, J.C., 2013. Implementation strategies: recommendations for specifying and reporting. Implementation Science, 8(1): 139. 2. Kilbourne, Amy M., et al. "Implementing evidence-based interventions in health care: application of the replicating effective programs framework." Implementation Science 2.1 (2007): 42. 3. Simpson, Kelly M., et al. "Tool for evaluating research implementation challenges: a sense-making protocol for addressing implementation challenges in complex research settings." Implementation Science 8.1 (2013): 2. 4. Powell, Byron J., et al. "A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project." Implementation Science 10.1 (2015): 21. 5. Sogolow ED, Poll LSKLS, Neumann MS. Strengthening HIV prevention: application of a research-to-practice framework. AIDS Education and Prevention. 2000;12:21-32. 6. Proctor EK, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health Ment Health Serv Res. 2011;38: 65-76. 7. Waltz TJ, Powell BJ, Matthieu MM, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implementation Science. 2015;10(1):109. 8. Zimmerman, L., Lounsbury, D. W., Rosen, C. S., Kimerling, R., Trafton, J. A., & Lindley, S. E. (2016). Participatory system dynamics modeling: Increasing stakeholder engagement and precision to improve implementation planning in systems. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 834- 849.

Contact Information Alexis K Huynh, PhD, MPH Center for the Study of Healthcare Innovation, Implementation & Policy (CSHIIP) VA Greater Los Angeles Healthcare System Los Angeles, CA 90073 310-478-3711 x 49862 alexis.huynh@va.gov

Back-up Slide-CV Toolkit Components

Methods: Strategy Matrix Application of Proctor's Rubric To Implementation Strategy Matrix Number Strategy Strategy Conceptual Cluster Pre-Condition Pre- Implementation Implementation Maintenance & Evolution S1 Involve executive boards Develop stakeholder Adapt pilot package for WV Review potential sites with partners Send trainers for Gateway training Re-tailor as needed with spread interrelationships S2 Build a coalition Develop stakeholder interrelationships Engage selected site Identify local champions Review business case S9 Develop formal implementation blueprint Use evaluative and iterative strategies Local tailoring with training & technical assistance of toolkit package Provide and elicit feedback to make modifications to implementation process

Back-up Slide-Results Table 2b. Frequency of Strategy Implementation by Conceptual Cluster Number Involve executive boards Build a coalition Inform local opinion leaders Identify and prepare champions Develop an implementation glossary Conduct local needs assessment Conduct cyclical small tests of change Assess for readiness and identify barriers and facilitators (local resources) Develop formal implementation blueprint Audit and provide feedback Purposely reexamine the implementation Conduct educational meetings Develop educational materials Tailor strategies Promote adaptability S16 Provide local technical assistance Strategy Strategy Conceptual Cluster Develop stakeholder interrelationships Develop stakeholder interrelationships Develop stakeholder interrelationships Develop stakeholder interrelationships Develop stakeholder interrelationship Use evaluative and iterative strategies Use evaluative and iterative strategies Use evaluative and iterative strategies Use evaluative and iterative strategies Use evaluative and iterative strategies Use evaluative and iterative strategies Train and educate stakeholders Train and educate stakeholders Adapt and tailor to context Adapt and tailor to context Provide interactive assistance S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15

Results-Mapping Strategies