Clinical Competency Committee Essentials

The Clinical Competency Committee Dean Seehusen, MD, MPH Associate Dean for Graduate Medical Education Professor of Family Medicine

Learning Objectives Upon completion of this session, participants should be able to: Define the minimal membership of the Clinical Competency Committee and who else may be included Describe preparation for a CCC meeting Describe at least two best practices for running a CCC meeting Name at least three factors that significantly contribute to positive or negative functioning of the CCC

Agenda Background on resident evaluation The benefits of the CCC Structure of the CCC CCC meetings Additional CCC tips and thoughts

Resident Evaluation

Competency: Who Decides and How Traditionally, competency was assumed at the end of a defined period of training This ignores individual variation among trainees The PD essentially bore the full weight of deciding Disagreements were difficult to resolve The ACGME started a movement toward competency- based evaluation in the 1990 6 core competencies Milestones Entrusted Professional Activities (EPA) in the future?

Origins of the CCC Some programs have always had a CCC, although it may have been called many different things The move to competency based assessment led the advent of many new evaluation tools Milestones and EPAs have only increased the amount of evaluation done The volume of data became overwhelming for the PD to sift through alone The ACGME now mandates that a committee play a key role in competency assessment

ACGME. Clinical Competency Committees 2nd Edition. Available at: https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf

Benefits of the CCC

From the PD Perspective Spreads the workload and responsibility Greater buy-in from faculty Increases credibility and reproducibility of evaluations comes with a larger number of voices at the table Higher quality and quantity of feedback ACGME. Clinical Competency Committees 2nd Edition. Available at: https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf

From the Program Perspective Develops a shared mental model of expectations Insures competency is adequately evaluated and reported Standardizes evaluation between residents Ensures each resident is progressing along a proper trajectory Identifies residents in difficulty Identifies program weaknesses Acts as a form of faculty development ACGME. Clinical Competency Committees 2nd Edition. Available at: https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf

From the Resident Perspective Higher quality and quantity of feedback Performance is measured against a national set of standards (milestones) Increases transparency of evaluations More perspectives lessens the likelihood of a skewed evaluation ACGME. Clinical Competency Committees 2nd Edition. Available at: https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf

CCC Structure

Minimal Requirements The PD must appoint the CCC Three faculty members Written description of responsibilities Advise the PD on resident progress, promotion, graduation, dismissal Review of residents and prepare milestone evaluations semi-annually ACGME. Common Program Requirements. Available online at: https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_2017-07-01.pdf

Size and Constitution The minimum is set (3) but the maximum is not set A program may have more than 1 CCC; for instance based on PGY level or competencies The literature suggests 5-7 members is ideal and more than 10 members becomes counterproductive Diverse groups make better decisions

CCC Structural Options Some specialties have additional rules (in anesthesia, for example, the PD cannot Chair the CCC) Additional physician faculty can come from other programs Other health professionals can be assigned if they have extensive contact with the residents Chief residents that have graduated the core program can be assigned to the CCC Program Coordinators may administratively attend the CCC; any input should only be provided through standard evaluation channels such as 360oevaluations A public member is allowable ACGME. Clinical Competency Committees 2nd Edition. Available at: https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf

CCC Meetings

Preparing for CCC Meetings The PD and CCC Chair, if different, should determine how the CCC will generally function The Chair, should develop a shared mental model with the CCC members covering: The milestones The role of the CCC The CCC process for that program Create a map, or matrix, showing where each milestone is taught and evaluated in the program s curriculum Make note of gaps discovered in the map Note tools that do not seem to add much to the evaluation process

Two Mapping Examples ICU rotation evaluation by faculty Clinic rotation evaluation by faculty IM Milestone ICS1: Communicates effectively with patients and caregivers. IM Milestone PC1: Gathers and synthesizes essential and accurate information to define each patient s clinical problem(s). ICU rotation evaluation by faculty OSCE Mini-CEX Ward rotation evaluation by faculty

Assessing Milestones Assigning one or more members to review certain residents, or certain data, before the meeting will save time The ACGME discourages simple use of statistical data or use of a single data point to assign make determinations If during a given period, not enough data on a milestone has been gathered, hold the prior milestone rating The goal is consensus; the ACGME discourages voting The CCC only advises; the PD has final authority for assessment, promotion and graduation ACGME. Clinical Competency Committees 2nd Edition. Available at: https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf

Additional CCC Duties Determine baseline milestones for fellows accepted into to a program that do not meet the standard eligibility requirements within 6 weeks of matriculation May help create and monitor remediation plans for residents in difficulty Identify potential improvements in the program s evaluation process ACGME. Clinical Competency Committees 2nd Edition. Available at: https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf

Running the Meeting Have enough time Use the same approach for every resident Have a method to ensure all voices are heard Consider starting with the junior person The Chair should speak last The PD should speak the least or not at all Consider designating a Devil s Advocate Have a solid documentation plan Have a solid communication plan ACGME. Clinical Competency Committees 2nd Edition. Available at: https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf

Two Broad CCC Approaches Problem Identification Model (by far the most common) Looking for red flags Most time spent on low performing residents Focus on global performance Provides less effective feedback to learners Developmental Model Data synthesis is very time consuming Focus on comparing performance to milestones Assessment more balanced and nuanced Feedback framed in developmental language Hauer KE, Chesluk B, Iobst W, Holmboe E, Baron RB, Boscardin CK, Cate OT, O'Sullivan PS. Reviewing residents' competence: a qualitative study of the role of clinical competency committees in performance assessment. Acad Med. 2015;90(8):1084-92. PMID: 25901876

CCC Food for Thought

Common Pitfalls Spending a disproportionate time on residents in difficulty Relying too heavily on the least reliable evaluation tools, such as end of rotation evaluations Too little, or too much, data Lack of confidentiality Lack of a solid communication strategy to the resident Groupthink

Groupthink Groupthink is a term first used in 1972 by social psychologist Irving L. Janis A psychological phenomenon in which people strive for consensus within a group. In many cases, people will set aside their own personal beliefs or adopt the opinion of the rest of the group. People who are opposed to the decisions or overriding opinion of the group as a whole frequently remain quiet, preferring to keep the peace rather than disrupt the uniformity of the crowd. Janis, I. L. "Groupthink". Psychology Today. 1971;5 (6): 43 46, 74 76.

Features of Groupthink Illusions of invulnerability lead members of the group to be overly optimistic and engage in risk-taking. Unquestioned beliefs lead members to ignore possible moral problems and ignore consequences of individual and group actions. Rationalizing prevents members from reconsidering their beliefs and causes them to ignore warning signs. Stereotyping leads members of the in-group to ignore or even demonize out-group members who may oppose or challenge the group's ideas. Self-censorship causes people who might have doubts to hide their fears or misgivings. "Mindguards" act as self-appointed censors to hide problematic information from the group. Illusions of unanimity lead members to believe that everyone is in agreement and feels the same way. Direct pressure to conform is often placed on members who pose questions, and those who question the group are often seen as disloyal or traitorous. Janis, I. L. "Groupthink". Psychology Today. 1971;5 (6): 43 46, 74 76.

Influences on Group Process Member characteristics Member homogeneity Size Understanding of the work Group leader Information sharing procedures Time pressures Hauer KE, Cate OT, Boscardin CK, Iobst W, Holmboe ES, Chesluk B, Baron RB, O'Sullivan PS. Ensuring Resident Competence: A Narrative Review of the Literature on Group Decision Making to Inform the Work of Clinical Competency Committees. J Grad Med Educ. 2016;8(2):156-64. PMID: 27168881

Competency Champions Department of Surgery, University of New Mexico, Albuquerque Six faculty designated Competency Champions one of each of the ACGME Core Competencies Functions of experts on those milestones Help write remediation plans for residents not functioning at the expected level in their competency Ketteler ER, Auyang ED, Beard KE, McBride EL, McKee R, Russell JC, Szoka NL, Nelson MT. Competency champions in the clinical competency committee: a successful strategy to implement milestone evaluations and competency coaching. J Surg Educ. 2014;71(1):36-8. PMID: 24411421

Some Last Thoughts Assess how well the CCC is functioning periodically Improving the CCC can be selected as an annual enhancement project for the PEC There are no term limits defined by the ACGME but you may consider them useful in your program Fresh blood brings new perspectives Serving on the CCC is a form of faculty development that would benefit all of your faculty Turnover means loss of expertise There is a cost to educating new members

The CCC Guidebook CCC Guidebook from the ACGME available at: https://www.acgme.org/Portals/0/ACGMEClinicalComp etencyCommitteeGuidebook.pdf A CCC knowledge quiz A step-by-step CCC creation guide An annotated bibliography Case studies ACGME. Clinical Competency Committees 2nd Edition. Available at: https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyCommitteeGuidebook.pdf

Learning Objectives Upon completion of this session, participants should be able to: Define the minimal membership of the Clinical Competency Committee and who else may be included Describe preparation for a CCC meeting Describe at least two best practices for running a CCC meeting Name at least three factors that significantly contribute to positive or negative functioning of the CCC

Bibliography

Bibliography ACGME. Clinical Competency Committees 2ndEdition. Available at: https://www.acgme.org/Portals/0/ACGMEClinicalCompetencyComm itteeGuidebook.pdf ACGME. Common Program Requirements. Available online at: https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/C PRs_2017-07-01.pdf French JC, Dannefer EF, Colbert CY. A systematic approach toward building a fully operational clinical competency committee. J Surg Educ. 2014;71(6):e22-7. PMID: 24878101 Hauer KE, Cate OT, Boscardin CK, Iobst W, Holmboe ES, Chesluk B, Baron RB, O'Sullivan PS. Ensuring Resident Competence: A Narrative Review of the Literature on Group Decision Making to Inform the Work of Clinical Competency Committees. J Grad Med Educ. 2016;8(2):156-64. PMID: 27168881

Bibliography Hauer KE, Chesluk B, Iobst W, Holmboe E, Baron RB, Boscardin CK, Cate OT, O'Sullivan PS. Reviewing residents' competence: a qualitative study of the role of clinical competency committees in performance assessment. Acad Med. 2015;90(8):1084-92. PMID: 25901876 Ketteler ER, Auyang ED, Beard KE, McBride EL, McKee R, Russell JC, Szoka NL, Nelson MT. Competency champions in the clinical competency committee: a successful strategy to implement milestone evaluations and competency coaching. J Surg Educ. 2014;71(1):36-8. PMID: 24411421 Chahine S, Cristancho S, Padgett J, Lingard L. How do small groups make decisions? : A theoretical framework to inform the implementation and study of clinical competency committees. Perspect Med Educ. 2017;6(3):192-198. PMID: 28534277