Improving Health and Healthcare Course Review for Year 1 Students

Review of Year 1 Patients & Populations: Improving Health and Healthcare Course occurs in all 3 terms of Year 1 Course Director Brenda Sirovich, MD, MS Course has 60 curricular hours Course was not reviewed previously since it is a new course

Course Objectives

Course Objectives Mapping Since this was a new course, the mapping of course objectives to Geisel competencies was completed recently and seemed appropriate The course director and subcommittee agreed that a few minor tweaks would be beneficial; these are listed on the next slide to be updated in Ilios

Course Objectives Mapping Session Objective New Mapping 31 2 11 31 4 7, 11 31 5 9, 11 31 6 11 34 1 1, 7 34 4 7, 8, 9 35 1 19, 20, 21 35 2 19, 20 35 3 21, 23 35 5 18, 21, 22

Course Objectives Comments The IH&H course was created by merging multiple disciplines, thus the course director reports that the list of 24 course objectives contains objectives that may only apply to one or several portions of the course. To simplify things, the 24 objectives were not presented all at one time, but piecemeal in the relevant modules. Additionally, a separate condensed set of objectives shown on the next slide was published on Canvas. Since this course continues in year 2 with similar objectives, footnotes were added to certain course objectives to indicate: a) that they would be covered over two years; or b) that they would only be addressed in year 2

Objectives published on Canvas

Format of Course & Session Objectives Two sets of objectives are provided to the students as previously discussed Course objectives are written in the correct format Session objectives are provided in the course materials for all sessions Session objectives are written in the correct format Note: one typo was found in objective #2 for the Statistical Inference session in the fall term

Issues of Redundancy We were unable to conduct a keyword search since the curriculum database does not include AY 15-16 at this time (and the course did not exist in AY 14- 15). However, the content of this course is unique with respect to other year 1 courses, thus we don t expect that issues of unplanned redundancy are a problem Students on our subcommittee agreed that redundancy was not an issue with other year 1 courses, however there was some redundancy within the course (primarily reflective assignments in winter term) that was not beneficial

Exploration of Health and Values Health and Values material includes healthcare ethics, cultural awareness, health equity, resilience and compassionate care There were many sessions that included material from the Health and Values theme, for example: The 30,000 Foot View session discussed issues of cultural awareness Variation in Healthcare and Health Across the U.S. discussed issues of health equity Session on Historical Context of Healthcare included information about health equity The Power of Choice session addressed ethical decision making Session about Patient Safety and Teamwork addressed issues of compassionate care Social Determinants of Health and Healthcare included health equity and cultural issues Student presentations during the Making a Difference project allowed students to discuss some of these topics

Summary regarding Objectives Course and session objectives are well-written using measurable verbs Currently there are two sets of course objectives a longer comprehensive list that was distributed in parts, and a short condensed version that was published on Canvas Session objectives are distributed on Canvas in a clear fashion Some redundancy exists in course assignments that did not seem planned or for a particular purpose The course has successfully incorporated health and values content

Course Learning Opportunities Large Group Interactive Sessions 23.25 hrs. (39%) 19 hrs Fall term; 13.25 Winter term; 6.75 Spring term Lectures 12 hrs. (20%) Small Group Conferences 13.5 hrs. (23%) 12 hrs. Fall term; 1.5 hrs Spring term Interprofessional Sessions 9.25 hrs. (15%) 2.75 Winter term; 6.5 Spring term Shadowing Experience 2 hrs. (3%) Optional reviews were offered at various times in the course to aid understanding prior to an assessment

Summary regarding Pedagogy The pedagogy in the course meets the Geisel policy on Active Learning vs. Lecture time, thus no changes are necessary Most course activities facilitate engaged learning and sessions such as Making a Difference are very novel The course provides valuable opportunities for interprofessional learning

Assessment Fall term Quizzes (2) 40% Final exam 60% Spring term Participation and Professionalism 20% Patient Safety Project & Worksheet 30% Make a Difference Project 30% Self-Assessments 20% Winter term Participation and Professionalism 20% Pre-Class Readings and Reflections 20% Personal Health Improvement Project 20% Team Improvement Project 20% Interprofessional Shadowing Experience 10% Monthly Self-Assessment 10%

Assessment for Course Objectives For each course objective, we were able to identify one or more methods in the course that is used to assess the objective. For example, objective #1, Describe the common methods of measurement in clinical medicine and research and how they are used. is assessed during small group discussions and on the quizzes/final exam. Assessment methods seem appropriate for this course

Summary regarding Assessment Due to the fact that each section of the course has unique aspects, assessment activities vary per term and are tailored to course goals; the students on our committee felt this was appropriate Every course objective is being assessed by one or more methods including quizzes, exams, written submissions or student participation

Measures of Quality Step I 2013* 2014* 2015* Mean TRADITIONAL CORE DISCIPLINES Biochemistry 0.20 0.22 0.03 0.15 Biostatistics/Epidemiology 0.40 N/A N/A 0.40 Biostatistics 0.08 0.29 0.19 Genetics 0.18 0.28 0.09 0.18 Gross anatomy/Embryology 0.26 0.14 0.16 0.19 Histology/Cell Biology 0.26 0.23 0.07 0.19 Microbiology/Immunology 0.47 0.39 0.02 0.29 Pathology 0.24 0.20 0.12 0.19 Pharmacology 0.22 0.12 -0.02 0.11 Physiology 0.35 0.25 0.11 0.24 *values reported for core disciplines are SD above the US/Can mean for Geisel mean scores

Measures of Quality Step I 2013* 2014* 2015* Mean SYSTEMS-BASED TOPICS Behavioral sciences 0.26 0.15 -0.1 0.10 Biostatistics, Epidemiology, Pop Health 0.01 0.22 0.12 Cardiovascular system 0.47 0.02 0.16 0.21 Endocrine system 0.39 0.24 0.32 Gastrointestinal system 0.59 0.54 0.2 0.45 General Principles of Foundational Sciences 0.25 0.12 0.19 Hematopoietic/lymph systems 0.18 0.09 0.10 0.12 Immune system 0.53 0.16 -0.07 0.21 Multisystem Processes & Disorders 0.23 -0.18 0.03 Musculoskeletal, skin, CT systems 0.34 -0.02 0.22 0.18 Nervous system 0.21 N/A N/A 0.21 Nervous system and Behavioral Health 0.06 -0.10 -0.02 Nutrition 0.39 0.22 -0.08 0.17 Renal/urinary system 0.21 0.23 0.02 0.15 Reproductive/endocrine systems 0.39 N/A N/A 0.39 Reproductive system 0.39 -0.03 0.18 Respiratory system 0.31 0.18 0.27 0.25 *values reported for core disciplines are SD above the US/Can mean for Geisel mean scores

Measures of Quality Course Evaluation Overall Quality AY 14-15 Overall Quality AY 15-16 Year 1 courses 4.34 4.40 Biochemical and Genetic Basis of Medicine 4.07 4.06 CTO 4.35 4.04 Human Anatomy and Embryology I 3.23 3.41 Physiology-Cardiovascular 3.34 3.41 Physiology-Respiration N/A 2.96 Patients and Populations - Fall On Doctoring not surveyed in Fall scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent]

Measures of Quality Course Evaluation Overall Quality AY 14-15 Overall Quality AY 15-16 Year 1 courses 4.57 4.43 Human Anatomy and Embryology II 3.67 3.94 Immunology 4.35 4.48 Metabolic Basis of Disease 4.19 3.63 Physiology-Renal 3.76 3.52 Physiology-Endocrine 3.77 3.83 Virology N/A 2.60 Patients and Populations - Winter On Doctoring not surveyed in Winter scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent]

Measures of Quality Course Evaluation Overall Quality AY 14-15 Overall Quality AY 15-16 Year 1 courses 3.94 4.10 Basic Science of Microbial Disease 3.59 3.32 Neuroscience 3.22 3.06 Pathology N/A 2.76 Patients and Populations - Spring 3.74 4.10 On Doctoring Spring scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent]

Measures of Quality Course Evaluation scale [1=poor; 2=fair; 3=good; 4=very good; 5=excellent] P&P Fall (87.1%)* P&P Winter (86.2%)* P&P Spring (72.3%)* Overall satisfaction of course 2.96 2.60 2.76 Clarity of learning objectives 3.39 2.81 2.98 Organization of the course 3.04 2.52 2.87 How well the course introduced me to this discipline 3.19 2.49 2.61 Congruence of assessment questions to material emphasized in course 3.82 2.50 3.24 *student participation rate on course evaluation

Fall Student Comments Strengths: Comprehensive notes for Biostats/Epi Module The printed lecture notes were a great teaching tool Like the binder- clearly organized Application sessions were well-received Liked the screening and more clinically relevant sessions much more I liked...the lecture [that was half on stats] and the other half was on a cool, real life application of the these concepts to [epidemics] Students appreciated summaries provided prior to Biostats exams that highlighted important information

Fall Student Comments Areas for Improvement: Required designation for small groups with learners of varying levels the small group is very useful for some students but not for (those with MPH degrees or backgrounds in public health). I think making those sessions optional so that people can decide if they want help or more contact time with the material, then they can. [regarding] small groups- thought that we could've done the problem sets on our own and didn't need 2 hours to basically do them and walk through them many times to teach the few students who didn't understand. I would've rather done them on my own and then had office hours or something to ask questions. Variable quality and preparedness among lecturers The first guest lecturer was terribly [unprepared] Some of the statistics material could be somewhat dry, so I think the lectures would be more well-received if these sessions were more interactive Find ways to accommodate variety of student backgrounds in biostats I already have an MPH so the course was greatly redundant for me and I didn't study for it much. This is a tough class to teach because some students had some exposure to stats, some had masters in stats, and some had no experience to stats.

Fall Student Comments Areas for Improvement: Student would appreciate more class time devoted to public health, health economics, health policy, etc. Dartmouth has a wealth of expertise in health care delivery science. The course would be more interesting if you...bring in [more individuals] who are doing cool, related biostats/epi research Organization of the course could be better communicated It feels scattered...it just has so many different instructors and styles of teaching and topics that it feels a little disjointed. Setting the stage of the course. I think the first lecture gave this big picture, somewhat fluffy and feel-good expectation of this class. However, the subsequent lectures did not meet this vision or expectation and it simply became a biostatistics course.

Winter Student Comments Strengths: Students found the Interprofessional Shadowing to be beneficial and complementary to the course Throughout the course, I really enjoyed the interdisciplinary work we did, the team building exercises in our small group, which I found helped me practice leadership and critical thinking skills in an applied way The shadowing experience was EXCELLENT and likely yielded far more of the desired results (interprofessional appreciation, communication skills, professional development) than any powerpoint. Students enjoyed the TIP and PHIP Projects as it allowed them to focus on personal wellbeing and teamwork: I think it's great that self-care and team improvement and communication are a part of our curriculum first year and I got a lot out of these projects. I actually really enjoyed the personal health improvement plan and that's something I know I need to continue emphasizing to keep myself healthy and happy throughout medical school and residency. Students found Bio/epistats monthly assessments to be helpful refreshers: I think the bio/epistats assignments were helpful to keep skills fresh. They could be improved by having a little mini summary of the content optional but available to review the concepts so that people could remind themselves before or after completing the questions.

Winter Student Comments Areas for Improvement: Students want more big picture lectures: Honestly, I thought that this class would be learning about health systems and populations. I want to know tangible and useful information: how to use electronic medical records the insurance system in our country the business behind healthcare, ethical situations we will face as physicians, etc. I hope that we will eventually learn some of these crucial skills that will make us more culturally sensitive, well-rounded, ethical physicians in the next quarter or next year in this course. Too much busy work : The reflections took up a tremendous amount of time and were very repetitive. There were too many busy-work assignments that made me feel frustrated by the course and did not help me work towards the goals of the course. This was unfortunate, because I do believe that teamwork is incredibly important. Too many reflections - if you are going to assign that much writing don't have the subject matter be so redundant. I felt like we were writing about the same thing over and over.

Winter Student Comments Areas for Improvement: Scheduling: the timing of the course for deliverables, projects, and mandatory sessions was extremely poor. Quizzes, assessments, and overall demands of other core courses should be considered; it seemed as if the IH&H course operated in a logistical vacuum with respect to the rest of the curriculum. Disliked the fact that we often had major assignments due on quiz days or before big exam weeks. Why was there a required 2 hour session after a quiz? Why did we have 6+ required assignments the week before finals? Pedagogy: Students expressed interest in alternate models, perspectives, and modalities of teaching leadership I think the concept of the leadership module was great; however, in practice, it was not well executed. Having been in different leadership trainings in my past, this module really lacked in terms of actionable items. We were constantly just told about theories and personal stories. We need to be provided with tools to become effective leaders and to navigate these situations appropriately. Dartmouth has a great resource in Tuck, which has successfully led series of leadership courses for years. Why not consult them and adapt their proven model to fit medical education?

Spring Student Comments Strengths: The MaD Project was a great way to see what our classmates were thinking about medicine and Geisel ..the Make a Difference project...reminded us that there was something greater out there we have to fight for and change, and rekindled passion and excitement in a lot of us. [MaD offered a] great opportunity to improve public speaking skills, think of creative ideas around positive change in healthcare, give and receive constructive feedback. Students especially enjoyed Healthcare Finance , Historical Context , and Social Determinants sessions I LOVED the finance in healthcare lecture by Dr. Paul Gardent. I wish we spent more time learning about the history of the American healthcare system, healthcare financing, and methods of insurance.

Spring Student Comments Areas of Improvement: Interprofessional Education (IPE) sessions While the point of the IPE sessions was to engage with students entering other health professions, too much of the time was dedicated toward lecturing. Rather than passively absorbing information while sitting next to someone from a different school, it would have been much more interesting and valuable to learn about the differences between our respective fields. The shadowing and discussion that followed was a good start to this. I didn't get a lot out of the interprofessional sessions...add in other disciplines (pharmacy, PT, respiratory therapy, etc.) so we actually get to learn about what these roles do. I m really really glad that IH&H is incorporating this into our curriculum. But I really think we need more non-MD students participating in these sessions. Maybe we could invite students from DO, Physical therapy, acupuncture, yoga therapy programs, reiki, chiropractic medicine as well as nurses and MAs. About 1/3rd of US adults use some form of alternative therapies. It would be really awesome and progressive if Dartmouth tried incorporating some inter- professional and patient cultural competent medical care training regarding integrative and alternative healing modalities.

Spring Student Comments Areas of Improvement: Course structure deserves some reconsideration. I'm not sure what the overall goal of the course was. There were too many little projects here and there. Additionally, we were all over the place discussing various [seemingly] unrelated aspects of healthcare. I think the structure of the course could be changed to make it more engaging from the start. It would have been great to start with learning about issues in healthcare related to finance and social determinants and then do the MaD project near the beginning of the course when we still had the bandwidth to be able to carry out projects that we proposed. Course communication to students; assignment reminders The emails that are sent out are way too long. A brief bullet point list of Here are the 2 things due this week would suffice without a paragraph explanation of what is expected. What and when assignments are due or assigned reading material is very unclear and requires searching. Other classes seem to have a better system for this, and it is easier not to miss deadlines.

Summary regarding Measures of Quality The data available from USMLE at this time is limited with regard to content covered in the course; a new category in future reports ( Systems-based practice and patient safety ) will provide information about student performance on these topics. Students rated the course in the fair to good range; the winter term received the most criticism and was rated the lowest

Summary regarding Measures of Quality At times the overall organization of the course (how all the topics were interrelated) and communication about assignments was unclear to students Some pedagogies seemed ineffective for various parts of the course, while others were rated more positively Many assignments in the winter term were perceived by students as redundant and not effective for learning useful skills or meeting course goals

Summary regarding Measures of Quality Many students feel that the content of this course is valuable and they appreciated that Geisel was including it in the curriculum, however pedagogical methods need improvement. The utility of IPE sessions was not clear to most students. Students would also like greater emphasis on the topics of public health, health economics, and health policy

Overall Summary We recognize that this is a new course and expected some hiccups getting it off the ground; we commend the course director for her efforts and feel the course has tremendous potential. We appreciate that the course director held focus groups with students during the last academic year to gather ideas regarding improving the course, and she is very receptive to student feedback. We anticipate that the course will benefit tremendously from multiple sources of input. Our recommendations follow on the next series of slides.

Recommendations Currently there are two sets of course objectives. The course should choose one (or revise and create a new list) and make it available in the course materials. The objectives in the abbreviated list were well-written high order objectives that are appropriate for course objectives; several objectives on the longer list may be more appropriate as session objectives The footnotes were distracting; the subcommittee suggests creating one list of objectives without footnotes that is used for both years 1 and 2

Recommendations The leadership portion of the course received the strongest criticism from the students and should be one of the primary targets for improvement; we feel that leadership may not be the best name for this module as it conveyed certain expectations to the students that were not realized. Many students felt this section was more about wellness and personal improvement perhaps it could be called something like Professional Development or Professional Formation

Recommendations One primary source of dissatisfaction related to numerous assignments that were not perceived as having any value. Course assignments should be re-examined to ensure that the work required is relevant for developing essential skills and meeting course objectives. The course leadership should communicate with Y1 course directors and the Y1 Dean to coordinate the scheduling of major assignments to avoid conflicts with quiz and exam periods

Recommendations We encourage the course to expand sessions on topics such as public health, health economics, and health policy if it fits with the goals of the course IPE session are important and should be preserved, however some improvement is needed to ensure the collaborative goals are realized Examine session objectives to see if H&V and Nutrition objectives acknowledged where appropriate. Communication with students regarding the organization of the course and assignments should be presented in a more clear fashion

Action Plan Sincere thanks to the course review team. The plan outlined, in development over the past year, is currently being implemented. The OVERARCHING GOAL: By the conclusion of the course, students will be empowered with the knowledge, capabilities, and motivation to be effective, collaborative problem solvers and change agents, who go on to make a difference in the health of communities and populations, and advance the effectiveness and value of healthcare.

Action Plan 1. Organization & Communication o Course objectives Six high level objectives to be now referred to as domains Each comprised of 1-7 learning objectives (total of 24) Data Context Leadership Systems Communication Synthesis o Restructured and rationally reordered content discrete units lead off with Context o Increased clarity and transparency Course structure, rationale, components, and how they fit together Course map* reinforced throughout Syllabus/Readiness test prior to Session 1

Action Plan 2. Content o Revised emphasis and approach across selected modules Context expands modestly Systems/Improvement & Teamwork/Leadership interwoven o o Highlighting of applications and applicability Taking advantage of: Session and assignment content that worked well Students foundational preparedness (via readiness assessments) to enhance session productivity Enriched familiarity with students backgrounds o Greater intentionality in communicating alignment between objectives, content, and local resources

Action Plan 3. Assessments & Assignments o Content Ensure well-mapped to learning objectives Two major assignments reworked, applied, and now nested Keeping workload at targeted level o Timing Graded escalation term by term* Intentional alignment of deadlines distinct from those of other courses (quizzes, finals, in particular) initiated last Spring o Communication Of connections between assigned work and learning objectives Of deadlines via multiple modalities*

New Objectives DATA 1. Describe common methods of measurement in clinical medicine and research and how they are used. Illustrate how statistical inference is used to address questions in health and medicine. Interpret the findings of a research study reported in the medical or health literature. Discuss how the findings of a research study apply to a clinical or public health scenario. Interpret patterns of disease and healthcare, including to identify populations, communities, or subgroups who are at increased risk of disease. (was #7) Examine the role of probability, variability, and uncertainty in clinical decision making, including decisions about diagnostic testing and screening. (was #5) Assess the quality of a medical research paper by appraising the study's design and execution, including potential sources of confounding and bias. (was #6) 2. 3. 4. 5. 6. 7.

New Objectives CONTEXT 1. Examine the relationships between the healthcare system and other societal systems and entities that impact population health. (was #9) Interpret patterns of variation in how healthcare is delivered and financed across geography and across time. (was #10) Characterize social determinants that impact the health of individuals and populations. (was #8) Discuss the context (e.g. financial, political, ethical) in which medicine is practiced in the United States, and how it influences healthcare delivery. 2. 3. 4.

New Objectives SYSTEMS 1. 2. Analyze a local clinical front line healthcare microsystem. (was #13) Plan, perform, and measure tests of change to demonstrate measurable improvement. (was #14) Demonstrate collaboration, mutual respect, and understanding while facilitating effective team performance. (reconfigure of #15 and #18) Examine challenges and facilitators to effective inter-professional collaboration while describing its importance in clinical and improvement work. (reconfigure of #15 and #18) Apply core patient safety concepts while recognizing the physician's role in monitoring and creating safe systems of care. Develop a sound plan to create meaningful and measurable change in a system. (was #20) Illustrate how quality improvement and systems change can result in unintended consequences in complex health systems. (new) 3. 4. 5. 6. 7.

New Objectives COMMUNICATION 1. Demonstrate clear and concise communication with individuals or larger audiences, using writing, speech and visual elements. (was #21) 2. Formulate and collaboratively refine a clearly articulated vision to effect change. (was #23) 3. Demonstrate high standards of professionalism, integrity, and collaborative teamwork. (new, incorporates #17, 18, 22)) SYNTHESIS 1. Evaluate a real world problem by effectively organizing, prioritizing, and synthesizing information into a plan for action. (was #24) (#12 and #19 have been omitted)

Action Plan Thank you. You are here