Excellence in Medical Education and Accreditation Conference 2024

Excellence in Medical Education and Accreditation International Conference on Accreditation for Quality Medical Education Dhaka, Bangladesh. September 30, 2024 Professor Ricardo Le n-B rquez MD, MSA President www.wfme.org admin@wfme.org

Introduction The World Federation for Medical Education aims to enhance the quality of medical education and to promote the highest standards. It is a partnership organisation of the world s six regional associations for medical education, also working with its two founding members WHO and WMA, and four executive members, JDN, IFMSA, AMEE and ECFMG. It was founded in 1972. Three main priorities among many others: promotion of accreditation through the WFME Recognition Programme raising the standards for BME, PGME and CPD maintaining the World Directory of Medical Schools www.wfme.org admin@wfme.org

Regional members of WFME www.wfme.org admin@wfme.org

ogo_ifmsa_small JUNIOR DOCTORS NETWORK Executive members of WFME www.wfme.org admin@wfme.org

WFME Executive Council www.wfme.org admin@wfme.org

Accreditation of Medical Schools around the world There is a large comparative literature on accreditation agencies around the world. In general*, it shows that: Most have a self-evaluation or internal quality assurance processes. Some operate at the level of institutions, others at the level of programmes Most are nationally based * Hou, A.Y.-C., Ince, M., Tsai, S. and Chiang, C.L. (2015) Quality assurance of quality assurance agencies from an Asian perspective: regulation, autonomy and accountability. Asia Pacific Education Review, 16 (1), pp. 95 106. Woodhouse, D. (2004) The quality of quality assurance agencies. Quality in Higher Education, 10 (2), pp. 77 87. www.wfme.org admin@wfme.org

Accreditation of Medical Schools around the world More than half have external reviews by a recognised authority, such as the government or an internationally accepted organisation with legal status They are reviewed every 5 or 6 years Transparency of the regulation of accreditation agencies needs to be strengthened e.g. by publication of internal and external review reports, and systematic communication with staff Staff training should be invested in A balance between autonomy and accountability should be achieved Quality assurance is now seen as a profession that should have codes of conduct www.wfme.org admin@wfme.org

A general background of concern about accreditation* Public concern about accountability of accreditation agencies has increased over the last 20 years. USA was first to regulate accreditation agencies for higher education by the Council for Higher Education (CHEA) Europe followed, in the form of the European Association for Quality Assurance in Higher Education (ENQA), and a register of accreditation agencies (European Quality Assurance Register for Higher Education: EQAR) In Asia, similar bodies that regulate the quality of accreditation agencies were established in The Philippines, along with the Asia-Pacific Quality Network: APQN * Hou, A.Y.-C., Ince, M., Tsai, S. and Chiang, C.L. (2015) Quality assurance of quality assurance agencies from an Asian perspective: regulation, autonomy and accountability. Asia Pacific Education Review, 16 (1), pp. 95 106. www.wfme.org admin@wfme.org

A general background of concern about accreditation* But there were no agencies that quality assured for a specific discipline. WFME has taken that role for medicine. * Hou, A.Y.-C., Ince, M., Tsai, S. and Chiang, C.L. (2015) Quality assurance of quality assurance agencies from an Asian perspective: regulation, autonomy and accountability. Asia Pacific Education Review, 16 (1), pp. 95 106. www.wfme.org admin@wfme.org

Medical school accreditation: The global landscape* Medical school accreditation began about 60 years ago, with considerable growth in the last 20 years (in 1980 there were only eight medical school accreditation agencies; now 128) All medical school accreditation systems try to ensure that medical graduates are ready for the next stage of their career: in postgraduate training, or in practice But only about 50% of medical schools have access to national accreditation that is specific to medicine. It is usually low-income countries that are without specific medical school accreditation This has led to a cross-border market in medical school accreditation www.wfme.org admin@wfme.org

Medical school accreditation: The global landscape* Where there are specific medical school accreditation agencies, their scope and organization vary considerably. WFME recognition tries to take this into account. * Bedoll D, van Zanten M, McKinley D Global trends in medical education accreditation Human Resources for Health (2021) 19(1) 70 www.wfme.org admin@wfme.org

Directory of Organizations that Recognize/Accredit Medical Schools - DORA FAIMER created a directory of international organizations that recognize, authorize, certify, or accredit medical schools and/or medical education programs. These organizations are often responsible for the establishment of national standards. They report 128 accreditation agencies www.wfme.org admin@wfme.org

Recognized Countries 47 in progress 11 September 2024 www.wfme.org admin@wfme.org

Acreditation agencies can serve one or several counties Sri Lanka - Sri Lanka Medical Council (SLMC) Mexico Consejo Mexicano para la Acreditaci n de la Educaci n M dica (COMAEM) Costa Rica Panam El Salvador Guatemala www.wfme.org admin@wfme.org

Acreditation agencies can serve one or several counties International Caribbian Accreditation Authority for Education of Medicine and other Health Professions (CAAM-HP) Antigua and Bermuda The Bahamas Barbados Belize Dominica Grenada Jamaica Guyana - Montserrat - Saint Lucia - Saint Kitts and Navis - Saint Vincent and the Granadines - Trinidad and Tobago - Anguilla - Turks and Caicos - Dominican Republic * www.wfme.org admin@wfme.org

WFME and accreditation WFME does not accredit medical schools Instead, WFME reviews the quality of the agencies that do accredit medical schools, through the WFME Recognition Programme So our influence on the quality of medical schools is indirect. www.wfme.org admin@wfme.org

Quality of medical school accreditation: Why is there a WFME programme for the recognition of accrediting agencies? The simple existence of an accreditation system does not guarantee that the system is working well and making trustworthy decisions about the quality of programmes in medical education. We should ensure the integrity of accrediting agencies in the evolving market for accreditation, where private firms and individual consultants offer their services in conducting a review and issuing an accreditation. These problems in accreditation suggest the need for a robust and independent system for the evaluation and recognition of accreditation agencies. www.wfme.org admin@wfme.org

THE WFME Recognition Programme and accreditation systems The Recognition Programme sets out the conditions that we believe accreditation agencies should display. Those conditions (which we call Criteria) allow agencies to make their own decisions about how they perform the necessary tasks that WFME sets out. So accreditation agencies around the world have different acceptable ways of performing their work. www.wfme.org admin@wfme.org

Recognition Criteria The WFME Recognition Programme evaluates how accrediting agencies or systems not individual medical schools meet the WFME Recognition Criteria against which accrediting agencies are evaluated. The criteria are summarised as follows: Background (scope of authority, acceptance of the accrediting agency by others, substantive changes). Accreditation standards (existence and availability of standards, type of standards, appropriateness of standards, review of standards). Accreditation process and procedures (medical school self-study, site visit, reports, qualification and training of individuals associated with the accrediting agency, accreditation decisions, activities subsequent to accreditation decisions, complaints). www.wfme.org admin@wfme.org

Recognition Criteria Accreditation agency policies and resources (controls against conflict of interest, controls against inconsistent application of standards and procedures, administrative and fiscal responsibilities, due process, maintenance of records, availability and dissemination of information). www.wfme.org admin@wfme.org

Recognition Criteria In April 2024 the Recognition Criteria were revised. The new version will take effect on 1 January 2025 and apply to newly applying agencies and those seeking renewal of Recognition Status after this date. www.wfme.org admin@wfme.org

WFME Policy on accrediting agencies that are eligible to apply for WFME Recognition The WFME Recognition Committee recognises accrediting agencies that have the authority to accredit education programmes or schools that award the MD degree or equivalent medical qualifying degree. The following types of agencies are eligible to apply for WFME recognition: a. Government entities b. Entities that are authorised or recognised by the government (either or both of the Ministry of Health and Ministry of Education) where the school is located c. Entities that are authorised or recognised by an appropriate professional or scientific association Recognition of an accrediting agency by the WFME Recognition Committee confers the understanding that an agency has been deemed to be credible in its policies and procedures to assure the quality of medical education in the programmes and schools that it accredits. www.wfme.org admin@wfme.org

Recognition criteria for agencies accrediting medical Schools April 2019 Introduction i. The World Federation for Medical Education (WFME) Recognition Programme is the process through which WFME evaluates the legal standing, accreditation/regulation ( accreditation ) process, post-accreditation monitoring, and decision-making processes of an accreditation/regulatory agency ( agency ) of basic medical education schools or programmes ( schools ). WFME Recognition Status of an agency confers the understanding that the quality of medical education in its accredited schools is to an appropriate and rigorous standard. ii. The criteria are the measures of quality by which WFME determines whether to award or re- award Recognition Status, as follows: Recognition Status awarded for a ten-year term Recognition Status denied or withdrawn www.wfme.org admin@wfme.org

Recognition criteria for agencies accrediting medical Schools April 2019 iii. Recognition Status may be accompanied with conditions, usually in the form of follow-up actions to an agreed timeline and post-Recognition Status monitoring. iv. Agencies do not need to meet all criteria in full to be awarded Recognition Status, provided that there is reasonable justification why a criterion is not appropriate in the context in which the agency operates. v. The award of Recognition Status confirms that the policies and procedures followed by an agency are appropriate in the region(s) where the agency operates. vi. The criteria are based on the WHO-WFME guidelines for accrediting agencies. The WHO- WFME guidelines are suggested good practice for an agency. The Recognition Criteria are the minimum basic requirements, and allow agencies to determine an appropriate and rigorous standard of performance for the context in which they operate www.wfme.org admin@wfme.org

Recognition criteria for agencies accrediting medical Schools April 2019 Part A. Eligibility 1. The agency accredits basic medical education and is: a government or inter-governmental entity, or an independent professional body that is authorised or recognised by the relevant national or state/provincial government (Ministry of Health or Ministry of Education or both), or the legislator (parliament), or an independent professional body that is authorised or recognised by a professional or scientific association with appropriate authority. 2. Where the agency operates in more than one country or region, its processes are endorse and outcomes are subject to adoption by the governments of each of these jurisdictions. It should also be able to demonstrate that the standards and procedures for accreditation of medical schools are appropriate to those countries and regions and applied in a consistently robust manner. www.wfme.org admin@wfme.org

Recognition criteria for agencies accrediting medical Schools April 2019 Part A. Eligibility 3. The accreditation decisions of the agency are made known to, and accepted by, other organisations such as professional licensing bodies, educational institutions and employers. 4. The agency operates within a framework that enables the establishment of agreements and the signing of contracts according to the laws of the country or countries in which the agency is seated and operates. www.wfme.org admin@wfme.org

Recognition criteria for agencies accrediting medical Schools April 2019 Part B. Accreditation Standards 5. The agency uses comprehensive standards for accreditation appropriate to basic medical education. 6. The agency makes publicly available the accreditation standards. 7. The agency has a system to determine that the standards are sufficiently rigorous and appropriate to evaluate the quality of the education and training provided at medical schools. 8. The agency has a system for periodically reviewing and updating the standards. www.wfme.org admin@wfme.org

Recognition criteria for agencies accrediting medical Schools April 2019 Part C. Accreditation process and procedures 9. Medical school self-evaluation 10. On site observation and evaluation 11. Reports 12. Individuals associated with the agency 13. Accreditation decisions 14. Activities subsequent to accreditation decisions 15. The agency has policies and procedures to investigate and act upon complaints regarding accredited medical schools. www.wfme.org admin@wfme.org

Recognition criteria for agencies accrediting medical Schools April 2019 Part D. Agency policies and resources 16. The agency has policies and procedures for managing actual or perceived conflicts of interest for all individuals involved in the accreditation and decision-making processes. 17. The agency has controls to ensure that the policies and procedures for accreditation of medical schools are applied consistently. 18. Administrative and fiscal responsibilities 19. Maintenance of records 20. Availability and dissemination of information www.wfme.org admin@wfme.org

Measuring the impact of accreditation of medical schools www.wfme.org admin@wfme.org

But before we continue thinking about how to measure the impact of accreditation of medical schools . We need to understand the challenges of social science research And we need to know what has been found by other researchers www.wfme.org admin@wfme.org

The Problem of Outcomes Research in Social Science In almost every area of education, there is no definitive, robust research to tell us what generalisable effect any intervention has. This is because education is a social science, and social science has challenges: Identifying variables (What shall we measure?) Measuring variables (How shall we measure it?) Identifying jeopardising variables i.e. being able to isolate the effect of the process we are looking at Comparing different contexts (What are the cultural and contextual difference between sites?) www.wfme.org admin@wfme.org

The Problem of Precision. It has been observed* that: Accreditation is an imprecise science It is based on judgement This makes a difficult base for precise research *Scrivens, E. (1997) Assessing the value of accreditation systems. The European Journal of Public Health, 7 (1), pp. 4 8. www.wfme.org admin@wfme.org

The Problem of Establishing Cause and Effect in Social Science Research Research questions often imply a relationship between an intervention and an outcome: How to measure the impact of medical school accreditation implies cause and effect BUT Being able to show cause in social science is very difficult Variables are too individual and complex to describe, define and replicate There are often jeopardising variables that cannot be isolated or generalised e.g. schools that choose to apply for accreditation may be better schools www.wfme.org admin@wfme.org

Examples of previous research www.wfme.org admin@wfme.org

Van Zanten, Norcini, Boulet and Simons review of accreditation* concluded that: Despite global trends indicating an increasing focus on the quality of education programmes, data linking accreditation processes to the production of more highly skilled doctors and, ultimately, better patient care are lacking. * Van Zanten, M., Norcini, J.J., Boulet, J.R. and Simon, F. (2008) Overview of accreditation of undergraduate medical education programmes worldwide. Medical Education, 42 (9), pp. 930 937. www.wfme.org admin@wfme.org

Blouin and Tekian* and others concluded that: Direct linkages between accreditation and education quality are .difficult to establish. student outcomes present challenges with regard to data availability, comparability, and contamination. The true impact of accreditation may well rest in its ability to promote continuous quality improvement (CQI) within medical education programs. Blouin, D. and Tekian, A. (2018) Accreditation of Medical Education Programs: Moving From Student Outcomes to Continuous Quality Improvement Measures. Academic Medicine, 93 (3), pp. 377 383. Blouin, D., Tekian, A., Kamin, C. and Harris, I.B. (2018) The impact of accreditation on medical schools processes. Medical Education, 52 (2), pp. 182 191. www.wfme.org admin@wfme.org

However. Minimal evidence about accreditation and CQI exists. Arja, S.B., White, B.A., Fayyaz, J. and Thompson, A. (2024) The impact of accreditation on continuous quality improvement process in undergraduate medical education programs: A scoping review. MedEdPublish, 14, p. 13. www.wfme.org admin@wfme.org

One study found short-term and temporary increases in student satisfaction as a result of preparing for external accreditation Al-Eyadhy, A. and Alenezi, S. (2021) The impact of external academic accreditation of undergraduate medical program on students satisfaction. BMC Medical Education, 21 (1), p. 565 www.wfme.org admin@wfme.org

There can also be negative effects of accreditation Although accreditation is perceived to have many benefits, it also has a number of unintended consequences, including on staff morale, student-teacher relationships, and teacher workloads. Medical teachers also have a number of concerns about the credibility of accreditation standards, assessors, and processes. Choa, G., Arfeen, Z., Chan, S.C.C. and Rashid, M.A. (2022) Understanding impacts of accreditation on medical teachers and students: A systematic review and meta-ethnography. Medical Teacher, 44 (1), pp. 63 70. www.wfme.org admin@wfme.org

The Frank International Consensus report* concluded that research is needed . By adopting a common taxonomy of elements, we can begin to compare, learn from, and build upon the diversity of Health Profession Education accreditation systems worldwide In other words, we can start to undertake research, if we agree on terminology. However, the diversity of accreditation practice means that generalisability may be low. Frank, J.R., Taber, S., Van Zanten, M., Scheele, F. and Blouin, D. (2020) The role of accreditation in 21st century health professions education: report of an International Consensus Group. BMC Medical Education, 20 (S1), p. 305. www.wfme.org admin@wfme.org

Effects on Accreditation The primary purposes of accreditation are to ensure the quality of medical education and to promote quality improvement, with the ultimate goal of providing optimal patient care. The accreditation-related activities steer medical education programmes towards the establishment of many processes, namely those related to governance, academic accountability, data collection and analysis, monitoring, documentation, and the creation and revision of policies. It also contributes to Continue Quality Improvment and faculty members engagement in the medical education programme. It triggers a complete overhaul of the curriculum and of the academic accountability scheme. Medical Education 2018: 52: 182 191 doi: 10.1111/medu.13461 www.wfme.org admin@wfme.org

Effects on Accreditation Although a direct link between the implementation or strengthening of medical schools processes and the improvement of the quality of medical education is difficult to ascertain, the identified processes, given their nature, appear inseparable from the quality and effectiveness of medical education programmes. The statment that accreditation drives medical schools to develop, implement and strengthen processes that support quality in medical education programmes is well accepted Medical Education 2018: 52: 182 191 doi: 10.1111/medu.13461 www.wfme.org admin@wfme.org

Qualitative statements 1 When questions are asked to members of accredited medical schools, these are samples of answers: We ve put in place more structure around our committees and subcommittees that oversee and manage the curriculum. As part of that, I think those committees have greater clarity around what their role is. There s been improved communication between the committees. I mentioned earlier the curricular mapping tool that we just bought. This is a huge expense for the School of Medicine and I think it s a good thing. If it hadn t been for accreditation, I am not sure we would have had the support of the university to buy this resource. Medical Education 2018: 52: 182 191 doi: 10.1111/medu.13461 www.wfme.org admin@wfme.org

Qualitative statements 2 It s actually an impediment or an imposition and a costly one at that but that actually distracts from the real activities of the organisation, medical education or whatever it might be. When you start spending huge amounts of time and money that seem to increase every cycle, I think that's part of the problem, that the standards seem to increase, if not ultimately, then in the amount of work it takes to satisfy the standard. You increase the workload, you increase the costs, and it's very difficult to determine cost effectiveness, especially as it relates to specific standards and maybe the whole process. Medical Education 2018: 52: 182 191 doi: 10.1111/medu.13461 www.wfme.org admin@wfme.org

Qualitative statements 3 Feedback to the WFME from recognized agencies, these are samples of statments: The benefit of AAA carrying out a recognition process is to have the feedback from an institution of international prestige to carry out the continuous improvement of our processes as an added value to our function as a regulator, evaluator and accreditor of medical careers in the country Through this complete review, this accreditation agency had noticed some issues to be addressed immediately. Recommendations based on critical and objective review are quite helpful for improving quality of external review and accreditation process by the accreditation agency. www.wfme.org admin@wfme.org

Qualitative statements 4 Feedback to the WFME from recognized agencies, these are samples of statments: It was a great opportunity to meet with external experts. We were able to share experiences and receive constructive feedback. The discussions and comments gave confidence to the council's members and helped us feel we are moving in the right direction. the letter was useful and contained suggestions to improve the quality of the Council s work. We have already begun working on some paragraphs as well as setting the policy for students participation in the accreditation process in addition to other valuable remarks. www.wfme.org admin@wfme.org

Qualitative statements 5 Feedback to the WFME from recognized agencies, these are samples of statments: It is a well-organized program that enhances the development of medical education programs by evaluating and following up on the work of the agencies that accredit these programs and ensure their quality. Using the criteria also motivated and encouraged the agency to become more self-reflective and aware of the key international facets of quality assurance. The information WFME provided in the cover letter with the quality improvement recommendations was very useful. The feedback will assist the agency in working towards continuous quality improvements in its quest for sustainable excellence. The agency has already started working on the suggestions. www.wfme.org admin@wfme.org

Qualitative research will enable WFME to understand the effects of our Recognition Programme www.wfme.org admin@wfme.org

The challenges of conducting meaningful research We have seen that there is no substantive research to date that shows any generalisable effects of medical school accreditation Because the variables are difficult to isolate and measure And cause and effect cannot be directly shown when there are jeopardising variables Inadequate curriculum Outdated content Lack of integration Insufficient resources Poor teaching quality, etc www.wfme.org admin@wfme.org