Methods for Stakeholder Engagement in Guideline Development: A Methodological Survey
Stakeholder engagement approaches in guideline development through a systematic survey of published methodological guidance documents. Gain insights into types of stakeholders engaged, levels of engagement, processes, and modes of engagement
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Methods for stakeholder Methods for stakeholder engagement in practice guideline engagement in practice guideline development: a methodological development: a methodological survey of published guidance survey of published guidance Joanne Khabsa1, Vanessa Helou2, Hussein Noureldine3, Ahmed Mohamed4, Pauline Campbell5, Elie A. Akl6,7
Background and Objective Incorporating stakeholder views helps in tailoring of recommendations and enhances their implementation Organisations that develop practice guidelines are developing programs that promote user and public participation The objective of this study is to describe stakeholder engagement approaches as depicted in methodological guidance on guideline development
Methods This study is part of a larger project addressing the methodology used in guideline development, according to published guidance on the subject We conducted a systematic search in July 2019 to identify a comprehensive sample of published methodological guidance documents on guideline development We included published documents in the form of handbooks , information posted on the organization s website, or peer reviewed articles by the organizations
Methods We abstracted data in duplicate and independently on General characteristics of the organizations General characteristics of the guidance Stakeholder engagement, mapped to the 18 items of the GIN-McMaster Guideline Development Checklist. For each step, we abstracted data on: Types of stakeholders engaged Level of engagement Processes for stakeholder identification and recruitment Mode of engagement Whether engagement is internal to the guideline group (e.g., stakeholders being part of the panel), or external to the guideline group
Preliminary results Total sample of guideline-producing organizations included: n=137 Total sample of guideline-producing organizations abstracted: n=89 Total sample of guideline-producing organizations with mention of stakeholder engagement: n=82
General characteristics of included organizations (N=82) Level of the organization Sub-national National Regional International Region of the organization North America Europe Australasia Latin America East Asia & Pacific Middle East & North Africa International n (%) 3 (4) 63 (77) 9 (11) 7 (8) Type of the organization Professional organization Governmental agency Disease- or population-specific organization Other Field of the organization Clinical Public health Health systems Unclear n (%) 53 (65) 16 (20) 7 (8) 6 (7) 42 (51) 25 (30) 3 (4) 3 (4) 1 (1) 1 (1) 7 (8) 64 (78) 16 (20) 1 (1) 1 (1)
General characteristics of guidance documents (N=82) Stakeholder engagement policy; n (%) Document completely dedicated to stakeholder engagement 1 (1) Stakeholder engagement is addressed in a dedicated section of the guidance 24 (29) Stakeholder engagement is addressed throughout the guidance 57 (70) Date of guidance document; median, IQR 2016 (2014 - 2018)
Stakeholder engagement per the 18 items of the GIN-McMaster Guideline Development Checklist
ITEM 1: Organization, budget, planning, training 1. Organization (n=0) 2. Budget (n=3, 4%) 3. Planning (n=9, 11%) 4. Training (n=20, 24%)
ITEM 1: Organization, budget, planning, training Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 3 (4) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) Budget (n=3, 4%) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) n (%) 1 (1) 1 (1) 0 (0) 2 (2) 1 (1) 1 (1) 0 (0) 0 (0) Patient representative organizations 1 (1) Public, community, consumers 0 (0) 3 (4) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors 2 (2) 1 (1) 0 (0) 2 (2) 0 (0) 1 (1) Stakeholders not otherwise specified 0 (0)
ITEM 1: Organization, budget, planning, training Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 6 (7) 1 (1) 3 (4) 1 (1) 0 (0) 0 (0) Planning (n=9, 11%) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) n (%) 1 (1) 1 (1) 1 (1) 7 (9) 0 (0) 1 (1) 1 (1) 0 (0) Patient representative organizations 3 (4) Public, community, consumers 2 (2) 7 (9) 2 (2) 0 (0) 2 (2) 0 (0) 4 (5) 0 (0) Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors 0 (0) 2 (2) 7 (9) 7 (9) 1 (1) 2 (2) Stakeholders not otherwise specified 0 (0)
ITEM 1: Organization, budget, planning, training Training (n=20, 24%) Stakeholders trained Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 5 (8) 0 (0) 5 (8) 7 (12) 5 (8) 11 (18) 0 (0) 0 (0) 3 (5) 0 (0) 3 (5) 0 (0) 1 (3) 5 (8) Stakeholders trained as Members of the panel Members of a specific group Not as members of any panel 19 (23) 2 (2) 1 (1)
ITEM 2: Priority Setting Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 19 (23) 1 (1) 1 (1) 0 (0) 1 (1) 0 (0) n = 22 (27%) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 2 (2) 1 (1) 2 (2) 9 (11) 5 (6) 15 (18) 2 (2) 0 (0) 7 (9) 0 (0) 9 (11) 0 (0) 6 (7) 1 (1) 16 (20) 4 (5) 0 (0) 3 (4) 0 (0) 1 (1) 12 (15) 11 (13) 10 (12) 4 (5) 10 (12)
ITEM 3: Guideline group membership Exclusion of stakeholders Stakeholders excluded from the group Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 1 (1) 0 (0) 1 (1) 3 (4) 0 (0) 0 (0) 1 (1) 0 (0) 0 (0) 7 (9) 1 (1) 0 (0) 0 (0) 0 (0) Exclusion from Executive committee Systematic review group Panel Peer review group Other n (%) 1 (1) 2 (2) 10 (12) 0 (0) 2 (2)
ITEM 3: Guideline group membership Inclusion of stakeholders Stakeholders included in the group Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 16 (20) 3 (4) 12 (15) 39 (48) 21 (26) 64 (78) 3 (4) 1 (1) 12 (15) 2 (2) 43 (52) 1 (1) 10 (12) 13 (16) Inclusion in Oversight/executive committee Systematic review group Panel Peer review group Other n (%) 17 (21) 5 (6) 70 (85) 6 (7) 11 (13)
ITEM 3: Guideline group membership Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Unclear Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Unclear n (%) 27 (33) 19 (23) 31 (38) 1 (1) 3 (4) 5 (6) Conditions on whether individuals are involved* Conditions on how individuals are involved* 51 (62) 42 (51) 27 (33) 37 (45) 29 (35) 18 (22) 2 (2) *Conditions were mainly with regards to COI
ITEM 4: Establishing guideline group processes n = 2 (2%) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 0 (0) 0 (0) 0 (0) 1 (1) 0 (0) 2 (2) 0 (0) 0 (0) 0 (0) 0 (0) 1 (1) 0 (0) 0 (0) 0 (0) Stakeholders engaged as Members of the panel Members of a specific group Not as members of any panel 2 (2) 0 (0) 0 (0)
ITEM 5: Identifying target audience and topic selection Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 4 (5) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) Identifying target audience: n = 4 (5%) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 1 (1) 0 (0) 2 (2) 1 (1) 2 (2) 3 (4) 0 (0) 0 (0) 2 (2) 0 (0) 3 (4) 0 (0) 1 (1) 1 (1) 3 (4) 0 (0) 0 (0) 0 (0) 1 (1) 0 (0) 2 (2) 2 (2) 4 (5) 1 (1) 1 (1)
ITEM 5: Identifying target audience and topic selection Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 31 (38) 2 (2) 3 (4) 2 (2) 7 (9) 1 (1) N=45 (55%) (topic nomination: n = 34 (41%); topic selection: n = 20 (24%)) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 8 (10) 2 (2) 4 (5) 15 (18) 10 (12) 37 (45) 1 (1) 1 (1) 8 (10) 2 (2) 12 (15) 0 (0) 13 (16) 10 (12) 24 (29) 3 (4) 3 (4) 18 (22) 2 (2) 1 (1) 24 (23) 24 (23) 18 (22) 12 (15) 27 (33)
ITEM 8: Question Formulation Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 39 (48) 2 (2) 3 (4) 5 (6) 0 (0) 2 (2) n= 49 (60%) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 12 (15) 0 (0) 11 (13) 24 (29) 16 (20) 38 (46) 3 (4) 0 (0) 8 (10) 1 (1) 19 (23) 0 (0) 4 (5) 10 (12) 28 (34) 9 (11) 5 (6) 12 (15) 2 (2) 1 (1) 19 (23) 40 (49) 38 (46) 5 (6) 14 (17)
ITEM 9: Considering importance of outcomes and interventions, values, preferences, and utilities Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 39 (48) 2 (2) 3 (4) 5 (6) 1 (1) 2 (2) n= 26 (32%); n=1: individuals who already have the disease are excluded Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 12 (15) 0 (0) 11 (13) 24 (29) 16 (20) 38 (46) 3 (4) 0 (0) 8 (10) 1 (1) 19 (23) 0 (0) 4 (5) 10 (12) 28 (34) 9 (11) 5 (6) 12 (15) 2 (2) 0 (0) 5 (6) 21 (26) 38 (46) 5 (6) 14 (17)
ITEM 10: Deciding what evidence to include and searching for evidence n= 46 (56%); employees of the pharmaceutical industry are explicitly excluded from the systematic literature review (n=1) Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 43 (52) 0 (0) 2 (2) 1 (1) 0 (0) 0 (0) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 13 (16) 2 (2) 9 (11) 21 (26) 8 (10) 37 (45) 1 (1) 0 (0) 6 (7) 1 (1) 23 (28) 0 (0) 5 (6) 7 (9) 35 (43) 6 (7) 3 (4) 3 (4) 1 (1) 3 (4) 5 (6) 37 (45) 39 (48) 6 (7) 5 (6)
ITEM 11: Summarizing evidence and considering additional information n = 14 (17%); employees of the pharmaceutical industry are explicitly excluded from the systematic literature review (n=1) Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 12 (15) 1 (1) 1 (1) 0 (0) 0 (0) 0 (0) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 3 (4) 0 (0) 5 (6) 6 (7) 3 (4) 11 (13) 0 (0) 0 (0) 3 (4) 0 (0) 5 (6) 0 (0) 0 (0) 4 (5) 11 (13) 2 (2) 2 (2) 1 (1) 0 (0) 0 (0) 3 (4) 11 (13) 12 (15) 1 (1) 2 (2)
ITEM 12: Judging quality, strength or certainty of a body of evidence n = 29 (35%); while it is not expected that patient/carer members will undertake the specific critical appraisal process for individual clinical questions, patient/carer members will be kept informed at all stages and invited to every meeting of the GDG (n=1) Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 27 (33) 0 (0) 1 (1) 0 (0) 0 (0) 1 (1) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 7 (9) 1 (1) 4 (5) 9 (11) 4 (5) 21 (26) 1 (1) 0 (0) 4 (5) 1 (1) 15 (18) 0 (0) 2 (2) 6 (7) 24 (29) 3 (4) 2 (2) 3 (4) 1 (1) 1 (1) 0 (0) 28 (34) 26 (32) 2 (2) 1 (1)
ITEM 13: Developing recommendations and determining their strength Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group *n=1: meetings open to the public n (%) 60 (73) 2 (2) 2 (2) 1 (1) 0 (0) 1 (1) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 18 (22) 4 (5) 9 (11) 29 (35) 13 (16) 52 (63) 2 (2) 0 (0) 10 (12) 3 (4) 28 (34) 0 (0) n = 64 (78%) individuals with expertise on specific topics or who are experts from other professions or disciplines or who are patient/family advocates provide input to the GWG as advisors. They do not participate in determining guideline recommendations or ratings (n=1) public panel members cannot vote (n=1) employees of the pharmaceutical industry are excluded from meetings of the consensus, even as observers (n=1) requires the presence of at least one patient or patient representative; if quorum is achieved draft recommendations may be formulated in their absence, then agreed upon with the patient or patient representative (n=1) 34 (41) 25 (30) 9 (11) 9 (11) 2 (2) 0 (0) 0 (0)* 63 (77) 6 (7) 8 (10) 60 (73) 5 (6)
ITEM 14: Wording of recommendations and of considerations about implementation, feasibility and equity n = 14 (17%) Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 13 (16) 1 (1) 0 (0) 0 (0) 0 (0) 0 (0) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 0 (0) 6 (7) 3 (4) 4 (5) 8 (10) 3 (4) 8 (10) 0 (0) 0 (0) 0 (0) 5 (6) 0 (0) 0 (0) 2 (2) 12 (15) 2 (2) 0 (0) 0 (0) 0 (0) 1 (1) 2 (2) 11 (13) 12 (15) 1 (1) 1 (1)
ITEM 15a: Reporting n = 6 (7%) Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 5 (6) 0 (0) 0 (0) 1 (1) 1 (1) 0 (0) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 1 (1) 1 (1) 2 (2) 3 (4) 3 (4) 4 (5) 1 (1) 1 (1) 3 (4) 0 (0) 1 (1) 0 (0) 1 (1) 0 (0) 11 (13) 2 (2) 2 (2) 1 (1) 0 (0) 1 (1) 1 (1) 4 (5) 5 (6) 1 (1) 1 (1)
ITEM 15b: Peer review n = 62 (76%); there no current process to facilitate review by patient stakeholders (n=1) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 9 (11) 2 (2) 5 (6) 23 (28) 30 (37) 37 (45) 3 (4) 0 (0) 20 (24) 1 (1) 27 (33) 1 (1) 17 (21) 15 (18) Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other/Unclear Engagement as Members of another group Not as members of any group n (%) 23 (28) 6 (7) 21 (26) 16 (20) 4 (5) 2 (2) 16 (20) 51 (62)
ITEM 16a: Dissemination n = 34 (41%) Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement As receivers Creating the dissemination material/plan Both Engagement as Members of the panel group Members of another group Not as members of any group n (%) 28 (34) 1 (1) 3 (4) 0 (0) 2 (2) 1 (1) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 8 (10) 1 (1) 3 (4) 12 (15) 10 (12) 23 (28) 0 (0) 1 (1) 7 (9) 1 (1) 11 (13) 0 (0) 3 (4) 10 (12) 12 (15) 7 (9) 3 (4) 18 (22) 4 (5) 8 (24) 11 (32) 15 (44) 13 (16) 3 (4) 21 (26)
ITEM 16b: Implementation n = 28 (35%) Engaged as implementers (n=7, 24%), in creating the implementation material/plan (n=11, 38%), both (n=16, 55%) Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 28 (34) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 12% 4% 6% 13% 7% 27% 5% 0% 11% 0% 10% 1% 7% 11% 18 (22) 3 (4) 0 (0) 8 (10) 3 (4) 2 (2) 4 (5) 22 (27) 10 (12) 4 (5) 16 (20)
ITEM 17: Evaluation and use n = 23 (28%) Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 21 (26) 0 (0) 0 (0) 2 (2) 1 (1) 0 (0) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 7 (9) 2 (2) 3 (4) 5 (6) 4 (5) 15 (18) 2 (2) 1 (1) 2 (2) 0 (0) 5 (6) 0 (0) 4 (5) 9 (11) 16 (20) 3 (4) 1 (1) 9 (11) 1 (1) 2 (2) 13 (16) 10 (12) 9 (11) 3 (4) 14 (7)
ITEM 18: Updating Recruitment process Not specified Open, with selection process Targeted, with selection process Open, without selection process Targeted, without selection process Other Mode of engagement Not specified Face to face Virtual, real-time (e.g. conference call) Virtual, not real-time(e.g. email) Other Level of engagement Not specified/unclear* Feedback Decision-making Engagement as Members of the panel group Members of another group Not as members of any group n (%) 32 (39) 0 (0) 2 (2) 2 (2) 0 (0) 2 (2) n = 38 (46%) Stakeholders engaged Patients with the condition Former patients Caregivers/Relatives (family) Patient representative organizations Public, community, consumers Providers Payers Purchasers Policymakers Product-makers Principal investigators Peer review editors Stakeholders not otherwise specified Other n (%) 10 (12) 1 (1) 7 (9) 14 (17) 6 (7) 33 (40) 0 (0) 0 (0) 7 (9) 0 (0) 17 (21) 0 (0) 5 (6) 5 (6) 31 (28) 1 (1) 2 (2) 6 (7) 0 (0) 5 (6) 11 (13) 24 (29) 27 (33) 7 (9) 10 (12) *Literature surveillance
