Comprehensive Guide to Post-Market Surveillance of POC NAT Assays

Post-Market Surveillance of IVDs: How to Detect Issues on POC NAT Assays and When/How to Fill an IVD Complaint Form Jeff Lemaire | 29 October 2019

Outline Critical elements of post-market surveillance POC NAT characteristics Remote monitoring and how to detect issues How to identify potential root causes, and which issues require an IVD product complaint How to fill, report and communicate an IVD complaint

Critical Elements of Post-Market Surveillance (PMS) Look for problems Act on advice from Mx (including FSN) Interrogate problem, try to replicate finding Report to Mx, NRA (and WHO) Document complaint

Device problem Adverse event Medical device problem is something that happens to the IVD e.g. unsealed device packaging, tear, rip or hole in device packaging Adverse event is something that happens to a person e.g. death or serious deterioration in health Indirect harm caused by Imprecise results Inadequate quality controls Inadequate calibration False positive results False negative results IMDRF guidance on IMDRF terminologies for categorized Adverse Event Reporting (AER): terms, terminology structure and codes. IMDRF/AE WG(PD1)/N43 FINAL:2019 (Edition 3) http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-190321-aer-n43-r3.docx

Point-of-Care (POC) Nucleic Acid Testing (NAT) Instruments POC technologies allow for decentralization of laboratory services and often allow testing of a variety of assays (TB, DR-TB, HIV EID, HIV VL, HCV VL, HPV, Ebola, etc.) The ease of use of POC NAT assays enable non-laboratory medical staff, such as nurses and phlebotomists, to perform these assays. While increasing patients access to testing services, the decentralization, multiplexing, and high number and variety of testing sites and end-users of POC NAT assays result in an increased need and complexity for monitoring quality of products/testing.

Examples of POC NAT Performance No IVD is ever 100% sensitive and 100% specific A certain proportion of false negative and false positive results is expected Internal quality control (IQC) failures (errors and invalids) are expected and critical to prevent release of dubious results A certain proportion of errors/invalids is expected and can help identify potential root cause of issues. Sample Type Invalid Rate Assay Evaluator Sensitivity (95% CI) Specificity (95% CI) WHO PQ CDC/NHLS WB 98.67% (95.27-99.84) 100.00% (97.59-100.00) 5.58% m-PIMA HIV-1/2 Detect EID Consortium WB 99.00% (96.45-99.88) 99.97% (99.83-100.00) 6.36% WHO PQ CDC/NHLS WB 98.86% (93.83-99.97) 100.00% (97.55-100.00) 2.2% Xpert HIV- 1 qual WHO PQ CDC/NHLS DBS 99.34% (96.40-100.00) 100.00%(97.60-100.00) EID Consortium WB 96.79% (92.68-98.95) 99.91%(99.76-99.97) 4.28%

Remote Monitoring of POC NAT Testing Investigation and corrective actions Good governance and appointed coordinating body is crucial Feedback communication with appointed focal points is key District-level support can be instrumental for follow-up and corrective action Competency assessment and refresher training might be needed Instrument repair might be needed Share information with supply chain teams to avoid stock outs Seek technical support from manufacturer if needed Post-market surveillance Focus on products (test cartridges, instruments) and adverse events Know how to identify potential problem in cartridge and instruments Similar trends seen across sites should raise a flag and prompt further investigation Sudden surge in problem following the use of a new lot should also raise a flag

Remote Monitoring: Critical Elements Regular data collection Enables identification of trends (consumption, positivity, IQC failures) Collected manually (labor intensive) or automatically (seamless and better data quality) through connectivity Frequent critical analysis of data Site level analysis is essential Trends analysis of IQC failures are key Requires a minimum quantity of data for each site Enables identification of outliers (e.g. a facility with higher error rates) when POC NAT implementation is at scale Root cause analysis using error codes Allows for targeted and efficient corrective actions Informs refresher training needs Helps identify and filter which issues are product related Helps identify for which issues an IVD product complaint should be made

Internal Quality Control (IQC) Failures IQC are built-in instrument and assay controls looking at various vital components that are linked to assay validation (instrument, cartridge, sample, environment, etc.) IQC failures occur when the instrument detects an irregularity in one of the vital components. This can be due to several different root causes. No assay result is generated, rather an error code is, providing insight on which potential root cause might be involved in each particular IQC failure

Analysis of IQC Failures 6.00% 5.20% 5.00% Analysis across the network has a limited ability to uncover issues. Severe problems may be overlooked. 4.00% 3.00% 2.00% 1.00% 0.00% Country 1 Site-level analysis enables the identification of outliers and better deploys targeted investigative efforts Strategizing which corrective action is required relies on analyzing the distribution of potential root causes using the error codes 10 jlemaire@pedaids.or g

Analysis of IQC Failures Trend analysis enables the identification of whether: The issue is recent and still of concern

Analysis of IQC Failures Trend analysis enables the identification of whether: There could be a relationship with staff turnover, supply chain distribution, seasonal changes, etc.

Analysis of IQC Failures Trend analysis enables the identification of whether: The recent corrective action is helping resolve the issue

When to Fill a Complaint All product-related problematic categories need to be reported to the manufacturer using their technical support email address (or as prescribed by manufacturer). Any serious or moderate adverse event needs to also be reported to your national regulatory authority and the World Health Organization (WHO) (diagnostics@who.int) Ensure proper documentation of a complaint using the WHO PQ User complaint form for reporting problems and/or adverse events https://www.who.int/diagnostics_laboratory/procurement/111121_user_complaint_form_for_adverse_events_and_product_problems_reporting_english.pdf?ua=1 Ensure you document the frequency and the rate occurrence of the problem using total test performed as denominator Review records and documentation kept at testing sites (testing logbook) Assay name Lot number, expiry date Instructions for use (IFU) version number and language Storage conditions (temperature records) IFU matches SOP/training Any additional testing results to support the investigation (E.g. operator, kit box, lot)

How to Fill an IVD Complaint https://www.who.int/diagnostics_laboratory/procurement/111121_user_complaint_form_for_adverse_events_and_product_problems_reporting_english.pdf?u a=1

Guidance for End-Users WHO webpage on post-market surveillance contains many resources: https://www.who.int/diagnostics_laboratory/postmarket/en/ Part I of WHO normative guidance on post-market surveillance for IVDs: http://apps.who.int/iris/bitstream/10665/255576/1/9789241509213-eng.pdf?ua=1 Complaints and product alerts are published on: http://www.who.int/diagnostics_laboratory/procurement/complaints/en/

Thank You! For more information, please contact: jlemaire@pedaids.org

Examples of training materials, algorithms, SOPs and job aids Daily IQC failure register

Examples of Training Materials, Algorithms, SOPs and Job Aids IQC Failure Monthly Reporting

This document was made possible thanks to Unitaid s support. Unitaid accelerates access to innovation so that critical health products can reach the people who most need them. For more information, please contact: The EGPAF Innovation and New Technology Team at (innovation@pedaids.org). www.pedaids.org | www.unitaid.org