Lifting of Under Reservation: Recorded Crime Statistics Update

Lifting of the Under Reservation designation from Recorded Crime Statistics 25 October 2023 Jim Dalton, Crime & Criminal Justice Statistics, CSO

Presentation outline 1. 2. Introduction. Background to the application of the Under Reservation designation to Recorded Crime Statistics. Why is the Under Reservation categorisation being lifted now? What time periods does the lifting of the Under Reservation apply to? How confident can CSO and the public be in the crime data now? 3. 4. 5. 2

1. Introduction The Central Statistics Office (CSO) compiles and publishes Recorded Crime statistics for Ireland. Recorded Crime statistics are intended to inform users about the set of criminal offences which are reported to, or become known to An Garda S och na (AGS), Ireland s national police and security service. The statistics are compiled using administrative data recorded by AGS on its PULSE (Police Using Leading Systems Effectively) crime incident database. 3

2. Background to the application of the Under Reservation designation to Recorded Crime Statistics 2014 Garda S och na Inspectorate report 2014 Crime Investigation highlighted quality issues with recording of data on PULSE such as Some crimes being reported to AGS but not recorded on PULSE. Poor timeliness in creation of PULSE records and incomplete record- keeping. Misclassification of crimes. Crimes being marked as detected when there is no corresponding sanction for the offender (e.g., a prosecution). 4

2. Background to the application of the Under Reservation designation to Recorded Crime Statistics 2015 -2017 CSO twice suspended publication of Recorded Crime Statistics. CSO conducted quality reviews for 2011 and 2015 data which highlighted similar issues to the Garda Inspectorate report. 5

2. Background to the application of the Under Reservation designation to Recorded Crime Statistics March 2018 CSO resumed publication of Recorded Crime Statistics to meet societal and policy needs. Categorised as Statistics Under Reservation to actively alert users to quality issues with the underlying PULSE data. 6

2. Background to the application of the Under Reservation designation to Recorded Crime Statistics July 2018 CSO Quality Improvement Proposal 2018 highlighted three key recommendations to AGS : Development of a formal Data Quality Management System. Appointment of a Senior Manager with responsibility for Data Quality. Provision of comprehensive and publicly available document explaining how crime is recorded. 7

2. Background to the application of the Under Reservation designation to Recorded Crime Statistics December 2018 - 2023 CSO conducted further reviews of quality of Crime Statistics in 2018 (based on 2017 data), in 2021 (based on 2020 data) and in 2023 (based on Q1 2023 data). Progressive improvement in data quality noted. 8

Examples of data quality improvements Figure 1 Percentage of recorded crimes incidents assigned to correct crime ICCS classification 100% 98% 96% 94% 92% 90% 88% 2011 2015 2017 2020 2023 9

Examples of data quality improvements Figure 2 Percentage of detected crime incidents with valid charge/summons attached 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 2011 2015 2017 2020 2023 10

Examples of data quality improvements Figure 3 Percentage of non-crime incidents correctly classified as non-crime 100% 98% 96% 94% 92% 90% 88% 2011 2015 2017 2020 2023 11

2. Background to the application of the Under Reservation designation to Recorded Crime Statistics The CSO Quality review in 2021 had two recommendations: 1. The pathway to lifting the Under Reservation is in building enhanced public confidence in the quality of PULSE crime data. Such confidence can only be generated through a trusted and transparent data quality verification process by An Garda S och na, rooted in a clear appreciation of the data quality risks involved in the crime reporting and recording process . 2. End to end review of AGS crime reporting and recording process. 12

2. Background to the application of the Under Reservation designation to Recorded Crime Statistics CSO Review 2023 notes: Evidence of consistent high data quality levels (see earlier graphics). Much enhanced sense within AGS of the risks to PULSE data quality and the controls and resources needed to manage those risks. AGS PULSE data is fit for official statistical purposes, while also generating improved data for operational policing purposes. Systematic data quality checking now in place with metrics on same being published. Taken as a package, these give a level of assurance to users that they can be confident in PULSE data. 13

3. Why is the Under Reservation categorisation being lifted now? CSO Recommendations have been implemented as follows: A more formal quality management system for PULSE data with a focus on risk management fit for official statistical purposes and improved data for operational needs. AGS commissioned an independent review which found evidence of good controls for the acknowledged risks to data collection and quality. There is now a senior manager within AGS who has overall responsibility for data quality (CIO appointed November 2019). The provision by AGS of a comprehensive and publicly available document explaining how crime is recorded (2020). A new Memorandum of Understanding between the CSO and AGS which explicitly outlines expectations concerning AGS quality checking and reporting of same. 14

Examples of AGS risks and controls Risk Control Roll-out of CAD2 system will mean increased integration between CAD and PULSE plus additional supervisory controls Crimes reported but not recorded on PULSE Mis-classification of crime incident either as non-crime or wrong crime type The creation-review-PAF data cycle represents a closed loop process where incident data reported by Garda members goes through multiple checks including being presented back to local management for performance monitoring and operational decisions. PULSE data is incoherent and conflicting AGS have developed a series of coherency checks and any errors or inconsistencies are flagged for review 15

4. What time periods does the lifting of the Under Reservation apply to? The Under Reservation designation is lifted from all recorded crime data disseminated by the CSO from Quarter 1 2023. AGS has worked over the past number of years to advance the assurance levels which can be provided around Garda PULSE data, and in particular the quality management developments of 2022 and 2023. This work has seen the development of quality assurance processes by AGS over the period which has culminated in the implementation of a more formal quality management system by AGS. The cumulative impact of this work has been progressive improvement in the Recorded Crime series over time with higher data quality levels noted in various CSO reviews in recent years. Users should exercise judgement for data produced in the earlier years of the recorded crime time series given the legacy quality issues which have been commented on in various reviews. For example, detections data pre and post 2018 are not comparable given the improved governance controls introduced in that year. The CSO will continue to inform users of the quality of the data they are using, and of any issues which may need to be noted around time series comparability as they arise. 16

5. How confident can CSO and the public be in the crime data now? The new AGS controls, quality checking, and reporting, with more to be added, coupled with the CSO reserving the right to do its own reviews, provides a level of assurance such that users can be confident in the data. The CSO has seen is a much-enhanced sense within AGS of the risks to PULSE data quality and the controls and resources needed to manage those risks. Note of caution as earlier years of the time series may still encounter some of the legacy data quality issues. The CSO will continue to update users around how to interpret the recorded crime time series, particularly with reference to comparability of the series over time. The CSO thanks the AGS for their collaboration and professionalism over the last number of years in addressing the issues. 17

Upcoming publications Circumstances of People linked to Justice Sanctions early November. Recorded Crime Detection 2022 20 November 2023. Recorded Crime Q3 2023 20 December 2023 Note that the landing page on CSO website will look a bit different now that the Under Reservation has been lifted. 18

Thank you 19

Appendices Garda Inspectorate Crime Investigation report: https://www.gsinsp.ie/wp- content/uploads/2019/07/Crime-Investigation-Full- Report.pdf CSO Quality Reviews of Recorded Crime Statistics: https://www.cso.ie/en/statistics/crimeandjustice/reviewoft hequalityofrecordedcrimestatistics/

Appendices CSO Quality Improvement Proposal July 2018 https://www.cso.ie/en/methods/crime/methodologydocuments/qualit yimprovementproposaljuly2018/ AGS Review of Quality of Pulse Crime data for 2022 https://www.garda.ie/en/information-centre/statistics/ags-review-of- the-quality-of-pulse-crime-data-for-2022.pdf

Appendices AGS Data Quality reporting: https://www.garda.ie/en/information-centre/statistics/ Independent review of the PULSE data quality process: ags-review-of-process-for-ensuring-data-quality-on-garda-pulse-system.pdf 22