Impact of MERIT Program on Re-Offending and Imprisonment

The impact of the MERIT program on re-offending and imprisonment Don Weatherburn*, Sara Rahman**, Suzanne Poynton**, Stephanie Todd***, Samantha Black***, Tanya Merinda*** & Michael Farrell* *National Drug and Alcohol Research Centre, **NSW Bureau of Crime Statistics and Research ***NSW Ministry of Health

What is MERIT? Program type: Pre-plea AOD treatment program for defendants in Local Court The aim of the program is to: reduce the harms associated with AOD [alcohol and other drug] use, improve the health and well- being of defendants and reduce their offending (NSW Health: MERIT Model of Care 2022) Program length: Three months Treatment options: detoxification, pharmacotherapy, residential rehabilitation, counselling, case management and welfare support Finalisation: Report on progress taken into account in sentencing

How does it work? Referrals can be made by the defendant, his/her lawyer, the magistrate or the police (rare) The relevant Local Court MERIT team assess each defendant in terms of: Risk to self or others Alcohol and other drug use If the team recommends the defendant is suitable for MERIT and the magistrate concurs, the defendant enters the program. If the defendant is deemed unsuitable or the magistrate opposes placement on MERIT, the case proceeds as a normal prosecution Further information: https://www.health.nsw.gov.au/aod/programs/Pages/merit-model-of- care.aspx

Who is eligible? To be eligible for MERIT, a defendant must be an adult; be released on bail; have a demonstrable and treatable drug problem; reside where you can participate in treatment programs; be charged with a drug-related offence; not be facing a strictly indictable offence not be facing a charge of sexual assault or serious violence; provide written and informed consent

Findings from earlier evaluations Pasey, J., Bolitho, Scantleton, J. & Flaherty, B. (2007): The Magistrates Early Referral into Treatment (MERIT) Pilot program: Court Outcomes and Recidivism. Australian & New Zealand Journal of Criminology, 40(2), 199 217. https://doi.org/10.1375/acri.40.2.199 Key finding: Lower rates of offending among program completers than non-completers NSW Health (2007). The Magistrates Early Referral into Treatment (MERIT) program: health outcomes. URL: http://www.merit.justice.nsw.gov.au/Documents/MERIT Health Outcomes.pdf. Key finding: Improvements in health outcomes as defendants progressed through the program Lulham, R. (2009). The Magistrates Early Referral into Treatment Program. Impact of program participation on re-offending by defendants with a drug use problem. Crime and Justice Bulletin 131. Sydney: NSW Bureau of Crime Statistics and Research. Key finding: No effect on ITT but evidence of effect among program completers McSweeney, T., Hughes, C.E. & Ritter, A. (2016). Tackling 'drug-related' crime: Are there merits in diverting drug-misusing defendants to treatment? Findings from an Australian Case Study. Findings from an Australian Case Study. Australian and New Zealand Journal of Criminology, 49(2): 198-220. Key finding: higher rates of offending among MERIT participants

Limitations with earlier evaluations Pasey et al. (2007). Completers vs non-Completers (risks selection bias) NSW Health (2007). No counterfactual Lulham (2009). Bivariate probit but questionable instrument (legal representation). McSweeney et al. (2016). Drug use not included in model for treatment selection + limited controls for risk of reoffending (e.g., no concurrent offence measure, no prior prison measure

The present evaluation Includes a more extensive set of controls for substance use Examines a broader range of outcomes Employs a more rigorous defence against selection bias Adjusts for exposure (i.e., time spent in custody) Employs a much larger sample than previous studies

Data sources The NSW Bureau of Crime Statistics and Research reoffending database (ROD) All persons appearing in a Local Court between 2008 and 2017 who: Had not been charged with a strictly indictable offence Had not been charged with sexual assault or an offence involving serious violence; Appeared in a Local Court where MERIT was available Had not had their charge(s) dismissed on mental health grounds The NSW Health: MERIT database The NSW Health APDC: AOD related admissions The NSW Health EDDC: AOD related admissions

Treatment and control groups Treatment group: All those eligible for MERIT who appeared in a MERIT court between 2007 and 2017, had been referred to and accepted into MERIT, and who had had: An AOD related ED admission, or an AOD related hospital admission Or both N = 10,294 Control group: All those eligible for MERIT who appeared in a MERIT court between 2007 and 2017,who had not been referred to MERIT and who had had: An AOD related ED admission, or an AOD related hospital admission Or both N = 391,486

Outcomes and controls Outcomes Whether the defendant was convicted of having committed a further offence within 12 months after the referral date (treatment group) or after the sentence date (control group) Whether the defendant was imprisoned for the offence Controls Sex, age, Indigenous status, offence type, concurrent offences, number of prior convictions, whether imprisoned in the last five years Also, whether convicted of a drug trafficking offence, whether convicted of assault, whether convicted of an indictable offence, whether had an OTP episode in the last five years, whether had a treatment (MDS) episode in the last five years, whether on OTP at first court appearance Note: OTP = opioid treatment plan, MDS = Minimum Data Set (MDS) for drug and alcohol treatment services (a marker of having had treatment for drug and/or alcohol problems)

Analysis Approach 1: 2SLS 1. Tried to carry out a residualised IV analysis (a la Dobbie, Goldin & Yang 2018) 2. Allocation of cases to magistrates not random conditional on court and time fixed effects Augmented inverse probability of treatment weighting (AIPW) Essentially a method for matching treatment subjects with control subjects in terms of factors likely to result in treatment General approach 1. Ran a series of crosstabs to identify correlates of treatment, re-offending and imprisonment 2. Ran logistic regression models to identify independent correlates of treatment, reoffending and imprisonment 3. Included significant predictors in the AIPW analysis 4. Checked covariate balance and propensity score overlap

Sample description Variable Frequency Percent Variable Frequency Percent Sex Prior imprisonment female male 84,569 282,100 23.06 76.94 No Yes 299,969 72,267 80.59 19.41 Indigenous Status Reconvicted in one year unknown 102,650 233,040 31,090 27.99 63.54 8.48 No Yes 270,180 96,046 73.77 26.23 non-Indigenous Indigenous Age group Reconvicted in two years 18-24 25-34 35-44 45+ 95,657 108,837 88,095 79,647 25.7 29.24 23.67 21.4 No Yes 232,662 132,644 63.69 36.31 Group status Control Treatment 391,486 10,294 94.02 2.56 Offence group violent 59,518 155,884 38,054 30,858 38,536 49,386 15.99 41.88 10.22 8.29 10.35 13.27 Imprisoned theft/fraud illicit drugs driving No Yes 304,846 19,396 81.09 5.88 justiceprocedure OTP episode in last five years other No Yes 299,215 25,027 92.28 7.72 concurrent offences none or one two or more 251,955 120,281 67.69 32.31 MDS episode in last 5 years No Yes 115,923 208,319 37.75 64.25 prior convictions none in last 5yrs one in last 5yrs 2/3 in last 5yrs 4+ in last five years 116,276 57,878 64,953 133,129 31.24 15.55 17.45 35.76 On OTP at index court app. No Yes 303,837 20,405 93.71 6.29

Sample description Variable Frequency Percent Variable Frequency Percent Sex Prior imprisonment female male 84,569 282,100 23.06 76.94 No Yes 299,969 72,267 80.59 19.41 Indigenous Status Reconvicted in one year unknown 102,650 233,040 31,090 27.99 63.54 8.48 No Yes 270,180 96,046 73.77 26.23 non-Indigenous Indigenous Age group Reconvicted in two years 18-24 25-34 35-44 45+ 95,657 108,837 88,095 79,647 25.7 29.24 23.67 21.4 No Yes 232,662 132,644 63.69 36.31 Group status Control Treatment 391,486 10,294 97.44 2.56 Offence group violent 59,518 155,884 38,054 30,858 38,536 49,386 15.99 41.88 10.22 8.29 10.35 13.27 Imprisoned theft/fraud illicit drugs driving No Yes 304,846 19,396 94.02 5.98 justiceprocedure OTP episode in last five years other No Yes 299,215 25,027 92.28 7.72 concurrent offences none or one two or more 251,955 120,281 67.69 32.31 MDS episode in last 5 years No Yes 115,923 208,319 37.75 64.25 prior convictions none in last 5yrs one in last 5yrs 2/3 in last 5yrs 4+ in last five years 116,276 57,878 64,953 133,129 31.24 15.55 17.45 35.76 On OTP at index court app. No Yes 303,837 20,405 93.71 6.29

Unadjusted outcomes by treatment status Re-convicted within 12 mths Treatment status Treatment status Imprisoned No Yes Total No Yes Total 230,240 73.06 5,187 60.79 235,427 72.73 84,902 26.94 3,346 39.21 88,248 27.26 297,237 94.15 7,609 89.09 304,846 94.02 18,465 5.85 931 10.90 19,396 5.98 Not referred Not referred 315,179 315,742 Referred & treated Referred & treated 8,530 8,537 Total Total 323,709 324,279

Bi-variate relationships with reoffending within one year Reconvicted in 12 mths No Reconvicted in 12 mths No Variable p-value Variable p-value Yes Yes Indictable offence sex No Yes 75.88 68.71 24.12 31.29 female 76.27 male 71.67 23.73 28.33 <0.001 <0.001 Concurrent offences Indigenous none or one two or more 76.03 66.18 23.97 33.82 non-Indigenous 72.11 Indigenous 55.46 27.89 44.54 <0.001 <0.001 Age group Convicted in last 12 months 18-24 69.15 25-34 69.85 35-44 45+ 82.48 30.85 30.15 27.9 17.52 No Yes 80.03 55.16 19.97 44.84 <0.001 <0.001 72.1 Prior convictions none in last 5yrs one in last 5yrs 2/3 in last 5yrs 4+ in last five years 89.29 81.11 73.79 55.15 10.71 18.89 26.21 44.85 Remoteness groups outer regional, remote, very remote 70.82 major cities, inner regional 73.24 Offence group <0.001 29.18 26.76 <0.001 Prior imprisonment violent 74.02 theft/fraud 79.24 illicit drugs 62.91 driving 66.24 Justice procedure 66.99 25.98 20.76 37.09 33.76 33.01 31.18 No Yes 78.33 48.6 21.67 51.4 <0.001 <0.001 OTP episode in last five years No Yes 74.65 47.22 25.35 52.78 <0.001 other 68.82 MDS episode in last five years Drug trafficking offence No Yes 64.89 77.09 35.11 22.91 No 72.68 Yes 77.93 27.32 22.07 <0.001 <0.001 On OTP at index court app. Assault offence No Yes 48.1 76.9 51.9 23.1 No 72.76 Yes 72.46 27.24 27.54 <0.001 0.276

Bi-variate relationships with reoffending within one year Reconvicted in 12 mths Variable p-value Variable Reconvicted in 12 mths p-value No Yes No Yes Indictable offence sex No Yes 75.88 68.71 24.12 31.29 <0.001 female 76.27 male 71.67 23.73 28.33 <0.001 Concurrent offences Indigenous none or one two or more 76.03 66.18 23.97 33.82 <0.001 non-Indigenous 72.11 Indigenous 55.46 27.89 44.54 <0.001 Convicted in last 12 months Age group No Yes 80.03 55.16 19.97 44.84 <0.001 18-24 69.15 25-34 69.85 35-44 45+ 82.48 30.85 30.15 27.9 17.52 <0.001 Prior convictions 72.1 none in last 5yrs one in last 5yrs 2/3 in last 5yrs 4+ in last five years 89.29 81.11 73.79 55.15 10.71 18.89 26.21 44.85 Remoteness groups outer regional, remote, very remote 70.82 major cities, inner regional 73.24 Offence group <0.001 29.18 26.76 <0.001 Prior imprisonment violent 74.02 theft/fraud 79.24 illicit drugs 62.91 driving 66.24 justiceprocedure 66.99 other 68.82 25.98 20.76 37.09 33.76 33.01 31.18 No Yes 78.33 48.6 21.67 51.4 <0.001 OTP episode in last five years <0.001 No Yes 74.65 47.22 25.35 52.78 <0.001 MDS episode in last five years Drug trafficking offence No Yes 64.89 77.09 35.11 22.91 No 72.68 Yes 77.93 27.32 22.07 <0.001 <0.001 On OTP at index court app. Assault offence No Yes 48.1 76.9 51.9 23.1 No 72.76 Yes 72.46 27.24 27.54 <0.001 0.276

Bi-variate relationships with acceptance into treatment p- Accepted into treatment Accepted into treatment Variable p-value Variable valu e No Yes No Yes indictable offence sex No Yes 98.47 95.97 1.53 4.03 <0.0 01 female 97.24 male 2.76 2.6 0.016 97.4 concurrent offences Indigenous none or one two or more 98.47 95.97 1.53 4.03 <0.0 01 97 95.3 non-Indigenous Indigenous 3 <0.001 4.7 convicted in last 12 mths age group 97.3 18-24 25-34 96.72 35-44 97.19 45+ 98.64 2.7 3.28 2.81 1.36 No Yes 97.64 96.72 2.36 3.28 <0.0 01 <0.001 prior convictions none in last 5yrs one in last 5yrs 2/3 in last 5yrs 4+ in last five years 99.15 98.25 97.22 95.61 0.85 1.75 2.78 4.39 remoteness groups outer regional, remote, very remote 95.68 major cities, inner regional 97.38 offence group <0.0 01 4.32 2.62 <0.001 prior imprisonment violent 97.65 theft/fraud 98.84 illicit drugs 93.91 driving 93.14 justiceprocedure 97.62 other 97.93 2.35 1.16 6.09 6.86 2.38 2.07 No Yes 97.86 95.25 2.14 4.75 <0.0 01 <0.001 OTP episode in last five years No Yes 98.2 93.3 1.8 6.7 <0.0 01 MDS episode in last 5 years drug trafficking offence No Yes 96.59 97.8 3.41 2.2 <0.0 01 No 97.51 Yes 83.54 2.49 16.46 <0.001 On OTP at index court app. Assault offence No Yes 93.58 98.01 6.42 1.99 <0.0 01 No 97.42 Yes 96.83 2.58 3.17 0.276

Bi-variate relationships with acceptance into treatment Accepted into treatment No Accepted into treatment Variable p-value Variable p-value Yes indictable offence No Yes No Yes 98.47 95.97 1.53 4.03 sex <0.001 female 97.24 male 2.76 2.6 0.016 97.4 concurrent offences Indigenous none or one two or more 98.47 95.97 1.53 4.03 <0.001 97 95.3 non-Indigenous Indigenous 3 <0.001 4.7 convicted in last 12 mths age group No Yes 97.64 96.72 2.36 3.28 <0.001 97.3 18-24 25-34 96.72 35-44 97.19 45+ 98.64 2.7 3.28 2.81 1.36 <0.001 prior convictions none in last 5yrs one in last 5yrs 2/3 in last 5yrs 4+ in last five years 99.15 98.25 97.22 95.61 0.85 1.75 2.78 4.39 remoteness groups outer regional, remote, very remote 95.68 major cities, inner regional 97.38 offence group <0.001 4.32 2.62 <0.001 prior imprisonment violent 97.65 theft/fraud 98.84 illicit drugs 93.91 driving 93.14 justiceprocedure 97.62 other 97.93 2.35 1.16 6.09 6.86 2.38 2.07 No Yes 97.86 95.25 2.14 4.75 <0.001 OTP episode in last five years <0.001 No Yes 98.2 93.3 1.8 6.7 <0.001 MDS episode in last 5 years drug trafficking offence No Yes 96.59 97.8 3.41 2.2 No 97.51 Yes 83.54 2.49 16.46 <0.001 <0.001 On OTP at index court app. Assault offence No Yes 93.58 98.01 6.42 1.99 No 97.42 Yes 96.83 2.58 3.17 <0.001 0.276

Odds ratios and confidence intervals for one year re-offending model 5.000 4.500 4.000 3.500 3.000 Odds Ratio 2.500 2.000 1.500 1.000 0.791 0.500 0.000

Odds ratios and confidence intervals for imprisonment model 8.000 7.000 6.000 5.000 Odds Ratio 4.000 3.000 2.000 1.000 0.687 0.000

AIPW results: re-offending at 12mths Robust std. error Lower 95% CI Upper 95% CI Re-offending at 12 Coefficient z P>|z| Average treatment effect Referred and accepted -0.046 0.016 -2.750 0.006 -0.079 -0.013 Potential outcomes mean Not referred 0.367 0.002 155.17 <0.001 0.363 0.372

AIPW results: imprisoned Robust std. error. Lower 95% CI Upper 95% CI Imprisoned Coefficient z P>|z| Average treatment effect Referred and accepted -0.013 0.012 -3.950 <0.001 -0.019 -0.0004 Potential outcomes mean Not referred 0.067 0.0006 119.55 <0.001 0.067 0.062

So much for the good news, now for the bad.

Propensity score overlap

Covariate balance Standardized differences Variable Variance ratio Raw -0.031 0.416 Weighted -0.006 0.085 Raw 1.039 0.720 Weighted 1.007 0.950 Male Indigenous Age group 0.170 0.039 -0.288 0.017 0.017 -0.029 1.127 1.046 0.561 1.015 1.020 0.956 25-34 35-44 45+ Overidentification test for covariate balance H0: Covariates are balanced chi2(17) = 277.97 Prob > chi2 = <0.001 Offence group -0.514 0.394 0.403 -0.046 -0.084 -0.054 0.017 0.085 0.013 -0.044 0.590 2.055 2.408 0.895 0.805 0.973 1.043 1.276 1.030 0.898 theft/fraud illicit drugs driving justice procedure other Concurrent offences 0.748 0.046 0.980 1.030 two or more Prior convictions -0.155 0.031 0.507 0.348 0.457 -0.244 0.377 -0.026 -0.016 0.187 0.147 0.097 -0.231 0.078 0.706 1.052 1.021 1.454 2.692 1.095 2.704 0.950 0.973 1.064 1.214 1.330 1.090 1.306 one in last 5yrs 2/3 in last 5yrs 4+ in last five years Prior imprisonment OTP episode in last five years MDS episode in last 5 years OTP at first appearance

Conclusions Treatment appears to reduce re-offending and imprisonment Lack of covariate balance makes this conclusion uncertain Issues yet to resolve: What really drives treatment entry? Does treatment affect some subgroups but not others? Are some treatments better than others? Are some treatment teams better than others? Key policy implication: None (this is a work in progress) Key take home message: Talk to the evaluator before rolling out a program