Improving Labour Market Outcomes Through Targeted Job Services in Australia

Targeting Labour Market Programs Towards the Disadvantaged An Analysis of Streaming in Job Services Australia Patrick Hamshere and Lixin Cai jobactive Evaluation Section Evaluation, Research and Evidence Branch Australian Government Department of Employment Disclaimer: the views expressed herein are those of the authors and do not necessarily reflect the views of the Australian Government Department of Employment. The analysis in this presentation is preliminary and subject to change.

Overview Background and motivation Policy environment Methodology and data Preliminary results Concluding remarks

Background Active Labour Market Programs (ALMPs) in Australia typically vary a job seekers level of treatment (a serviceintensity ) depending on their degree of labour market disadvantage. A way of targeting resources within a program to where it will be utilised most effectively. True for the current program (jobactive) and its predecessor model (Job Services Australia; JSA).

This Paper This paper is concerned with measuring the effect of different service intensities (i.e. targeting of services) on labour market outcomes in the second contract period of JSA (2012 to 2015). Uses Regression Discontinuity Design (RDD) to measure effects as a whole and by subgroup: income support type gender age.

Motivation To measure howmuch labour market outcomes improved with the increase in service intensity. Implications for value for money / justify spending. Provides insight on how ALMPs can achieve labour market outcomes more broadly and for whom. Helps to identify subpopulations where this type of targeting is/is not effective helps to reconfigure our targeting instruments.

JSA Policy Environment Each person who registered as a job seeker was placed into one of four Streams . Streaming was the primary instrument used to target assistance in JSA. Essentially, a job seekers Stream represented their degree of service intensity.

JSA Policy Environment Job seekers were allocated to Streams based on their difficulty in gaining and maintaining employment. Allocation to Streams 1-3 depended on their score from Jobseeker Classification Instrument (JSCI). Allocation to Stream 4 depended on a further assessment to establish they had personal issues (e.g. alcohol or drug problem).

JSA Policy Environment The JSCI questionnaire was used to identify job seekers, at the time of service registration, with the greatest risk of long-term unemployment. It is based off a logistic regression model that estimated the relative weights of the risk factors that have been identified by policy makers as being associated with long-term unemployment. By combining the weights and the risk factors, a job seeker gets a JSCI score.

JSA Policy Environment We focus the analysis on the first three Streams. RDD can only measure the effect for those Streams where allocation was a function of the JSCI score only. More on this later.

JSCI Scores and Streams 1 76 Least Most disadvantaged disadvantaged Each job seeker received a JSCI score. Set of integers: {1 76}. Ordered by degree of labour market disadvantage.

JSCI Scores and Streams Stream 1 Stream 2 Stream 3 19 20 28 29 1 76 Least Most disadvantaged disadvantaged Allocation to Streams 1-3 depended on the JSCI cut- off scores.

Stream Differences Maximum outcome fee ($) Maximum service fee ($) Employment Pathway Fund ($) Stream 1 1,258 581 11 Stream 2 2,064 885 550 Stream 3 5,880 1,120 1,100 Outcome fees: paid to providers for an employment outcome. Service fee: paid to providers to service job seekers. Employment Pathway Fund: expenditure for training.

Average Expenditure per Job Seeker in Their First 12 Months of JSA 1400 Stream 3 Stream 2 Stream 1 1200 1000 800 600 400 200 0 $ 11 13 15 17 19 21 23 25 27 29 31 33 35 37 JSCI score

Methodology We use a Regression Discontinuity Design (RDD) to estimate the effect of Stream placement on outcomes. Outcome used is the off income support rate at 12 months from commencement (other outcome rates are also checked more on this later).

Methodology RDD relies on the fact that Stream placement depends only on the JSCI score and there is a cut-off point in JSCI such that Stream allocation changes. Underlying idea is that job seekers just below cut-off point (who did not receive treatment) are good comparisons to those just above (who did receive treatment).

Methodology Methodology is implemented by: estimating the function mapping JSCI scores to outcomes measuring the change in this function at the JSCI boundary score. RDD controls for jobseeker characteristics because it controls for changes in JSCI score.

Illustrative Example Outcome RDD measures difference in the function (solid black line) at boundary. Stream 1 Stream 2 Distance from JSCI cut-off 0 (cut-off) Solid black line is function mapping distance to outcomes. Dotted black line is prediction using function of other Stream.

Methodology If more services and assistance causes job seekers to achieve better outcomes then there should be a discontinuous changes in the level and/or slope of function at the JSCI cut-off point. RDD relies on three underlying assumptions.

Assumption 1 Assumption 1: there would not have been a jump in outcomes at the cut-off score if not for the change in Stream. In other words, the JSCI questionnaire accurately captured disadvantage. This assumption was largely satisfied except for a few sub-populations (described later).

Outcome Rate By JSCI Score 80 Stream 1 Stream 2 70 60 50 40 30 20 10 0 % -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7 8 9 Distance from cut-off score

Assumption 2 Assumption 2: job seekers can not perfectly manipulate their JSCI score. We do not find evidence of manipulation (graph). Important population excludes job seekers who changed Streams during service (more on study population soon). to note that our study

Job Seekers By JSCI 40,000 No evidence of a spike in job seeker numbers at boundary scores. 30,000 20,000 10,000 28 29 19 20 0 Number of job seekers 11 13 15 17 19 21 23 25 27 29 31 33 35 37 JSCI Scores

Functional Form Assumption 3: proposed functional form is a suitable fit to the JSCI score/outcome relationship. Most controversial assumption because true functional form is unobservable. Estimates will be bias if this assumption is violated.

Functional Form To minimize chance of choosing the wrong specification we did not rely on only one functional form. Instead, tested variety of functional forms. Results that were stable across equally plausible specifications were regarded as more reliable than those sensitive to minor changes in specification.

Functional Form We test six specifications:

Ranking Functional Forms We follow other researchers in reporting results for the specification with the lowest Akaike Information Criteria. The AIC is a measure of the functional form s goodness of fit .

Data We administrative data and Department of Human Services income support data. Specifically: use Department of Employment gender and age income support type JSCI scores and Stream.

Study Population Second contract period of JSA (2012 to 2015): first three months of study period are excluded to reduce transition effects last 12 months are excluded because it would not be possible to assess a jobseekers 12 month outcome rate.

Study Population A number of jobseekers are excluded from analysis in order to isolate the pure Stream effect. Includes jobseekers with: Stream changes during service Streaming that was not based on JSCI scores (e.g. Stream 4) Income support types other than Newstart Allowance (NSA) or Youth Allowance (Other) (YA(O)). Around 400,000 job seekers in total.

Preliminary Results For NSA recipients a moderate Stream effect was found regardless of functional form: Stream 2: +6.1 ppts (robust) Stream 3: +1.3 ppts (robust). For YA(O) recipients the effect is smaller: Stream 2: +0.7 ppts (robust) Stream 3: +2.5 ppts (not robust).

Preliminary Results Why was Streaming less effective for YA(O) recipients? YA(O) recipients may have different behaviour to NSA recipients because of their lower age. Looking at the off-IS rate may mask the Stream effect for young people. What about study? Including movements to YA(S) in the outcome rate increased the Stream effect slightly and made it significant.

Male and Female (NSA) For males a large Stream effect was found: Stream 2: +9.3 ppts (robust) Stream 3: +5.3 ppts (robust). For females the effect is smaller and less robust: Stream 2: +2.9 ppts (robust) Stream 3: -0.2 ppts (not robust) Stream 3: +2.0 ppts (robust) when partial off-IS rate included in the outcome rate.

Male and Female The results are consistent with the findings of other studies that show females drivers to obtain work may be very different to males. This suggests other issues external to a concept of disadvantage are at play when looking for work and finding work that has implications on the way to service different cohorts.

Age Stream effects largest for prime-aged workers (they made up most of the NSA recipients). For job seekers aged over 55 years old, the JSCI did not capture accurately. Hence, assumption (1) that there are no jumps in the outcome rate at non-threshold JSCI scores not satisfied. Identification strategy questionable for this cohort. disadvantage

Implications We concluded that differentiating service intensity by level of disadvantage can be important in addressing unemployment amongst the most disadvantaged. However, such policies may require tailoring to the characteristics of individual subpopulations in order to improve their effectiveness.

Implications For example, the relatively low Stream effect detected for females may be linked to the availability, affordability and quality of formal childcare. Improvements in childcare policies may help increase the effectiveness of labour market assistance for females.

Future Research Quantifying implications for value for money but we have not undertaken analysis on this yet. the Stream effect has For example: did the reduction in the income support bill (from increased outcomes) more than offset the marginal cost of the higher Stream placement?

Future Research Placebo testing to check the identification strategy. Involves implementing the RD procedure on non-threshold JSCI scores to check whether it discontinuities. picks up any

Questions and comments :)