A theory-driven approach to improving ALMPs

Previous work has shown that a theory-driven approach to improving labour market outcomes can lead to better outcomes (Liu et al. Reference Liu, Huang and Wang2014; Malmber-Heimonen et al. Reference Malmber-Heimonen, West and Vuori2017). One challenge in taking a theory-driven approach is that one needs to choose a theory which can drive the approach. This choice is not simple. There are numerous theories which one can choose to use, and little work has been undertaken to look at which theories explain the most variance of outcomes across the diverse set of ALMPs used (Manroop and Richardson Reference Manroop and Richardson2016).

Manroop and Richardson (Reference Manroop and Richardson2016) describe the numerous psychological, social and emotional factors that affect job search, which are typically examined by psychologists and sociologists, as well as the prevailing labour market characteristics and policies that are usually explored by labour economists (Layard et al. Reference Layard, Layard, Nickell and Jackman2005; Arntz and Wilke Reference Arntz and Wilke2009). The psychological theories commonly used to explain the job search effectiveness are extensions of broader theories of behaviour change such as the Theory of Reasoned Action (Fishbein and Ajzen Reference Fishbein and Ajzen1975) and Social Cognitive Theory (Bandura Reference Bandura1977; Lent et al. Reference Lent, Brown and Hackett1994). These will largely focus on goal orientation, self-efficacy and emotional self-regulation. This focus allows researchers to divorce the factors that affect job search effectiveness from the policy decisions that are studied by labour economists. However, by separating the two, we miss the important interplay between the psychological and the bureaucratic factors that influence employment outcomes.

In this article, we advocate a multidisciplinary approach that leverages multiple theories to develop effective labour market policies. In the next section, we describe how we view specific components of the UK ALMP through a theoretical lens to lead to better employment outcomes for jobseekers. We have specifically flagged the following factors that can be improved by behavioural insights interventions: cognitive frictions and “sludge”, goals and emotional self-regulation. These were chosen as we found specific opportunities to implement alterations to the current ALMP to affect these theoretical constructs.

Removing friction and sludge

A recurring finding from behavioural science is that relatively small costs in the present can strongly influence our decisionmaking and behaviour, even when stacked against disproportionately larger benefits in the longer term. This concept has recently been formalised by Sunstein (Reference Sunstein2019), as “sludge”. As the book Nudge (Thaler and Sunstein Reference Thaler and Sunstein2008) described how small changes to policies and services can encourage people to undertake behaviours, the corollary of this is that small frictions can stop people from undertaking behaviours. Sunstein (Reference Sunstein2019, p. 1844) formally describes “sludge” as “friction, reducing access to important licenses, programs, and benefits”.

Mortensen (Reference Mortensen2011) identified that search frictions can prevent the proper functioning of markets, including job search markets, even in the absence of nonstandard discounting, which would serve to exacerbate the effects. However, less attention has been paid to the frictions that are created by labour market policies themselves. As described in section “previous studies”, jobseekers are often required to complete numerous forms that can be described as “sludge”. By reducing this sludge, it is possible that jobseekers’ outcome can be improved.

Anchoring, goal orientation and implementation intentions

One of the principal strands of our research concerned how goal-setting and anchoring might help jobseekers find a job. Goals feature heavily in the social cognitive models of job search (Kanfer and Bufton Reference Kanfer and Bufton2015; Manroop and Richardson Reference Manroop and Richardson2016). However, less attention has been paid to specific ways in which the implementation of ALMPs can influence the generation of goals. In ALMPs which include a conditionality regime, a specific requirement for the number of activities has to be set (Crépon and Van den Berg Reference Crépon and Van den Berg2016). In this article, we explore how the requirements of a conditionality regime could potentially have iatrogenic effects due to anchoring.

Anchoring describes the tendency to “anchor” to an initial piece of information, which then influences subsequent judgements, even when this initial piece of information should be irrelevant. We were particularly interested in the extent to which the requirement for jobseekers to conduct three job searches (as was the case in the JSA regime) each week might anchor jobseekers to a suboptimal level of job-searching activities.

Similarly, the goal-setting literature suggests that having a clearly defined goal is a strong predictor of job search intensity, which is a predictor of the number of job interviews people receive, which, in turn, is a predictor of the number of job offers jobseekers receive (Creed et al. Reference Creed, King, Hood and McKenzie2009). In other studies, low targets have been found to lower motivation and performance (Cervone and Peake Reference Cervone and Peake1986), and loose deadlines have been shown to lead to slower work pace than tight ones (Latham and Locke Reference Latham and Locke1975).

The team also examined the literature on how to follow through on a goal that has been set. One of the core strands of the literature that the research team drew upon here concerned “implementation intentions” (Gollwitzer Reference Gollwitzer1999). In broad terms, this literature shows that people are more likely to follow through on an intention if they are able to state where, when and how they are going to do it. A review of 94 studies found significantly higher success rates for a range of goals when the technique was used (Gollwitzer and Sheeran Reference Gollwitzer and Sheeran2006). This technique has been shown to help people eat more fruit, take more public transport, increase exercise and quit smoking (Gollwitzer and Sheeran Reference Gollwitzer and Sheeran2006) and vote (Nickerson and Rogers Reference Nickerson and Rogers2010). This technique seems to be effective because it encourages individuals to make associations between certain cues (e.g. get home from work) and the action required to follow through on their intentions (e.g. go for a run).

Implementation intentions have also been shown to reduce cognitive load (Brandstätter et al. Reference Brandstätter, Lengfelder and Gollwitzer2001) which is likely to be heightened by the stress of being unemployed (Brown et al. Reference Brown, Balluz, Ford, Giles, Strine, Moriarty, Croft and Mokdad2003; Mullainathan and Shafir Reference Mullainathan and Shafir2013) and the sludge designed in some ALMPs. Therefore, it is hypothesised that the use of implementation intentions can improve job search effectiveness of ALMPs. Indeed, since the launch of our interventions, goal-setting has been proven to be an effective counter for youth unemployment (Abel et al. Reference Abel, Burger, Carranza and Piraino2017).

Stress, resilience and self-efficacy

Studies have shown that there is a strong link between jobseekers’ anxiety, stress and depression, and their employment status (Turner et al. Reference Turner, Kessler and House1991). For example, Barber et al. (Reference Barber, Hollenbeck, Tower and Phillips1994) found that job search decreased over time as a result of stress, and there is evidence that addressing poor mental health can lead to better employment outcomes (Vinokur et al. Reference Vinokur, van Ryn, Gramlich and Price1991).

In a UK policy context, this has led to a greater emphasis on the provision of mental health treatment for the long-term unemployed (Gyani et al. Reference Gyani, Shafran, Layard and Clark2013; Layard and Clark Reference Layard and Clark2014). However, it has also been recognised that a preventative approach might be more helpful in the long term than a treatment approach. One way in which this might be done is to build resilience in individuals facing challenging circumstances. A study conducted on the United States (US) Army’s Master Resilience Training (MRT) course, a 10-day programme of study that teaches resilience skills to noncommissioned officers, has been found to significantly reduce depressive symptomatology (Gillham et al. Reference Gillham, Reivich, Freres, Chaplin, Shatté, Samuels and Seligman2007; Brunwasser et al. Reference Brunwasser, Gillham and Kim2009). The MRT course has been found to be effective in increasing resilience and psychological health, especially among individuals aged 18–24 (Lester, Harms et al. Reference Lester, Harms, Herian, Krasikova and Beal2011; Lester, McBride et al. Reference Lester, McBride, Bliese and Adler2011; Reivich et al. Reference Reivich, Seligman and McBride2011).

Another area of research that the team drew upon to develop the interventions concerned self-confidence and self-efficacy (Bandura Reference Bandura1977). One study found that encouraging people to identify their strengths using the VIA Signature Strengths Survey online and then instructing them to use those strengths in a new way led to a decrease in depressive symptomatology immediately afterwards, and these gains were sustained after six months (Seligman et al. Reference Seligman, Steen, Park and Peterson2005).

Similarly, writing about past traumas, known as “expressive writing”, has been successfully used in a variety of settings, proving to be effective in improving participants’ health outcomes (Baikie and Wilhelm Reference Baikie and Wilhelm2005). It has also been shown to facilitate returning to work after an unemployment spell (Spera et al. Reference Spera, Buhrfeind and Pennebaker1994).

Finally, the implementation intentions described in section “Anchoring, goal orientation and implementation intentions” could also have an effect on reducing stress, as they enforce a structure on the inherently unstructured process of job search (Brown et al. Reference Brown, Cober, Kane, Levy and Shalhoop2006). An imposition of structure on job search has been shown to reduce stress and anxiety (Wanberg Reference Wanberg1997). This has the potential to create a virtuous cycle that will increase resilience and improve employment outcomes.

Evaluating labour market policies

There is a precedent for carrying out field experiments in labour markets (Dolton and O’Neill Reference Dolton and O’Neill1996; Micklewright and Nagy Reference Micklewright and Nagy2005; Hägglund Reference Hägglund2007; Crépon et al. Reference Crépon, Duflo, Gurgand, Rathelot and Zamora2013). Much of the experimental and other work in this area is summarised in Card et al. (Reference Card, Kluve and Weber2010) meta-analysis of labour market interventions. More recently, the experimental approach has started to incorporate elements of behavioural sciences, for instance testing how simplified and better-framed information can help to motivate jobseekers. Altmann et al. (Reference Altmann, Armin, Jäger and Zimmermann2015) report the results of a trial aimed at reducing long-term unemployment in Germany: jobseekers experienced significantly increased employment and earnings when they received a brochure informing them about job search strategies and the consequences of unemployment, and when they received motivation to actively look for new employment. Similarly, an experiment by Belot et al. (Reference Belot, Kircher and Muller2016) demonstrates how suggesting alternative occupations and associated vacancies to jobseekers helps them broaden their job search and secure a higher number of job interviews. These studies complement previous evaluations suggesting that a more human-centric focus can help to increase the effectiveness of ALMP programmes (see, among others, Crépon et al. Reference Crépon, Dejemeppe and Gurgand2005; Price and Vinokur Reference Price, Vinokur, Klehe and Van Hooft2014).

The next section of this article explains in detail the evaluation methodology we employed to measure the impact of this mix of behavioural insights interventions. The first pilot followed a quasi-randomisation approach, while the large-scale trial was designed following a cluster-randomised controlled trial design. While the benefits of using experimental and quasi-experimental approaches in evaluating public policy are well documented, this approach still receives a proportionally lower level of resources compared to more traditional models of public policy evaluation (Haynes et al. Reference Haynes, Goldacre and Torgerson2012). This may partially be due to institutional constraints, among other reasons. The following section provides details on how we overcame some of these barriers, providing insights on public policy evaluation approaches beyond the findings from our experiments.

Design

Jobcentres allocate jobseekers to the next available adviser in their local area when their claim is processed. This is conducted off-site and without knowledge of the jobseeker’s abilities or prior performance. In our pilot, staff on the first floor of Loughton Jobcentre delivered our new bundle of interventions (described below). Staff on the ground floor delivered our control interventions. The Jobcentre managers indicated to researchers that allocation was arbitrary and that there were no rules or guidelines that would produce differences between the groups allocated to each floor.Footnote ³ Jobseekers who started their claims prior to the launch of the trial were treated in the same manner as the control group and were not included in our sample.

Interventions

Interventions included in the Loughton trial were categorised under three headings:

1. (1) reformatted first interview;

2. (2) jobseeker commitment pack; and

3. (3) resilience-building exercises.

Expressive writing

Twelve weeks after jobseekers’ claims had started, they were invited to undertake expressive writing. Advisers were given discretion over the jobseekers to whom they offered the protocol. Jobseekers were told that this would be done voluntarily and that they would not have to tell advisers whether or not they had undertaken the writing. This exercise was based on a study by James Pennebaker and colleagues (Spera et al. Reference Spera, Buhrfeind and Pennebaker1994). In the original exercise, 50% of those who wrote about their emotions found work within six months, compared to just 20% of those who wrote about how they used their time. This was despite there being no difference between the two groups in terms of how many jobs they applied to: the impact was on application quality.

The protocol was developed in consultation with James Pennebaker. The phone number of a local psychological service was included on the sheet given to jobseekers in light of concerns that the expressive writing exercise might evoke some distress in a small group of jobseekers. Whether jobseekers had been given the protocol and whether they had completed the exercise was not recorded. In all, the addition of these new processes amounted to a substantive change to the jobseeker’s journey. Figure 1 shows the differences between the two processes. Besides our interventions, the new process we introduced also had implications for the timeframes between steps, and it shifted the focus of the conversations with jobseekers. The change in timeframes was designed to allow jobseekers to spend as much time as possible focusing on effective job searches, rather than the receipt of benefits (the former being an effortful process which the jobseeker has control over and the latter being a process that is done to them). In the previous process, there was a wait of two weeks until the focus on effortful job searches began.

Figure 1.

Differences between the two jobseeker journeys.

Data

Our data contain 2,383 observations of individuals starting a claim in Loughton during the seven months of the pilot study, and the 6,701 individuals in the baseline (preexperimental) period. For participants in the trial, we observe whether or not they were assigned to treatment or to control. For all individuals, we observe whether or not they had flowed off benefits at 13 weeks – that is, if they had ceased claiming benefits within 13 weeks of first “signing on”. As mentioned, this outcome measure was selected as it was a key performance indicator for Jobcentres at the time of the trial and the data were routinely collected.

Baseline data cover the 19 months preceding the beginning of our trial. These data were fully anonymised and extracted from the Loughton Jobcentre management information system.

Empirical strategy and results

Our main results are reported in Table 1. For our initial analysis, we began with a simple linear prediction model of our outcome measure on our treatment conditions, using only the experimental sample, as in column 1 of Table 1. In column 2, a linear time trend (in months) is added to the specification, and in column 3 this is supplemented by an interaction term between the time trend and our treatment variable.

Table 1.

Main results, Experiment 1 (linear prediction model)

Note: Standard errors in parentheses.

*p < 0.01, *p < 0.05, ***p < 0.001.

Column 4 estimates the same model as column 3, with the addition of baseline data. Columns 5 and 6 use baseline data but contain binary variables indicating the experimental period. Finally, column 6 contains a full set of month fixed effects.

The results reported in Table 1 are striking. In all of our specifications, there is a significant and positive effect of our bundle of treatments on off-flow at 13 weeks. The most relevant of these specifications (Column 1), which makes use only of experimental data, suggests a 5-percentage point increase in off-flow as a result of our treatments. Estimates of the effectiveness of treatment in columns 3–6 are conflated with the dynamic variables included in the model.

Other findings of interest can be seen from the data. Within our experimental period, we detected a significant decline in off-flow rates in successive cohorts compared with the first, and this fall was (insignificantly) greater for participants in the treatment group.

When the data are expanded to include pretrial individuals in this Jobcentre, the time trend is only economically and statistically significant (and negative) for the treatment groups, but the addition of an indicator variable for the experimental period shows that there was a significant above-trend increase in off-flow for the control group during this period. Both the deteriorating effect of treatments over time and the spike at the initiation of the trial are indicative of possible experimental artefacts (shown in Figure 2).

Figure 2.

Off-flow by treatment and control groups over time, Experiment 1.

Employees of the Jobcentre received repeated and regular visits from members of the experimental team both before and during the trial. In addition, those administering the treatment condition received considerable additional training. It seems likely that, as well as aiding in the administration of the trial, this extra attention may have had a positive effect on the motivation and wellbeing of employees and increased their enthusiasm for their work, which may have been reflected in their better-than-expected performance in both conditions. We must therefore suspect that at least part of the effectiveness of the treatment was a result of Hawthorne effects. It is also possible that the front-loading of the active parts of the ALMP interventions contributed to the effects.

Weaknesses of the pilot study

Following the completion of this trial, issues were brought to light concerning its implementation. In particular, analysis of more detailed data on participant characteristics held by the Department for Work and Pensions (DWP) and not accessible to the research team suggested that participants were significantly imbalanced in terms of their closeness to the labour market (their past work history), with participants in the treatment group being deemed significantly more likely to get a job than participants in the control group.Footnote ⁴ Based on this finding, and given the scarcity of our own data preventing the inclusion of unobserved participant characteristics as covariates, the results of this trial were deemed to be not wholly reliable in themselves. This also emphasised the importance of implementing randomisation within the research team.

We are also concerned about displacement effects. Crépon et al. (Reference Crépon, Dejemeppe and Gurgand2005) conducted a quasi-experimental analysis of a counselling programme administered to jobseekers in France with a similar objective. As opposed to those who did not receive job counselling, jobseekers in that study who received job counselling were significantly more likely to find work and were significantly more likely to still be in employment 12 months later.

A risk of evaluating labour market interventions is the difficulty in finding a “clean” control group that is not influenced by the interventions. Within a local area, such as that in our study, it is possible that additional employment is caused either by the treated unemployed taking jobs that would otherwise have been taken by the untreated unemployed or by the treated unemployed taking jobs that would have been taken by employed people currently engaged in job-to-job transition. Crépon et al. (Reference Crépon, Duflo, Gurgand, Rathelot and Zamora2013) investigated this possibility by varying the proportion of unemployed participants who received job search support (0, 25, 50, 75 and 100%) in 235 areas of France. Although they found positive treatment effects on the treated, they concluded that these could primarily be explained by displacement effects. We attempted to address these issues in Experiment 2.

Design

This trial was conducted in 12 Jobcentres in Essex (i.e. all Jobcentres in the county, excluding Loughton and its sister site, Brentwood, which had adopted the treatment condition from the previous trial already). This study was designed as a pragmatic stepped-wedge trial and ran from January 2013 to January 2014. The 12 Jobcentres form 9 clusters, with some smaller Jobcentres merged together by the DWP under a single manager, named a customer service operations manager. Intuitively, we can therefore think of the unit of randomisation as the manager of either single or paired Jobcentres. Jobcentres taking part in the trial were assigned readiness scores by their regional manager, indicating how quickly they would be able to change their processes to accommodate the treatment. Clusters were therefore rated as “ready to receive intervention”, “nearly ready to receive intervention” or “not ready to receive intervention”. This was done on an ad hoc basis without a formalised instrument, due to time and administrative constraints on the experiment.

When following a stepped-wedge design, all clusters in an experiment are eventually treated (Hussey and Hughes Reference Hussey and Hughes2007), with the order of roll-out randomised. In this trial, the implementation took into account the readiness ratings assigned by the regional manager. Each readiness rating contained three clusters of Jobcentres. More “ready” clusters were endogenously selected to receive the intervention earlier, with the order of implementation randomised within each readiness rating.

The quality of the randomisation was necessarily weakened by the small number of clusters for randomisation and the endogenous sequencing of the roll-out. However, we argue that this was justified by the practical necessities of the trial and by the benefit of this method for the trial’s external validity, as our method closely resembled the method that would ordinarily be used to roll out an intervention in a Jobcentre context, with the benefit of randomisation within readiness rating. Randomising over time periods, with training in the intervention only delivered to a cluster at the time when it is due to begin the intervention, minimises spillovers between treatment and control individuals, as well as allocation failures. All jobseekers who started their claims prior to the launch of the trial were treated in the same manner as the control group.

Finally, the stepped-wedge nature of the design, which ensured that we observed both treated and untreated individuals in each cluster during the trial, at least partially mitigated the consequences of any imbalance.

In this design, clusters are switched from the control to treatment condition in the order defined by their readiness rating and randomisation. In our intervention, switching took place roughly once a month, based on the timing of “cohorts” reported in the management information data produced by the Jobcentres. Table C1 in Supplementary material provides details of Jobcentres in the trials, their readiness ratings and when they switched from control to treatment.Footnote ⁵

Differences in the interventions between experiments

Our intervention was implemented almost identically in the two experiments. In Experiment 2, however, the VIA survey and expressive writing components of the intervention were not used. The decision to change the intervention between pilot and large-scale experiment was taken due to concerns about the public perception of the test, particularly due to unfavourable coverage which may have reduced participants’ confidence in the VIA survey. Jobcentre advisers in the first two clusters to be treated, Colchester and Chelmsford, were given training on the survey prior to the trial’s launch but were subsequently instructed not to use it with jobseekers. The expressive writing exercise was also removed as it was felt that this intervention might be viewed in the same negative light.

Empirical strategy and results

In the interests of fullness, our primary analysis will be described in a series of stages, with each stage introducing new controls for robustness. To conduct our primary analysis, we followed the model specified in our trial protocol. This model was specified as follows:

$${{\it{Y}}_{{\it{ict}}}} = \alpha + \beta {\rm{1}}{T_{ct}} + \beta {\rm{2}}{C_i} + \beta {\rm{3}}{\theta _t} + {u_{ict}}$$

where Y _ict is our binary outcome measure: whether an individual i in cluster c and cohort t has flowed off at 13 weeks. T_ct is our treatment variable, set to 0 if an individual is in a cluster–cohort pair that is not currently treated and 1 if that individual is in a cluster–cohort pair that is treated. C _i is a cluster-specific fixed effect, θ_t is a linear time trend and u_ict is an error term clustered to account for correlations within a cluster. We report the results of this primary analysis in Table 3.

Table 3.

Treatment effects on off-flow at 13 weeks, over time, Experiment 2 (LPM)

Note: Clustered standard errors in parentheses.

*p < 0.01, **p < 0.05, ***p < 0.001.

Column 1 shows a linear prediction model regression on all participants, with errors clustered at the level of the Jobcentre and a linear time trend. This column is our primary result. Column 2 shows the same analysis but excluding our baseline data. Although this reduces our statistical power by reducing our sample size, the point estimate of our result remains consistent with that in column 1, and the result is statistically significant at the 7% level. Column 3 consists of a model with time fixed effects and column 4 consists of a variable capturing the time since a cluster was treated.

We found a significant and positive effect of treatment in our main analysis, the point estimate of which is largely unchanged by the exclusion of historical data.

Although these figures represent a reduction in the estimated effectiveness of our treatment compared with the pilot study, the effect remains substantial in real terms, and importantly it survives the introduction of controls that reduce statistical power considerably. Given the relatively small number of clusters, we also conducted randomisation inference analysis, which offers a more conservative estimate of statistical significance, yielding p = 0.10Footnote ⁶ .

Column 3 of Table 2 is concerned with whether there is a relationship between the size of the treatment effect and the amount of time for which it had been active. In contrast to Experiment 1, we found that this effect was actually positive and statistically significant, with no significant impact of the treatment in the first month but increasing effects thereafter. This may reflect the fact that less support was given to Jobcentres in rolling out the trial in Experiment 2 compared to Experiment 1, which may in turn have alleviated Hawthorne effects while also indicating that the new processes took time to “bed in”. Given the pragmatic way in which this trial was implemented, we now proceed to robustness checks.

Table 2.

Effect of treatment on off-flow at 13 weeks, Experiment 2 (LPM)

Note: Clustered standard errors in parentheses.

*p < 0.01, **p < 0.05, ***p < 0.001.

Robustness checks: Controlling for blocks of randomisation

Jobcentres (clusters) were randomly allocated to treatment from within groups of Jobcentres based on their readiness to accommodate the treatment. Clusters were rated according to three levels of readiness, with the first (most ready) block beginning the trial first and the least ready beginning the trial last. Hence, we repeated our primary analysis (column 1 of Table 2) with additional controls added for this level of readiness, and with readiness ratings interacted with the treatment indicator. The results of these analyses can be found in Table A1 in Supplementary material.

Readiness controls did not substantively alter the shape of our result, in large part due to controls for cluster-level fixed effects. In our interaction analyses, we found that the less ready centres (readiness scores of 2 and 3) responded slightly better to treatment than more-ready clusters did; however, this effect was not significant. When taking this into account, the point estimate of our effect rises to 2.8 percentage points (p < 0.01, CI 1.4–0.41%).

Displacement

As discussed in section “Previous studies”, we were interested in the source of the observed increases in off-flow. In particular, we were interested in discovering whether treated participants were genuinely getting new jobs or whether the benefits to treated individuals were coming at the expense of untreated individuals.

To test this, we began with a model in which we included an interaction term between time trend and treatment. In the event that treated individuals in a cluster were depriving untreated individuals from the same cluster of jobs they would otherwise have got, we would expect the effect of treatment to be higher the fewer people were treated in that cluster, as the relative magnitude of their competitive advantage would be greater.

This single model is reported in Table 3, which shows a negative average effect of treatment, significant at the 1% level and a negative time trend. Interestingly, we found a positive and significant effect of the treatment interacted with time. This is not consistent with a hypothesis of pure displacement, as the effect of treatment on the treated increased as their competitive advantage declined. Instead, this suggests that during the first month or second, while procedures were being changed, clusters’ performance actually fell, and then it rose after the treatment had settled in. This is markedly different from the suggestion that Hawthorne effects were present in Experiment 1 (in which treatment effects deteriorated over time). As before, however, we believe that this is most likely an interaction effect with the later clusters to be treated and the time trend.

Given that these data suggest there was a “bedding-in” period once the intervention had been launched, it is perhaps not clear whether treated individuals represent the best place to look for evidence of displacement. We therefore turn our attention instead to control individuals. Here, it is possible to exploit two facts: that cohorts existed simultaneously within each cluster and that our outcome was measured after three months. Hence, the last control cohort in each cluster faced competition from two cohorts of treated individuals: one for one month, and one for two months.

As an additional check for displacement, we replicated the primary specifications estimated in Table 2, with the inclusion of a binary variable that indicated whether a cohort was the last within its cluster not to be treated. These results are in Table A2 in Supplementary material. We did not find that last control cohorts performed significantly worse than other cohorts. Again, although this is not conclusive, the available evidence does not suggest that the benefits of treatment to the treated came at the expense of the untreated unemployed within their own cluster.

Time spent on benefits

As well as our primary outcome measure of off-flow by 13 weeks, data are also available on off-flow at each week prior to this, from which we can impute (at week level) durations of benefit spells. A Cox regression yielded estimates, mechanically, that our treatment was associated with a 4% increase in the off-flow hazard rate in any given week.

From a policy perspective, we are interested in participants’ speed of off-flow. Considering only those who flow off benefits within the 13-week window leads to biased estimates, as we have already established that the likelihood of an individual flowing off within this window is affected by our treatment. To correct this, we took the most conservative estimate possible of future off-flows and assumed that all participants who did not flow off benefits within the first 13 weeks did so immediately afterwards, at the 14-week point. We were then able to estimate the same empirical specification as before, replacing our binary outcome measure with a discrete one capturing the number of weeks before a participant flows off benefits. This analysis suggested that participants in the treatment group flowed off benefits 22% of a week (1.54 days) faster than participants in the control group (p = 0.045). These results seem to counter the hypothesis that the effect of the treatment was purely due to participants having a work-focused interview two weeks sooner. However, it cannot conclusively be said that this was not the case.