BY ELEANOR CARTER & CLARE FITZGERALD
Evaluation should test key SIB promises to government: greater collaboration, prevention and innovation, argue two researchers from Oxford University’s Government Outcomes Lab.
To discover whether Social Impact Bonds work, we would be wise, first of all, to ask: “Work for whom?” Our response is: “For public service commissioners.” This answer acknowledges a reality: if SIBs don’t make sense to government, they are unlikely to be more than a passing fashion.
This standpoint cuts through a myriad of promises that typically surround SIBs and other payment by results programmes. It also helps focus how to evaluate them. We should measure SIBs by looking at what, if anything, they can deliver that public service commissioners seek.
That’s what we’re doing in the Government Outcomes Lab – GO Lab – at the Blavatnik School of Government, where our work is currently co-sponsored by the School and the UK government.
The dominant narrative around SIBs, born in an age of austerity, is that they can deliver cashable savings to government. Of course, every government seeks such dividends. However, this claim, far from proven – at least at scale – may or may not be true. Either way, we believe SIBs go well beyond promising the efficiency and entrepreneurialism claimed by conventional outsourcing. SIBs make a more fundamental offer that should be rigorously tested: that they provide an opportunity to remake the public sector itself.
Public sector reform has sat at the heart of every government’s agenda – Conservative and Labour – for nearly 40 years, since the election of Margaret Thatcher in 1979. Its goals – aside from making savings – have included three principles that SIBs directly claim to address.
SIBs address 3 principles of public sector reform
The first principle is to improve collaboration in the commissioning and provision of services so that the collective effort is better focused on the needs of the service user.
The second principle is to shift activity to a more preventive footing, so that service provision is moved upstream of social challenges and towards earlier identification of risk.
The third principle is to make innovation less risky for commissioners so they move to evidence-based procurement, decommissioning underperforming services and becoming better at reaching those, often experiencing the most complex mix of social issues, who may be poorest served by existing provision.
We should, therefore, question, in our evaluations, whether SIB models are improving collaboration, shifting services towards prevention and creating spaces in which public service commissioners and delivery organizations can innovate.
Principles largely untested by evaluations
Our review of evaluations suggests that they rarely explicitly or rigorously test any these three principles by comparing a SIB commissioning approach with a grant, fee-for-service or even in-house delivery for a given population. Evaluations also face the “doughnut challenge”. That’s the difficulty, particularly with impact evaluations, in teasing apart the effects of the commissioning architecture (the dough) from the effects of frontline services operating in different ways (the jam). Indeed, most evaluation material is quite descriptive and exploratory. It doesn’t typically offer or test a mechanism by which a particular SIB is expected to work in practice.
Collaboration is seen
However, where we have tried to read these three principles into the evaluations that have been produced, we see glimmers of each logic being present in SIBs. So Peterborough’s SIB – focused on reducing ex-offender recidivism – saw front-line organisations pull together in cohesive ways, wrapping around the service user.
We’ve also seen multiple commissioners joining forces as, for example, where the Department for Education, Cabinet Office and the Department for Work and Pensions came together for a fund-based approach to support young people vulnerable to disengaging from school and work. We see top-up funding approaches from the Cabinet Office to acknowledge that there might be wider public service benefits springing from a SIB that are not captured within the outcome payer’s domain.
“Step-down services”, more than prevention
As far as prevention is concerned, very early intervention programmes are rare in the SIB field, but there is a focus on “step-down” services. The Birmingham Foster Care SIB is a good example that is already demonstrating cashable savings.
A growing number of SIBs try to de-escalate situations of family conflict or address unstable conditions for children and young people. Typically, for example, the Department for Work and Pensions has worked with people only once they are employed. Now, through SIB programmes, the Department, through the Young Engagement Fund, is becoming involved in programmes that try to understand and anticipate future needs. It is working with school children to prepare them for a smooth transition into work.
Space for innovation
There are also examples of SIBs supporting innovation and helping to dissipate cultural inertia. Typically commissioning has a cycle – there’s the planning stage, the doing phase and then there’s the evaluation. A SIB can help reduce risk at any or all of these stages, allowing SIB commissioners flexibility to think differently about their work. So, for example, with an innovative programme such as the GLA Rough Sleeping project, a SIB allowed commissioners to ask of themselves: ‘Do I know how to be the best contract holder, or would I benefit from bringing in a third party that might have a better appreciation or knowledge base about that side of things?’
SIBs can also enable a certain amount of experimentation by providing quicker feedback through performance management. It’s this feedback that allows frontline organisations to change the way they are doing things to better meet the needs of their clients. A well-designed SIB also creates impetus for high quality impact evaluation within a payment mechanism. That inclusion can, in the long run, only help to improve the quality of future commissioning.
Time to test public sector reform potential
In conclusion, we cannot demonstrate, on the basis of existing evaluation evidence, that SIBs fulfil the many and varied promises that their champions make for them. But observing SIBs through the lens of public sector reform – and their possible role in it – we offer an approach to evaluating whether they might be a tool for remaking the public sector.
Eleanor Carter and Clare FitzGerald are research fellows at the Government Outcomes Lab (GO Lab)at Oxford University’s Blavatnik School of Government.
If you wish to receive our weekly blog on SIBs, please email Paula.Fry@lshtm.ac.uk and we will add you to our subscriber list.