Comment / Paying for outcomes: rescuing promise from the rubble of hype

09 July 2024 Fraser Battye

The first effect of policy is on expectations. In every case I can think of, the effect is inflationary. And policies created on the promise that payments can be linked to outcomes are a stand-out example. In this case, the hype-reality gap is more like a tectonic chasm. 

It’s difficult to be precise, but the decade after the election of the coalition government in 2010 probably saw the peak of enthusiasm for policies based on this approach. 

Programmes came with different labels: Payment by Results (not to be confused with the NHS ‘PbR’ scheme); Outcomes Based Commissioning; Results Based Financing; Social Impact Investing; Pay for Success; (etc). And initiatives mushroomed across the policy landscape: appearing in supported housing, foreign aid, prisoner rehabilitation, unemployment, locating illegal immigrants, (etc). 

The fever of the times is captured nicely in a 2010 ‘think piece’ from advisory and audit firm, KPMG. It suggested: ‘Payment by results should be implemented across the public sector without exception… Where payment by results exists it should be made enhanced and where it does not exist it should be hurried into existence, even if it is crude to start with.’

The fever didn’t catch quite as widely as KPMG hoped. But crude hurry certainly did – and at a pace that meant evidence from evaluation would never catch up.

The NHS was not immune. Examples began to emerge where the promise of paying providers for outcomes was failing real world tests. Often under the guise of ‘capitated outcome-based incentivised contracts’ (or COBICs), procurements stalled, contracts were suspended or altered, investigations were ordered, and lessons were noted. Policy hopes were dashed on the delivery rocks of Staffordshire, Cambridgeshire and Peterborough, Oxfordshire, Bedfordshire, Sussex and elsewhere.

Policy ideas never quite die. But outcome-based commissioning began to look very badly wounded. Why? Because the whole enterprise was doomed from the outset? 

High profile and expensive failures will have as many explanations as denials. Lessons from the schemes noted here are in various evaluation and official ‘having recovered the flight recorder’ type reports. This isn’t the place to rehearse these details. 

Instead, my rough and broad explanation is that what seems intuitively so obvious – that people should be incentivised and rewarded according to the results they achieve – is actually astonishingly difficult to implement in practice. 

One clue as to why this might be so is in the very word ‘outcome’. My dictionary says that an outcome is ‘something that follows as a result or consequence’. It is here that the rocks begin to show. We must be making a causal claim: a result occurs because of an action. 

At this point, anyone with a background in evaluation is starting to shrink slightly. It is notoriously difficult to show a causal relationship between an action and its outcomes. We can wish this problem away – or we can try and ignore it – but, if we lose sight of this causal relationship, talk of ‘results’ becomes increasingly meaningless. 

And this was where many schemes failed. Crude hurry couldn’t cope with analytical subtlety. Instead, it led straight to expensive and sticky endings. 

I'd like to think the Strategy Unit is immune to hype. Our contribution to these debates will always be rooted in careful analytical work. So, is it possible to accept all the analytical challenges that come with outcome-based commissioning and still make use of it? Or is it time to abandon hope (until the next time)?

We think hope can be salvaged from the hype. Two of our past contributions seem valuable to me. The first, looking at risk and reward sharing for integrated care systems, sets out a route to realising the benefits of such arrangements, while navigating inherent analytical and contractual complexities. The second builds on this and examines the use of risk and reward to incentivise system-level reductions in emergency activity.
And now, following work with Luigi Siciliani (University of York) and Matthew Revell (The Royal Orthopaedic Hospital NHS Foundation Trust), we have provided a detailed look at how payments could be structured to incentivise providers to increase the quality of elective knee-replacement surgery. 

The report comprehensively fails the hype test. It is a careful and forensically-reasoned piece of work that goes from basic concept (why do this?) up to contractual conditions (how?). It suggests that the way forward may be to trial an outcomes-based contract for a specific service and sets out how just such a contract could operate for elective knee replacement surgery. Fundamentally, we argue that the relative simplicity of this case is what makes it attractive. 

In many ways, outcome-based commissioning needs to be saved from its promoters. If this approach is ever going to make it out of the trough of disillusionment (see the Gartner hype cycle), then it needs an empirical hand more than a hypeman.

The Strategy Unit will be hosting an online event looking at these issues on 3 October. Further details  via X (@Strategy_Unit) and LinkedIn. The Strategy Unit is a research and analysis consultancy hosted by Midlands and Lancashire Commissioning Support Unit.