Tag Archives: Financial incentives

Payment by Results: More Evidence Against

The CLAHRC WM Director is something of a pro-market libertarian. From this ideological standpoint he is sympathetic towards the idea of payment by results to supplement for lack of market incentives in the supply of non-market services. Just one problem – they don’t work. Well, they often don’t work. A recent, elegant paper uses threshold analysis to examine the effect of the Medicaid / Medicare Services payment by results – their ‘Value Based Modifier’.[1] This is the largest health service in the world – much bigger than the NHS – so there are plenty of data points. The threshold used in studies was the size of the practice, at which various rewards and penalties cut in. So, in outline, the graphs look like this:

100 DCi - Payment by Results Fig 1

If the incentive worked then we would expect to see this:

100 DCi - Payment by Results Fig 2

Instead we see this:

100 DCi - Payment by Results Fig 3

In other words, the incentives did not work. Assuming that the threshold does not coincide exactly with a counterfactual break, this disproves any benefit (or disbenefit) of performance-based payment in this or similar contexts. But CLAHRC WM did find that an incentive was effective in increasing uptake of home haemodialysis.[2] So here is an hypothesis: incentive systems only work when there is a clear objective that can be achieved by specific management / clinical action. We have said this before – Lilford’s rule if you insist – “incentives do not work when the people at whom the incentive is targeted do not have a clear and correct idea of how the objective may be achieved.” I also think this is a lovely example of threshold analysis, a topic mentioned in a recent News Blog.[3]

— Richard Lilford, CLAHRC WM Director


  1. Roberts ET, Zaslavsky AM, McWilliams JM. The Value-Based Payment Modifier: Program Outcomes and Implications for Disparities. Ann Intern Med. 2018; 168: 255-65.
  2. Combes G, Allen K, Sein K, Girling A, Lilford R. Taking hospital treatments home: a mixed methods case study looking at the barriers and success factors for home dialysis treatment and the influence of a target on uptake rates. Implement Sci. 2015; 10: 148
  3. Lilford RJ. Discontinuities in Data – a Neat Statistical Method to Detect Distorted Reporting in Response to Incentives. NIHR CLAHRC West Midlands News Blog. 1 September 2017.

Declining Readmission Rates – Are They Associated with Increased Mortality?

I have always been a bit nihilistic about reducing readmission rates to hospitals.[1][2] However, I may have been overly pessimistic. A new study confirms that it is possible to reduce readmission rates by imposing financial incentives.[3] Importantly, this does not seem to have caused an increase in mortality – as might occur if hospitals were biased against re-admitting sick patients in order to avoid a financial penalty. “False null result” (type two error), do I hear you ask? Probably not, since the data are based on nearly seven million admissions. In fact, 30 day mortality rates were slightly lower among hospitals that reduced readmission rates.

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ. If Not Preventable Deaths, Then What About Preventable Admissions? NIHR CLAHRC West Midlands News Blog. 6 May 2016.
  2. Lilford RJ. Unintended Consequences of Pay-For-Performance Based on Readmissions. NIHR CLAHRC West Midlands News Blog. 13 January 2017.
  3. Joynt KE, & Maddox TM. Readmissions Have Declined, and Mortality Has Not Increased. The Importance of Evaluating Unintended Consequences. JAMA. 2017; 318(3): 243-4.

Getting Evidence into Practice

In the early days of CLAHRCs, ‘getting evidence into practice‘ was an important objective. We set about closing the T2 gap and used implementation science to get doctors to prescribe evidence-based care, dentists to use tooth protecting resins, and nurses to make regular observations. That is to say, we were concerned with how to make practitioners comply with standards over which they had complete jurisdiction. Theories of individual behaviour change were invoked, and rather then choose a theory on the basis of its impressive sounding title (e.g. prospect theory, social network theory), a framework was developed to identify barriers and facilitators of change.[1]

But practitioners increasingly follow the evidence when it is compelling and when the evidence-based standard is in their gift.[2] So, the big (and much more interesting) problem now is how to change the service in a generic way rather than simply to increase performance on a specific measure – we are becoming more concerned with draining the swamp than zapping individual mosquitoes.[3] In our CLAHRC we recently evaluated a compound (multi-component) intervention to improve home dialysis rates, having promulgated a guideline supporting improved access to such a service. We showed that agreement with the proposed change among stakeholders, an agreed

Implementation plan, managerial support, and product champions all facilitated the success of the intervention in taking West Midlands from the worst to the best performing region in England. However, the king of all intervention components was a financial incentive.[4] Fulop and colleagues have now published a similar multi-methods evaluation of an arguably even more complex intervention to improve access to acute stroke care.[5] The findings are very similar, save that we found more emphasis on financial incentives and also more problems in communication with patients; something that would perhaps not stand out in the hyper acute stroke context. The Fulop paper is an advance on ours in (at least) two respects. First, they compare and contrast across two regions/CLAHRCs and I always think controls should be used if possible; even one is better than none. Second, they illustrate the causal model with diagrams that make the theoretical framework they are using clear, a practice that is helpful in communicating the very real distinctions between the intervention as planned, its implementation/adaption, its upstream effects (e.g. staff knowledge/morale), its downstream effects (at the patient ‘level’), and the context in which all takes place.[6] People muddle these concepts and hence fall over their feet , but Fulop and colleagues have shown themselves to be sure-footed!

— Richard Lilford,


  1. Michie S, van Stralen M, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventionsImplement Sci. 2011; 6: 42.
  2. Johnson N, Sutton J, Thornton JG, Lilford RJ, Johnson VA, Peel KR. Decision analysis for best management of mildly dyskaryotic smear. Lancet. 1993;342(8863):91-6
  3. Lilford RJ, Chilton PJ, Hemming K, Girling AJ, Taylor CA, Barach P. Evaluating policy and service interventions: framework to guide selection and interpretation of study end pointsBMJ. 2010; 341: c4413.
  4. Combes G, Allen K, Sein K, Girling A, Lilford R. Taking hospital treatments home: a mixed methods case study looking at the barriers and success factors for home dialysis treatment and the influence of a target on uptake ratesImplement Sci. 2015; 10: 148.
  5. Fulop NJ, Ramsay AIG, Perry C, et al. Explaining outcomes in major system change: a qualitative study of implementing centralised acute stroke services in two large metropolitan regions in England. Implement Sci. 2016; 11: 80.
  6. Lilford RJ, Chilton PJ, Hemming K, Girling AJ, Taylor CA, Barach P. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ. 2010; 341: c4413.

Experimental Study of Income Inequality

Haushofer and colleagues report on a randomised trial of cash transfers to poor people in Kenyan villages.[1] They find that when some people receive a material cash transfer this adversely affects those who do not, on the cognitive, but not affective components of psychological wellbeing. However, the effect is short-lived, dissipating over the 15-month follow-up period in the study. The authors are careful to point out that there is no reason to avoid making the transfers, which, among other things, empower women and reduce violence. In any case, how can society function if people are not allowed to escape from poverty for fear of upsetting their neighbours? All the same, conspicuous consumption is distasteful because it does adversely affect those around you, and should thus be avoided.

— Richard Lilford, CLAHRC WM Director


  1. Haushofer J, Reisinger J, Shapiro J. Your Gain Is My Pain: Negative Psychological Externalities of Cash Transfers. 2015.

Penalties for Readmissions Reduce Readmissions at New York Hospitals, but at the Cost of Increasing Attendance at Emergency Departments.

The CLAHRC WM Director is always concerned about possible perverse effects of incentives. Sure, do all you can to ensure a decent handover of care, but only a small proportion of readmissions are avoidable. Anyway, a recent difference in difference analysis in New York State [1] suggests that financial penalties, while reducing readmissions, were associated with a small but significantly increased rate of attendance at the Emergency Department. Again the law of unintended consequences.

— Richard Lilford, CLAHRC WM Director


  1. McGarry B, Blankley A, Li Y. The Impact of the Medicare Hospital Readmission Reduction Program in New York State. Med Care. 2016; 54(2): 162-71.

Important Evidence on Pay for Performance

Evidence is accumulating that pay for performance does not perform. The problems are both endogenous (what happens to the particular performance targets at which the intervention is targetted) and exogenous (what happens to performance criteria beyond the designated targets).

First, the endogenous effects. Pay for performance does not seem to work (not even on its own terms) or its effects are short lived. Studies on both sides of the Atlantic have found that initially positive effects were short lived.[1] [2]

Second, a recent article highlights the negative exogenous effects of payment for performance.[3] The interventions particularly penalise hospitals at the bottom of the financial difficulty league, who tend to serve disadvantaged populations. They are unfair, since patients at these hospitals have greater morbidity (with more opportunities for error).[4] Their patients are not as mobile as better-off people, and so they cannot as easily ‘vote with their feet’. Worse, there is a substantial volume of psychological work showing that financial rewards/penalties are demotivating, especially to intrinsically motivated people. Specific targets sometimes work,[5] but burdensome lists of tick-box targets are often not effective in the long term, and can have negative spill-over effects.

— Richard Lilford, CLARHC WM Director


  1. Jha AK, Joynt KE, Orav EJ, Epstein AM. The long-term effect of premier pay for performance on patient outcomes. N Engl J Med. 2012; 366: 1606-15.
  2. Kristensen SR, Meacock R, Turner AJ, et al. Long-term effect of hospital pay for performance on mortality in England. N Engl J Med. 2014; 371: 540-8.
  3. Woolhandler S and Himmelstein DU. Collateral Damage: Pay-for-Performance Initiatives and Safety-Net Hospitals. Ann Intern Med. 2015; 163(6): 473-4.
  4. Brown C, Hofer T, Johal A, Thomson R, Nicholl J, Franklin BD, Lilford RJ. An epistemology of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement. Qual Saf Health Care. 2008; 17(3): 170-7.
  5. Combes G, Allen K, Sein K, Girling A, Lilford R. Taking hospital treatments home: a case study of barriers and success factors. Implement Sci. [Submitted].