League tables – not always bad

Health services have become used to report cards on performance. These are valid if the signal to noise ratio is favourable (waiting times, vaccination rates, patient satisfaction), but invalid when the signal is overwhelmed by noise (standardised mortality ratios, readmission rates).[1] [2] School performance on national tests seems to have a reasonably good signal to noise ratio, especially if adjusted. So what is the effect of league tables on provider performance of schools and how does that differ between:

  1. Public vs private providers.
  2. Schools with good vs bad to relative baseline performance.

Andrabi et al. [3] carried out an RCT of 112 Pakistani villages, all of which contained a mix of fee paying and public schools. Test score results improved after the introduction of report cards showing consolidated results across each school. Parents were aware of the reports and took them seriously.  School performance increased across the board, costs of private schools fell on average (and the worst performing closed down) and equity improved since schools at baseline improved most.

Private schools in the above study may have responded to financial incentives since their pupils could vote with their feet. But the interesting thing is that public providers also improved.  This might reflect the pure power of comparative information. Alternatively, providers may have responded to the implicit threat to livelihoods from the availability of comparative data in the context of rapidly increasing provision of private education.

Another example of the availability of comparative data improving quality can be found in the UK catering industry, following the introduction of a scheme requiring restaurants and cafes to make their food hygiene inspection ratings publically available, and visible on doors and shop fronts. Customers preferred to buy food from outlets with higher ratings, and competition among food businesses on hygiene standards resulted in an increase in the proportion of food premises that complied with hygiene standards.[4]

What implications does this have for healthcare? Hospital performance seems to improve under the influence of league tables, even when reimbursement is not affected by the results. However, Fotaki says that the impact of initiatives such as NHS Choices, designed to provide public access to comparative data on hospital performance, consultant outcomes, and user satisfaction, may have little effect. She argues that the impact of informed choice on efficiency and quality is limited at best, and may even have negative consequences for equity: pre-existing inequalities of income and education influence patients’ access to information and ability to choose.[5] We would welcome comments on this enigma.

— Richard Lilford, CLAHRC WM Director

References:

  1. Lilford R, Mohammed MA, Spiegelhalter D, Thomson R. Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet. 2004;363(9415):1147-54.
  2. Girling AJ, Hofer TP, Wu J, Chilton PJ, Nicholl JP, Mohammed MA, et al. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study. BMJ Quality & Safety. 2012;21:1052-6
  3. Andrabi T, Das J, Khawaja AI. Report cards: the impact of providing school and child test scores on educational markets. Social Science Research Network. RWP14-052. June 2014.
  4. Salis S, Jabin N, Morris S. Evaluation of the impact of the Food Hygiene Rating Scheme and the Food Hygiene Information Scheme on food hygiene standards and food-borne illnesses. Food Standards Agency. March 2015.
  5. Fotaki M. What market-based patient choice can’t do for the NHS: The theory and evidence of how choice works in health care. 2014.
Advertisements

One thought on “League tables – not always bad”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s