Tag Archives: Care

Measuring Quality of Care

Measuring quality of care is not a straightforward business:

  1. Routinely collected outcome data tend to be misleading because of very poor ratios of signal to noise.[1]
  2. Clinical process (criterion based) measures require case note review and miss important errors of omission, such as diagnostic errors.
  3. Adverse events also require case note review and are prone to measurement error.[2]

Adverse event review is widely practiced, usually involving a two-stage process:

  1. A screening process (sometimes to look for warning features [triggers]).
  2. A definitive phase to drill down in more detail and refute or confirm (and classify) the event.

A recent HS&DR report [3] is important for two particular reasons:

  1. It shows that a one-stage process is as sensitive as the two-stage process. So triggers are not needed; just as many adverse events can be identified if notes are sampled at random.
  2. In contrast to (other) triggers, deaths really are associated with a high rate of adverse events (apart, of course, from the death itself). In fact not only are adverse events more common among patients who have died than among patients sampled at random (nearly 30% vs. 10%), but the preventability rates (probability that a detected adverse event was preventable) also appeared slightly higher (about 60% vs. 50%).

This paper has clear implications for policy and practice, because if we want a population ‘enriched’ for high adverse event rates (on the ‘canary in the mineshaft’ principle), then deaths provide that enrichment. The widely used trigger tool, however, serves no useful purpose – it does not identify a higher than average risk population, and it is more resource intensive. It should be consigned to history.

Lastly, England and Wales have mandated a process of death review, and the adverse event rate among such cases is clearly of interest. A word of caution is in order here. The reliability (inter-observer agreement) in this study was quite high (Kappa 0.5), but not high enough for comparisons across institutions to be valid. If cross-institutional comparisons are required, then:

  1. A set of reviewers must review case notes across hospitals.
  2. At least three reviewers should examine each case note.
  3. Adjustment must be made for reviewer effects, as well as prognostic factors.

The statistical basis for these requirements are laid out in detail elsewhere.[4] It is clear that reviewers should not review notes from their own hospitals, if any kind of comparison across institutions is required – the results will reflect the reviewers rather than the hospitals.

Richard Lilford, CLAHRC WM Director


  1. Girling AJ, Hofer TP, Wu J, et al. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling studyBMJ Qual Saf. 2012; 21(12): 1052-6.
  2. Lilford R, Mohammed M, Braunholtz D, Hofer T. The measurement of active errors: methodological issues. Qual Saf Health Care. 2003; 12(s2): ii8-12.
  3. Mayor S, Baines E, Vincent C, et al. Measuring harm and informing quality improvement in the Welsh NHS: the longitudinal Welsh national adverse events study. Health Serv Deliv Res. 2017; 5(9).
  4. Manaseki-Holland S, Lilford RJ, Bishop JR, Girling AJ, Chen YF, Chilton PJ, Hofer TP; UK Case Note Review Group. Reviewing deaths in British and US hospitals: a study of two scales for assessing preventability. BMJ Qual Saf. 2016. [ePub].

I Know that Cracks in Care Between Institutions Undermine Patient Safety, but How Can I Rectify the Problem?

Cracks between institutions

It is well known that danger arises when care is fragmented over many organisations (hospital, general practice, community care, social services, care home, etc.). With the rise in the proportion of patients with chronic and multiple diseases, fragmented care may have become the number one safety issue in modern health care. Confusion of responsibility, silo thinking, contradictory instructions, and over and under treatment are all heightened risks when care is shared between multiple providers – patients will tell you that. The risk is clearly identified, but how can it be mitigated? There is a limit to what can be achieved by structural change – Accountable Care Organisations featured in a recent blog, for instance.[1] Irrespective of the way care is structured, front line staff need to learn how to function in a multidisciplinary, inter-agency setting so that they can properly care for people with complex needs. Simply studying different ways of organising care, as recommended by NICE,[2] does not get to the heart of the problem in our view. The business aphorism “culture eats strategy for breakfast” applies equally to inter-sectoral working in health and social care. Studying how care givers in different places can better work in teams to provide integrated care is hard, but the need to do so cannot be ignored; we must try. We propose first, a method to enhance performance at the sharp end of care, and second, a system to sustain the improvement.

Improving performance

Improving performance of clinicians who need to work as a team, when the members of the team are scattered across different places, and patients have different, complex needs, is a challenge. For a start, there is no fixed syllabus based on ‘proverbial knowledge’. Guidelines deal with conditions one at a time.[3] There can be no set of guidelines that reconciles all possible combinations of disease-specific guidelines for patients suffering from many diseases.[4] [5] Everything is a matter of balance – the need to avoid giving patients more medicines than they can cope with is in tension with the need to provide evidence-based medicines for each condition. The greater the number of medicines prescribed, the lower is the adherence rate to each prescribed medicine, but it is not possible to pre-specify where the optimal prescribing threshold lies.[6] The lack of a specifiable syllabus does not mean performance cannot be enhanced – it is not just proverbial knowledge that can be enhanced through education, tacit knowledge can be too.[7-9] There is an extensive theoretical and empirical literature concerning the teaching of tacit skills; the central idea is for people to work together in solving the kinds of problems they will encounter in the real world.[10] In the process some, previously tacit, knowledge may be abstracted from deliberations to become proverbial (for an example, see box). Management is a topic that is hard to codify. So (highly remunerated) business schools use case studies as the basis for discussion in the expectation that tacit knowledge will be enhanced. We plan to build on theory and experience to implement learning in facilitated groups to help clinical staff provide better integrated care – we will create opportunities for staff of different types to work through scenarios from real life in facilitated groups. We will use published case studies [11] as a template for further scenario development. Group deliberations will be informed by published guidelines that aim to enhance care of patients with multi-morbidity (although these have been written to guide individual consultations rather than to assist management across sectors).[11-13] In the process group members will gain tacit knowledge (and perhaps some proverbial knowledge will emerge as in the example in the box). CLAHRC WM is implementing this method in a study funded by an NIHR Programme Development grant.[14] But how can it be made sustainable?

Box: Hypothetical Scenario Where Proverbial Knowledge Emerges from Discussion of a Complex Topic

The topic of conflicting information came up in a facilitated work group. A general practitioner argued that this was a difficult problem to avoid, since a practitioner could not know what a patient may have been told by another of their many care-givers. One of the patient participants observed that contradictory advice was not just confusing, but distressing. A community physiotherapist said that he usually elicited previous advice from patients so that he would not inadvertently contradict, or appear to contradict, previous advice. The group deliberated the point and concluded that finding out what advice a patient had received was a good idea, and should be included as a default tenet of good practice.


Again we turn to management theory – there are lots to choose from, but they embody similar ideas. We will take for Ferlie and Shortell.[15] To make a method stick, three organisational levels must be synchronised:

  1. Practitioners at the sharp end who must implement change. They will be invited to join multi-disciplinary groupings and participate in the proposed work groups, as above.
  2. The middle level of management who can facilitate or frustrate a new initiative must make staff development an on-going  priority, for example by scheduling team-building activities in time tables. Our CLAHRC is conducting a project on making care safer in care homes, where much can be done to reduce risk at interfaces in care.
  3. The highest levels of management, who can commit resources and drive culture change by force of personality and the authority of high office, must be engaged. This includes hospitals at board levels and local authorities. Patients have a big role to play – they are the only people who experience the entire care pathway and hence who are experts in it. They can campaign for change and for buy-in from top managers.

CLAHRC WM has deep commitment from major participating hospitals in the West Midlands, from Clinical Commissioning Groups, and local authorities. These organisations are all actively engaged in improving interfaces in care, and the draft Sustainability and Transferability Partnership strategy for Birmingham and Solihull includes plans to better integrate care. We will build on these changes to promote and sustain bottom-up education, supported by the Behavioural Psychology group at Warwick Business School, to drive forward this most challenging but important of all initiatives – improving safety across interfaces in care.

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ. Accountable Care Organisations. NIHR CLAHRC West Midlands. 11 November 2016.
  2. National Institute for Health and Care Excellence. Multimorbidity: clinical assessment and management. London, UK: NICE, 2016.
  3. Wyatt KD, Stuart LM, Brito JP, et al. Out of Context: Clinical Practice Guidelines and Patients with Multiple Chronic Conditions. A Systematic Review. Med Care. 2014; 52 (3s2): s92-100.
  4. Lilford RJ. Multi-morbidity. NIHR CLAHRC West Midlands. 20 November 2015.
  5. Boyd CM, Darer J, Boult C, et al. Clinical Practice Guidelines and Quality of Care for Older Patients With Multiple Comorbid Diseases: Implications for Pay for Performance. JAMA. 2005; 294(6): 716-24.
  6. Tinetti ME, Bogardus ST, Agostini JV. Potential Pitfalls of Disease-Specific Guidelines for Patients with Multiple Conditions. New Engl J Med. 2014; 351: 2870-4.
  7. Patel V, Arocha J, Kaufman D. Expertise and tacit knowledge in medicine. In: Tact knowledge in professional practice: researcher and practitioner perspectives. Sternberg RJ (ed). Mahwah, NJ: Lawrence Erlbaum Associates, 1999.
  8. Nonaka I, von Krogh G. Tacit knowledge and knowledge conversion: Controversy and advancement in organizational knowledge creation theory. Organ Sci. 2009; 23(3): 635-52.
  9. Eraut M. Non-formal learning and tacit knowledge in professional work. Brit J Ed Psychol. 2000; 70: 113-36.
  10. Lilford RJ. Tacit and Explicit Knowledge in Health Care. NIHR CLAHRC West Midlands. 14 August 2015.
  11. American Geriatrics Society Expert Panel on the Care of Older Adults with Multimorbidity. Guiding Principles for the Care of Older Adults with Multimorbidity: An Approach for Clinicians. J Am Geriatr Soc. 2012; 60(10): E1-25.
  12. Muth C, van den Akker M, Blom JW, et al. The Ariadne principles: how to handle multimorbidity in primary care consultations. BMC Medicine. 2014; 12: 223.
  13. American Geriatrics Society Expert Panel on the Care of Older Adults with Diabetes Mellitus. Guidelines Abstracted from the American Geriatrics Society Guidelines for Improving the Care of Older Adults with Diabetes Mellitus: 2013 Update. J Am Geriatr Soc. 2013; 61(11): 2020-6.
  14. Lilford, Combes, Taylor, Mallan, Mendelsohn. Improving clinical decisions and teamwork for patients with multimorbidity in primary care through multidisciplinary education and facilitation. NIHR Programme Grant. 2016-2017.
  15. Ferlie E, & Shortell S. Improving the quality of health care in the United Kingdom and the United States: a framework for change. Milbank Quart. 2001;79(2): 281-315.

Accountable Care Organisations

Accountable Care Organisations have been introduced in many settings in the USA. Evaluations are few and far between, but a recent overview [1] finds that while they do not save money, they are associated with improved processes of care (e.g. increased rates of cancer screening), and intermediate outcomes (e.g. HbA1c and blood pressure in people with diabetes). Attempts to create something similar in the UK by merging hospital and primary care budgets are underway in England, notably in Manchester. Before and after studies, such as those typically used in evaluations, are prone to exaggerate effectiveness of quality improvement initiatives,[2] thanks to the rising tide phenomenon.[3] Moreover, merging budgets is not the only way to improve coordination of care across providers, as discussed in a previous post.[4] That said, merged budgets do align provider financial incentives with patient need and core professional values, and we have not reached the end of history on this topic – not nearly.

— Richard Lilford, CLAHRC WM Director


  1. Song Z, Fisher ES. The ACO Experiment in Infancy – Looking Back and Looking Forward. JAMA. 2016; 316(7): 705-6.
  2. Eccles M, Grimshaw J, Campbell M, Ramsay C. Research designs for studies evaluating the effectiveness of change and improvement strategies. Qual Saf Health Care. 2003; 12: 47-52.
  3. Chen YF, Hemming K, Stevens AJ, Lilford RJ. Secular trends and evaluation of complex interventions: the rising tide phenomenon. BMJ Qual Saf. 2015. [ePub].
  4. Lilford RJ. Polycentric Organisations. NIHR CLAHRC West Midlands. 25 July 2014.

A Disappointing Article

All that glitters in the fabled New England Journal of Medicine is not gold. A recent article by Dale and colleagues is a masterclass in producing pleasing-sounding statements, and truisms that go precisely nowhere, but impress the undiscerning reader.[1] They write an article in favour of using quality metrics to improve care. Then they show that process measures may focus attention on things that can be counted at the expense of more important things that cannot. So they say we should count “what’s important to patients”. Then they point out that the signal to noise ratio will not emerge in most cases where outcomes are used – patients value not dying from cancer, but you can never judge your clinician’s performance in screening by cancer death rates. They advocate a ‘balanced mixture’ of measures and advertise their own. But they do not say or prove that they have the right balance. And they admit that using payment to change behaviour is effete. But they say it is a good idea. The whole thing is a muddle. Truth is, no one knows how to use metrics in performance management. But we advocate for task-based (clinical process) measures to ensure that the essentials are in place. We think outcome measures are a poor idea except for patient satisfaction and maybe outcomes of a very small number of highly technical procedures.[2]

— Richard Lilford, CLAHRC WM Director


  1. Dale CR, Myint M, Compton-Phillips AL. Counting Better – the Limits and Future of Quality-Based Compensation. New Engl J Med. 2016; 375(7): 609-11.
  2. Lilford RJ. Risk Adjusted Outcomes – Again! NIHR CLAHRC West Midlands News Blog. 24 April 2015.

Care that is Not Just Unskilled but Abusive

Maternal care is disrespectful to the point of abuse in many of the countries of the world.[1] How can it be that members of the caring professions can so abuse their position of trust? This short editorial argues that a culture of poor care can develop among perfectly ordinary people – indeed, we know this from the iconic experiments of Zimbardo [2] and Milgram.[3] As the Good Samaritan experiment shows,[4] people are exquisitely sensitive to their environment, especially their social environment.[5] So, here is my model for how an abusive culture develops:

— Richard Lilford, CLAHRC WM Director


  1. The Lancet. Achieving respectful care for women and babies. Lancet. 2015. 385:1366.
  2. Haney C, Banks C, Zimbardo P. A Study of Prisoners and Guards in a Simulated Prison. Washington, D.C.: Office of Naval Research. 1973.
  3. Milgram S. Behavioral Study of Obedience. J Abnorm Psychol. 1963; 67(4):371-8.
  4. Darley JM, Batson D. “From Jerusalem to Jericho”: A Study of Situational and Dispositional Variables in Helping Behavior. J Pers Soc Psychol. 1973; 27(1): 100-8.
  5. Bandura A, Ross D, Ross SA. Transmission of aggression through imitation of aggressive modelsJ Abnorm Soc Psychol. 1961; 63: 575-82.