Tag Archives: Patient safety

Patient Involvement in Patient Safety: Null Result from a High Quality Study

Most patient safety evaluations are simple before and after / time series improvement studies. So it is always refreshing to find a study with contemporaneous controls. Lawton and her colleagues report a nice cluster randomized trial covering 33 hospital wards in five hospitals.[1] They evaluate a well-known patient safety intervention based on the idea of giving patients a more active role in monitoring safety on their ward.

The trial produced a null result, but some of the measures of safety were in the right direction and there was a correlation between the enthusiasm/fidelity with which the intervention was implemented and measures of safety.

Safety is hard to measure (as the authors state), and improvement often builds on a number of small incremental changes. So, it would be very nice to see this intervention replicated, possibly with measures to generate greater commitment from ward staff.
Here is the problem with patient safety research; on the one hand the subject of patient safety is full of hubristic claims made on the basis of insufficient (weak) evidence. On the other hand, high quality studies, such as the one reported here, often fail to find an effect. In many cases, as in the study reported here, there are reasons to suspect a type 2 error (false negative result). Beware also the rising tide – the phenomenon that arises where a trial occurs in the context of a strong secular trend – this trend ‘swallows up’ the headroom for a marginal intervention effect.[2] What is to be done? First, do not declare defeat too early. Second, be prepared to either carry out larger studies or replication studies that can be combined in a meta-analysis. Third, make multiple measurements across a causal chain [3] and synthesise this disparate data using Bayesian networks.[4] Fourth, further to the Bayesian approach, do not dichotomise results on the standard frequentist statistical convention into null and positive. It is stupid to classify a p-value of 0.06 as null if other evidence supports an effect, or to classify a p-value of 0.04 as positive if other data point the opposite way. Knowledge of complex areas, such as service interventions to improve safety, should take account of patterns in the data and information external to the index study. Bayesian networks provide a framework for such an analysis.[4] [5]

— Richard Lilford, CLAHRC WM Director


  1. Lawton R, O’Hara JK, Sheard L, et al. Can patient involvement improve patient safety? A cluster randomised control trial of the Patient Reporting and Action for a Safe Environment (PRASE) intervention. BMJ Qual Saf. 2017; 26: 622-31.
  2. Chen YF, Hemming K, Stevens AJ, Lilford RJ. Secular trends and evaluation of complex interventions: the rising tide phenomenon. BMJ Qual Saf. 2016; 25: 303-10.
  3. Lilford RJ, Chilton PJ, Hemming K, Girling AJ, Taylor CA, Barach P. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ. 2010; 341: c4413.
  4. Watson SI & Lilford RJ. Essay 1: Integrating multiple sources of evidence: a Bayesian perspective. In: Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Southampton (UK): NIHR Journals Library, 2016.
  5. Lilford RJ, Girling AJ, Sheikh, et al. Protocol for evaluation of the cost-effectiveness of ePrescribing systems and candidate prototype for other related health information technologies. BMC Health Serv Res. 2014; 14: 314.

Yet More Evidence that Patient Safety Culture Measures, Measure Nothing

Searching for a patient safety culture measurement ‘tool’ is like a search for the Holy Grail. I think that it is a search for something that does not exist. If safety culture exists, then safety practices should correlate within organisations. They don’t.[1] [2] Meddings, et al. [3] seem slightly surprised by their finding that bloodstream infections improved in many US hospitals with absolutely no change in safety culture. Of course, if you teach ‘safety culture’ to staff then their scores will improve as they have learned the correct answers. But there is little evidence that, absent such a change, improvements in safety are related to a measurable construct of culture.

— Richard Lilford, CLAHRC WM Director


  1. Wilson B, Thornton JG, Hewison J, Lilford RJ, Watt I, Braunholtz D, Robinson M. The Leeds University Maternity Audit Project. Int J Qual Health Care. 2002; 14(3): 175-81.
  2. Jha A, & Pronovost P. Toward a Safer Health Care System. The Critical Need to Improve Measurement. JAMA. 2016; 315(17): 1831-2.
  3. Meddings J, Reichert H, Green MT, et al. Evaluation of the association between Hospital Survey on Patient Safety Culture (HSOPS) measures and catheter-associated infections: results of two national collaboratives. BMJ Qual Saf. 2017; 26: 226-35.

I Know that Cracks in Care Between Institutions Undermine Patient Safety, but How Can I Rectify the Problem?

Cracks between institutions

It is well known that danger arises when care is fragmented over many organisations (hospital, general practice, community care, social services, care home, etc.). With the rise in the proportion of patients with chronic and multiple diseases, fragmented care may have become the number one safety issue in modern health care. Confusion of responsibility, silo thinking, contradictory instructions, and over and under treatment are all heightened risks when care is shared between multiple providers – patients will tell you that. The risk is clearly identified, but how can it be mitigated? There is a limit to what can be achieved by structural change – Accountable Care Organisations featured in a recent blog, for instance.[1] Irrespective of the way care is structured, front line staff need to learn how to function in a multidisciplinary, inter-agency setting so that they can properly care for people with complex needs. Simply studying different ways of organising care, as recommended by NICE,[2] does not get to the heart of the problem in our view. The business aphorism “culture eats strategy for breakfast” applies equally to inter-sectoral working in health and social care. Studying how care givers in different places can better work in teams to provide integrated care is hard, but the need to do so cannot be ignored; we must try. We propose first, a method to enhance performance at the sharp end of care, and second, a system to sustain the improvement.

Improving performance

Improving performance of clinicians who need to work as a team, when the members of the team are scattered across different places, and patients have different, complex needs, is a challenge. For a start, there is no fixed syllabus based on ‘proverbial knowledge’. Guidelines deal with conditions one at a time.[3] There can be no set of guidelines that reconciles all possible combinations of disease-specific guidelines for patients suffering from many diseases.[4] [5] Everything is a matter of balance – the need to avoid giving patients more medicines than they can cope with is in tension with the need to provide evidence-based medicines for each condition. The greater the number of medicines prescribed, the lower is the adherence rate to each prescribed medicine, but it is not possible to pre-specify where the optimal prescribing threshold lies.[6] The lack of a specifiable syllabus does not mean performance cannot be enhanced – it is not just proverbial knowledge that can be enhanced through education, tacit knowledge can be too.[7-9] There is an extensive theoretical and empirical literature concerning the teaching of tacit skills; the central idea is for people to work together in solving the kinds of problems they will encounter in the real world.[10] In the process some, previously tacit, knowledge may be abstracted from deliberations to become proverbial (for an example, see box). Management is a topic that is hard to codify. So (highly remunerated) business schools use case studies as the basis for discussion in the expectation that tacit knowledge will be enhanced. We plan to build on theory and experience to implement learning in facilitated groups to help clinical staff provide better integrated care – we will create opportunities for staff of different types to work through scenarios from real life in facilitated groups. We will use published case studies [11] as a template for further scenario development. Group deliberations will be informed by published guidelines that aim to enhance care of patients with multi-morbidity (although these have been written to guide individual consultations rather than to assist management across sectors).[11-13] In the process group members will gain tacit knowledge (and perhaps some proverbial knowledge will emerge as in the example in the box). CLAHRC WM is implementing this method in a study funded by an NIHR Programme Development grant.[14] But how can it be made sustainable?

Box: Hypothetical Scenario Where Proverbial Knowledge Emerges from Discussion of a Complex Topic

The topic of conflicting information came up in a facilitated work group. A general practitioner argued that this was a difficult problem to avoid, since a practitioner could not know what a patient may have been told by another of their many care-givers. One of the patient participants observed that contradictory advice was not just confusing, but distressing. A community physiotherapist said that he usually elicited previous advice from patients so that he would not inadvertently contradict, or appear to contradict, previous advice. The group deliberated the point and concluded that finding out what advice a patient had received was a good idea, and should be included as a default tenet of good practice.


Again we turn to management theory – there are lots to choose from, but they embody similar ideas. We will take for Ferlie and Shortell.[15] To make a method stick, three organisational levels must be synchronised:

  1. Practitioners at the sharp end who must implement change. They will be invited to join multi-disciplinary groupings and participate in the proposed work groups, as above.
  2. The middle level of management who can facilitate or frustrate a new initiative must make staff development an on-going  priority, for example by scheduling team-building activities in time tables. Our CLAHRC is conducting a project on making care safer in care homes, where much can be done to reduce risk at interfaces in care.
  3. The highest levels of management, who can commit resources and drive culture change by force of personality and the authority of high office, must be engaged. This includes hospitals at board levels and local authorities. Patients have a big role to play – they are the only people who experience the entire care pathway and hence who are experts in it. They can campaign for change and for buy-in from top managers.

CLAHRC WM has deep commitment from major participating hospitals in the West Midlands, from Clinical Commissioning Groups, and local authorities. These organisations are all actively engaged in improving interfaces in care, and the draft Sustainability and Transferability Partnership strategy for Birmingham and Solihull includes plans to better integrate care. We will build on these changes to promote and sustain bottom-up education, supported by the Behavioural Psychology group at Warwick Business School, to drive forward this most challenging but important of all initiatives – improving safety across interfaces in care.

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ. Accountable Care Organisations. NIHR CLAHRC West Midlands. 11 November 2016.
  2. National Institute for Health and Care Excellence. Multimorbidity: clinical assessment and management. London, UK: NICE, 2016.
  3. Wyatt KD, Stuart LM, Brito JP, et al. Out of Context: Clinical Practice Guidelines and Patients with Multiple Chronic Conditions. A Systematic Review. Med Care. 2014; 52 (3s2): s92-100.
  4. Lilford RJ. Multi-morbidity. NIHR CLAHRC West Midlands. 20 November 2015.
  5. Boyd CM, Darer J, Boult C, et al. Clinical Practice Guidelines and Quality of Care for Older Patients With Multiple Comorbid Diseases: Implications for Pay for Performance. JAMA. 2005; 294(6): 716-24.
  6. Tinetti ME, Bogardus ST, Agostini JV. Potential Pitfalls of Disease-Specific Guidelines for Patients with Multiple Conditions. New Engl J Med. 2014; 351: 2870-4.
  7. Patel V, Arocha J, Kaufman D. Expertise and tacit knowledge in medicine. In: Tact knowledge in professional practice: researcher and practitioner perspectives. Sternberg RJ (ed). Mahwah, NJ: Lawrence Erlbaum Associates, 1999.
  8. Nonaka I, von Krogh G. Tacit knowledge and knowledge conversion: Controversy and advancement in organizational knowledge creation theory. Organ Sci. 2009; 23(3): 635-52.
  9. Eraut M. Non-formal learning and tacit knowledge in professional work. Brit J Ed Psychol. 2000; 70: 113-36.
  10. Lilford RJ. Tacit and Explicit Knowledge in Health Care. NIHR CLAHRC West Midlands. 14 August 2015.
  11. American Geriatrics Society Expert Panel on the Care of Older Adults with Multimorbidity. Guiding Principles for the Care of Older Adults with Multimorbidity: An Approach for Clinicians. J Am Geriatr Soc. 2012; 60(10): E1-25.
  12. Muth C, van den Akker M, Blom JW, et al. The Ariadne principles: how to handle multimorbidity in primary care consultations. BMC Medicine. 2014; 12: 223.
  13. American Geriatrics Society Expert Panel on the Care of Older Adults with Diabetes Mellitus. Guidelines Abstracted from the American Geriatrics Society Guidelines for Improving the Care of Older Adults with Diabetes Mellitus: 2013 Update. J Am Geriatr Soc. 2013; 61(11): 2020-6.
  14. Lilford, Combes, Taylor, Mallan, Mendelsohn. Improving clinical decisions and teamwork for patients with multimorbidity in primary care through multidisciplinary education and facilitation. NIHR Programme Grant. 2016-2017.
  15. Ferlie E, & Shortell S. Improving the quality of health care in the United Kingdom and the United States: a framework for change. Milbank Quart. 2001;79(2): 281-315.

Need to Improve Measurement

The patient safety movement is bogged down. The reason is that it cannot measure its central concept – safety – and measurement is a necessary (albeit not sufficient) condition for a subject to advance. Jha and Pronovost [1] make the excellent point that billing systems (read routine data in an NHS context) are not up to the job as they are subject to surveillance bias (better institutions report more incidents), and are heavily gamed. Signal-to-noise ratios are often poor.[2] They correctly point out that clinical data are needed, and these are now becoming available on both sides of the Atlantic as hospitals implement electronic clinical records. The authors give nice examples of adverse events that are contingent on the clinical situation and that can be harvested from notes. But they do not emphasise process measures that provide one of the richest source of data for quality enhancement.[3-5]

— Richard Lilford, CLAHRC WM Director


  1. Jha A, & Pronovost P. Toward a Safer Health Care System. The Critical Need to Improve Measurement. JAMA. 2016; 315(17):1831-2.
  2. Girling AJ, Hofer TP, Wu J, et al. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study. BMJ Qual Saf. 2012; 21(12): 1052-6.
  3. Coleman JJ, Hodson J, Brooks HL, Rosser D. Missed medication doses in hospitalised patients: a descriptive account of quality improvement measures and time series analysis. Int J Qual Health Care. 2013; 25(5): 564-72.
  4. Coleman JJ, Hemming K, Nightingale PG, et al. Can an electronic prescribing system detect doctors who are more likely to make a serious prescribing error? J R Soc Med. 2011; 104(5): 208-18.
  5. Lilford RJ, Chilton PJ, Hemming K, Girling AJ, Taylor CA, Barach P. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ. 2010; 341: c4413.

A Proper Large-Scale Quality Improvement Study in a Middle-Income Country

The vast majority of studies testing an intervention to improve quality/safety of care are conducted in high-income countries. However, a cluster RCT of 118 Brazilian ICUs (6,761 patients) has recently been reported.[1] The intervention was compound (multi-component), involving goal setting, clinician prompting, and multi-disciplinary ward rounds. Although mortality and other patient outcomes were not improved, clinical processes (e.g. use of appropriate settings on the ventilator and avoidance of heavy sedation) did improve. The nub of my argument is that clinical outcomes are insensitive indicators of improved practice, and we should be content with showing improved adherence to proven care standards – the argument is laid out numerically elsewhere.[2] The safety and quality movement is doomed so long as we insist on showing improvements in patient level outcomes.

— Richard Lilford, CLAHRC WM Director


  1. Writing Group for the CHECKLIST-ICU Investigators and the Brazilian Research in Intensive Care Network (BRICNet). Effect of a Quality Improvement Intervention with Daily Round Checklists, Goal Setting, and Clinician Prompting on Mortality of Critically Ill Patients. JAMA. 2016;315(14):1480-90.
  2. Lamont T, Barber N, de Pury J, et al. New approaches to evaluating complex health and care systems. BMJ. 2016; 352: i154.

Are People Who Are Deeply Religious More Altruistic Than Secular Controls?

CLAHRC WM takes a large interest in the clinician-patient relationship – the Director has a special interest in the doctor-patient relationship. Moreover, CLAHRC WM has developed a research protocol (led by Prof Julian Bion) to evaluate methods to augment compassion in acute medical care. But the doctors and clinicians are under broader influences than their immediate work environment and their post-professional education. Despite a similar education and environment some give much more of themselves than others. There are broader personal and cultural influences at work. So one may suppose that doctors who are very religious might give more than their secular peers. Well, any research on that lies in the future. But as far as human beings as a whole are concerned, the Economist provides a synopsis [1] of a fascinating study.[2] They studied altruistic responses using a variant of the well-studied Dictator Game, which is a validated test of altruism. The investigators interviewed 1,170 children, one per family, across six countries. About half of the families turned out to be religious, and half of these were ‘highly observant’. So were the children of pious families more altruistic than their peers? Were they equally altruistic? Could it be that they were less altruistic? Well it turned out that children of non-religious families were more altruistic than their peers. What’s going on here? Is there a flaw in the study? If not, how can the results be explained? The CLAHRC WM Director is surprised by this result and has no answer to these questions.

— Richard Lilford, CLAHRC WM Director


  1. The Economist. Matthew 22:39. The Economist. 07 Nov 2015.
  2. Decety J, Cowell JM, Lee K, et al. The Negative Association between Religiousness and Children’s Altruism across the World. Curr Biol. 2015; 25(22): 2951-5.

Step Wedge Cluster Design for Service Delivery Interventions Comes to New England Journal of Medicine

Dreischulte and colleagues randomised 34 practices in clusters using a step-wedge design, to evaluate a complex intervention to reduce prescription errors in primary care.[1] The intervention included an educational component and informatics (as in the famous PINCER trial [2]), but also included a financial incentive. There was a marked drop in the types of high-risk prescribing targeted in the intervention. In addition, admissions decreased in the hypothesised direction. Adjustments were made for intra-class correlations at various time points, but what about temporal trends? Is this the first step wedge cluster study in the world’s top medical journal?

— Richard Lilford, CLAHRC WM Director


  1. Dreischulte T, Donnan P, Grant A, et al. Safer Prescribing – A Trial of Education, Informatics, and Financial Incentives. New Engl J Med. 2016; 374(11): 1053-64.
  2. Hemming K, Chilton PJ, Lilford RJ, Avery A, Sheikh A. Bayesian cohort and cross-sectional analyses of the PINCER trial: a pharmacist-led intervention to reduce medication errors in primary care. PLoS One. 2012; 7(6): e38306.

Patient Safety Really is Improving

Research carried out by CLAHRC WM colleagues showed, mainly on the basis of process measures, that hospital care in the UK became safer over the ‘Blair Decade’.[1] [2] Now an even larger Dutch study, 2005-2013,[3] has produced corroborating findings with respect to adverse events. Both studies were based on case-note review. The Dutch study found an approximately one-third reduction in adverse events on retrospective review of nearly 16,000 case-notes. So, there are now two separate studies that have used a consistent methodology over time and both suggest that care is becoming safer. This is probably the result of national initiatives and diffusion of safety ideas among clinicians. Indeed one of the reasons put forward for failure to find a statistically significant effect from the Safer Patients Initiative in the UK was the system-wide temporal trend, or ‘rising tide’.[4] There are good arguments to conduct a further follow-up of safety in UK hospitals to see if the improvement noted over the first decade of the millennium has been sustained. This might be the last chance, since case-note review may become more difficult as the future case record is fragmented across hospital IT systems.

— Richard Lilford, CLAHRC WM Director


  1. Benning A, Ghaleb M, Suokas A. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ. 2011; 342:d195.
  2. Benning A, Dixon-Woods M, Nwulu U, et al. Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ. 2011; 342:d199.
  3. Baines R, Langelaan M, de Bruijne M, Spreeuwenberg P, Wagner C. How effective are patient safety initiatives? A retrospective patient record review study of changes to patient safety over time. BMJ Qual Saf. 2015; 24: 561-71.
  4. Chen Y, Hemming K, Stevens AJ, Lilford RJ. Secular trends and evaluation of complex interventions: the rising tide phenomenon. BMJ Qual Saf. 2015. [ePub].

Patient safety in hospitals: errors decline in UK and now in the US

The CLAHRC WM Director and colleagues demonstrated a reduction in error rates over the previous decade by means of in-depth case reviews in 19 UK hospitals. Infection rates, hand washing rates, vital sign observations, the quality of medical history taking, adherence to various tenets of evidence-based practice, and patient satisfaction all showed an improved trend.[1]

Now AHRQ has demonstrated improving safety in US Hospitals in the first half of the current decade, including pressure ulcers, bloodstream infections, and drug errors.[2] [3] The authors think the reasons are multi-faceted, but financial incentives may have played a large part. Increased spending on the NHS over the years of the Blair Government is an obvious candidate explanation for similar, but earlier, improvements in the UK. It would be interesting to see if the improvement in the safety of UK hospitals in the last decade has been sustained or augmented in the current decade.

— Richard Lilford, CLAHRC WM Director


  1. Cohn J. A Picture of Progress on Hospital Errors. Milbank Quarterly. 2015. 93(1); 36-9.
  2. Benning A, Dixon-Woods M, Nwulu U, et al. Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ. 2011; 342:d199.
  3. Agency for Healthcare Research and Quality. Interim update on 2013 annual hospital-acquired condition rate and estimates of cost savings and deaths averted from 2010 to 2013. Rockville, MD: Agency for Healthcare Research and Quality. 2014.

Can We Do Without Heroism in Health Care?

Two icons of patient safety, Peter Pronovost (Time magazine’s Top 100, 2008) and Ara Darzi (British House of Lords) have recently penned a manifesto. It calls for a systems approach to safety assurance and less reliance on heroism to make up for weak systems.[1] The document is well written, persuasive and wrong. Well, not wrong, just not right!

Let’s start with the systems point. The argument put forward by Pronovost and Darzi echoes a constant refrain from safety pundits that goes something like this:

Health care is beset by safety problems; Adverse events (AEs) are more likely when weak systems leave gaping holes that link up to cause adverse events, as per Reason’s “Swiss Cheese” model; Systems approaches have massively reduced AEs in other industries; But two decades of trying to replicate these achievements have yielded little progress in health care; By Jupiter, we must try harder to improve the system!

However, it would seem more reasonable to conclude that health care is not like other industries – as argued in a previous post, only about one quarter of AEs arise from archetypal system problems in the sense that their root cause lies in the host organisation. The remainder follow diagnostic errors (broadly defined) and procedure-related errors. These both lie in the province of front-line doctors failing to exhibit sufficient skill. Of course manifest skill can be influenced by local conditions, for instance if the doctor has to care for more patients than she can cope with. And they emanate from a broader system concerned with selection, training and so on (see ‘Bring Back the Ward Round’ below). But it is hard for a hospital to indemnify itself against a surge in demand, and recruiting and training doctors plays out over decades.

Things that can be systemised are being addressed to good effect. Hospital-acquired infections are massively down; severe pressure ulcers down; medication error heading south; and wrong site surgery is right down.

The patient safety ‘industry’ needs to move on from its preoccupation with infection, falls, medication errors, pressure ulcers, and rare egregious errors. And to be fair, Pronovost has campaigned elsewhere for more emphasis on diagnosis,[2] while Darzi is a world leader in surgical training.[3]

And that is where we need to turn to this issue of heroism. Here the story goes something like this:

It took heroes, such as Charles Lindbergh, to establish aviation; But a modern airline pilot with a barn-storming attitudeis a “bloody nuisance”; Medicine also needed adventurous doctors like Christiaan Barnard in its heroic phase; But now we need bland team players who can follow guidelines

Now I am not arguing for the return of Sir Lancelot Spratt and fully understand that it is patients, not doctors, who put their lives on the line. But treating health care workers like office clerks is wrong – again and again clinicians have to go above and beyond, doing hard cognitive, physical and emotional work – a good doctor, has to ‘give of herself’.

A good doctor who has done a hard night in A&E has not just spent the evening following guidelines; she has made sound judgements under uncertainty, maintained her composure under abuse, coped with a stream of patients arriving faster than they can be seen, and she has kept a cheerful demeanour throughout. She may have helped hit the government’s four hour target. Doctors in many parts of the world feel demoralised, but medicine cannot go back to the hands-off approach of yesteryear – scrutiny and regulations are here to stay for very good reason. But don’t let the pendulum swing too far – recognise that the work is hard, that it cannot be completely codified, and that it is deeply personal, and leave space for just a little heroism.

— Richard Lilford, CLAHRC WM Director


  1. Pronovost PJ, Ravitz AD, Stoll RA, Kennedy SB. Transforming Patient Safety: A Sector-wide Systems Approach. Doha, Qatar: World Innovation Summit for Health. 2015.
  2. Newman-Toker DE, & Pronovost PJ. Diagnostic Errors – The Next Frontier for Patient Safety. JAMA. 2009; 301(10): 1060-2.
  3. Singh P, & Darzi A. Surgical training. Br J Surg. 2013; 100: 307-9.