Tag Archives: Adverse events

Diagnosis, Diagnosis, Diagnosis

A new study from Ann Arbor confirms the crucial importance of diagnosis.[1] This study, based on analysis of no less than 63,000 settled inpatient malpractice claims shows that 22% were diagnosis-related. Moreover, the diagnosis-related claims were about twice as likely as other types of settled claim to result in permanent harm or death. Of course, malpractice claims represent only a small portion of all safety incidents. Nevertheless, this paper should be assimilated with other evidence, such as the high proportion of misdiagnoses at post mortem, even among thoroughly scrutinised patients who died in intensive care.[2] Some diagnostic errors result from a lack of vigilance, but diagnosis is hard when a condition is uncommon, or when a common condition presents in an uncommon way. And beware the unknown unknowns. At one time the BMJ produced a list of ‘easily missed diagnoses’, such as myasthenia gravis and Hirschsprung’s disease. Some eminently treatable diseases that are easily missed were listed in a previous blog [3] – Addison’s disease, mastocytosis, Lyme disease, Kawasaki’s disease, leaking aneurisms, and right-sided endocarditis, for example. You might want to add your own. TB, the great mimicker; drug overdose / interactions; and nutritional deficiencies are often missed.

— Richard Lilford, CLAHRC WM Director


  1. Gupta A, Snyder A, Kachalia A, Flanders S, Saint S, Chopra V. Malpractice claims related to diagnostic errors in the hospital. BMJ Qual Saf. 2018; 27: 53-60.
  2. Winters B, Custer J, Galvagno SM, et al. Diagnostic errors in the intensive care unit: a systematic review of autopsy studies. BMJ Qual Saf. 2012; 21: 894-902.
  3. Lilford RJ. Diagnosis, Diagnosis, Diagnosis. Richard Lilford’s Friday Blog. February 15, 2013.

“We seek him here, we seek him there, Those Frenchies seek him everywhere.”

The notorious weekend mortality effect is every bit as elusive as the Scarlet Pimpernel. Recent studies have delved deeper into the possibility that the weekend effect is an artefact of admission of sicker patients at the weekend than on week days.[1] First, it has been shown that the mortality of all who present to the emergency department (i.e. admitted plus sent home) is the same over the weekend as over the rest of the week.[2] Second, patients who arrive by ambulance are generally much sicker than patients arriving by other means and the proportion who arrive by ambulance is higher over the weekend than over weekdays.[3] When controlling for method of arrival, most of the weekend effect disappears. Most, but not all. This paper provides further evidence that most estimates of the weekend effect are at least overestimates. Through Professor Julian Bion’s HiSLAC Study [4] we are evaluating the effect of weekend admission, not just on mortality, but also on the quality of care and the overall adverse event rate. We will use a Bayesian network to synthesise information across the causal chain and come up with a refined estimate of the effect of weekend admission, not only on mortality, but also on other adverse events.

— Richard Lilford, CLAHRC WM Director


  1. Bray BD, Steventon A. What have we learnt after 15 years of research into the ‘weekend effect’? BMJ Qual Saf. 2017; 26: 607-10.
  2. Aldridge C, Bion J, Boyal A, et al. Weekend specialist intensity and admission mortality in acute hospital trusts in England: a cross-sectional study. Lancet. 2016. 388: 178-86.
  3. Anselmi L, Meacock R, Kristensen SR, Doran T, Sutton M. Arrival by ambulance explains variation in mortality by time of admission: retrospective study of admissions to hospital following emergency department attendance in England. BMJ Qual Saf. 2017; 26: 613-21.
  4. Chen Y, Boyal A, Sutton E, et al. The magnitude and mechanisms of the weekend effect in hospital admissions: A protocol for a mixed methods review incorporating a systematic review and framework synthesis. Syst Rev. 2016; 5: 84.

Measuring Quality of Care

Measuring quality of care is not a straightforward business:

  1. Routinely collected outcome data tend to be misleading because of very poor ratios of signal to noise.[1]
  2. Clinical process (criterion based) measures require case note review and miss important errors of omission, such as diagnostic errors.
  3. Adverse events also require case note review and are prone to measurement error.[2]

Adverse event review is widely practiced, usually involving a two-stage process:

  1. A screening process (sometimes to look for warning features [triggers]).
  2. A definitive phase to drill down in more detail and refute or confirm (and classify) the event.

A recent HS&DR report [3] is important for two particular reasons:

  1. It shows that a one-stage process is as sensitive as the two-stage process. So triggers are not needed; just as many adverse events can be identified if notes are sampled at random.
  2. In contrast to (other) triggers, deaths really are associated with a high rate of adverse events (apart, of course, from the death itself). In fact not only are adverse events more common among patients who have died than among patients sampled at random (nearly 30% vs. 10%), but the preventability rates (probability that a detected adverse event was preventable) also appeared slightly higher (about 60% vs. 50%).

This paper has clear implications for policy and practice, because if we want a population ‘enriched’ for high adverse event rates (on the ‘canary in the mineshaft’ principle), then deaths provide that enrichment. The widely used trigger tool, however, serves no useful purpose – it does not identify a higher than average risk population, and it is more resource intensive. It should be consigned to history.

Lastly, England and Wales have mandated a process of death review, and the adverse event rate among such cases is clearly of interest. A word of caution is in order here. The reliability (inter-observer agreement) in this study was quite high (Kappa 0.5), but not high enough for comparisons across institutions to be valid. If cross-institutional comparisons are required, then:

  1. A set of reviewers must review case notes across hospitals.
  2. At least three reviewers should examine each case note.
  3. Adjustment must be made for reviewer effects, as well as prognostic factors.

The statistical basis for these requirements are laid out in detail elsewhere.[4] It is clear that reviewers should not review notes from their own hospitals, if any kind of comparison across institutions is required – the results will reflect the reviewers rather than the hospitals.

Richard Lilford, CLAHRC WM Director


  1. Girling AJ, Hofer TP, Wu J, et al. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling studyBMJ Qual Saf. 2012; 21(12): 1052-6.
  2. Lilford R, Mohammed M, Braunholtz D, Hofer T. The measurement of active errors: methodological issues. Qual Saf Health Care. 2003; 12(s2): ii8-12.
  3. Mayor S, Baines E, Vincent C, et al. Measuring harm and informing quality improvement in the Welsh NHS: the longitudinal Welsh national adverse events study. Health Serv Deliv Res. 2017; 5(9).
  4. Manaseki-Holland S, Lilford RJ, Bishop JR, Girling AJ, Chen YF, Chilton PJ, Hofer TP; UK Case Note Review Group. Reviewing deaths in British and US hospitals: a study of two scales for assessing preventability. BMJ Qual Saf. 2016. [ePub].

Electronic Health Record System and Adverse Outcomes

Does introducing an electronic health record system in a hospital result in an increase in adverse outcomes? Seventeen hospitals were tracked over the ‘go live’ date to see if electronic records would affect risk-adjusted mortality or readmission rates.[1] The result was null – care neither improved nor deteriorated on the end-points observed. The CLAHRC WM Director’s comment is that these end-points are not sensitive to change, since the noise to signal ratio is poor. And, of course, the range of outcomes narrow – no mention here of any stress to patients or staff, for example. Nevertheless, these results are broadly reassuring, especially as the NHS is planning a big push on electronic notes following the Wachter review.[2]

— Richard Lilford, CLAHRC WM Director


  1. Barnett ML, Mehrotra A, Jena AB. Adverse inpatient outcomes during the transition to a new electronic health record system: observational study. BMJ. 2016; 354: i3835.
  2. Wachter R. Making IT work: harnessing the power of health information technology to improve care in England. London: Department of Health. 2016.

Patient Safety Really is Improving

Research carried out by CLAHRC WM colleagues showed, mainly on the basis of process measures, that hospital care in the UK became safer over the ‘Blair Decade’.[1] [2] Now an even larger Dutch study, 2005-2013,[3] has produced corroborating findings with respect to adverse events. Both studies were based on case-note review. The Dutch study found an approximately one-third reduction in adverse events on retrospective review of nearly 16,000 case-notes. So, there are now two separate studies that have used a consistent methodology over time and both suggest that care is becoming safer. This is probably the result of national initiatives and diffusion of safety ideas among clinicians. Indeed one of the reasons put forward for failure to find a statistically significant effect from the Safer Patients Initiative in the UK was the system-wide temporal trend, or ‘rising tide’.[4] There are good arguments to conduct a further follow-up of safety in UK hospitals to see if the improvement noted over the first decade of the millennium has been sustained. This might be the last chance, since case-note review may become more difficult as the future case record is fragmented across hospital IT systems.

— Richard Lilford, CLAHRC WM Director


  1. Benning A, Ghaleb M, Suokas A. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ. 2011; 342:d195.
  2. Benning A, Dixon-Woods M, Nwulu U, et al. Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ. 2011; 342:d199.
  3. Baines R, Langelaan M, de Bruijne M, Spreeuwenberg P, Wagner C. How effective are patient safety initiatives? A retrospective patient record review study of changes to patient safety over time. BMJ Qual Saf. 2015; 24: 561-71.
  4. Chen Y, Hemming K, Stevens AJ, Lilford RJ. Secular trends and evaluation of complex interventions: the rising tide phenomenon. BMJ Qual Saf. 2015. [ePub].

Ranking Hospitals on Preventable Deaths – an Article that Everyone Should Read

The UK government plans to rank hospitals on avoidable mortality based on case reviews of 2,000 deaths in English hospitals each year. They plan to use the method developed by Hogan et al.,[1] which designates a death as preventable when the reviewer concludes that the probability that the death could have been prevented exceeds 50% (P>0.5).

A recent article [2] criticises the use of a single threshold of preventability (e.g. P< or >=0.5) to determine whether or not a death was preventable. CLAHRC WM, in collaboration with Tim Hofer of University of Michigan, has advocated the use of a six-point Likert or a sliding scale to overcome the loss of information from dichotomising preventability.[3]

While preventable mortality rates provide real information on hospital performance, reviewer reliability is rather low (i.e. inter-observer variability is high).[3] This means that the signal can easily be obscured by noise.[4] Further modelling work may shed light on the extent to which this reduces the accuracy of league table approaches to identify outliers.Meanwhile, it is clear that while measuring preventable deaths overcomes some of the problems associated with measuring all deaths,[5] it nevertheless is no panacea. Measurement of preventability should probably be used as a learning tool, rather than as a performance metric.

— Richard Lilford, CLAHRC WM Director


  1. Hogan H, Healey F, Neale G, et al. Preventable deaths due to problems in care in English acute hospitals: a retrospective case record review study. BMJ Qual Saf. 2012; 21(9): 737-45.
  2. Abel G, Lyratzopoulos G. Ranking hospitals on avoidable death rates derived from retrospective case record review: methodological observations and limitations. BMJ Qual Saf. 2015; 24: 554-7.
  3. Manaseki-Holland S, Lilford RJ, Bishop J, et al. Reviewing deaths in British and US hospitals: a study of case-note reviews. 2015. [Submitted].
  4. Girling AJ, Hofer TP, Wu J, et al. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study. BMJ Qual Saf. 2012; 21(12): 1052-6.
  5. Lilford R & Pronovost P. Using hospital mortality rates to judge hospital performance: a bad idea that just won’t go away. BMJ. 2010; 340: c2016.

High Side-Effect Rates with Statins in Ordinary Practice but not in RCTs – the Enigma is Explained

Why do such a high proportion of patients who take statins complain of muscle pain in ordinary practice, while RCTs find little increase in this unintended effect? Could it be because the RCTs recruited a low-risk population? No – over 170,000 patients have been entered into high-quality RCTs, many had multi-morbidities and this was not a ‘sanitised’ population. Could it be because unintended effects were not assiduously recorded in RCTs? Again, no.

A recent paper in JAMA [1] examines re-introduction studies and find that the great majority of patients are able to tolerate treatment when it is restarted (admittedly, often in a different form). A particularly interesting study was carried out among people who had been ‘intolerant’ of statins. They were randomised to statin and non-statin treatments following a preparation phase where they were given placebo and could withdraw if intolerable symptoms recurred. Muscle pains were no more prevalent among those randomised to receive statins than among those in the non-statin group. It would appear that the very high incidence of muscle pain in standard practice is the result of psychological expectation, not pharmacological action. What do readers think?

— Richard Lilford, CLAHRC WM Director


  1. Newman CB, & Tobert JA. Statin Intolerance: Reconciling Clinical Trials and Clinical Experience. JAMA. 2015. 313(10); 1011-2.

Patient safety in hospitals: errors decline in UK and now in the US

The CLAHRC WM Director and colleagues demonstrated a reduction in error rates over the previous decade by means of in-depth case reviews in 19 UK hospitals. Infection rates, hand washing rates, vital sign observations, the quality of medical history taking, adherence to various tenets of evidence-based practice, and patient satisfaction all showed an improved trend.[1]

Now AHRQ has demonstrated improving safety in US Hospitals in the first half of the current decade, including pressure ulcers, bloodstream infections, and drug errors.[2] [3] The authors think the reasons are multi-faceted, but financial incentives may have played a large part. Increased spending on the NHS over the years of the Blair Government is an obvious candidate explanation for similar, but earlier, improvements in the UK. It would be interesting to see if the improvement in the safety of UK hospitals in the last decade has been sustained or augmented in the current decade.

— Richard Lilford, CLAHRC WM Director


  1. Cohn J. A Picture of Progress on Hospital Errors. Milbank Quarterly. 2015. 93(1); 36-9.
  2. Benning A, Dixon-Woods M, Nwulu U, et al. Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ. 2011; 342:d199.
  3. Agency for Healthcare Research and Quality. Interim update on 2013 annual hospital-acquired condition rate and estimates of cost savings and deaths averted from 2010 to 2013. Rockville, MD: Agency for Healthcare Research and Quality. 2014.

The Weekend Effect

It is well known that the mortality rate of patients admitted to hospitals over the weekend is higher than that for patients admitted during the week. Whether, or to what extent, this ‘weekend effect’ is caused by case-mix factors vs. care quality factors is one of the big unknowns. This is being investigated by a CLAHRC WM-associated HS&DR grant led by Prof Julian Bion with economic support from Sam Watson, the CLAHRC WM Director and Jo Lord. We were thus provoked by a recent article by Meacock at al [1] investigating the health economics of providing increased consultant support over the weekend. The health gain is calculated on the basis of avoiding all of the excess in deaths and this is offset against the cost of providing a seven-day service. Based on their calculation, the authors find that even if the weekend effect could be eliminated, it would not justify the cost of the service at the NICE willingness-to-pay threshold. In other words, the opportunity cost is such that it would be better to leave the money doing what it is currently doing (if no new money), or to allocate it elsewhere (if new money). However, preventable deaths are merely the top of the adverse event severity pyramid and if the adverse events come down roughly in proportion to deaths, then the gains are much greater and the cost much lower than estimated in the paper. CLAHRC WM collaborators have produced a model to estimate the costs and benefits of reducing adverse events.[2] [3] We hope to collaborate with the authors of the Meacock paper in developing this research.

–Richard Lilford, CLAHRC WM Director


  1. Meacock R, Doran T, Sutton M. What are the costs and benefits of providing comprehensive seven-day services for emergency hospital admissions? Health Economics. 2015. [ePub].
  2. Yao GL, Novielli N, Manaseki-Holland S, Chen Y-F, van der Klink M, Barach P, Chilton PJ, Lilford RJ. Evaluation of a predevelopment service delivery intervention: an application to improve clinical handovers. BMJ Qual Saf. 2012; 21(s1):i29-38.
  3. Lilford RJ, Girling AJ, Sheikh A, et al. Protocol for evaluation of the cost-effectiveness of ePrescribing systems and candidate prototype for other related health information technologies. BMC Health Serv Res. 2014; 14: 314.

Very Fresh Blood for Transmission: Probably Not Worth the Effort

This is the largest trial (over 1,200 patients per group) to compare extremely fresh (less than a week old on average) with standard (about three weeks old) blood transfusion in sick adult patients.[1] ‘Old’ blood has lower oxygen carrying capacity than fresh blood, and accumulates potentially harmful metabolites. However, there are costs associated with trying to give everyone fresh blood. In the end this study showed a null result with the point estimate favouring old blood. The death rate was high (over one third) so the trial ‘excluded’ the rather large adverse effect of a 16% increased death rate found in observational studies.

— Richard Lilford, CLAHRC WM Director


  1. Lacroix J, Hébert PC, Fergusson DA, et al. Age of Transfused Blood in Critically Ill Adults. New Engl J Med. 2015;372:1410-8.