Tag Archives: Mortality

Measuring Quality of Care

Measuring quality of care is not a straightforward business:

  1. Routinely collected outcome data tend to be misleading because of very poor ratios of signal to noise.[1]
  2. Clinical process (criterion based) measures require case note review and miss important errors of omission, such as diagnostic errors.
  3. Adverse events also require case note review and are prone to measurement error.[2]

Adverse event review is widely practiced, usually involving a two-stage process:

  1. A screening process (sometimes to look for warning features [triggers]).
  2. A definitive phase to drill down in more detail and refute or confirm (and classify) the event.

A recent HS&DR report [3] is important for two particular reasons:

  1. It shows that a one-stage process is as sensitive as the two-stage process. So triggers are not needed; just as many adverse events can be identified if notes are sampled at random.
  2. In contrast to (other) triggers, deaths really are associated with a high rate of adverse events (apart, of course, from the death itself). In fact not only are adverse events more common among patients who have died than among patients sampled at random (nearly 30% vs. 10%), but the preventability rates (probability that a detected adverse event was preventable) also appeared slightly higher (about 60% vs. 50%).

This paper has clear implications for policy and practice, because if we want a population ‘enriched’ for high adverse event rates (on the ‘canary in the mineshaft’ principle), then deaths provide that enrichment. The widely used trigger tool, however, serves no useful purpose – it does not identify a higher than average risk population, and it is more resource intensive. It should be consigned to history.

Lastly, England and Wales have mandated a process of death review, and the adverse event rate among such cases is clearly of interest. A word of caution is in order here. The reliability (inter-observer agreement) in this study was quite high (Kappa 0.5), but not high enough for comparisons across institutions to be valid. If cross-institutional comparisons are required, then:

  1. A set of reviewers must review case notes across hospitals.
  2. At least three reviewers should examine each case note.
  3. Adjustment must be made for reviewer effects, as well as prognostic factors.

The statistical basis for these requirements are laid out in detail elsewhere.[4] It is clear that reviewers should not review notes from their own hospitals, if any kind of comparison across institutions is required – the results will reflect the reviewers rather than the hospitals.

Richard Lilford, CLAHRC WM Director


  1. Girling AJ, Hofer TP, Wu J, et al. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling studyBMJ Qual Saf. 2012; 21(12): 1052-6.
  2. Lilford R, Mohammed M, Braunholtz D, Hofer T. The measurement of active errors: methodological issues. Qual Saf Health Care. 2003; 12(s2): ii8-12.
  3. Mayor S, Baines E, Vincent C, et al. Measuring harm and informing quality improvement in the Welsh NHS: the longitudinal Welsh national adverse events study. Health Serv Deliv Res. 2017; 5(9).
  4. Manaseki-Holland S, Lilford RJ, Bishop JR, Girling AJ, Chen YF, Chilton PJ, Hofer TP; UK Case Note Review Group. Reviewing deaths in British and US hospitals: a study of two scales for assessing preventability. BMJ Qual Saf. 2016. [ePub].

An Interesting Report of Quality of Care Enhancement Strategies Across England, Germany, Sweden, the Netherlands, and the USA

An interesting paper from the Berlin University of Technology compares the quality enhancement systems across the above countries with respect to measuring, reporting and rewarding quality.[1] This paper is an excellent resource for policy and health service researchers. The US has the most developed system of quality-related payments (P4P) of the five countries. England wisely uses only process measures to reward performance, while the US and Germany include patient outcomes. The latter are unfair because of signal to noise issues,[2] and the risk-adjustment fallacy.[3] [4] Above all, remember Lilford’s axiom – never base rewards or sanctions on a measurement over which service providers do not feel they have control.[5] It is true, as the paper argues, that rates of adherence to a single process seldom correlate with outcome. But this is a signal to noise problem. ‘Proving’ that processes are valid takes huge RCTs, even when the process is applied to 0% (control arm) vs. approaching 100% (intervention arm) of patients. So how could an improvement from say 40% to 60% in adherence to clinical process show up in routinely collected data?[6] I have to keep on saying it – collect outcome data, but in rewarding or penalising institutions on the basis of comparative performance – process, process, process.

— Richard Lilford, CLAHRC WM Director


  1. Pross C, Geissler A, Busse R. Measuring, Reporting, and Rewarding Quality of Care in 5 Nations: 5 Policy Levers to Enhance Hospital Quality Accountability. Milbank Quart. 2017; 95(1): 136-83.
  2. Girling AJ, Hofer TP, Wu J, et al. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study. BMJ Qual Saf. 2012; 21: 1052-6.
  3. Mohammed MA, Deeks JJ, Girling A, et al. Evidence of methodological bias in hospital standardised mortality ratios: retrospective database study of English hospitals. BMJ. 2009; 338: b780.
  4. Lilford R, & Pronovost P. Using hospital mortality rates to judge hospital performance: a bad idea that just won’t go away. BMJ. 2010; 340: c2016.
  5. Lilford RJ. Important evidence on pay for performance. NIHR CLAHRC West Midlands News Blog. 20 November 2015.
  6. Lilford RJ, Chilton PJ, Hemming K, Girling AJ, Taylor CA, Barach P. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ. 2010; 341: c4413.

Can Thinking Make It So?

When we think of risk factors for mortality we properly think behaviours (e.g. smoking / obesity) or genetics (e.g. family history). What about psychological factors – can unhappiness increase your risk of risk of cancer? Well, Batty and colleagues [1] have tackled this problem as follows:

  1. They assembled 16 prospective cohort studies where behaviours and psychological state had been measured and in which participants were followed up to see if cancer developed.
  2. They obtained the raw data and obtained an individual patient meta-analysis.
  3. They adjusted for the usual things known to increase risk of cancer (obesity, smoking, etc).
  4. They calculated relative risk of cancer according to antecedent psychological state.

They found a positive correlation between psychological distress and risk of cancer. But causality might have run the other way – (occult) cancers may have been the cause of psychological distress, not the other way round. So:

  1. They ‘left censored’ the data, thereby widening the gap between the point in time where the psychological state was measured and the point where cancer supervened.

The association between psychological state and cancer death persisted, even when they were separated by many years. What is the explanation?

  1. Failure to fully control for all behaviours (although behaviour could be the mechanism through which the cancer risk is increased in people with depression, in which case they ‘over-controlled’).
  2. Reduced natural killer cell function.
  3. Increased steroid levels, which can apparently affect DNA repair in some way.
  4. Some mechanism yet to be discovered.

In any event, the findings are intriguing, for all that practical implications may be limited.

— Richard Lilford, CLAHRC WM Director


  1. Batty GD, Russ TC, Stamatakis E, Kivimäki M. Psychological distress in relation to site specific cancer mortality: pooling of unpublished data from 16 prospective cohort studies. BMJ. 2017; 356: j108.

Thyroid Cancer: Another Indolent Tumour Prone to Massive Over Diagnosis

Park and colleagues, writing in the BMJ, document a massive (80 times) rise in the incidence of thyroid cancer in South Korea over the past two decades.[1] What is going on here? An epidemic of thyroid cancer in South Korea? No, the mortality from cancer of the thyroid has remained absolutely flat over the study period. The rise in the incidence of cancer is due entirely to screening uncovering cancers that would have otherwise remained occult. It turns out that the great majority of thyroid cancers are entirely innocent. As with prostate cancer and intraductal breast cancer, thyroid cancer tends to have a very long lead time, such that the patient is most likely to die with, rather than from, the disease.

— Richard Lilford, CLAHRC WM Director


  1. Park S, Oh C-M, Cho H, et al. Association between screening and the thyroid cancer “epidemic” in South Korea: evidence from a nationwide study. BMJ. 2016; 355: i5745.

Are Female Doctors Better Than Male Doctors?

There seems to be two models of feminism:

  1. Women and men are constitutionally identical (in everything but size and appearance), and any observed differences are entirely patterned by social influences.
  2. Differences between men and women are more than just anatomy and physiology, and women bring useful and unique attributes to society.

I am inclined to the second opinion. It is already known that female doctors are more likely to adhere to clinical guidelines, provide more preventative advice, and are better listeners than male doctors. Well they also seem to save more lives, according to a brilliant study from Yasuke Tsugawa and colleagues.[1] They compared outcomes from Medicare beneficiaries treated by general internists in hospital. These were really sick patients with a death rate of over 11%. The patients treated by female doctors had a risk-adjusted difference in mortality of nearly 0.5% and also lower risk of readmission. Findings were similar if the analysis was restricted to patients treated by ‘hospitalists’ – a general physician on call for emergencies. Patients hospitalised for an emergency medical condition are less likely to select their physician than patients who are admitted electively, and severity and condition profiles were well balanced between male and female physicians. The authors claim that this means their study was ‘quasi-randomised’. The results are congruent with other studies; across many industries it has been shown that men, compared to women, are “less deliberate in their approach to solving complex problems.”

In ‘My Fair Lady’ Henry Higgins sings “Why can’t a women by more like a man?” In the context of clinical care it seems that the song should go “Why can’t a man be more like a woman?

— Richard Lilford, CLAHRC WM Director


  1. Tsugawa Y, Jena AB, Figueroa JF, et al. Comparison of Hospital Mortality and Readmission Rates for Medicare Patients Treated by Male vs Female Physicians. JAMA Intern Med. 2016.

How Much Exercise Do You Need?

We know exercise reduces the incidence of cancer, diabetes and cardiovascular disease, but how much is needed? The WHO answer to this question is at least 600 on a standardised measure of total (work-related plus leisure) exercise called Metabolic Equivalent Tasks, or METs. This is the ratio of energy expenditure while performing an activity to expenditure at rest. Running for 75 minutes per week, atop of an otherwise sedentary life, yields the WHO standard of 600 METs. Better than nothing, but not enough according to a massive and sophisticated meta-analysis [1] – 2,000 to 4,000 METs are necessary to achieve material benefit (250 – 500 minutes of running). After this threshold, further gains with yet more exercise are nugatory. So lots of exercise is ideal, but excessive exercise is a fetish that wastes time. I aim to do two hours of ‘spinning’ and 90 minutes of doubles tennis each week. Let’s say spinning has an MET of 10, then I spend 1200 MET minutes spinning. If doubles tennis consumes 3 METs, then I spend 270 MET minutes. So my total METs is 1470 – not quite optimal. This study does not shed light on whether 2,000 METs spent in short bursts is better or worse than the same energy expenditure doing something really tedious, like lengths in a swimming pool. For that we need a study comparing “weekend warriors” with people who take similar amounts of exercise, but spread more evenly over the week.[2] The study was based on answers to questionnaires sent to participants in two huge cohort studies – the Health Survey for England and the Scottish Health Survey. The study replicates a link between exercise and overall mortality, cardiovascular mortality, and cancer mortality. However, the pattern of exercise does not seem to make much difference to the risk reduction.

— Richard Lilford, CLAHRC WM Director


  1. Kyu HH, Bachman VG, Alexander LT, et al. Physical activity and risk of breast cancer, colon cancer, diabetes, ischemic heart disease, and ischemic stroke events: systematic review and dose-response meta-analysis for the Global Burden of Disease Study 2013. BMJ. 2016; 354: i3857.
  2. O’Donovan G, Lee I-M, Hamer M, et al. Association of “Weekend Warrior” and Other Leisure Time Physical Activity Patterns With Risks for All-Cause, Cardiovascular Disease, and Cancer Mortality. JAMA Intern Med. 2017.

Yet More Evidence That Public Reporting of Outcomes is Without Benefit

The Centers for Medicare and Medicaid Services Hospital Compare Program added to its list of clinical process measures with mortality outcomes for three index conditions – pneumonia, heart attack, and heart failure. Across 3,970 hospitals (of sufficient size to participate) there was no improvement in mortality for the three index conditions when compared to non-index conditions. Mortality for both index and non-index conditions was declining gradually before they were added to the publically reported measures in 2008.[1] Far from an acceleration in the rate of decline post-2008, it actually slowed down. Our article on incentives suggested why public reporting is effete.[2]

— Richard Lilford, CLAHRC WM Director


  1. Joynt KE, Orav J, Zheng J, Jha AK. Public Reporting of Mortality Rates for Hospitalized Medicare Patients and Trends in Mortality for Reported Conditions. Ann Intern Med. 2016; 165: 153-60.
  2. Lilford RJ. Financial Incentives for Providers of Health Care: The Baggage Handler and the Intensive Care Physician. NIHR CLAHRC West Midlands News Blog. 25 July 2014.

Telemedicine in the Intensive Care Unit

I promised in the last blog to provide a brief summary of a review article on the above topic in the journal of Critical Care Medicine.[1] Telemedicine is “the practice of medicine when the doctor and patient are widely separated using two-way voice and visual communication”.[2] I was surprised to learn that 11% of hospital ICU beds in the US are served by a continuously monitoring telemedicine programme, and that this proportion continues to increase. The review found that implementation of telemedicine is associated with reduced ICU length of stay (just over half a day) and reduced ICU mortality (20% decrease in relative risk). However, the studies all use before and after designs and the article is not ‘reflective’ – it does not discuss the possibility of a ‘rising tide phenomenon’,[3] or regression to the mean (see previous blog). Malpractice claims following implementation of telemedicine dipped precipitously in a single quoted study, but this could be subject to publication bias. However, process measures observed sporadically among some studies appear to improve when telemonitoring is introduced, so there is a plausible basis for improved outcomes. On balance I think this was a rather uncritical review, but, pending better quality studies, my interim conclusion is that telemedicine is more likely to do good than harm.

— Richard Lilford, CLAHRC WM Director


  1. Lilly CM, Zubrow MT, Kempner KM, et al. Critical Care Telemedicine: Evolution and State of the Art. Crit Care Med. 2014; 42(11): 2429-36.
  2. Merriam-Webster, Inc. Telemedicine. Merriam-Webster.com.
  3. Chen YF, Hemming K, Stevens AJ, Lilford RJ. Secular trends and evaluation of complex interventions: the rising tide phenomenon. BMJ Qual Saf. 2015. [ePub].

More on Fats and Their Effect on Cholesterol, Heart Disease, and Death

The accumulating evidence on the lack of association between eating saturated fat and heart disease has featured in previous posts.[1] [2] An intriguing re-analysis of an RCT carried out in nursing homes and hospitals for mental illness has recently been published in the BMJ.[3] In this trial saturated fats were replaced in the diet by polyunsaturated fats. The now familiar story was confirmed; yes, the polyunsaturated fat is associated with lower cholesterol levels, but no, there was no hint of a decrease in heart attack or all-cause mortality in the low fat group. The authors then carried out a systematic review, finding five RCTs examining the same hypothesis. They provided strikingly similar results; the meta-analysis corroborated the nursing home study. One intriguing point made in an accompanying editorial [4] is that the climate was so heavily slanted towards the fat and cholesterol hypothesis that the trial, which ended in 1973, was not published until 1989. But opinion has eventually caught up with the evidence and US dietary guidelines have finally removed dietary cholesterol and fat from the list of foods that should be avoided.[5] But note this point – the fact that saturated fats are no worse than polyunsaturated fats does not mean that there are not yet better sources of calories. And, yes, plants are better than meat, butter, etc. They are boring to eat, of course, but they are probably the best source for most of our calories.

— Richard Lilford, CLAHRC WM Director


  1. Lilford R. More on Diet. NIHR CLAHRC WM News Blog. 14 August 2015.
  2. Lilford R. On Diet Again. NIHR CLAHRC WM News Blog. 23 October 2015.
  3. Ramsden CE, Zamora D, Majchrzak-Hong S, et al. Re-evaluation of the traditional diet-heart hypothesis: analysis of recovered data from Minnesota Coronary Experiment (1968-73). BMJ. 2016; 353: i246.
  4. Veerman JL. Dietary fats: a new look at old data challenges established wisdom. 2016; 352: i1512.
  5. US Department of Health and Humans Services and US Department of Agriculture. 2015-2020 Dietary Guidelines for Americans. 8th Washington, D.C.: USDA, 2015



Inequalities: Your Next Exciting Instalment

One month ago we cited the majestical study of health and wealth published in JAMA.[1] A fortnight ago we cited Angus Deaton’s insightful commentary on this study.[2] This week we draw your attention to a study of wealth and health inequalities, based on panel data (derived from national censuses) in eleven European countries covering two decades from 1990 to 2010.[3] The study was designed to look for associations between socio-economic class recorded in the censuses and deaths, overall and in major categories, such as cardiovascular disease and cancer. They also re-categorised deaths in classes that may indicate behaviours, such as smoking and alcohol. An overall reduction in age-specific mortality was observed over the study period. The study also showed that inequalities were growing wider when relative risks were compared, but absolute differences declined in nine of the eleven countries (including England and Wales). Absolute inequalities in smoking related deaths declined, but they increased for alcohol-related deaths.

— Richard Lilford, CLAHRC WM Director


  1. Chetty R, Stepner M, Abraham S, et al. The Association Between Income and Life Expectancy in the United States, 2001-2014. JAMA. 2016; 315(6):1750-66.
  2. Deaton A. On Death and Money. History, Facts, and Explanations. JAMA. 2016; 315(16): 1703-5.
  3. Mackenbach JP, Kulhánovâ I, Artnik B, et al. Changes in Mortality Inequalities over Two Decades: Register Based Study of European Countries. BMJ. 2016; 353: i1732.