Tag Archives: Director’s Choice – From the Journals

Another Interesting Trial of an Educational Intervention – This Time Concerning Access

Young people from disadvantaged backgrounds are less likely to apply to elite universities, both in the UK and the US, than those from economically better-off backgrounds. This finding applies even after controlling for exam results prior to application – i.e. the GCSE results in England. So Sanders and co-authors from the Behavioural Insights Team and the English Department for Education did an inexpensive trial of an inexpensive intervention.[1] The outcomes were application to, and acceptance into, an elite university (defined as belonging to the Russell Group). The intervention consisted of a letter sent to students from disadvantaged backgrounds who were on track to attend an elite university given their GCSE grades. Eligible schools were randomised to control conditions or one of three interventions: to receive a letter written by a pseudonymous male student (Ben) at Bristol University on Department for Education note paper; to receive a similar letter from a female student (Rachel) at the same university; or to receive letters from both Ben and Rachel. Three hundred schools (clusters) and 11,104 students participated. It was then a simple matter to collect the outcomes from the agency that supervises the admission process (the Universities and Colleges Admissions Service, UCAS). Receipt of a letter was associated with a non-significant increase in applications, and eventual admission to, an elite university. The increase was greatest and statistically significant for students who received both letters – from 8.5% acceptance among controls, to 11.4% in the ‘double dose’ intervention group – an increase of 2.9 percentage points (or 34 percent relative risk). Certainly, these results add to growing evidence concerning aspirations in education – see recent News Blogs on keeping children back a year [2], streaming [3], and the Michelle Obama effect.[4]

— Richard Lilford, CLAHRC WM Director

References:

  1. Sanders M, Chande R, Selley E. Encouraging People into University. London: Department for Education; 2017.
  2. Lilford RJ. Keeping a Child Back at School. NIHR CLAHRC West Midlands News Blog. 10 March 2017.
  3. Lilford RJ. Evidence-Based Education (or How Wrong the CLAHRC WM Director was). NIHR CLAHRC West Midlands News Blog. 15 July 2016.
  4. Lilford RJ. More on Education. NIHR CLAHRC West Midlands News Blog. 16 September 2016.

Length of Hospital Stay

The average length of hospital stay for patients has ‘plummeted’ over the last thirty years, from 10 days in 1983 to 5 days in 2013.[1] However, the proportion of patients discharged to a nursing facility has quadrupled over this same period.[2] So, from the point of view of the patient, the stay away from home has not changed as much as it might be inferred from an uncritical analysis of inpatient stays. So, how have home-to-home times changed? This was assessed by Barnett et al.[3] on the basis of Medicare administration claims for 82 million hospitalisations over the years 2004 to 2011 inclusive.

Yes, the mean length of hospital stay declined (from 6.3 to 5.7 days), but the mean length of stay in post-acute care facilities increased from 4.8 to 6 days. Total home-to-home time increased from 11.1 to 11.7 days. This is not necessarily a bad thing, but it must be taken into account in assessing costs and benefits of care. The risk of iatrogenic harm and costs are lower in nursing facilities than hospitals. However, the article cited here does not consider the possibility that these risks and costs are not lower for the group of people in nursing facilities who would otherwise be cared for in hospital.

— Richard Lilford, CLAHRC WM Director

References:

  1. Centers for Medicare and Medicaid Services. CMS program statistics: 2013 Medicare Utilization Section. 2017.
  2. Tian W (AHRQ). An All-Payer View of Hospital Discharge to Postacute Care, 2013. Rockville, MD: Agency for Healthcare Research and Quality; 2016.
  3. Barnett ML, Grabowski DC, Mehrotra A. Home-to-Home Time – Measuring What Matters to Patients and Payers. N Engl J Med. 2017; 377: 4-6.

Update on Scientists Being Held Accountable for Impact of Research

I recently wrote a news blog on the dangers of researchers being advocates for their own work. Readers may be interested in an article from an authoritative source that I chanced upon recently, published in BMC Medicine (Whitty JM. What Makes an Academic Paper Useful for Health Policy? BMC Med. 2015; 10: 301).

— Richard Lilford, CLAHRC WM Director

Reducing the Global Burden of Diagnostic Errors

A recent issue of the BMJ Quality and Safety carried an interesting review on the global burden of diagnostic errors in primary care.[1] The review looked at the most common symptoms and conditions affected by such errors; summarised the current interventions; and suggested what could be done next to reduce the global burden of diagnostic errors. The authors found that:

  • Typically there are multiple ‘root causes’ for errors, including both cognitive errors, such as failing to synthesise evidence, and system flaws, such as communication issues.
  • The most common categories of harmful diagnostic errors are infections, cardiovascular disease, cancer, and diseases in children.
  • Very few interventions to reduce errors have been tested empirically.
  • In order to reduce errors successfully there is likely to be a need for multiple interventions.

They go on to propose eight themes for interventions to measure and reduce diagnostic error:

  1. Improving diagnostic reasoning.
  2. Encouraging government policies that support primary care.
  3. Improving information technology.
  4. Involving patients.
  5. Improving access to diagnostic tests.
  6. Developing methods to identify and learn from diagnostic errors.
  7. Optimising diagnostic strategies in primary care.
  8. Providing systematic feedback to clinicians about their diagnoses.

The authors call on the World Health Organization to bring together concerned bodies to address the many challenges that are common across all countries and the opportunities that can be taken to reduce diagnostic error. CLAHRC WM collaborators are working on a more detailed classification system for the theoretical basis for diagnostic error.

— Peter Chilton, Research Fellow

Reference:

  1. Singh H, Schiff GD, Graber ML, Onakpoya I, Thompson MJ. The global burden of diagnostic errors in primary care. BMJ Qual Saf. 2017; 26: 484-94.

Mindfulness for Low Back Pain

Lower back pain is fast becoming a major public health problem. Perhaps because of our increasingly sedentary life style it affects around 75% of the population at some point during their lives. However, there is no optimum clinical treatment. In light of this, many people turn to complementary therapies. A recent systematic review by Anheyer and colleagues [1] looked at the effectiveness of such a therapy, mindfulness-based interventions. Mindfulness-based stress reduction programmes (MBSR), and mindfulness-based cognitive therapy (MBCT) (see main article) usually involve activities such as meditation, yoga, and focusing attention on different parts of the body. The authors identified seven RCTs involving 864 patients, and found that MBSR was associated with statistically significant short-term improvements in pain, compared to standard care, though these weren’t sustained in the long term, and could not be deemed clinically meaningful. However, there were no significant differences when compared to active comparators, such as health education programmes. These were only a limited number of RCTs and there is still a need for long-term RCTs that compare MBSR against active treatments.

— Peter Chilton, Research Fellow

Reference:

  1. Anheyer D, Haller H, Barth J, et al. Mindfulness-based stress reduction for treating low back pain. A systematic review and meta-analysis. Ann Intern Med. 2017; 166: 799-807.

A Drug Treatment for Autism

Autism affects 1-2% of children. These children may have problems with social interaction, adhere to strict routines, have repetitive behaviours, restricted interests, poor self-care, and/or heightened sensory experiences. A very wide array of genetic mutations and environmental exposures interact to produce the phenotype. It is a neurological disease and one theory, the “cell danger hypothesis”, holds that certain neurological pathways are prone to become over-activated and respond as though they were under ‘threat’. Purines released from mitochondria leech through the cell membrane where they play a role in activating microglia and affecting synaptic remodelling – a topic covered in other News Blogs.[1][2] A drug called suramin inhibits the action of purines such as ATP. It is used in high doses to control trypanosomiasis (sleeping sickness). It is toxic at high dose, but might it be effective at a lower dose for autism? A very small, double-blind trial has been carried out in which five matched pairs of autistic children were randomised to a single intravenous dose of suramin or saline.[3] Metabolic pathways were affected as expected, and the treatment was associated with improvement on a standard score two days after the infusion. It is early days, but it is just possible that we are entering a period where autism will be added to the growing list of neuro/psychiatric disorders that can be mitigated by pharmacological therapy based on an improved understanding of molecular pathogenesis.

— Richard Lilford, CLAHRC WM Director

References:

  1. Lilford RJ. Okay Then, There is a Fourth Period of Whole-Scale Synaptic Pruning in the Grey Matter of the Brain. NIHR CLAHRC West Midlands News Blog. 13 January 2017.
  2. Lilford RJ. A Fascinating Account of the Opening up of an Area of Scientific Enquiry. NIHR CLAHRC West Midlands News Blog. 11 November 2016.
  3. Naviaux RK, Curtis B, Li K, et al. Low-dose suramin in autism spectrum disorder: a small, phase I/II, randomized clinical trial. Ann Clin Transl Neurol. 2017.

Lancet Leader on a Complex Systems Model of Evidence – an Opportunity Missed

A recent paper argued for use of models in the evaluation of complex interventions where:

  • correlations are not linear,
  • components interact,
  • feedback loops are incorporated, and
  • they adapt over time.[1]

But they leave it there – they do not say how to model the components, still less how parameters can be derived from such models for use in decision models, such as health economic models. CLAHRC WM has developed and published on the use of such models in policy and service delivery research. We show how causal chains can be mapped and how probabilities can be propagated across such causal chains.[2-4] Along with Alec Morton (University of Strathclyde) and Gavin Stewart (Newcastle University), we are leading a workshop on Bayesian causal models at the forthcoming Society of Social Medicine meeting, and will give examples of this work in forthcoming issues of the News Blog.

— Richard Lilford, CLAHRC WM Director

References:

  1. Rutter H, Savona N, Glonti K, et al. The Need for a Complex Systems Model of Evidence for Public Health. Lancet. 2017.
  2. Watson SI & Lilford RJ. Essay 1: Integrating multiple sources of evidence: a Bayesian perspective. In: Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Southampton (UK): NIHR Journals Library, 2016.
  3. Lilford RJ, Girling AJ, Sheikh, et al. Protocol for evaluation of the cost-effectiveness of ePrescribing systems and candidate prototype for other related health information technologiesBMC Health Serv Res. 2014; 14: 314.
  4. Watson SI, Taylor CA, Chen Y-F, Lilford RJ. A Framework for the Evaluation of Service Delivery Interventions. J Health Econ. [Submitted].

A Cluster RCT of an Internet-Based Programme to Promote Activity and Reduce Postpartum Calorie Intake in Poor Hispanic Women

When I read the introduction and methods section of a research paper I often try to guess the result before I read on. In the case of the paper above [1] I guessed a null result. I guessed wrong. In this cluster RCT (12 clusters, 371 patients), a carefully designed and piloted internet-based intervention to nudge women to healthy living reduced mean mass by a statistically significant 2.3kg compared to standard care. There was no effect on exercise as assessed by a pedometer. The authors express surprise that there was ‘no’ reduction in calorie intake, but they over-interpret this finding. The variance in measured calories was very wide and the p-value was 0.06. They make the mistake of reifying the 95% limits on the confidence interval.

The 2.3kg mean intervention effect may strike some as nugatory. However, a relatively small reduction in mass can have a worthwhile metabolic and health effect, as we showed in a study of liver function tests.[2] Postpartum weight loss is important because it is associated with long-term obesity, is largely truncal, and increases risk in any subsequent pregnancy. Dr Ponnusamy Saravanan from CLAHRC WM is collaborating with Prof Kamlesh Khunti (Director of CLAHRC East Midlands) in tackling the problem.

— Richard Lilford, CLAHRC WM Director

References:

  1. Phelan S, Hagobian T, Brannen A, et al. Effect of an Internet-Based Program on Weight Loss for Low-Income Postpartum Women: A Randomized Clinical Trial. JAMA. 2017; 312(23): 2381-91.
  2. Lilford RJ, Bentham L, Girling A, et al. Birmingham and Lambeth Liver Evaluation Testing Strategies (BALLETS): a prospective cohort study. Health Technol Assess. 2013; 17(28): 1-307.

Numbers and the Doctor/Patient Relationship

I have always been interested in communicating scientific information and probability. A paper co-authored by CLAHRC WM colleague Eivor Oborn [1] therefore caught my eye. The paper concerns numbers and their ‘performativity’, by which the authors mean how the numbers affect doctors, patients, and the interaction between doctors and patients. They use medical consultations in a Swedish rheumatology clinic to explore the issue, since this is a ‘data-rich’ environment. By this I mean charts are used to plot long-run numerical data relating to patient-reported outcomes, medical assessments, and laboratory data. The study shows that the numbers have high salience for patients who generally find graphical representation of long-run data useful. Doctors also find graphical display of trends useful in spotting threats to patient health. However, patients sometimes feel that the data on display take precedence over how they actually feel. That is to say, the doctor tends to focus on the numbers while the patient’s main symptom might not be captured in the numbers. Of course, there is no counterfactual, so how much of this dissatisfaction is caused by availability of numbers is uncertain. Also I felt that more could be said about the extent to which patients, and indeed doctors, really understand the meaning of the numbers they were seeing. Many people have poor numeracy skills and draw erroneous inferences from data. For instance, people tend to over-interpret improving trends following a run of high-values – the issue of regression to the mean, covered in the Method Matters section of a previous News Blog.

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Essén A & Oborn E. The performativity of numbers in illness management: The case of Swedish Rheumatology. Soc Sci Med. 2017; 184: 134-43.

The Problem with ‘Red, Amber, Green’

I have always thought that the so-called traffic light system, which classifies service quality as ‘red, amber, green’ is so crude as to be otiose. A recent article in BMJ Quality and Safety explicates the logic behind my intuition.[1] There are two problems with the so-called traffic light system. First, it focuses just on one episode in time and says nothing about trends. Second, it (usually) relies on thresholds set externally. For both of these reasons, the authors argue (and I agree) that the traffic light system is inimical to safety. It does not distinguish between common cause variation (play of chance) and special cause variation (likely to be due to some specific, potentially remediable factor). The point is made that tackling common cause variation, even if it comes up red against some externally set threshold, is likely to lead nowhere. If one wants to improve outcomes from systems that are fluctuating randomly, then it is necessary to look for a common cause, not a cause specific to a particular time and space. Control charts analyse trends and hence distinguish between common cause variation and special course variation; the latter requires a focussed approach. For instance most English Accident and Emergency departments would show up red if judged against the four hour waiting time target. A red rating does not therefore suggest a problem specific to a particular hospital, but failure across the hospital system. It therefore needs a systemic approach across hospitals. This would be self-evident if a funnel plot were used. Such a chart would distinguish outliers where a targeted diagnosis and intervention would be appropriate from the generality of hospitals where a more systematic approach is more likely to bear fruit. CLAHRC WM is trying to enhance uptake of control charts by hospitals based on our previous work that shows they are seldom used.[2]

— Richard Lilford, CLAHRC WM Director

References:

  1. Anhøj J, Hellesøe A-MB. The problem with red, amber, green: the need to avoid distraction by random variation in organisational performance measures. BMJ Qual Saf. 2017; 26: 81-4.
  2. Schmidtke KA, Poots AJ, Carpio J, Vlaev I, Kandala N-B, Lilford RJ. Considering chance in quality and safety performance measures: an analysis of performance reports by boards in English NHS trusts. BMJ Qual Saf. 2017; 26: 61-9.