A Trio of BMJ Systematic Reviews

In Director’s Choice this week I single out not one, not two, but three papers that were all published in a recent issue of the BMJ (17 May 2014). I find this a most timely and interesting journal – second only to this News Blog! All three of the selected papers are systematic reviews, now the most widely cited form of clinical research.[1]

The first concerns prevention of asthma exacerbation by means of long-term inhaled medication.[2] First, this is a very interesting and important clinical topic, especially given the recent survey of national asthma deaths.[3] Second, because it uses network meta-analyses where treatments are compared indirectly through a common comparator. We are going to have to get used to this type of analysis because of “question inflation,” as the Director described in a previous blog.[4] In any event, this analysis produces a strong result showing that adding long-acting ß-agonists to low-dose corticosteroids was a considerably more effective therapy than standard therapy of low-dose corticosteroids alone.

The second paper concerns effectiveness of non-surgical interventions to sustain initial weight-loss.[5] This is an important topic and the effect size of all interventions were small, amounting to a mean difference of less than 2kg. There is however, evidence that even small reductions in mass are beneficial.[6] [7] [8] Behavioural interventions that cover both diet and exercise produced a significant result, as did the medicine orlistat, which works to reduce fat absorption from the gut. Of course the latter comes at the cost of side-effects. Nutritional supplements and food replacements produced null results.

The last paper concerns an intervention that many people might consider warranted, irrespective of any evidence – screening women for intimate partner violence in healthcare settings.[9] There are 11 randomised or quasi-randomised trials on this topic. Importantly, screening programmes that involved structured interventions as part of the programme (i.e. a compound intervention of both screening and therapy) were excluded. This left the above 11 trials, which included over 13,000 participants. Hardly surprisingly the intervention did result in improved identification of violence. As a result referrals to domestic violence support services also increased, but the confidence limits were wide and not quite significant at the 0.05 level (a one-tailed test might have been more appropriate here?). Only two studies reported on reduction in intimate partner violence, both finding a null result. One study reported on psychological and physical health and again yielded a null result. I enjoyed the review but did not agree with the conclusion that there was “no evidence that universal screening is warranted.” I do not believe that worthwhile effects have been excluded. Yet again, no evidence of effect is conflated with evidence of no effect. This fallacy can be avoided by more careful interpretation of frequentist statistics, but a Bayesian approach would be better still. This is because Bayesian approaches explicitly incorporate indirect evidence.[10] Thus, although there is no direct comparitive evidence that screening for partner violence is warranted, there might be plenty of indirect evidence that can inform decision making, pending larger head-to-head trials. The evidence-based practice movement consistently gets ahead of itself by conflating “direct comparative evidence” with “evidence” as I shall describe in a strongly-worded argument in the next News Blog.

–Richard Lilford, Director CLAHRC WM


  1. Patsopoulos NA, Analatos AA, Ioannidis JP. Relative citation impact of various study designs in the health sciences. JAMA. 2005; 293(19):2362-2366.
  2. Loymans RJB, Gemperli A, Cohen J, Rubinstein SM, Sterk PJ, Reddel HK, Jüni P, ter Reit G. Comparative effectiveness of long term drug treatment strategies to prevent asthma exacerbations: network meta-analysis.
    BMJ. 2014; 348:g3009.
  3. Royal College of Physicians. Why asthma still kills. The National Review of Asthma Deaths (NRAD). London: Healthcare Quality Improvement Partnership. 2014.
  4. Lilford RJ. The end of the hegemony of randomised trials. 2012 Nov 30. [Online].
  5. Dombrowski SU, Knittle K, Avenell A, Araújo-Soares V, Sniehotta FF. Long term maintenance of weight loss with non-surgical interventions in obese adults: systematic review and meta-analyses of randomised controlled trials. BMJ. 2014; 348:g2646.
  6. Goldstein DJ. Beneficial health effects of modest weight loss. Int J Obes Relat Metab Disord. 1992; 16(6): 397-415.
  7. Lilford R, et al. Birmingham and Lambeth Liver Evaluation Testing Strategies (BALLETS): a prospective cohort study. Health Technol Assess. 2013; 17(28).
  8. Vidal J. Updated review on the benefits of weight loss. Int J Obes. 2002; 26(s2): s25-8.
  9. O’Doherty LJ, Taft A, Hegarty K, Ramsay J, Davidson LL, Feder G. Screening women for intimate partner violence in healthcare settings: abridged Cochrane systematic review and meta-analysis. BMJ. 2014;348:g2913
  10. Lilford RJ, Thornton JG, Braunholtz D. Clinical trials and rare diseases: a way out of a conundrum. BMJ. 1995; 311(7020): 1621-5.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s