Statistics is Far Too Important to Leave to Statisticians

“P values, the ‘gold standard’ of statistical validity, are not as reliable as many scientists assume.”
R. Nuzzo. Scientific method: Statistical Errors. Nature. 2014; 506:150-2.

Phase 1: Theoretical Practice

When I was at medical school, the prevailing idea was that all that was needed for the sound practice of medicine was a deep understanding of physiology and pathology. Our teachers had reason to put faith in this idea. They emanated from what is sometimes called the golden age of discovery. Improved understanding of physiology, alongside technical developments, had placed in their hands powerful treatments, such as oral contraception, the breathing machine, kidney dialysis, cardio-pulmonary bypass, and cancer chemotherapy. Patients could be rescued from the clutches of death in the intensive care unit by following the sound principles of living physiology.
However, these heady discoveries soon gave way to a more deliberative process of trial and error to improve the use of generic treatment types.

Phase 2: Evidence-Based Practice

The effects of these second order interventions were not self-evident. Again and again, randomised trials showed that our intuitions, no matter how well-based in physiology and pathology, were often completely and utterly wrong. In short, we did not know enough about physiology and pathology to be able to predict which treatments would do more good than harm. At first the medical profession was non-plussed by this type of direct evidence, but good arguments gradually displace bad. Evangelists such as Archie Cochrane were followed by early adopters, such as David Sackett, Iain Chalmers and Thomas C. Chalmers, and the Evidence-Based Medicine movement was born. Pick up any of the major six journals now and you will most likely be treated to a pageant of randomised trials and systematic reviews of RCTs. RCTs continue to produce iconic results – for example the magnificent CRASH-2 trial [1] and the endovascular aneurysm trials.[2] [3] [4]

However, more and more RCTs are inconclusive, even when they have been of considerable size and very well-funded. Mainly, this is because the head-room for improvement is gradually being eroded by the success of modern medicine; if you halve the absolute effect size you are looking for, then you quadruple the necessary sample size, other things remaining unchanged. Also, as pointed out in a previous blog,[5] we face ‘question inflation’; every question we answer in science spawns a string of subsidiary questions and the science base produces an ever-increasing number of therapeutic targets.[6] This is unmasking an epistemological problem at the heart of the current evidence-based healthcare movement. The results of standard statistical tests have been treated as a decision rule. As argued before, frequentist statistics does not provide a decision rule that can be used in the interpretation of a particular result.[7] [8] This was emphasised by the founding fathers of frequentist statistics Neyman and Fisher. Frequentist statistics does not yield probabilistic estimates that are required for an axiomatic theory of decision-making.[9] [10] Yet, practitioners of evidence-based medicine reify the p-value and confidence limits, and use them as the basis for decisions. Confidence limits take account of the play of chance, while various procedural roles, such as randomisation, take care of bias, according to this view of the world. Statisticians, the only people who really understand the problem, keep silent – Bayesian statistics that provides the probabilities of treatment effects, was something “you do not do in front of the children”. This led Steven Goodman (himself a statistician) to make his insightful remark – “…statistics has become too important to leave only to statisticians”.[10]

Phase 3: Integrating Multiple Sorts of Evidence

Of course none of this mattered when RCTs produced iconic results that swept all before them. Under those circumstances misinterpreting a frequentist confidence limit as a Bayesian credible limit does no harm at all – they are virtually the same thing. It is only now as we enter the fuzzier world of small effect sizes and multiple objectives and the need to combine data of different sorts, that the intellectual flaws in using standard statistical method as a decision rule are assuming practical importance.

The only way out of the mess [11] is to think completely differently. We should start with a formal analysis of the decision problem (using expected utility theory or its elaboration into cost-utility analysis). We should then collect the relevant data and analyse it in a Bayesian paradigm, so that it can provide the kind of probabilities we need (probabilities of events if we follow one decision or the other) and so that the different probabilities and utilities can be reconciled.

When I was a medical student, evidence-based medicine was not given nearly enough prominence. Subsequently, a very simplified version, based on frequentist statistics, was presented as a one-stop shop to clinical decisions. But the best solution is one which synthesises prior knowledge, and all direct comparative evidence, to yield probabilities that are directly referable to the decision problem. Knowledge about the theory of treatments and of how other similar treatments have fared is also part of the evidence for evidence-based medicine, as Bradford Hill pointed out in his famous lecture.[12]

Personal Reflection

I have been proselytising for Bayesian statistics for 25 years. At one point in my career I spoke to a statistician who, like me, had attracted a certain amount of ridicule. His response had been to back off. Recently, a distinguished statistician who I like and admire, told me I should stop banging on about Bayes – the argument, he said, was widely accepted intellectually. Be that as it may, the world plows on regardless, with doctors, nurses, psychologists and many others misunderstanding conventional statistics, and statisticians, with a few exceptions, remaining silent. The CLAHRC WM Director has absolutely no intention to stop banging on about Bayes!

— Richard Lilford, CLAHRC WM Director.

  1. CRASH-2 Trial Collaborators.Effects of tranexamic acid on death, vascular occlusive events, and blood transfusion in trauma patients with significant haemorrhage (CRASH-2): a randomised, placebo-controlled trial. Lancet. 2010; 376(9734): 23-32.
  2. Prinssen M, et al. A randomized trial comparing conventional and endovascular repair of abdominal aortic aneurysms. NEJM. 2004; 351: 1607-18.
  3. EVAR Trial Participants. Endovascular aneurysm repair versus open repair in patients with abdominal aortic aneurysm (EVAR trial 1): randomised controlled trial. Lancet 2005; 365(9478): 2179-86.
  4. EVAR Trial Participants. Endovascular aneurysm repair and outcome in patients unfit for open repair of abdominal aortic aneurysm (EVAR trial 2): randomised controlled trial. Lancet 2005; 365(9478): 2187-92.
  5. Lilford RJ. The End of the Hegemony of Randomised Trials. 30 Nov 2012. [Online].
  6. Lord JM, et al. The systemic immune response to trauma: an overview of pathophysiology and treatment. Lancet. 2014. 384: 1455-65.
  7. Lilford RJ, Thornton JG, Braunholtz D. Clinical trials and rare diseases: a way out of a conundrum. BMJ. 1995; 311: 1621.
  8. Lilford RJ, Braunholtz D. Who’s afraid of Thomas Bayes? J Epidemiol Community Health. 2000; 54: 731-9.
  9. Lindley DV. The philosophy of statistics. Statistician. 2000; 49(3): 293-337.
  10. Goodman SN. Toward evidence-based medical statistics. 1: The P value fallacy. Ann Intern Med. 1999; 130(12): 995-1004.
  11. Lilford RJ. The Messy End of Science. 16 April 2014. [Online].
  12. Hill AB: The environment and disease: Association or causation? Proc R Soc Med. 1965;  58(5): 295-300.

A Mysterious Disease with Unknown Cause

At the recent CAHRD (Centre for Applied Health Research and Delivery) Consultation in Liverpool, the CLAHRC WM Director was reminded of a curious tropical disease, which he would like to share with readers of our blog.

Children being tied to a tree, or chained in their home while their parents work in the nearby fields. Neglect? Or a sad necessity in some parts of Africa where a little known disease has been devastating communities? A growing number of children in late childhood/early adolescence are being afflicted by nodding disease – suffering from, among other things, seizures and cognitive difficulties. Many parents fear that without taking drastic measures they will return to find their child has wandered into the bush, unlikely to ever return, or has accidentally injured herself. Crucially, the medical community have not been able to pinpoint a definite cause, let alone a cure.

Nodding disease has only been diagnosed in three, non-contiguous rural areas of Africa – western and central South Sudan, southern Tanzania and northern Uganda (Figure 1).[1] Estimates suggest 3,000–8,000 cases have already been reported [1] – figures higher than recorded Ebola cases.[2] Although nodding disease has only recently come to the attention of the wider medical community (the Center for Disease Control (CDC) began to study the disease in 2009), it first emerged in the 1960s.[1] [3]

Map of South Sudan, Uganda and Tanzania
Figure 1. Map of South Sudan, Uganda and Tanzania

Its name comes from the characteristic symptom of the disease – an uncontrollable nodding of the head that is a direct result of seizures causing a brief lapse in muscle tone. These seizures are of various severities, and often begin when the child eats or the ambient temperature drops, generally not stopping until the child has finished eating or has warmed up. In latter stages of the disease this can lead to severe malnutrition as children are no longer able to eat. (Curiously it has been reported that no seizures occur when an affected child is given an unfamiliar food, such as chocolate.)[4] Over time the symptoms worsen – from a few times a week, the nodding episodes begin to happen daily, then every few hours, and are followed by increasingly worse bouts of confusion, lethargy and convulsions. Eventually there is permanent and completely stunted growth of the body and the brain, leading to mental retardation.[1] MRI scans of affected children have shown significant brain atrophy and damage to the hippocampus and glial cells.[3]

There is currently no cure, with treatment confined to managing the symptoms. Anticonvulsants, such as sodium valproate,[5] have been administered in an attempt to control the nodding, and although there has been some indication that this has helped, with children regaining some degree of normality – once again being able to talk or walk – the underlying damage remains and the condition continues to worsen.[6]

A number of causes have been looked into, and subsequently ruled out – exposure to wartime chemicals, consumption of meat from monkeys, ingestion of toxic substances (e.g. coated seeds distributed by relief agencies that were meant for planting), or contaminated relief foods.[1] [7]

A potential cause that is still being considered is a possible autoimmune response linked to infection with microfilaria parasites – the age range of sufferers (5-15 years old), is similar to certain parasitic infections and epilepsy syndromes.[1] Originally researchers thought it could be linked to Onchocerca volvulus, a parasitic worm that is carried by black fly. Victims have generally been concentrated near fast-flowing rivers, such as the Yei in South Sudan, which are home to black fly.[7] Further, infection with O. volvulus is prevalent in areas of outbreak, with 93% of sufferers in some areas estimated to harbour the parasite, a figure significantly higher than those without the disease.[8] O. volvulus also causes river blindness (onchocerciasis), which has been tentatively linked to other forms of epilepsy,[9] as well as stunted growth.[10] However, O. volvulus is very common in many areas which have not seen the disease,[1] and a study in Tanzania found no significantly elevated levels of antibodies in the cerebrospinal fluid of patients.[3] Perhaps the cause lies in a closely-related or variant strain of the parasite, or there is an unidentified co-factor that is needed in combination with O. volvulus infection. A second potential cause that is being looked into is linked to malnutrition (many of the affected populations have recently been displaced, or are amongst the poorest in the region)[1] and/or vitamin B6 (pyridoxine) deficiency.[1] A trial administering vitamin B6 to patients has been set up to study this possibility.[11]

Although the local governments are trying to help – the Uganda Ministry of Health allocated $1.4 million of their budget to mount a response, and deployed teams of health workers to affected areas to distribute anticonvulsants [6] – outside donors and non-governmental organisations are needed if there is to be a solution any time soon.

— Peter Chilton, CLAHRC WM Research Associate


  1. Dowell SF, Sejvar JJ, Riek L, et al. Nodding Syndrome. Emerg Infect Dis. 2013; 19(9): 1374-84.
  2. Centers for Disease Control and Prevention. Chronology of Ebola Hemorrhagic Fever Outbreaks. 2014. [Online].
  3. Winkler AS, Friedrich K, König R, et al. The head nodding syndrome – Clinical classification and possible causes. Epilepsia. 2008; 49 (12): 2008-15.
  4. World Health Organization. World Health Organization joins other partners to support Nodding Disease investigations in Southern Sudan. 2011. [Online].
  5. Idro R, Musubire KA, Mutamba BB, et al. Proposed guidelines for the management of nodding syndrome. Afr Health Sci. 2013; 13(2): 219-32.
  6. Kielty M. Nodding Syndrome: A Devastating Medical Mystery in Uganda. National Public Radio. 2013. [Online].
  7. Vogel G. Mystery Disease Haunts Region. Science. 2012; 336(6078): 144-6.
  8. Harding A. ‘Nodding disease’ hits Sudan. BBC News. 2003. [Online].
  9. Kaiser C, Pion SDS, Boussinesq M. Case-control Studies on the Relationship between Onchocerciasis and Epilepsy: Systematic Review and Meta-analysis. PLoS Negl Trop Dis. 2013; 7(3): e2147.
  10. Ovuga E, Kipp W, Mungherera M, Kasoro S. Epilepsy and retarded growth in a hyperendemic focus of onchocerciasis in rural western Uganda. East Afr Med J. 1992; 69(10): 554-6.
  11. Dowell S, Sejvar J, Bunga S, Idro R. Treatment of Nodding Syndrome – A Randomized Blinded Placebo-Controlled Crossover Trial of Oral Pyridoxine and Conventional Anti-Epileptic Therapy, in Northern Uganda — 2012. Clinical Trials Registry. 2012. [Online].

The Debate on Confidentiality of Interim Trial Results

A recent paper has looked at the ethics of keeping interim RCT results confidential.[1] Many hold the view that releasing interim results compromises the integrity of the trial and is against the interests of the patient, but there is little evidence for this. The reasons for confidentially seem to stem from only a few sources of evidence – a study that may have been stopped early due to a significant difference in deaths between arms, which had become non-significant by the end of the trial; and a comparison of trials that found that those that released interim data generally had a decline in accrual, though there were many other differences.

Stephens et al. argue that, under specific circumstances, it is possible to release interim results without compromising integrity or credibility, and that this can enable difficult trials to continue and be successfully completed. They provide two examples of this – the QUARTZ trial, which looked at radiotherapy for patients with inoperable brain metastases; and the GRIT trial, which compared two obstetric strategies for delivering foetuses. They provide a number of factors that may be considered when assessing whether a trial should release interim results. It appears that there is a need for further debate regarding the current views on releasing interim data.

— Richard Lilford, Director CLAHRC WM


  1. Stephens RJ, Langley RE, Mulvenna P, Nankivell M, Vail A, Parmar MKB. Interim results in clinical trials: Do we need to keep all interim randomised clinical trial results confidential? Lung Cancer. 2014. [In Press].

A Culture of Quality: Join the Debate

There are two theories about hospital level culture:

  1. The general view that is probably overstated. This says that a person can walk into a hospital and within ten minutes they will accurately discern whether or not it is a safe place to be treated. Yet evidence for this latent variable called culture is weak at the hospital level. Certainly I know of no evidence that a person can just walk in and ‘smell’ the culture. My peer CLAHRC Director, Prof Derek Bell of Northwest London, has recently reviewed the literature on the association between patient experience on the one hand and certain outcomes and clinical processes on the other.[1] These two sets of variables were statistically correlated in many cases, but the paper does not say how large the associations were. (These would have been hard to summarise from a diverse literature.) However, Jha and colleagues, in a massive survey of US hospitals,[2] found that the difference in adherence to quality standards between the highest and lowest quartiles for satisfaction varied by less than three percentage points – a result that was statistically, but not practically, significant. Likewise, an extensive review of the relationship between clinical processes (technical safety and quality) and standardised mortality rates showed a correlation, but again it was very weak, such that one could not be a useful diagnostic test for the other. Jaipaul and Rosenthal [3] studied the link between various measures of patient satisfaction and the tractable domain of standardised hospital mortality. While they too show associations, the correlations are weak (all are lower than 0.4). Perhaps most telling of all, there is there is little or no correlation between adherence to the quality of clinical care standards themselves.[4] [5] It is of more than passing interest that the now notorious Stafford Hospital did not show up as an outlier on the patient survey. So pervasive is the belief in the cultural explanation that it is more difficult to get an article accepted if it is null than if it is positive, as I have found at an anecdotal level.
  2. The Director’s view. This holds that the idea of a zeitgeist that can be manipulated to simultaneously improve performance across a hospital is a seductive, but over-used concept. The Good Samaritan experiment [6] is a good example of the ‘situated’ nature of human behaviour, meaning that local environment has a persuasive impact on what we do.[7] Albert Bandura is a ground-breaking researcher on how behaviour is strongly influenced by what other people are doing in the immediate vicinity, rather than a set of fixed values driving behaviour across a social group.[8] A recent systematic review of intervention studies did “not identify any effective strategies to change organisational culture”,[9] while another association study concluded “current policy prescriptions which seek service improvement through cultural transformation are in need of a more secure evidential base”.[10] The poor or non-existent correlations cited above support the conclusion that if you want to improve hospital care, make the right thing to do the easy thing to do and do not obsess over this notion of a magic, cultural bullet that will make everything all right.

Please post your views on evidence on the above important issue.

–Richard Lilford, CLAHRC WM Director


  1. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013; 3: e001570
  2. Jha AK, Orav EJ, Zheng J, Epstein AM. Patients’ Perception of Hospital Care in the United States. NEJM. 2008; 359: 1921-31.
  3. Jaipaul CK, Rosenthal GE. Do hospitals with lower mortality have higher patient satisfaction? A regional analysis of patients with medical diagnoses. Am J Med Qual. 2003; 18(2): 59-65.
  4. Jha AK, Li Z, Orav EJ, Epstein AM. Care in US hospitals – the Hospital Quality Alliance program. NEJM. 2005; 353(3): 265-74.
  5. Wilson B, Thornton JG, Hewison J, Lilford RJ, Watt I, Braunholtz D, Robinson M. The Leeds University Maternity Audit Project. Int J Qual Health Care. 2002; 14(3): 175-81.
  6. Darley JM, Batson D. “From Jerusalem to Jericho”: A Study of Situational and Dispositional Variables in Helping Behavior. J Pers Soc Psychol. 1973; 27(1): 100-8.
  7. Bandura A, Ross D, Ross SA. Transmission of aggression through imitation of aggressive models. J Abnorm Soc Psychol. 1961; 63: 575-82.
  8. Bandura A. Social Learning Theory. Englewood Cliffs, NJ: Prentice Hall, 1977.
  9. Parmelli E, Flodgren G, Beyer F, Baillie N, Schaafsma ME, Eccles MP. The effectiveness of strategies to change organisational culture to improve healthcare performance: a systematic review. Implement Sci. 2011; 6: 33.
  10. Scott T, Mannion R, Marshall M, Davies H. Does organisational culture influence health care performance? A review of the evidence. J Health Serv Res Policy. 2003; 8(2): 105-17.

Assisted reproduction to address global infertility – is access a human right?

Most of you may have seen a picture of me with my fraternal / dizygotic twin girls (16 May 2014). To cut a long story short, they were the result of in-vitro fertilisation (IVF), or more specifically intracytoplasmic sperm injection (ICSI). I did my first degree in medical science and was always particularly impressed that one could get pregnant via a ‘test tube’ in the laboratory… and I thought, well that won’t happen to me with my appetite for alpha-males, surely not?! Of course, the term ‘test tube’ isn’t totally accurate; there are lot of drugs you have to self-inject into either buttocks or thigh to regulate your cycle and then to over-stimulate your ovaries, and then there is the whole extraction and return procedure, but I will save that story for another rainy day.

I am now officially a Mum of Multiples (as Tamba, the Twins and Multiple Births Association, refers to us) and as such I seem to surround myself with other twin mums. In fact, we have formed a mini-society in Lichfield and we meet on a Friday in a children’s centre and watch our children cause mischief and havoc, while drinking tea and supporting each other – which mainly consists of telling each other how hard it is and how we are all super mums for having two babies/toddlers simultaneously. However, most of us, I would say at a guess around 80% of us, ‘chose our own destiny’ through the wonder of medical science having had some form of fertility treatment. I do have some interesting stories about same-sex fertility treatment options/decisions after quizzing a few members about how they chose who would carry the child/children and where does one get their sperm from. All totally fascinating I can assure you. Anyway, even though all our journeys here were different, you might say that we are the lucky ones – by hook or by crook our fertility treatment had been a success and our screaming, whining and snotty-nosed pairs were evidence of our victories.

This got me thinking that not everybody was so lucky. I have had friends struggle to get access to fertility treatment provided by our NHS as they did not fulfil certain criteria – they had moderate to high BMIs; they had had ectopic pregnancies within the last two years; they had been pregnant before but they had miscarried and so on. NICE has recently (Feb 2013) updated their guidance on fertility treatment (from previous 2004 guidance) and they still recommend that couples must have been trying to conceive for a total of two years, but that this can now include up to a one year before their fertility investigations commenced. If couples were struggling to gain access to fertility treatment here, what was the situation like further afield?

Data from demographic and health surveys collected by the World Health Organisation from 47 low-income countries, including countries in Sub-Saharan Africa, North Africa, Central and South-eastern Asia and Latin America, showed that more than 180 million couples have infertility problems and are childless involuntarily.[1] Fertility treatment is not available for these groups because they happen to reside in a country that has limited resources, where government strategies seem preoccupied with managing population growth, and where prevention against infectious diseases, sexually-transmitted diseases and malnutrition remain key public health priorities. I had to wonder, is withholding fertility treatment for this group right from an ethical and social-cultural perspective? Indeed, the social consequences of being childless in the developing world appears more problematic, with women being blamed, ostracised and even assaulted by their families.[2]

So what is the answer? We hand back to the medical scientists and ask them to further develop or simplify the current techniques so they are more affordable. Savings can be made by lowering the ‘laboratory costs’ associated with fertilisation and culture of eggs and embryos, for example, by using something called a ‘humidicrib’ – literally a plastic box instead of laminar flow hood.[2] Such developments are supported by the non-profit foundation, the Low-Cost IVF Foundation, established in 2007 with a mandate to bring down the material cost of an IVF cycle to €200.[3] Of course there are many more barriers to implementation of such procedures in the developing world, but at least it is now on the radar – infertility should be considered a global health problem and surely we should all be able to have access to treatments – isn’t it a basic human right to be able to procreate? Otherwise, what’s it all for?

— Nathalie Maillard, CLAHRC WM Head of Programme Delivery


  1. Rutstein SO, Shah IH. Infecundity, infertility, and childlessness in developing countries. DHS Comparative Reports No. 9. Calverton, Maryland: ORC Macro and WHO. 2004.
  2. Ombelet W. Global access to infertility care in developing countries: a case of human rights, equity and social justice. Facts Views Vis Obgyn. 2011; 3(4):257-66.
  3. Vayena E, Peterson HB, Adamson D, Nygren KG. Assisted reproductive technologies in developing countries: are we caring yet? Fertil Steril. 2009; 92(2): 413-6.


Context to Influence Behaviour

It has long been recognised that influencing behaviour is central to key policy changes. This isn’t just the case in health, it is also recognised in finance and even climate change. Typically – in economics and psychology – attempts to change behaviour have been based on influencing the way people think, through information and incentives (cognitive model).

Ivo Vlaev, recently appointed Professor of Behavioural Science at Warwick Business School, has developed the mnemonic, MINDSPACE to ‘change contexts’ by influencing the environments in which people act (context model).

MINDSPACE gathers together the nine most robust effects that influence behaviour:

Messenger – who communicates the information.
Incentives – shaped by predictable mental shortcuts, e.g. avoiding loss.
Norms – what others do.
Defaults – going with the pre-set options.
Salience – what is novel and relevant to us.
Priming – sub-conscious cues.
Affect – emotional associations.
Commitments – consistency with public promise, and reciprocation.
Ego – what makes us feel better about ourselves.

Policy-makers can put MINDSPACE into practice following a simple, structured process, extending and enhancing traditional ways of influencing behaviour.

Policies that change the context and ‘nudge’ people in a particular directions have captured the imagination of academics and policy-makers. Indeed MINDSPACE is the topic of both an article in the Journal of Economic Psychology [1] and a report to the Cabinet Office.[2]

 –Richard Lilford, Director CLAHRC WM


  1. Dolan P, Hallsworth M, Halpern D, King D, Metcalfe R, Vlaev I. Influencing behaviour: the mindspace way. J Econ Psychol. 2012; 33: 264-77.
  2. Dolan P, Hallsworth M, Halpern D, King D, Vlaev I. MINDSPACE: Influencing behaviour through public policy.  London, UK: Institute for Government. 2010.

Do the Social Health Features of Slums Correlate to the West Midlands?

As we were chewing the fat over a cup of coffee, Richard Lilford, CLAHRC WM Director, told me about his recent visit to Kenya and a grant application he was consulting on. The application plans to look at interventions which aim to improve the diet of the people who lived in the slums around Nairobi. Many people who live in such slums buy their food from street vendors, as this minimises the effort and cost of cooking. In 2002 The International Food Policy Research Institute based in Washington, produced a report looking at the challenges and options for poor people living in cities in developing countries.[1] One study it looked at found that city residents in Nigeria spent up to 50% of their total food expenditure on street food and would often have less time available for buying and preparing food, greater exposure to advertising, and easier access to street food compared to poor people living in rural areas. People buying this street food are constrained by what the street vendors are selling, which usually contains high levels of salt, saturated fat and sugar.

This got me thinking – can we claim to have better diets in higher income countries? Is there any difference between me popping into my local fast food restaurant on the way home from work or ordering a takeaway because I cannot make the effort to cook, and the poor people in Nairobi who buy their dinner from street vendors? Gavin Rudge of the CLAHRC WM recently carried out a pilot study looking at the relationship between fast food provision and neighbourhood characteristics in the metropolitan boroughs of Sandwell and Dudley. Initial findings suggest that there is an association between neighbourhood deprivation and the density and proximity of fast food outlets. The density of fast food outlets increases with deprivation, in the most deprived areas your nearest fast food outlet is less than a 2 minute walk away. While local authorities in this country are already moving to place restrictions on new applications for fast food outlets, some areas are already saturated with them – this seems to be an idea that has come far too late. Instead, we need to change people’s individual behaviour and their food choices in order to improve their diet. Although Sandwell is currently saturated in fatty and greasy food choices, if residents decide to eat healthier then eventually the demand for fast food will go down and outlets will be forced to close or diversify into healthier options.

It seems to me that whether you are living in the slums of Nairobi or in an inner city within the UK, the social features around food choices are the same. I will be interested to see how the CLAHRC WM Director tries to tackle this problem in Africa – hopefully he can bring some insights back.

— Jo Sartori, CLAHRC WM Head of Programme Delivery


1: International Food Policy Research Institute. Living in the City: Challenges and Options for the Urban Poor. 2002. [Online].

Interdisciplinary Research in Health Care

Health care systems, by nature, are inherently complex and multifaceted. The challenges they pose for patients, clinicians, managers and policymakers extend beyond the visible clinical, organizational and financial elements to include a mix of social, cultural, economic, political, ethical and legal factors. The design and conduct of rigorous science-based research to improve clinical practice and to inform health policy require contributions from multiple academic disciplines and health professions. In essence, health systems research necessitates an interdisciplinary approach for solving thorny problems and finding appropriate solutions.

Interdisciplinary research is a collaborative enterprise that draws upon the talents of investigators grounded in different disciplinary perspectives. For more than 20 years, I have had the privilege of directing the Robert Wood Johnson Foundation Scholars in Health Policy Research Program, a two-year postdoctoral fellowship for outstanding young economists, political scientists, and sociologists. The program fosters research collaboration among these social science disciplines as well as with investigators in medicine, nursing, law, bioethics, epidemiology, public health, public policy, psychology, history, management science, and social work. The program stresses the importance of each investigator “learning the language” of other disciplines by internalizing the conceptual frameworks and analytic methods employed to address health care issues. In some cases, alumni have become multidisciplinary, incorporating the perspectives and methodological approaches of other disciplines in their research.

Because no single discipline or health profession can solve the many problems plaguing health care systems, a collaborative, interdisciplinary approach is becoming the norm in most nations. The NIHR CLAHRC West Midlands embodies this type of approach and offers unparalleled opportunities for academic and clinical researchers to join with practitioners, managers, local authorities, and patients in the community to translate research findings into best practices that will improve health and social services and result in better patient outcomes for the population of the West Midlands region and beyond.

Prof Alan B. Cohen, Professor of Health Policy and Management, Boston University School of Management

Speaking Truth Unto Power

How often do you hear someone saying “Policy makers don’t use  evidence?” That’s cobblers. They don’t always use evidence and they seldom respond immediately, but that is not the same as not using it. South Africa provides the worst and best examples.

The worst example was the tragic refusal of Thabo Mbeki‘s government to acknowledge the cause of AIDS and  the consequent culpable failure to intervene in the epidemic.

The best example is the current government’s rapid response to evidence – firstly that the prevalence of hypertension in South Africa is one of the highest in the world, and secondly, that the salt content of bread and processed food is very high. The government met with academics and industry representatives and successfully reduced salt levels in processed food through regulation. The process is described in an excellent article,[1] which Prof Alex Ezeh, Director of the African Population and Health Research Center, drew to my attention.

–Richard Lilford, Director CLAHRC WM


  1. Hofman KJ & Tollman SM. Population health in South Africa: a view from the salt mines. Lancet Global Health. 2013; 1(2): e66-7.

A Trio of BMJ Systematic Reviews

In Director’s Choice this week I single out not one, not two, but three papers that were all published in a recent issue of the BMJ (17 May 2014). I find this a most timely and interesting journal – second only to this News Blog! All three of the selected papers are systematic reviews, now the most widely cited form of clinical research.[1]

The first concerns prevention of asthma exacerbation by means of long-term inhaled medication.[2] First, this is a very interesting and important clinical topic, especially given the recent survey of national asthma deaths.[3] Second, because it uses network meta-analyses where treatments are compared indirectly through a common comparator. We are going to have to get used to this type of analysis because of “question inflation,” as the Director described in a previous blog.[4] In any event, this analysis produces a strong result showing that adding long-acting ß-agonists to low-dose corticosteroids was a considerably more effective therapy than standard therapy of low-dose corticosteroids alone.

The second paper concerns effectiveness of non-surgical interventions to sustain initial weight-loss.[5] This is an important topic and the effect size of all interventions were small, amounting to a mean difference of less than 2kg. There is however, evidence that even small reductions in mass are beneficial.[6] [7] [8] Behavioural interventions that cover both diet and exercise produced a significant result, as did the medicine orlistat, which works to reduce fat absorption from the gut. Of course the latter comes at the cost of side-effects. Nutritional supplements and food replacements produced null results.

The last paper concerns an intervention that many people might consider warranted, irrespective of any evidence – screening women for intimate partner violence in healthcare settings.[9] There are 11 randomised or quasi-randomised trials on this topic. Importantly, screening programmes that involved structured interventions as part of the programme (i.e. a compound intervention of both screening and therapy) were excluded. This left the above 11 trials, which included over 13,000 participants. Hardly surprisingly the intervention did result in improved identification of violence. As a result referrals to domestic violence support services also increased, but the confidence limits were wide and not quite significant at the 0.05 level (a one-tailed test might have been more appropriate here?). Only two studies reported on reduction in intimate partner violence, both finding a null result. One study reported on psychological and physical health and again yielded a null result. I enjoyed the review but did not agree with the conclusion that there was “no evidence that universal screening is warranted.” I do not believe that worthwhile effects have been excluded. Yet again, no evidence of effect is conflated with evidence of no effect. This fallacy can be avoided by more careful interpretation of frequentist statistics, but a Bayesian approach would be better still. This is because Bayesian approaches explicitly incorporate indirect evidence.[10] Thus, although there is no direct comparitive evidence that screening for partner violence is warranted, there might be plenty of indirect evidence that can inform decision making, pending larger head-to-head trials. The evidence-based practice movement consistently gets ahead of itself by conflating “direct comparative evidence” with “evidence” as I shall describe in a strongly-worded argument in the next News Blog.

–Richard Lilford, Director CLAHRC WM


  1. Patsopoulos NA, Analatos AA, Ioannidis JP. Relative citation impact of various study designs in the health sciences. JAMA. 2005; 293(19):2362-2366.
  2. Loymans RJB, Gemperli A, Cohen J, Rubinstein SM, Sterk PJ, Reddel HK, Jüni P, ter Reit G. Comparative effectiveness of long term drug treatment strategies to prevent asthma exacerbations: network meta-analysis.
    BMJ. 2014; 348:g3009.
  3. Royal College of Physicians. Why asthma still kills. The National Review of Asthma Deaths (NRAD). London: Healthcare Quality Improvement Partnership. 2014.
  4. Lilford RJ. The end of the hegemony of randomised trials. 2012 Nov 30. [Online].
  5. Dombrowski SU, Knittle K, Avenell A, Araújo-Soares V, Sniehotta FF. Long term maintenance of weight loss with non-surgical interventions in obese adults: systematic review and meta-analyses of randomised controlled trials. BMJ. 2014; 348:g2646.
  6. Goldstein DJ. Beneficial health effects of modest weight loss. Int J Obes Relat Metab Disord. 1992; 16(6): 397-415.
  7. Lilford R, et al. Birmingham and Lambeth Liver Evaluation Testing Strategies (BALLETS): a prospective cohort study. Health Technol Assess. 2013; 17(28).
  8. Vidal J. Updated review on the benefits of weight loss. Int J Obes. 2002; 26(s2): s25-8.
  9. O’Doherty LJ, Taft A, Hegarty K, Ramsay J, Davidson LL, Feder G. Screening women for intimate partner violence in healthcare settings: abridged Cochrane systematic review and meta-analysis. BMJ. 2014;348:g2913
  10. Lilford RJ, Thornton JG, Braunholtz D. Clinical trials and rare diseases: a way out of a conundrum. BMJ. 1995; 311(7020): 1621-5.