Tag Archives: Richard Lilford

More on the Hygiene Hypothesis and Exposure to Coliform Organisms from the Birth Canal

The putative advantages of a deep draught of coliform organisms during a baby’s journey into the world has been discussed in a previous News Blog, with respect to prevention of allergy.[1] It now seems that it is not just allergy, but also cancer – more specifically the acute leukaemia of childhood – that is influenced by the process of birth.[2] And again, bypassing the birth canal by means of Caesarean section increases risk. The mechanism seems to conform with the three hits hypothesis, described in a past News Blog.[3] Here the hits might be:

  1. Genetic predisposition.
  2. Failure to ‘benefit’ from exposure to coliforms during birth.
  3. Subsequent severe infection.

Regarding the third ‘hit’ above, it is known that acute lymphoblastic leukaemia of children occurs in semi-epidemic fashion, suggesting that an acute infection is the trigger.

Some decades ago I carried out a decision-analysis that argued that when the risk of intra-partum C-section exceeded a threshold of around 35%, then a planned C-section was the best option for mother and baby.[4] For the mother because intra-partum C-section is more risky than planned C-section; and for the baby because situations where intra-partum C-section is common usually imply that the baby is also at increased risk – for example if the baby is coming by the breach. However, I can now see that my decision-analysis was incomplete – maybe I should have factored in the ‘unknown unknowns’.

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ. Exposure of the Baby to a Rich Mixture of Coliform Organisms from the Birth Canal. NIHR CLAHRC West Midlands News Blog. 22 April 2016.
  2. Greaves M. A causal mechanisms for childhood acute lymphoblastic leukaemia. Nat Rev Cancer. 2018.
  3. Lilford RJ. Three Hits Hypothesis. NIHR CLAHRC West Midlands News Blog. 7 April 2017.
  4. Lilford RJ, van Coeverden de Groot HA, Moore PJ, Bingham P. The relative risks of caesarean section (intrapartum and elective) and vaginal delivery: a detailed analysis to exclude the effects of medical disorders and other acute pre-existing physiological disturbances. Br J Obstet Gynaecol. 1990; 97(10): 883-92.

Health Economics and Access to Care: Are We Using the Wrong Model?

I woke one morning, many years ago, to the voice of a famous economist sounding off on my bedside radio. He spoiled the equanimity of my morning with his argument that the value of primary care should be evaluated by comparing the costs of the service with the health gain achieved by the service (in terms of quality adjusted life years [QALYs]). That is cobblers! Quite apart from the facile idea that the health gain from primary care can be calibrated with any kind of accuracy, the economist’s health economic model bypasses much of the purpose of healthcare. In this model, health care is simply as an instrument to improve health status. But a little thought will immediately show that health gain is a very incomplete understanding of the reason that people consult doctors. Health care serves a deep psychological need; human beings have turned to healers from the time that we became human beings. Health practitioners are not only valuable for the health gain they can now achieve, but also because they provide human warmth and support. The need for comfort, information, magic, and cure are all entangled. Not only do we need someone to turn out at times of mental or physical distress, but crucially, we also need to know that someone will be there for us when our time comes. And we need this assurance, even when we are perfectly healthy. We could perhaps wrap in the avoidance of catastrophic loss and call this the ‘insurance value’. Nor should the value of information – news about your own body – be underestimated. Berwick & Weinstein found that half of the benefit of an antenatal scan was simply to get a picture of the baby and had nothing to do with its medical purpose.[1]

The classical health economic model of cost utility analysis is well adapted to rationing demand once the patient’s condition has been defined. At that point calculating the relative value of different treatment options is a relatively straightforward issue (Figure 1). However, calculating the return-on-investment from simply providing access to healthcare is a different matter altogether. First, there is the extraordinarily difficult instrumental question of how to hypothecate the treatment effect over the full range of health conditions (Figure 2). Second, there is a need to factor in the value of:

  1. Information.
  2. Solace, comfort, support.
  3. Knowing that access will be available when required – the insurance value.

At the very least, it should recognise that cost-utility analysis for a calculation of a QALY or a DALY is not up to the task. The topic of access is one area where health economics raises many unsolved problems. In a recent news blog we discussed another issue that exposes some of the deep philosophical conundrums at the heart of health economics – the thorny issue of infertility.[2]

Fig 1. Management of a Specific Condition: a task for standard health economics

105 DCB - Health economics Fig 1

Fig 2. Providing Access to Healthcare: Health benefits are diffuse and hence hard to capture

105 DCB - Health economics Fig 2 

— Richard Lilford, CLAHRC WM Director


  1. Berwick DM, Weinstein MC. What do patients value? Willingness to pay for ultrasound in normal pregnancy. Med Care. 1985; 23(7): 881-93.
  2. Lilford RJ. The Health Economics of Infertility Treatment. NIHR CLAHRC West Midlands News Blog. 9 March 2018.

Mediating Variables

I have long argued that service delivery research, especially when generic interventions are evaluated, should examine the entire causal chain from intervention uptake to patient outcome.[1] Such an approach, of course, includes observation of mediating variables – variables (often residing in people’s hearts and minds) that form part of the above causal chain. The advantages of such ‘medication analysis’ seems to be catching on – two papers in the journal ‘Implementation Science’ have covered this topic in the last year.[2][3] In one case the mediating variables explained most of the effect (Anselmi, et al.), while in the other (Lee, et al.) they explained none of it, suggesting that the intervention was working through a theoretical domain not previously considered. These papers used structural equations. However, I prefer Bayesian networks that our CLAHRC is pioneering.[4-6] This is for two reasons:

  1. They can capture information from outside of the index study.
  2. They can meld qualitative and quantitative information through elicitation of informative probability densities.

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ, Chilton PJ, Hemming K, Girling AJ, Taylor CA, Barach P. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ 2010; 341: c4413.
  2. Anselmi L, Binyaruka P, Borghi J. Understanding causal pathways within health systems policy evaluation through mediation analysis: an application to payment for performance (P4P) in Tanzania. Implement Sci. 2017; 12: 10.
  3. Lee H, Hall A, Nathan N, et al. Mechanisms of implementing public health interventions: a pooled causal mediation analysis of randomised trials. Implement Sci. 2018; 13: 42.
  4. Hemming K, Chilton PJ, Lilford RJ, Avery A, Sheikh A. Bayesian cohort and cross-sectional analyses of the PINCER trial: a pharmacist-led intervention to reduce medication errors in primary care. PLoS One. 2012;7(6):e38306
  5. Watson SI & Lilford RJ. Essay 1: Integrating multiple sources of evidence: a Bayesian perspective. In: Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Southampton (UK): NIHR Journals Library, 2016.
  6. Watson SI, Chen YF, Bion JF, Aldridge CP, Girling A, Lilford RJ; HiSLAC Collaboration. Protocol for the health economic evaluation of increasing the weekend specialist to patient ratio in hospitals in England. BMJ Open. 2018; 8(2): e015561.

Too Much Performance Measurement

The United States National Quality Measures Clearinghouse (NQMC) now lists over 2,500 performance measures for healthcare. Enough already!

Writing in the New England Journal of Medicine colleagues from Ann Arbor have audited the 271 measures in the Medicare quality payment program.[1] Expenditure on collecting quality measures costs the average US physician about $40,000 per year. However, only 32 of the above measures were found to be validated by a consensus process. There was no real evidence for most of the measures. The notion that health care can be monitored comprehensively like widgets on a production line Is a modern holy grail. In the last analysis we need well-informed, articulate, compassionate and, above all, committed clinicians. Let’s concentrate more on producing them and less on statistical process control.[2]

— Richard Lilford, CLAHRC WM Director


  1. MacLean CH, Kerr EA, Qaseem A. Time Out – Charting a Path for Improving Performance Measurement. New Engl J Med. 2018; 378: 1757-61.
  2. Lilford RJ. Comparing Statistical Process Control and Interrupted Time Series. NIHR CLAHRC West Midlands News Blog. 11 March 2016.

A Framework to Improve Access to Acute Care in Low- and Middle-Income Countries

Universal healthcare is an important goal in global health, as described in previous News Blogs.[1] Key to the concept of universal healthcare is the question of access to health. I lead one of three work packages in the NIHR Global Health Unit for Surgery, directed by Dion Morton. Along with Dr Dmitri Nepogodiev, I have recently returned from a series of meetings with influential doctors, policy-makers and community leaders in Oyo and Osun states in Nigeria. Our host was Dr Wally Adisa of Ife University, to whom we extend our sincere thanks.

Dmitri has reviewed the literature on how barriers may be overcome and access to healthcare facilitated. In Figure 1 we have attempted to synthesise the barriers from the literature and meetings in Nigeria. We use the famous Grossman model,[2] which recognises four phases on the pathway linking symptoms to effective treatment: recognition of the need for help; seeking help; transport to a place where appropriate care can be delivered; and then obtaining care in the healthcare institution. Many of the barriers we have identified could have been discerned though intuition: lack of money; poor understanding of disease and how it can be remedied; reliance on traditional healers; etc. However our investigations have identified certain factors we had not anticipated. For instance, many people are reluctant to call an ambulance, even when available, because they are superstitious of entering such a vehicle.[3]

While poverty is an important factor, limiting access, people who need acute care usually make it to hospital eventually. When we probe the reasons for delay among people who eventually did make it to hospital, we find that delay occurs because resources are not in the right place at the right time. In countries with a low tax base, more use should be made of existing networks and community ‘assets’ , to short-cut the barriers.[4] [5]

Figure 1. Barriers and Facilitators on the Pathway to Acute Care

104 DCB - A Framework to Improve Access to Care in Low - Figure - v2

During our meetings, and in the literature, there is much agreement about what the barriers are: poverty, superstitious beliefs, perceptions (not always erroneous) of poor care in hospitals, lack of facilities with need for further transfer, and so on. Where there is much less agreement, between stakeholders and within the literature, is on the relative importance of the various factors or how they may vary by clinical scenario – obstetric emergency, acute abdomen, trauma, childhood illness, and so on. While Desmond (personal communication) found severe constraints in access due to inadequate transport or ability to pay for transport (phase 3 in Grossman’s model) in a paediatric context in Malawi, Orji found that all the problems were in phase 1 and 2 or 4 in an obstetric context in Nigeria.[6] It is also known that no emergency transport systems are available in 33 (61%) of 54 African countries that answered a recent survey, and many only covered trauma or obstetric emergencies, while few were country-wide. Overall, only 8.7% of the population need could be met.[7] Nor should it be assumed that transport costs are negligible compared to health costs.[8] It is lack of facilities for transport that is the most important problem, not poor roads or hospitals too widely dispersed. Sheer distance can be a problem in some countries, such as Sudan, and lack of roads in other places, such as Ethiopia, but over two-thirds of Africa’s population live within two hours of hospital.[9]

We have produced a list of possible measures to improve access, classified according to whether they stimulate demand or supply. None of these interventions are easy to implement or evaluate. For this reason we plan to engage stakeholders to see what might be feasible, review the literature on what has been tried before, and then develop a health economic model to evaluate the cost-effectiveness of different potential solutions. In a future News Blog we will describe our approach to health economic modelling of this complex, but important, topic.

Factors that may be Tackled by Interventions to Improve Access

Demand for transfer Supply of the means for transfer
Knowledge of treatable illness, such as meningitis, typhoid, snakebite. Tackled through education. For example, messages targeted at misconceptions were massively influential in eliminating the Ebola epidemic. Many superstitions and beliefs are cultural, so different messages will be needed in different places. People can be influenced through local community and religious leaders, as well as through feedback from people who have experienced services.[10] Since most people do reach services, promotion of risk-sharing community schemes or ‘electronic’ wallets to provide resources when and where needed. Women’s participatory groups can also encourage autonomy, making women less reliant on husbands for money or permission.
Use of eHealth in general and eConsulting in particular to help translate awareness of symptoms to the intention to seek help. Public / NGO provision of inexpensive motorcycle taxis, successfully used for labour care in Sierra Leone [11] and Malawi.[12]
Interaction with traditional healers to recognise illnesses responsive to ‘modern medicine’. Encouraging / investing in small enterprises to promote transport, e.g. Uber-taxi style ambulances deployed in Nairobi.[13]

— Richard Lilford, CLAHRC WM Director & Dmitri Nepogodiev, Doctoral Research Fellow in Public Health.


  1. Lilford RJ. A Heretical Suggestion! NIHR CLAHRC West Midlands News Blog. 9 February 2018.
  2. Grossman M. The demand for health: A theoretical and empirical investigation. Cambridge, MA: NBER Books; 1972.
  3. Wilson A, Hillman S, Rosato M, Skelton J, Costello A, Hussein J, MacArthur C, Coomarasamy A. A systematic review and thematic synthesis of qualitative studies on maternal emergency transport in low- and middle-income countries. Int J Gynecol Obstet. 2013; 122: 192-201.
  4. Nielson K, Mock C, Joshipura M, Rubiano AM, Rivara F. Assessment of the Status of Prehospital Care in 13 Low- and Middle-Income Countries. Prehosp Emerg Care. 2012; 16(3): 381-9.
  5. Lilford RJ. Pre-Payment Systems for Access to Healthcare. NIHR CLAHRC West Midlands News Blog. 18 May 2018.
  6. Orji EO, Ogunlola IO, Onwudiegwu U. Brought-in maternal deaths in south-west Nigeria. J Obstet Gynaecol. 2002; 22:4:385-8.
  7. Mould-Millman N-K, Dixon JM, Sefa N, Yancey A, Hollong BG, Hagahmend M, Ginde AA, Wallis LA. The State of Emergency Medical Services (EMS) Systems in AfricaPrehosp Disaster Med. 2017; 32(3):273-83.
  8. Jan S, Laba T-L, Essue BM, Gheorghe A, Muhunthan J, Engelgau M, Mahal A, Griffiths U, McIntyre D, Meng Q, Nugent R, Atun R. Action to address the household economic burden of non-communicable diseases. Lancet. 2018; 391:2047-58.
  9. Ouma PO, Maina J, Thuranira PN, Macharia PM, Alegana VA, English M, Okiro EA, Snow RW. Access to emergency hospital care provided by the public sector in sub-Saharan Africa in 2015: a geocoded inventory and spatial analysis. Lancet Glob Health. 2018; 6:e342-50.
  10. Mould-Millman N-K, Rominski SD, Bogus J, Ginde AA, Zalariah AN, Boatemaah CA, Yancey AH, Akoriyea SK, Campbell TB. Barriers to Accessing Emergency Medical Services in Accra, Ghana: Development of a Survey Instrument and Initial Application in Ghana. Glob Health. 2015; 3(4):577-90.
  11. Bhopal SS, Halpin SJ, Gerein N. Emergency Obstetric Referral in Rural Sierra Leone: What Can Motorbike Ambulances Contribute? A Mixed-Methods Study. Matern Child Health J. 2013; 17: 1038-43.
  12. Hofman JJ, Dzimadzi C, Lungu K, Ratsma EY, Hussein J. Motorcycle ambulances for referral of obstetric emergencies in rural Malawi: Do they reduce delay and what do they cost? Int J Gynecol Obstet. 2008; 102: 191-7.
  13. Moh C. How a speedy emergency services app is saving lives. BBC News. 24 November 2017.

Which are the Most Cost-Effective Surgical Operations for Low- and Middle-Income Countries?

I recently came across a Lancet review on essential surgery: operations that should be available to even very poor communities.[1] The authors sensibly divide the list into those that should be available in first-level hospitals, and those that should be available in specialist (tertiary) hospitals.

Obstetrical and gynaecological procedure’s rank high in the first list, including caesarean section, surgery for ectopic pregnancy, and tubal ligation. There are a large number of operations related to trauma, including chest drain, complex fracture reduction, laparotomy, amputation, and debridement. Also included for the first-level hospitals are a range of general surgical procedures, such as repair of perforations, appendectomy, hernia repair, and colostomy.

Specialist procedures that should be available in tertiary hospitals include congenital abnormalities (especially cleft lip and club foot) and eye surgery (cataract extraction or eyelid repair for trachoma).

But how cost-effective are all these procedures? A recent study in PLOS One summarises some of the recent evidence.[2] The most cost-effective surgery is emergency caesarean section and voluntary male circumcision. This is followed in increasing order by: cataract surgery, cleft lip and palate surgery, hernia repair, breast cancer surgery, trauma surgery, colorectal surgery, and non-emergency orthopaedic conditions. I think vesicovaginal fistula should be high on the list.

One notes that most, but not all, of the highly cost-effective surgical conditions relate to acute presentations rather than chronic conditions. This underlines the critical importance of access to first-level hospital care. Such access is crucial, not only for surgical conditions, but also acute medical conditions such as meningitis, malaria and snake bites.

— Richard Lilford, CLAHRC WM Director


  1. Mock CN, Donkor P, Gawande A, Jamison DT, Kruk ME, Debas HT, for the DCP3 Essential Surgery Author Group. Essential surgery: key messages from Disease Control Priorities, 3rd edition. Lancet. 2015; 385: 2209-19.
  2. Horton S, Gelband H, Jamison D, Levin C, Nugent R, Watkins D. Ranking 93 health interventions for low- and middle-income countries by cost-effectiveness. PLoS ONE. 2017; 12(8): e0182951.

RCTs versus Observational Studies: This Time in the Advertisement Industry

There is a substantive body of medical methodological research in which the results of RCTs for a given treatment are compared to the results of observational studies for that same treatment. They show that, as compared to the RCTs, effect sizes in the observational studies are similar to those in RCTs, but that they are widely scattered around the gold standard (i.e. RCT) estimate.[1] [2]

A similar result was obtained in a study of RCTs versus observational studies in the economics literature back in the 1980s, except that in their study RCTs yield more conservative estimates than those in observational studies.[3] Now, a similar study has been carried out in the advertising industry using advertisements carried on Facebook as the basis for a field experiment.[4] The results of 15 RCTs of advertisements on Facebook were compared to the results of observational studies in which standard statistical methods were used to control for potential identifiable confounders. The findings of this methodological study corroborate those of earlier studies in economics. The observational studies, even after risk adjustment, showed a wide scatter of results around those of the corresponding RCTs and the observational studies tend to produce more strongly positive results. This contradicts the prevailing view in the advertisement industry that observational studies produce reliable estimates of the effectiveness of advertisements. Interestingly, if just one factor accounted for all of the difference between the observational studies and the RCTs, then this single factor would account for more explanatory power than all other variables taken together.

The selection bias in the case of advertising likely relates to a link between exposure to the environment featuring the advertisement and responding to the ad when observed. To put this another way, people who are exposed to an advertisement are already pre-disposed to respond to the advertisement. In health care we would say that exposure and response are on the same causal chain. Economists would say that they were ‘endogenous’.

— Richard Lilford, CLAHRC WM Director


  1. Benson K & Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med. 2000; 342(25): 1878-86.
  2. Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev. 2014; 4: MR000034.
  3. Banerjee AV, Duflo E, Kremer M. The Influence of Randomized Controlled Trials on Development Economics Research and on Development Policy. 2016
  4. Gordon BR, Zettelmeyer F, Bhargava N, Chapsky D. A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook. 2018.

Gluten Sensitivity but no Antibodies?

Consider the case of my good friend who developed gluten sensitivity in midlife. Subsequently he went on a gluten-free diet – his wife found this a terrible nuisance. So she surreptitiously re-introduced wheat to his diet. Within no time my friend complained and that he had been wrong, his symptoms had reoccurred despite no apparent exposure to wheat. He was disappointed with his wife when she had to confess to her clandestine challenge to his physiology. But I think she behaved like a true scientist!

The single case represented by my friend has been repeated on a larger-scale many times. The results have been the same; many people with gluten sensitivity manifest symptoms when challenged in blind studies.[1] Furthermore, unlike many types of putative psychosomatic illness, people with gluten sensitivity do not manifest different responses on psychological testing for depression or anxiety compared with those of the general population.

So what is the cause of this somatopsychic condition? It turns out that there are two main theories each with some evidence in their favour.[2] The theory that I prefer is called FODMAPs, based on the idea that wheat is a potent source of fermentable, short chain carbohydrates. These carbohydrates are poorly absorbed and thus ferment in the gut causing the typical symptoms of bloating, distention and discomfort. The alternative theory is that wheat, perhaps in the presence of certain alterations in the microbiome, causes an inflammatory reaction in the liver that is associated with symptoms.

It will be important to discern the cause, since treatment of excessive fermentation would consist of a more general reduction of foods containing large proportions of fermentable carbohydrates.

— Richard Lilford, CLAHRC WM Director


  1. Skodje GI, Sarna VK, Minelle IH, Rolfsen KL, Muir JG, Gibson PR, Veierød MB, Henriksen C, Lundin KEA. Fructan, Rather Than Gluten, Induces Symptoms in Patients With Self-Reported Non-Celiac Gluten Sensitivity. Gastroenterol. 2018; 154: 529-39.
  2. Servick K. The war on gluten. Science. 2018; 360: 848-51.

Interim Guidelines for Studies of the Uptake of New Knowledge Based on Routinely Collected Data

CLAHRC West Midlands and CLAHRC East Midlands use Hospital Episode Statistics (HES) to track the effect of new knowledge from effectiveness studies on implementation of the findings from those studies. Acting on behalf of CLAHRCs we have studied uptake of findings from the HTA programme over a five year period (2011-15). We use the HES database to track uptake of study treatments where the use of that treatment is recorded on the HES database – most often these are studies of surgical procedures. We conduct time series analyses to examine the relationship between publication of apparently clear-cut findings and the implementation (or not) of those findings. We have encountered some bear traps in this apparently simple task, which must be carried out with an eye to detail. Our work is ongoing, but here we alert practitioners to some things to look out for based on the literature and our experience. First, note that the use of time series to study clinical practice based on routine data is both similar and different from the use of control charts in statistical process control. For the latter purpose, News Blog readers are referred to the American National Standard (2018).[1] Here are some bear-traps/issues to consider when using databases for the former purpose – namely to scrutinise databases for changes in treatment for a given condition:

  1. Codes. By a long way, the biggest problem you will encounter is the selection of codes. The HTA RCT on treatment of ankle fractures [2] described the type of fracture in completely different language to that used in the HES data. We did the best we could, seeking expert help from an orthopaedic surgeon specialising in the lower limb. Some thoughts:
    1. State the codes or code combinations used. In a recent paper, Costa and colleagues did not state all the codes used in the denominator for their statistics on uptake of treatment for fractures of the lower radius.[3] This makes it impossible to replicate their findings.
    2. Give the reader a comprehensive list of relevant codes highlighting those that you selected. This increases transparency and comparability, and can be included as an appendix.
    3. When uncertain, start with a narrow set of codes that seem to correspond most closely to indications for treatment in the research studies, but also provide results for a wider range – these may reflect ‘spill-over’ effects of study findings or miscoding. Again, the wider search can be included as an appendix, and serves as a kind of sensitivity analysis.
    4. If possible, examine coding practice by examining local databases that may contain detailed clinical information with the routine codes generated by that same institution. This provides empirical information on coding accuracy. We did this with respect to use of tight-fitting casts to treat unstable ankle fracture (found to be non-inferior to more invasive surgical plates [4]) and found that the procedure was coded in different ways. We combined these three codes in our study, although this increases measurement error (reducing the signal) on the assumption that these codes are not specific.
  2. Denominators.
    1. In some cases denominators cannot be ascertained. We encountered this problem in our analysis of surgery for oesophageal reflux, where surgery was found more effective than medical treatment.[5] The counterfactual here is medical therapy that can be delivered in various settings and that is not specific for the index condition. Here we simply had to examine the effects of the trial results on the number of operations carried out country-wide. Seasonal effects are a potential problem with denominator-free data.
    2. For surgical procedures, the procedure should be combined with the counterfactual procedure from the trial to create a denominator. The denominator can also be expanded to include other procedures for the same operation if this makes sense clinically.
  3. Data-interval. The more frequent the index procedure, then the shorter the appropriate interval. If the number of observations falls below a certain threshold, then the data cannot be reported to protect patient privacy, and a wider interval must be used. A six month interval seemed suitable for many surgical procedures.
  4. Of protocols and hypotheses. We have found that the detailed protocol must emerge as an iterative process including discussion with clinical experts. But we think there should be a ‘general’ prior hypothesis for this kind of work. So we specified the dates of publication of the HTA report as our pre-set time point – the equivalent of the primary hypothesis. We applied this date line for all of the procedures examined. However, solipsistic focus on this data line would obviously lead to an impoverished understanding, so we follow a three phase process inspired by Fichte’s thesis-antithesis-synthesis-thesis model [6]:
    1. We test the hypothesis that a linear model fits the data using a CUSUM (cumulative sum) test. The null hypothesis is that the cumulative sum of recursive residuals has an expected value of 0. If it wanders outside the 95% confidence band at any point in time, this indicates that the coefficients have changed and a single linear model does not fit the data.
    2. If the above test indicates a change in the coefficients, we use a Wald test to identify the point at which the model has a break. We estimate two separate models before and after the break data and the slopes/intercepts are compared.
    3. Last we ‘check by members’ and discuss with experts who can fill us in on when guidelines emerged and when other trials may have been published – ideally a literature review would complement this process.
  5. Interpretation. In the absence of contemporaneous controls, cause and effect inference must be cautious.

This is an initial iteration of our thoughts on this topic. However, increasing amounts of data are being captured in routine systems, and databases are increasingly constructed in real time since they are used primarily as a clinical tool. So we thought it would be helpful to start laying down some procedural rules for retrospective use of data to determine long-term trends. We invite readers to comment, enhance and extended this analysis.

— Richard Lilford, CLAHRC WM Director

— Katherine Reeves, Statistical Intelligence Analyst at UHBFT Health Informatics Centre


  1. ASTM International. Standard Practice for Use of Control Charts in Statistical Process Control. Active Standard ASTM E2587. West Conshohocken, PA: ASTM International; 2018.
  2. Keene DJ, Mistry D, Nam J, et al. The Ankle Injury Management (AIM) trial: a pragmatic, multicentre, equivalence randomised controlled trial and economic evaluation comparing close contact casting with open surgical reduction and internal fixation in the treatment of unstable ankle fractures in patients aged over 60 years. Health Technol Assess. 20(75): 1-158.
  3. Costa ML, Jameson SS, Reed MR. Do large pragmatic randomised trials change clinical practice? Assessing the impact of the Distal Radius Acute Fracture Fixation Trial (DRAFFT). Bone Joint J. 2016; 98-B: 410-3.
  4. Willett K, Keene DJ, Mistry D, et al. Close Contact Casting vs Surgery for Initial Treatment of Unstable Ankle Fractures in Older Adults. A Randomized Clinical Trial. JAMA. 2016; 316(14): 1455-63.
  5. Grant A, Wileman S, Ramsay C, et al. The effectiveness and cost-effectiveness of minimal access surgery amongst people with gastro-oesophageal reflux disease – a UK collaborative study. The REFLUX trial. Health Technol Assess. 2008; 12(31): 1–214.
  6. Fichte J. Early Philosophical Writings. Trans. and ed. Breazeale D. Ithaca, NY: Cornell University Press, 1988

A Randomised Trial of the Effect of Theological Training on Health and Welfare Outcomes: Whatever Next

The CLAHRC WM Director’s heroes, Adam Smith and Max Weber, argued that religiosity promotes diligence and wealth.[1] [2] But how to separate the effect of religion from the effect of being the kind of person who is religious? Only a randomised trial could do this. And, yes, it has been done.[3]

One hundred and sixty pastors were recruited for this study, which was based in the Philippines. The pastors each provided 15 weekly meetings to a total of 6,276 poor Filipino households that were randomised to either receive the programme or not. The intervention group had increased religiosity and income, but they were no more satisfied with life. The study suggested that intervention households had improved their levels of hygiene but discord within the family also seemed to increase. What does the CLAHRC WM Director make of this? Firstly, human beings are primed to be receptive to religious messages – they affect us and it was ever thus. However, the effects are not all necessarily beneficial. And, of course, religious instruction introduced later in life is not the same as growing up in a religious family.

— Richard Lilford, CLAHRC WM Director


  1. Smith A. An Inquiry into the Nature and Cause of the Wealth of Nations. London, UK: W Strahan & T Cadell; 1776.
  2. Weber M. The Protestant Ethic and the Spirit of Capitalism. London, UK: Unwin Hyman; 1930.
  3. Bryan GT, Choi JJ, Karlan D. Randomizing Religion: The Impact of Protestant Evangelism on Economic Outcomes. NBER Working Paper Series. Working Paper No. 24278. 2018.