Tag Archives: Director & Co-Directors’ Blog

Interim Guidelines for Studies of the Uptake of New Knowledge Based on Routinely Collected Data

CLAHRC West Midlands and CLAHRC East Midlands use Hospital Episode Statistics (HES) to track the effect of new knowledge from effectiveness studies on implementation of the findings from those studies. Acting on behalf of CLAHRCs we have studied uptake of findings from the HTA programme over a five year period (2011-15). We use the HES database to track uptake of study treatments where the use of that treatment is recorded on the HES database – most often these are studies of surgical procedures. We conduct time series analyses to examine the relationship between publication of apparently clear-cut findings and the implementation (or not) of those findings. We have encountered some bear traps in this apparently simple task, which must be carried out with an eye to detail. Our work is ongoing, but here we alert practitioners to some things to look out for based on the literature and our experience. First, note that the use of time series to study clinical practice based on routine data is both similar and different from the use of control charts in statistical process control. For the latter purpose, News Blog readers are referred to the American National Standard (2018).[1] Here are some bear-traps/issues to consider when using databases for the former purpose – namely to scrutinise databases for changes in treatment for a given condition:

  1. Codes. By a long way, the biggest problem you will encounter is the selection of codes. The HTA RCT on treatment of ankle fractures [2] described the type of fracture in completely different language to that used in the HES data. We did the best we could, seeking expert help from an orthopaedic surgeon specialising in the lower limb. Some thoughts:
    1. State the codes or code combinations used. In a recent paper, Costa and colleagues did not state all the codes used in the denominator for their statistics on uptake of treatment for fractures of the lower radius.[3] This makes it impossible to replicate their findings.
    2. Give the reader a comprehensive list of relevant codes highlighting those that you selected. This increases transparency and comparability, and can be included as an appendix.
    3. When uncertain, start with a narrow set of codes that seem to correspond most closely to indications for treatment in the research studies, but also provide results for a wider range – these may reflect ‘spill-over’ effects of study findings or miscoding. Again, the wider search can be included as an appendix, and serves as a kind of sensitivity analysis.
    4. If possible, examine coding practice by examining local databases that may contain detailed clinical information with the routine codes generated by that same institution. This provides empirical information on coding accuracy. We did this with respect to use of tight-fitting casts to treat unstable ankle fracture (found to be non-inferior to more invasive surgical plates [4]) and found that the procedure was coded in different ways. We combined these three codes in our study, although this increases measurement error (reducing the signal) on the assumption that these codes are not specific.
  2. Denominators.
    1. In some cases denominators cannot be ascertained. We encountered this problem in our analysis of surgery for oesophageal reflux, where surgery was found more effective than medical treatment.[5] The counterfactual here is medical therapy that can be delivered in various settings and that is not specific for the index condition. Here we simply had to examine the effects of the trial results on the number of operations carried out country-wide. Seasonal effects are a potential problem with denominator-free data.
    2. For surgical procedures, the procedure should be combined with the counterfactual procedure from the trial to create a denominator. The denominator can also be expanded to include other procedures for the same operation if this makes sense clinically.
  3. Data-interval. The more frequent the index procedure, then the shorter the appropriate interval. If the number of observations falls below a certain threshold, then the data cannot be reported to protect patient privacy, and a wider interval must be used. A six month interval seemed suitable for many surgical procedures.
  4. Of protocols and hypotheses. We have found that the detailed protocol must emerge as an iterative process including discussion with clinical experts. But we think there should be a ‘general’ prior hypothesis for this kind of work. So we specified the dates of publication of the HTA report as our pre-set time point – the equivalent of the primary hypothesis. We applied this date line for all of the procedures examined. However, solipsistic focus on this data line would obviously lead to an impoverished understanding, so we follow a three phase process inspired by Fichte’s thesis-antithesis-synthesis-thesis model [6]:
    1. We test the hypothesis that a linear model fits the data using a CUSUM (cumulative sum) test. The null hypothesis is that the cumulative sum of recursive residuals has an expected value of 0. If it wanders outside the 95% confidence band at any point in time, this indicates that the coefficients have changed and a single linear model does not fit the data.
    2. If the above test indicates a change in the coefficients, we use a Wald test to identify the point at which the model has a break. We estimate two separate models before and after the break data and the slopes/intercepts are compared.
    3. Last we ‘check by members’ and discuss with experts who can fill us in on when guidelines emerged and when other trials may have been published – ideally a literature review would complement this process.
  5. Interpretation. In the absence of contemporaneous controls, cause and effect inference must be cautious.

This is an initial iteration of our thoughts on this topic. However, increasing amounts of data are being captured in routine systems, and databases are increasingly constructed in real time since they are used primarily as a clinical tool. So we thought it would be helpful to start laying down some procedural rules for retrospective use of data to determine long-term trends. We invite readers to comment, enhance and extended this analysis.

— Richard Lilford, CLAHRC WM Director

— Katherine Reeves, Statistical Intelligence Analyst at UHBFT Health Informatics Centre


  1. ASTM International. Standard Practice for Use of Control Charts in Statistical Process Control. Active Standard ASTM E2587. West Conshohocken, PA: ASTM International; 2018.
  2. Keene DJ, Mistry D, Nam J, et al. The Ankle Injury Management (AIM) trial: a pragmatic, multicentre, equivalence randomised controlled trial and economic evaluation comparing close contact casting with open surgical reduction and internal fixation in the treatment of unstable ankle fractures in patients aged over 60 years. Health Technol Assess. 20(75): 1-158.
  3. Costa ML, Jameson SS, Reed MR. Do large pragmatic randomised trials change clinical practice? Assessing the impact of the Distal Radius Acute Fracture Fixation Trial (DRAFFT). Bone Joint J. 2016; 98-B: 410-3.
  4. Willett K, Keene DJ, Mistry D, et al. Close Contact Casting vs Surgery for Initial Treatment of Unstable Ankle Fractures in Older Adults. A Randomized Clinical Trial. JAMA. 2016; 316(14): 1455-63.
  5. Grant A, Wileman S, Ramsay C, et al. The effectiveness and cost-effectiveness of minimal access surgery amongst people with gastro-oesophageal reflux disease – a UK collaborative study. The REFLUX trial. Health Technol Assess. 2008; 12(31): 1–214.
  6. Fichte J. Early Philosophical Writings. Trans. and ed. Breazeale D. Ithaca, NY: Cornell University Press, 1988

On Integrated Care

Integrated care is a big issue promoted in the NHS and throughout the world. This push to integrate care includes low- and middle-income countries (LMICs). This is so, despite the demonstrated and often spectacular success of vertical programmes to tackle diseases, such as HIV and malnutrition, in LMICs.[1] Yet, there are compelling reasons to integrate care:

  1. A greater proportion of people now survive to suffer multiple chronic diseases affecting multiple organ systems.
  2. Solipsistic focus on specific (vertical) programmes can lead to neglect and poor quality of the generality of care.
  3. Vertical programmes imply that the diagnosis has been made, yet health services need to cater for people with undiagnosed symptoms.

But what about empirical evidence for integrated programmes? One set of programmes commonly integrated are HIV programmes and programmes targeting maternal and neonatal health. A review by the Cochrane HIV/AIDS group,[2] commissioned by USAID, tackled this particular issue back in 2011 – a time when HIV care was still a critical issue. The results were generally positive. For example, pregnancy rates declined when HIV and family planning services were integrated, and recovery rates from malnutrition improved in studies that examined this outcome. The review also identified factors associated with more or less successful migration from vertical to integrated programmes. Better results can be achieved by upfront investment in the integration process itself, focusing on staff education, preparation of appropriate case-notes and community engagement. In my opinion these are generic success factors for any programme of change.

Integrated care is a pervasive theme in NIHR CLAHRC West Midlands. We have recently completed an authoritative overview of 80 systematic reviews on this topic.[3] [4] Our work has stressed the importance of human resources in effecting service change, most particularly the importance of committed middle managers with high emotional intelligence,[5] and the role of ‘expectancy’, by which we mean that targets or incentives should only be used when the people at whom the target is aimed believe that they know how to achieve the target.[6] Our sister centre, the NIHR Global Health Unit on Improving Health in Slums, is also examining optimal health service configurations in slum areas of Africa and Asia,[7] where we will be studying integrated services using tools developed in CLAHRC WM.

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ. A Heretical Suggestion! NIHR CLAHRC West Midlands News Blog. 9 February 2018.
  2. Kennedy G, Kennedy C, Lindegren ML, Brickley D. Systematic review of integration of maternal, neonatal and child health and nutrition, family planning and HIV. Report No. 11-01-303-02. Washington, D.C.: Global Health Technical Assistance Project; 2011.
  3. Damery S, Flanagan S, Combes G. Does integrated care reduce hospital activity for patients with chronic diseases? An umbrella review of systematic reviews. BMJ Open. 2016; 6: e011952.
  4. Lilford RJ. Future Trends in NHS. NIHR CLAHRC West Midlands News Blog. 25 November 2016.
  5. Burgess N & Currie G. The Knowledge Brokering Role of the Hybrid Middle Level Manager: the Case of Healthcare. Br J Manage. 2013; 24(s1): s132-42.
  6. Lilford RJ. Financial Incentives for Providers of Health Care: The Baggage Handler and the Intensive Care Physician. NIHR CLAHRC West Midlands News Blog. 25 July 2014.
  7. Lilford RJ. Measuring the Quality of Health Care in Low-Income Settings. NIHR CLAHRC West Midlands News Blog. 18 August 2017.

Patient Reported Outcome Measures: A Tool for Individual Patient Management, Not Just for Research

Research and practice are often thought of as totally different types of activity. For instance, research is governed by an extensive set of procedural requirements that do not apply to standard practice. However, many inquiries that would have counted as research in earlier times are now embedded in management and governance practices. Take, for instance, outcomes of treatments and procedures. When the CLAHRC WM Director was a young doctor a study of the outcomes of, say, 50 Wertheim’s hysterectomies would be a typical research undertaking. Now, this may be considered a standard audit and may take place as part of a hospital’s prudent monitoring of its work. CLAHRCs have a long tradition of building data analysis into routine practice.

One active area of practice concerns patient-reported outcome measures (PROMs).  Often thought of as endpoints for research, CLAHRC WM collaborator Melanie Calvert and members of the Centre for Patient Reported Outcomes Research, are working closely with clinicians at University Hospital Birmingham NHS Foundation Trust to capture electronic PROMs which are used for real-time monitoring of patient symptoms and to tailor care to individual patient needs. Prof Calvert notes that these data have the potential to be used for multiple purposes: aggregated data may be used to inform and improve service delivery, whilst individual patient data may be used alongside remote clinical monitoring to guide frequency of outpatient appointments. Attendance at the hospital can be flexible and dictated by patient need and response to therapy.

You can hear more about their work and exciting new developments in PROMs research at the forthcoming PROMS conference, sponsored by CLAHRC WM and hosted by the University of Birmingham on Wednesday 20 June 2018. You can register online by clicking here. Registration is open until 11 June.

— Richard Lilford, CLAHRC WM Director

— Melanie Calvert, Professor of Outcomes Methodology

Evaluation of High vs. Low Cost Service Interventions

Generic service interventions vary considerably in their costs. Human resource interventions, such as improving the nurse to patient ratio or making more specialists available over the weekend, tend to be expensive. Other service interventions, such as an educational intervention to improve team working in multi-disciplinary clinical teams, are less expensive. The cost-effective effect size is smaller for lower cost interventions than for those which are more expensive. The axiom, that the cost of an intervention determines the effectiveness threshold at which it becomes cost-effective, has profound implications for the design and analysis of evaluative studies. The nub of the argument is that the size of the health effect that would justify deployment of service interventions may be too small to be detected by affordable or logistically feasible studies when the cost of that intervention is low. Before developing this argument further, let me be clear that by cost I mean net cost (not just the cost of the intervention itself), and that costs must be compared with respect to a common denominator – e.g. cost per patient, cost per 1,000 patients, etc.

Let us imagine that we wish to improve consultant cover at weekends. This is a very expensive intervention (whether measured in terms of the cost of hiring new consultants or the opportunity costs of re-allocating consultant time).[1] Such an intervention would need to provide considerable health gain to justify its substantial cost. In such a case it is reasonable to expect – indeed require – that any evaluative study should be able to detect patient benefit, say in terms of lives saved and adverse events avoided. If no improvement in health gain is detected, then we must conclude that either the study was ‘underpowered’ OR that any effects are too small to justify the intervention costs. If the study was not underpowered – that is to say if the sample size was sufficient to detect health benefits sufficient to justify cost of the intervention – then we conclude that the intervention does not promise good value for money. We leave aside the issue of exactly how the threshold effect (which justifies an intervention cost) can be determined, save to point out that methods to do so exist and that we have advocated use of such methods (prospective health economic modelling) for sometime.[1-3]

Take, as an opposite extreme, an intervention to promote hand-washing – perhaps using ‘nudge theory’. The intervention here is likely to be nugatory – say having a sticker with an illustration of a ‘watching eye’ placed over hospital sinks, for example.[4] Harms are unlikely and intervention costs are low. It follows that there is not much downside to intervening. That is to say, even if the intervention was totally ineffective, no real harm would result. A massive trial with an endpoint such as hospital-acquired infection rates would be overkill in such a scenario. This is because the threshold effect to justify the intervention is much smaller than the minimal difference detectable in any affordable / logistically feasible study. Using ‘upstream’ endpoints, such as “was the intervention deployed?” and “did it increase use of hand-washing materials?” (necessary but not sufficient conditions for effectiveness) would suffice in an evaluation. Many interventions are rather more expensive than promotion of hand-washing, but much less expensive than large HR initiatives; the above mentioned educational intervention to promote team-work, for example. Here it might be too much to expect, or require, quality of life or mortality to change sufficiently for any change to be detectable (statistically) in an affordable trial. However, one might expect to pick up a broad range of other signals that an intervention effect was likely. For example, it may be observed that team working and patient satisfaction had improved, as well as that the intervention was adopted and supported by staff. That one might have to rely on such proxies for QoL and life years has been referred to as “an inconvenient truth in service delivery research.”[5] It is important that grant awarding panels should not follow a one-size fits all approach to service delivery research, but rather that they should tailor their requirements according to the cost of intervention concerned. Likewise, they should be prepared to integrate many sources of evidence in their assessment of health benefit parameters, as argued elsewhere,[1] [6-8] and in the report of a recent study later in this issue of your News Blog.

— Richard Lilford, CLAHRC WM Director


  1. Sutton M, Birbeck SG, Martin G, Meacock R, Morris S, Sculpher M, Street A, Watson SI, Lilford RJ. Economic analysis of service and delivery interventions in health care. Health Serv Del Res. 2018; 6(5).
  2. Girling A, Liflord R, Cole A, Young T. Headroom Approach to Device Development: Current and Future Directions. Int J Technol Assess Health Care. 2015; 31(5): 331-8.
  3. Yao GL, Novielli N, Manaseki-Holland S, Chen YF, van der Klink M, Barach P, Chilton PJ, Lilford RJ, European HANDOVER Research Collaborative. Evaluation of a predevelopment service delivery intervention: an application to improve clinical handovers. BMJ Qual Saf. 2012; 21(s1):i29-i38.
  4. King D, Vlaev I, Everett-Thomas R, Fitzpatrick M, Darzi A, Birnbach DJ. “Priming” Hand Hygiene Compliance in Clinical Environments. Health Psychol. 2016; 35(1): 96-101.
  5. Lilford RJ, Chilton PJ, Hemming K, Girling AJ, Taylor CA, Barach P. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ. 2010; 341: c4413.
  6. Watson SI & Lilford RJ. Essay 1: Integrating multiple sources of evidence: a Bayesian perspective. In: Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Southampton (UK): NIHR Journals Library, 2016.
  7. Watson SI, Chen YF, Bion JF, Aldridge CP, Girling A, Lilford RJ; HiSLAC Collaboration. Protocol for the health economic evaluation of increasing the weekend specialist to patient ratio in hospitals in England. BMJ Open. 2018; 8(2): e015561.
  8. Lilford RJ, Girling AJ, Sheikh A, Coleman JJ, Chilton PJ, Burn SL, Jenkinson DJ, Blake L, Hemming K. Protocol for evaluation of the cost-effectiveness of ePrescribing systems and candidate prototype for other related health information technologies. BMC Health Serv Res. 2014; 14:314

Private Providers are Consulted More Often than Public Providers in Slums

This finding comes from a number of studies across many parts of the world, including:

  1. India [1] – where private providers were both preferred over public providers and consulted more often. Private providers were more accessible in terms of distance from residence.
  2. Kenyan maternity care [2] – women preferred private over public providers, even though the private providers were rated as ‘inappropriate’ by government.
  3. Dhaka slums [3] – this is an important study because it divides health facilities according to Ahmed’s classification.[4] Most commonly consulted were pharmacies (43%), followed by government hospitals (14%), then private hospitals (4%), independent medical practitioners (3%), informal providers (3%), and traditional healers (1%). Dissatisfaction was highest with government hospitals (25%) and lowest with informal providers and pharmacists.
  4. Accra’s Sodom and Gomorrah slum [5] – the facilities accessed were similar to those in Dhaka; 61% pharmacies and 33% hospitals. In this study lack of insurance was a major factor limiting access, while distance from facilities was not.
  5. Mumbai slums [6] ­– this study did not look at pharmacies specifically, but overall local private providers were the most widely used facilities. The use of public providers rose in proportion to the seriousness of the disorder, from 15% at low categories to 42% for serious illness, and 60% for maternal health.

One important conclusion from the above literature is that facilities should be classified to capture those inside a slum and external to it, and that pharmacies / drug stores should have their own stratum and not be elided with informal or private providers. Private allopathic providers should be classified as medical, other registered health professional (nurse / medical officer), community health worker (with formal links to the public service), and informal non-qualified provider. In studies that cross slum boundaries, multi-level modelling should be used to allow for correlations within clusters and avoid an ecological fallacy / Simpson’s paradox.[7]

The above studies are all based on population/household-based questionnaires. Another Dhaka based study takes a different approach [8] – instead of asking people who live in slums where they go for their health care, Adams and colleagues mapped health facilities across six urban slums. They found that 80% of the 1041 facilities identified in their spatial survey were privately operated. Unlike NGO- and government-funded clinics, private health care delivery clinics operate in the evenings. Only a third of staff in these private clinics have a medical qualification. Overall, the ‘density’ of health delivery points across the six slums was 1.5 per 10,000 of population. The average distance to a major government hospital offering outpatient services was 3km.

In our NIHR Global Health Research Unit on Improving Health in Slums we will be combining supply-side surveys of facilities with demand-side household surveys of use and satisfaction. We plan to go further by examining the socio-political structures that have determined patterns of provision and that may facilitate or impede the future development of a more accessible and high-quality service. We will then model the costs and benefits of alternative logistically and politically viable options using an iterative approach. In developing these models we will work closely with residents of slums and with those who control the purse strings.

— Richard Lilford, CLAHRC WM Director


  1. Banerjee A, Bhawalkar JS, Jadhav SL, Rathod H, Khedkar DT. Access to Health Services Among Slum Dwellers in an Industrial Township and Surrounding Rural Areas: A Rapid Epidemiological Assessment. J Family Med Prim Care. 2012; 1(1): 20-6.
  2. Fotso JC & Mukiira C. Perceived quality of and access to care among poor urban women in Kenya and their utilization of delivery care: harnessing the potential of private clinics. Health Policy Plan. 2012; 27: 505-15.
  3. Khan MMH, Grübner O, Krämer A. Frequently used healthcare services in urban slums of Dhaka and adjacent rural areas and their determinants. J Public Health. 2012; 34(2): 261-71.
  4. Ahmed SM, Tomson G, Petzold M, Kabir ZN. Socioeconomic status overrides age and gender in determining health-seeking behaviour in rural Bangladesh. Bull World Health Organ. 2005; 83: 109-17.
  5. Owusu-Ansah F, Tagbor H, Afi Togbe M. Access to health in city slum dwellers: The case of Sodom and Gomorrah in Accra, Ghana. Afr J Prim Health Care Fam Med. 2016; 8(1): a822.
  6. Naydenova E, Raghu A, Ernst J, Sahariah SA, Gandhi M, Murphy G. Healthcare choices in Mumbai slums: A cross-sectional study. Wellcome Open Research 2017; 2: 115.
  7. Lilford RJ. Simpson’s Paradox and Discrimination. NIHR CLAHRC West Midlands News Blog. 28 November 2014.
  8. Adams AM, Islam R, Ahmed T. Who serves the urban poor? A geospatial and descriptive analysis of health services in slum settlements in Dhaka, Bangladesh. Health Policy Plan. 2015; 30: i32-45.

Electronic Patient Notes and Patient Safety

In a previous news blog we drew attention to the psychological consequences of insinuating a computer into the clinician-patient consultation. The deleterious effect of computers on the clinician-patient interaction was published by the CLAHRC WM Director over three decades ago.[1] This concern has been corroborated in Robert Wachter’s recent book.[2]

In this news blog we will focus on another disadvantage of the current generation of electronic clinical notes. This is the threat to patient safety that results from the inchoate nature of electronic clinical record systems. In short, they do not reflect the heuristic patterns that determine appropriate clinical care. This failure to follow medical logic is most dangerous in the context of diagnosis.

While clinical diagnosis may sometimes be clear from a single episode of care, the correct diagnosis frequently depends on the pattern of data as it emerges over time. Just as an understanding of politics requires an understanding of history, so a clinical diagnosis frequently requires an understanding, not just of the current symptoms and signs, but also of their provenance.

Building on this safe premise, one has to conclude that the way clinical record systems are organised can either facilitate or hinder accurate diagnosis. Diagnostic errors are the single largest threat to patient safety [3]; it is all very well to promote evidence-based care and safe prescribing, but if the patient is on the wrong pathway, then they are heading for a big disaster. Diagnosis lies at the heart of clinical medicine and a system that impedes accurate diagnosis is likely to do more harm than good. A recent study of electronic notes carried out by Aziz Sheikh, in collaboration with CLAHRC WM, showed that big reductions in medication error could be achieved by means of electronic prescribing and decision support [in press]. However, careful qualitative research showed that the electronic prescribing systems disrupted the normal flow of knowledge, and that doctors had to implement numerous ‘work-arounds’. This way be dragons!

The problems of disorganised clinical information have been the subject of investigation for over half a century; Professor Laurence Weed’s system of problem-orientated notes inspired the CLAHRC WM Director when he was a young doctor (a long time ago)! The most urgent requirement in modern computing is not to saturate the health service with electronic records, but to develop them in a way that preserves the logic of medical practice. In the meantime we should rely on structured paper-based notes, such as those recommended by Rupert Fawdry,[5] and confine the use of computers to things that they really are good at, such as electronic prescribing and disseminating medical images.

— Richard Lilford, CLAHRC WM Director


  1. Brownbridge G, Lilford RJ, Tindale-Biscoe S. Use of a computer to take booking histories in a hospital antenatal clinic. Acceptability to midwives and patients and effects on the midwife-patient interaction. Med Care. 1988;26(5):474-87.
  2. Wachter R. The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age. New York, NY: McGraw-Hill Education. 2015.
  3. Singh H, Schiff GD, Graber ML, Onakpoya I, Thompson MJ. The global burden of diagnostic errors in primary care. BMJ Qual Saf. 2017; 26: 484-94.
  4. Lilford RJ. The WISDAM of Rupert Fawdry. NIHR CLAHRC West Midlands News Blog. 5 September 2014.

The Health Economics of Infertility Treatment

Recently I found myself holding forth on the above topic in a plenary talk at the International Federation of Fertility Societies combined meeting with the African Fertility Society in Kampala. I made the point that the health economics of infertility raises a number of issues that are not generally considered in the standard canon for health economic assessment of health technology assessments (HTA).[1] Four issues stand out:

  1. The benefits of infertility treatment are more difficult to capture on a single quality of life (QoL) scale than in the case for standard HTA.
  2. The standard practice of discounting benefits can be questioned.
  3. The beneficiaries are more diverse, potentially extending to many family members.
  4. The issue of whether the lifelong utility of the potential child should be include is controversial.

I shall briefly consider these in turn.

  1. Benefit – generic quality of life (QoL) scales do not seem up to the job. First, it is very difficult to capture the benefits over a lifetime. The ‘area under the curve’ is the important relevant quantity and this is not well captured in cross-sectional studies. Second, QoL deteriorates when a sub-fertile couple have a baby, as it does for fertile people. I discovered this many years ago in a collaborative study with the Health Economics department at the University of York (unpublished). This finding reinforces the importance of a lifetime perspective. Third, it is doubtful that maximisation of the dimensions captured in a generic QoL scale are the things that people wish to maximise when they decide to have children – there is a deeper purpose in play. So, a utility function based on a direct trade-off would be preferable to a standard generic QoL scale, such as the SF-12 or EQ-5D. This way, the respondent can take a lifetime perspective and factor in all the valued benefits and disbenefits of treatment. Torrance used a standard gamble on a large study of US citizens and measured a disutility of 0.07 (utility 0.93).[2] That is to say, the average respondent would run up to a 7% risk of death to enable them to have a first child. Such a standard gamble method would likely underestimate the utility loss for those who actually experience infertility for reasons David Arnold and I explicated elsewhere.[3] A perhaps better method to capture the benefit over a lifetime would be willingness-to-pay studies and here, in addition to studies at the population level (say using discrete choice methods), studies of revealed preferences are possible. This is because much IVF takes place in an entirely private market. This enables the ‘market clearing’ price for infertility services to be observed (ideally in relation to disposable income). The high proportions of disposable incomes infertile people allocate to infertility treatment, sometimes amounting to catastrophic losses,[4] provides some evidence that Torrance’s study underestimates the trade-offs people will make in order to have children.
  2. Choice of discount rate – The fact that benefits continue to accrue, and may increase, over time, suggests that discounting may not be normative. Given that disbenefits generally precede benefits, it makes little sense to discount from the point of intervention. That said, it is important to factor disbenefits of treatment and downstream costs into the analysis. Disbenefits include the cost and discomfort of treatment and knock-on costs, for example, resulting from an increased risk of prematurity. Conversely, there may be hidden benefits beyond the joys of parenthood – for example, in reduced Intimate Partner Violence.[5]
  3. Diverse beneficiaries – In ‘normal’ health economics, benefits are hypotheticated on the affected person, even though loved ones also stand to benefit. Loved ones benefit through the improved health of the affected person. Ignoring third party benefits can be condoned on the ‘level playing field’ principle – in a comparison across diseases of middle-age, beneficiaries of various alternative treatments are in a roughly similar position – they have similar numbers of loved ones on average. On this basis, the decision tree can be ‘pruned’. It could be argued that this argument breaks down when comparisons are made across generational lines. In the particular case of infertility, mothers and fathers get direct benefits, as do grandparents and others, not only through the ‘affected’ person, but directly from the child that results from the treatment. For instance, the father is just as much a beneficiary as the mother. Grandparents are not far behind – I can attest to that. On the other hand, factoring these beneficiaries into the equation seems to tilt the playing field too far the other way, i.e. towards infertility services. Factoring the benefits that accrue to all these people would weight services for children in general, and infertility services in particular, very strongly. This is a topic requiring more philosophical analysis and, perhaps, empirical investigation.
  4. What about the child who would otherwise not have existed – the question of the utility of the hypothetical lives is vexed. Certainty, no-one counts the utility loss from contraception, even when no later ‘replacement child’ is envisaged. On the other hand, the utility of neonatal survival is included in standard economic practice. My preference is not to include this utility, but I am hard-pressed to defend this on a bottom-up, philosophical basis. Richard Hare, the famous Oxford philosopher, did attempt such an analysis and his conclusions support my instinct. Certainly, including the lifetime utility of the child massively improves the cost-benefit ratio of infertility services,[6-8] and if the tax return from the child is included, then a treatment such as IVF becomes a ‘no-brainer’ since it ‘dominates’ – it saves money and yields benefit down to a very low success rate (<6% of so).[9]

What would happen if we:

  1. Accepted a utility function of 0.9 (close to that of Torrance).
  2. Ignored other beneficiaries, including the child?

We present such an analysis below. Even under these relatively conservative constraints, IVF is cost-effective in most countries, and could be cost-effective in LMICs if some new idea, such as incubation within the vagina, were used.

Say we gave infertility a disutility of 0.1 over 50 years, undiscounted.
Then, the utility in an infertile couple successfully treated = 5 QALYs undiscounted and 1.1 discounted at 3%.
Let’s say society will pay $100 per QALY.
Then a treatment with a 25% success rate can have a net cost of up to $125 undiscounted, but less than $28 discounted.

— Richard Lilford, CLAHRC WM Director

I thank Sheryl van der Poel for sending some of the references quoted in this article.


  1. Drummond MF. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press. 2005.
  2. Torrance GW. Measurement of Health State Utilities for Economic Appraisal. J Health Econ. 1986; 5: 1-30.
  3. Arnold D, Girling A, Stevens A, Lilford R. Comparison of direct and indirect methods of estimating health state utilities for resource allocation: review and empirical analysis. BMJ; 2009; 339: b2688.
  4. Wu AK, Odisho AY, Washington III SL, Katz PP, Smith JF. Out-of-Pocket Fertility Patient Expense: Data from a Multicenter Prospective Infertility Cohort. J Urology. 2014; 191(2): 427-32.
  5. Stellar C, Garcia-Moreno C, Temmerman M, van der Poel S. A systematic review and narrative report of the relationship between infertility, subfertility, and intimate partner violence. Int J Gynecol Obstet, 2016; 133: 3-8.
  6. Connolly MP, Pollard MS, Hoorens S, Kaplan BR, Oskowitz SP, Silber SJ. Long-term Economic Benefits Attributed to IVF-conceived Children: A Lifetime Tax Calculation. Am J Manag Care. 2008; 14(9): 598-604.
  7. Svensson A, Connolly M, Gallo F, Hägglund L. Long-term fiscal implications of subsidizing in-vitro fertilization in Sweden: A lifetime tax perspective. Scand J Pub Health. 2008; 36: 841-9.
  8. Fragoulakis V & Maniadakis N. Estimating the long-term effects of in vitro fertilization in Greece: an analysis based on a lifetime-investment model. Clinicoecon Outcomes Res. 2013; 5: 247-55.
  9. Baird DT, Collins J, Egozcue J, et al. Fertility and Ageing. Hum Reprod Update. 2005; 11(3): 261-76.

Traditional Healers and Mental Health

The case for traditional healers in mental health

There are two arguments for traditional healer involvement in mental health provision; one pragmatic and one theoretical. The pragmatic argument turns on the huge shortfall in human resources to deal with mental health problems in low- and middle-income countries (LMICs).[1] Traditional healers could make up for this shortage in human resources in the formal sector. A theoretical argument for the role of traditional healers turns on cultural factors. The argument here is that traditional healers are ideally placed to intervene in conditions with social origins, or when symptoms are coloured by cultural assumptions. Traditional healers, one might suppose, can tap into the beliefs and expectation of local people to reach parts of the mind that are simply inaccessible under a ‘medical model’. According to this argument modern medicine is the appropriate vehicle for the diagnosis and management for the conditions that are mainly of the body. It would be unwise, for example, to rely on traditional healers for the treatment of an acutely febrile child, or for provision of contraceptive advice. However, the traditional healer might be the appropriate first port of call for people with conditions of the mind.

The case against traditional healers in mental health

An argument against the above position is that the most serious types of mental health condition, psychotic illnesses, require modern pharmacotherapy, at least to stabilise patients. While all psychiatric conditions are of both brain and mind, psychotic conditions can be closer in form to those of standard medical diseases and the effects of properly targeted chemotherapy can be dramatic. There are many well documented cases where access to appropriate pharmacological therapy was denied or cruelly delayed while patients were treated unsuccessfully by traditional healers. From this perspective one should no more consult a traditional healer for a mental illness than for suspected malaria.

Reconciling the case for and against: a topic for investigation and research

On the one hand, traditional healers can offer culturally sensitive treatment for non-psychotic conditions, while on the other hand, severe mental illness requires medical services. It could be argued that traditional and modern medical services should be integrated so that traditional healers could treat the majority of patients, i.e. those with non-psychotic diseases, while allopathic clinicians treat the more severe cases. Moreover, different people have different preferences, and individuals may wish to receive care from both types of providers, even for the same illness. These would seem to be further arguments to integrate traditional and allopathic services within the same system and, indeed, in an integrated reimbursement system. Before implementing such a system it would surely be sensible to evaluate the effectiveness of traditional healers in the treatment of various psychiatric conditions and to ensure that, with the appropriate education, they would be able to refer cases that need medical treatment.

Philosophical problems in collaboration between traditional healing and modern medicine

The CLAHRC WM Director is keen to explore the relative effectiveness of traditional and allopathic treatments for non-psychotic mental illness but he is concerned that there may be irreconcilable philosophical differences in the traditional versus allopathic approach. This concern arises from different ontologies that underpin the different kinds of service. That is to say these traditions have different views on what counts as truth. Modern medical practice is very much a product of what might be called ‘enlightenment thinking’; practice built on an understanding of biological mechanisms / scientific explanations.[2] Such a world view is a far cry from the assumptions that underpin traditional healing, and which are guided by a set of traditional beliefs, often of a religious nature. So the question is whether it is possible to truly integrate systems with such different sets of underpinning assumptions? This is partly an empirical question – different systems could be examined to understand how well they can work together. The CLAHRC WM Director understands that moves are afoot to integrate allopathic medicine with traditional Chinese medicine in China, and in Ayurvedic medicine in India. It would be interesting to make independent studies of these systems. But in the meantime I would suggest a thought experiment. Let us imagine a proposed trial of rose-hip water vs. anti-depressant medication taking place in an integrated hospital. The allopathic practitioners present this as a placebo-controlled trial, while the traditional healers present this as a trail of two effective alternatives – the underlying belief systems determine how the treatments are presented. The CLAHRC WM Director suspects that it is very difficult to really integrate two systems based on very different philosophical premises. It is one thing to make irenic statements about mutual respect and so on, but another to supress tensions that seem likely to arise from fundamentally irreconcilable philosophical assumptions.

Living with contradictions

The question of integrating these different systems of thought is, perhaps, unresolvable. The systems have existed side by side for a hundred years or more. In high-income countries there is a thriving industry in complementary therapies and the list of alternative methods is almost too long to recite. Likewise traditional medicine and modern medicine have existed side by side quite happily in Africa, South Asia and China for many years. The populations in all these countries seem, on the whole, pretty savvy at working out which method is more appropriate for them in which condition. I have never heard of anyone going to a homeopath for their family planning needs. But systems co-existing in society is one thing, integrating them in common administrative and reimbursement systems is another. Every now and then there is an attempt to unite religion and science around a common purpose – the Lancet commission is currently involved in such a process.[3] [4] However, it may be the case that like religion and science; traditional and allopathic medicine can live happily side by side within the same community and within the same individual. Whether and how they can really be brought together in a structural / organisational sense, for example in the same institution or within the same reimbursement system, is a matter for analysis and exploration. One thing I am sure of is that policy should not be made as though this were a technical issue and without considering the very different world views that lie behind each type of provision. Maybe the best that can be accomplished is for the systems to become more aware of each other and cross-refer when necessary, but to continue to make their own independent contributions?

— Richard Lilford, CLAHRC WM Director


  1. Rathod S, Pinninti N, Irfan M, Gorczynski P, Rathod P, Gega L, Naeem F. Mental Health Service Provision in Low- and Middle-Income Countries. Health Serv Insights. 2017; 10:
  2. Spray EC. Health and Medicine in the Enlightenment. Jackon M (ed). The Oxford Handbook of the History of Medicine. 2011.
  3. Horton R. When The Lancet went to the Vatican. Lancet. 2017; 389: 1500.
  4. Lee N, Remuzzi G, Horton R. The Vatican-Mario Negri-Lancet Commission on the value of life. Lancet. 2017; 390: 1573.

50 Year Anniversary of the First Human Heart Transplant: Lessons for Today

On 3 December we commemorated the 50 year anniversary of the world’s first heart transplant. The operation took place in the early hours of a Saturday morning at the Groote Schuur hospital in Cape Town, South Africa. Christiaan Barnard sutured Denise Darvall’s donated heart into the chest of the recipient, Louis Washkansky. Barnard restarted the new heart with an electric shock and then tried to wean the recipient off the heart and lung machine. But the new heart could not take the strain and Washkansky had to go back on the machine. The second attempt also failed, but when the heart and lung machine was turned off for the third time the recipient’s blood pressure started to climb. It kept on climbing, and soon Denise Darvall’s small heart had taken over the perfusion of Louis Washkansky’s large frame. Later that morning the world woke to the news of the world’s first heart transplant. Looking back over fifty years what should we make of Barnard’s achievement?

The transplant in an historical perspective

The two decades preceding the heart transplant have sometimes been referred to as the golden age of medical discovery.[1] The transplant can be ‘fitted’ retrospectively as the culmination of this golden age just as Neil Armstrong’s moon walk, two years later, can be seen as the crowning achievement of the space race. They belong to a number of technical achievements, including the first “test tube” baby and the first man in space, which are emblematic of human progress. They generate great public interest and media attention, but differ from more fundamental intellectual discoveries, such as the double helix in DNA or Higgs boson, that are rewarded with Nobel prizes.

The heart transplant in the ‘heroic’ medical age

In his book ‘One Life’ Barnard provides an interesting cameo of the power and autonomy of the medical profession in his time.[2] He recalls writing up the routine operation note that must follow any surgical procedure. The anaesthetist, ‘Oz’, suggested that Dr Jacobus Burger, the hospital superintendent, should be informed. Barnard asked whether he should wake him so early in the morning, but Oz replied that the night’s events warranted such an intrusion. At first the befuddled Dr Burger, aware if work in the animal lab, thought that he was being informed about another heart transplant in dogs. However, even when he learned that the transplant involved a human heart, he cryptically thanked the surgeon and replaced the receiver. Nowadays, the idea of carrying out a procedure of such novelty, cost and risk without formal sanction would be unfathomable. The vignette from the doctor’s tearoom vividly illustrates how the relationship between the medical profession and the broader society has changed over one generation. Rene Amalberti argues [3] that many professions progressed through a heroic age in the twentieth century before gradually becoming more formalised and regulated – aviation followed a similar trajectory following Charles Lindbergh’s dramatic flight across the Atlantic in 1927.

Gradually changing ethical norms

The ethics of heart transplants relate mainly to organ donation. In ‘One Life’ Barnard describes the tense atmosphere in the operating room as the team waited for the donor heart to stop after turning off Darvall’s ventilator. In fact, they did not wait, and Barnard’s brother Marius has stated he persuaded Christiaan to stop the donor heart by injecting a concentrated dose of potassium in order to give Washkansky the best chance of survival. Today two different doctors need to independently carry out tests to confirm the donor is brain stem dead before the heart can be removed, as opposed to waiting for death by the whole-body standard, i.e. when there is brain death and the heart has stopped beating.

Public views of heart transplants, then and now

Following the operation the exhausted Barnard went home for a sleep. In the afternoon he returned to the hospital where he was surprised to find his route obstructed by a large crowd of reporters. He had unleashed a tide of publicity and acclaim that resonated for many decades, but dissenting voices were also heard. Some, notably Malcolm Muggeridge, the editor of Punch magazine, attacked the operation on the basis of a near mystical reverence for the human heart and to this Barnard had a succinct response: “it’s merely a pump.” Others worried about the allocation of scarce resources to such a high-tech solution when people were dying from malnutrition and malaria. Defence of the procedure came, albeit years later, from the economics profession when it was shown that the operation has a highly favourable cost-to-benefit ratio (at least in a high-income country).[4] The procedure not only extends life by many years on average, but greatly improves the quality of that life. In fact, patients feel much better from the moment they regain consciousness after the operation despite pain from the sternotomy. The operation is now uncontroversial and is performed routinely in high-income countries. It was long predicted that a mechanical pump would supplant the need for transplantation. Mechanical hearts have improved,[5] but they are largely seen as a bridge to transplantation, rather than a better alternative.

If Christiaan Barnard had not performed his operation, heart transplants would have developed anyway (the second transplant was carried out independently by Adrian Kantrowitz in the USA on 6 December). I was a school boy with hopes of getting into medical school when Washkansky received his new heart. I was among the many millions who were swept up in the wonder of the event and it still stirs my imagination half a century later. And my family knows that I wish to donate my own heart if the circumstances arise.

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ. Future Trends in NHS. NIHR CLAHRC West Midlands. 25 November 2016.
  2. Barnard C & Pepper CB. One Life. Toronto, Canada: Macmillan; 1969.
  3. Amalberti R. The paradoxes of almost totally safe transportation systems. Saf Sci. 2001; 37(2-3): 109-26.
  4. O’Brien BJ, Buxton MJ, Ferguson BA. Measuring the effectiveness of heart transplant programmes: Quality of life data and their relationship to survival analysis. J Chron Dis. 1987; 40(s1): s137-53.
  5. Girling AJ, Freeman G, Gordon JP, Poole-Wilson P, Scott DA, Lilford RJ. Modeling payback from research into the efficacy of left-ventricular assist devices as destination therapyInt J Technol Assess Health Care. 2007; 23(2): 269-77.

Is Research Productivity on the Decline Internationally?

I have written previously on the so-called ‘golden age of medical research,’ [1] which coincides roughly with the first two decades of my life – 1950-1970. The premise of a golden age entails the conclusion that it is followed by a less spectacular age where marginal returns are lower per unit of input – say per researcher. So, where does the truth lie – is research becoming ever more efficient, or is the productivity of research declining? This subject has been carefully examined by a number of scholars, most recently by Bloom and others.[2] First they looked at aggregate supply of researchers and economic output across the US economy, and they found a relationship that looks like this:

091 DCB Figure 1

So, productivity per researcher appears to decline with time and does so quite rapidly – the graph uses log scales. The drop in unit productivity has been fully compensated by growth in the number of researchers.

Given the obvious problems of studying this phenomenon at the aggregate level, the researchers turn to individual topics, such as number of transistors packed onto a single chip. It turns out that keeping Moore’s law going takes a rapidly increasing number of researchers. However, diminishing returns are not just observed in electronics, the authors found the same phenomenon in agriculture and medicine. Research productivity in the pharmaceutical industry is one-tenth of what it was in 1970, and mortality gains have peaked in cancer and in heart disease. To some extent one can see this effect in the number of authors of medical papers, such as those in genetic epidemiology – they often run literally into hundreds. It would appear that ideas really are getting harder to find and/or when found they portend smaller gains.

I have previously made the obvious point that improved care reduces the headroom for future improvements.[3] Of course, economic growth and further improvement in health still turn on new knowledge and technology without which the supply-side of the economy must stagnate. The phenomenal growth of some emerging economies has been possible because of the non-rivalrous nature of previous discoveries made elsewhere. But we need to continue to advance for all that advances are hard to make. One of these advances concerns making optimal use of existing knowledge, and that is where CLAHRCs come into their own – we trade in knowledge about knowledge.

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ. Future Trends in NHS. NIHR CLAHRC West Midlands. 25 November 2016.
  2. Bloom N, Jones CI, Van Reenen J, Webb M. Are Ideas Getting Harder to Find? Centre for Economic Performance Discussion Paper No. 1496. 2017.
  3. Lilford RJ. Patient Involvement in Patient Safety: Null Result from a High Quality Study. NIHR CLAHRC West Midlands. 18 August 2017.