Traditional epidemiology was concerned with the causes of disease – many of the great medical discoveries, from malaria to the effects of smoking, can be credited to classical epidemiology. The subject continues to make great strides thanks to modern developments, such as genome-wide association studies and Mendelian randomisation. Approximately 70 years ago Austin Bradford Hill ushered in the days of clinical epidemiology. Epidemiological methods were used to study the diagnosis and treatment of disease, rather than simply the causes and prognosis. Randomised trials and systematic reviews became the ‘stock in trade’ of the clinical epidemiologist.
As more and more effective treatments were discovered, people started to worry about large variations in practice and in the quality of care. Service delivery health research and the ‘quality movement’ were born. Researchers naturally felt the need to measure quality. Progress was slow, however. First, quality improvement was initially dominated by management research; a subject that does not have a strong tradition of measurement, as I have reported elsewhere. Second, the constructs that quality researchers were dealing with were much harder to measure than clinical outcomes. For example, an attempt was made to correlate the safety culture with standardised mortality rates across intensive care units. The result was null, but this might have resulted entirely from measurement error; mortality rates suffer from unavoidable signal to noise problems, while the active ingredient in culture is hard to capture in a measurement. As the subject of the quality of care seemed to become bogged down with measurement issues, the patient safety movement became dominant. Initially people focused on psychology and organisational science. However, no science can mature without, at some point, making its central concepts quantifiable. As Galileo (allegedly) said, “Measure what can be measured, and make measurable what cannot be measured.” So it became necessary to try to measure safety, and all the problems of quality measurement re-surfaced.
Most sensible people now realise that impatience does more harm than good; shortcuts lead nowhere and we simply have to work away, measuring and mitigating measurement error as bast we can. As stated, and as I have argued elsewhere, clinical outcomes are insensitive to many service interventions. This is a lesson that those of us with a background in classical or clinical epidemiology have been slow to learn. Trying to copy clinical epidemiology, and to rely entirely on clinical endpoints, has driven service delivery research into two camps – qualitative researchers who eschew quantification, and quantitative researchers who want to apply rules of evidence that served them well in clinical research. However, there really is a third way. This method is based on observations across the causal chain linking intervention to clinical outcome. I have long argued that it is the pattern of data (qualitative and quantitative) across a causal chain that should be analysed. Since then, people have started to pay attention, not just to the outcome at the human level, but also to mediating variables. More recently still, I have argued for the use of Bayesian networks to synthesise information from the causal chain in a particular study, along with evidence from reviews in salient topics. Note that while coming from the same, realist, epistemology as ‘mixed-methods’ research, mediator variable analysis and Bayesian networking developed mixed-methods to another level, since they enable data of different sorts to be captured in a clinical outcome of sufficient importance to populate a decision model. The use of proxy outcomes acquired a bad reputation in clinical epidemiology. However, carrying this idea over into service delivery research is extremely limiting. It is also unscientific, since science is dependent on induction, and induction can only be carried out if the causal mechanisms behind the results obtained are understood.
— Richard Lilford, CLAHRC WM Director
- Hill AB. The environment and disease: Association or causation? Proc R Soc Med. 1965; 58(5): 295-300.
- Lilford RJ, Dobbie F, Warren R, Braunholtz D, Boaden R. Top-rated British business research: Has the emperor got any clothes? Health Serv Manage Res. 2003; 16(3): 147-54.
- Girling AJ, Hofer TP, Wu J, Chilton PJ, Nicholl JP, Mohammed MA, Lilford RJ. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study. BMJ Qual Saf. 2012; 21(12): 1052-6.
- Mannion R, Davies H, Konteh H, Jung T, Scott T, Bower P, Whalley D, McNally R, McMurray R. Measuring and Assessing Organisational Culture in the NHS (OC1). 2008.
- Lilford RJ, Chilton PJ, Hemming K, Girling AJ, Taylor CA, Barach P. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ 2010; 341: c4413.
- Watson SI & Lilford RJ. Essay 1: Integrating Multiple Sources of Evidence: a Bayesian Perspective. In: Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Southampton (UK): NIHR Journals Library, 2016.