Beyond Logic Models

It has become common in systematic reviews and, increasingly, in research papers to present a logic model – a trend we entirely applaud. Logic models are usually “graphical depictions of processes” that “describe logical linkages among program resources, activities, outputs… and outcomes.” The ultimate purpose is to depict ‘if-then‘ causal relationships between elements of the programme.[1] Good examples of logic models can be found in the Cochrane Reviews on slum regeneration,[2] and housing improvements.[3] In health care, the logic model explicates the putative causal pathway linking cause to its ultimate effect at the patient/client level. It may be said to encapsulate programme theory. Here is a simple logic model, adapted from one published previously,[4] relating to the effects of electronic prescribing systems.

062 DCB - Beyond Logic Models Fig 1

The advantage of such logic models are manifold – they explicate theory, provide information on salient endpoints,[5] clarify research questions, and improve communication.[6]

While logic models lay bare the nodes in a causal chain where data may be collected, they do not entail a method to synthesise all this information. The default approach in healthcare is to synthesise information implicitly – the literature refers to ‘triangulation’. However, the mental working of implicit ‘triangulation’ are opaque, at least to others. Equally important, they do not yield estimates of the effect of interest to a decision maker – quantities that are required to inform decision models (such as health economic models). But this limitation can be overcome by making use of a method that is well known in other disciplines – development economics, molecular biology, agriculture to name but a few. We refer here to Bayesian networks. A Bayesian network is a representation of the joint probability distribution and conditional independence assumptions. It enables information collected at each node in the chain to be synthesised to estimate the effects of the intervention on outcomes of interest. Qualitative information can be incorporated through probability distributions elicited from experts. It is even possible to adjust for bias in such a model by specifying a probability distribution for bias (which can be used to ‘update’ quantitative estimates) according to the model of Turner and Spiegelhalter.[7] External evidence, say from the literature, can also be incorporated, again by making use of elicited probability densities. The interactions between nodes do not have to be linear, but causality is in one direction only. Clearly, if you think that reverse causality will apply to a material degree within a given time-frame, then a more complex model, such as a dynamic event simulation, would be necessary.

We have systematically reviewed the literature of service delivery / health services research and find no examples where the powerful technique of Bayesian networking has been used or even advocated,[8] (apart from in our own papers). It is not mentioned, for example, in important articles on the systematic reviews of complex interventions in health services.[9] [10] Since the method has been used to good effect in other disciplines involving complex interventions, we think the time is propitious to explore its use in our field.

— Richard Lilford, CLAHRC WM Director

References:

  1. McCawley PF. The Logic Model for Program Planning and Evaluation. CIS 1097. Moscow, ID: University of Idaho, 1997.
  2. Turley R, Saith R, Bhan N, Rehfuess E, Carter B. Slum upgrading strategies involving physical environment and infrastructure interventions and their effects on health and socio-economic outcomes. Cochrane Database Syst Rev. 2013; 1: CD010067.
  3. Thomson H, Thomas S, Sellstrom E, Petticrew M. Housing improvements for health and associated socio-economic outcomes. Cochrane Database Syst Rev. 2013; 2: CD008657.
  4. Watson SI & Lilford RJ. Essay 1: Integrating multiple sources of evidence: a Bayesian perspective. In: Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Southampton (UK): NIHR Journals Library, 2016.
  5. Lilford RJ, Chilton PJ, Hemming K, et al. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ. 2010; 341: c4413.
  6. Anderson L, Petticrew M, Rehfuess E, et al. Using logic models to capture complexity in systematic reviews. Res Synth Methods. 2011; 2(1): 33-42.
  7. Turner RM, Spiegelhalter DJ, Smith GC, Thompson SG. Bias modelling in evidence synthesis. J R Stat Soc Ser A Stat Soc. 2009; 172(1):21-47.
  8. Chen Y-F, Uthman OA, Leamon S, Watson SI, Lilford RJ. Potential use of Bayesian networks in healthcare service delivery and quality improvement research. [In preparation].
  9. Petticrew M, Anderson L, Elder R, et al. Complex interventions and their implications for systematic reviews: a pragmatic approachJ Clin Epidemiol. 2013; 66(11): 1209-14.
  10. Petticrew M, Rehfuess E, Noyes J, et al. Synthesizing evidence on complex interventions: how meta-analytical, qualitative, and mixed-method approaches can contribute. J Clin Epidemiol. 2013; 66(11): 1230-43.
Advertisements

2 thoughts on “Beyond Logic Models”

  1. Hmmm, I think its a bit simplistic to reduce to mathematics quite yet. Are you aware of the ‘assumption-querying approach?’ I think you’d like the fact that the causal pathways are laid bare, but it uses mixed methods and tries to examine the problem of ‘context’ (ie what works this much in this way in context A, might work that much in opposite way in context B). I will send a report with a nice picture to you on email.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s