Important Notice: A New Online Repository for Research Results

Such a repository has now been launched – The Wellcome-Gates repository, established by the world’s second largest and largest medical research charities respectively, and run by a firm called F1000.[1] Research funded by Gates can only be published here. This is another big milestone in the gradual shake-up of the scientific publication sector.

— Richard Lilford, CLAHRC WM Director

Reference:

  1. The Economist. The findings of medical research are disseminated too slowly. The Economist. 25 March 2017.

Wrong Medical Theories do Great Harm but Wrong Psychology Theories are More Insidious

Back in the 1950s, when I went from nothing to something, a certain Dr Spock bestrode the world of child rearing like a colossus. Babies, said Spock, should be put down to sleep in the prone position. Only years later did massive studies show that children are much less likely to experience ‘cot death’ or develop joint problems if they are placed supine – on their backs. Although I survived prone nursing to become a CLAHRC director, tens of thousands of children must have died thanks to Dr Spock’s ill-informed theory.

So, I was fascinated by an article in the Guardian newspaper, titled ‘No evidence to back the idea of learning styles’.[1] The article was signed by luminaries from the world of neuroscience, including Colin Blakemore (who I knew, and liked, when he was head of the MRC). I decided to retrieve the article on which the Guardian piece was mainly based – a review in ‘Psychological Science in the Public Interest’.[2]

The core idea is that people have clear preferences for how they prefer to receive information (e.g. pictorial vs. verbal) and that teaching is most effective if delivered according to the preferred style. This idea is widely accepted among psychologists and educationalists, and is advocated in many current textbooks. Numerous tests have been devised to diagnose a person’s learning style so that their instruction can be tailored accordingly. Certification programmes are offered, some costing thousands of dollars. A veritable industry has grown up around this theory. The idea belongs to a larger set of ideas, originating with Jung, called ‘type theories’; the notion that people fall into distinct groups or ‘types’, from which predictions can be made. The Myers-Briggs ‘type’ test is still deployed as part of management training and I have been subjected to this instrument, despite the fact that its validity as the basis for selection or training has not been confirmed in objective studies. People seem to cling to the idea that types are critically important. That types exist is not the issue of contention (males/females; extrovert/introvert), it is what they mean (learn in different ways; perform differently in meetings) that is disputed. In the case of learning styles the hypothesis of interest is that the style (which can be observed ex ante) meshes with a certain type of instruction (the benefit of which can be observed ex post). The meshing hypothesis holds that different modes of instruction are optimal for different types of person “because different modes of presentation exploit the specific perceptual and cognitive strengths of different individuals.” This hypothesis entails the assumption that people with a certain style (based, say on a diagnostic instrument or ‘tool’) will experience better educational outcomes when taught in one way (say, pictorial) than when taught in another way (say, verbal). It is precisely this (‘meshing’) hypothesis that the authors set out to test.

Note then that finding that people have different preferences does not confirm the hypothesis. Likewise, finding that different ability levels correlate with these preferences would not confirm the hypothesis. The hypothesis would be confirmed by finding that teaching method 1 is more effective than method 2 in type A people, while teaching method 2 is more effective than teaching method 1 in type B people.

The authors find, from the voluminous literature, only four studies that test the above hypothesis. One of these was of weak design. The three stronger studies provide null results. The weak study did find a style-by-treatment interaction, but only after “the outliers were excluded for unspecified reasons.”

Of course, the null results do not exclude the possibility of an effect, particularly a small effect, as the authors point out. To shed further light on the subject they explore related literatures. First they examine aptitude (rather than just learning style preference) to see whether there is an interaction between aptitude and pedagogic method. Here the literature goes right back to Cornbach in 1957. One particular hypothesis was that high aptitude students fare better in a less structured teaching format, while those with less aptitude fare better where the format is structured and explicit. Here the evidence is mixed, such that in about half of studies, less structure suits high ability students, while more structure suits less able students – one (reasonable) interpretation for the different results is that there may be certain contexts where aptitude/treatment interactions do occur and others where they do not. Another hypothesis concerns an aspect of personality called ‘locus of control’. It was hypothesised that an internal locus of control (people who incline to believe their destiny lies in their own hands) would mesh with an unstructured format of instruction and vice versa. Here the evidence, taken in the round, tends to confirm the hypothesis.

So, there is evidence (not definitive, but compelling) for an interaction between personality and aptitude and teaching method. There is no such evidence for learning style preference. This does not mean that some students will need an idea to be explained one way while others need it explained in a different way. This is something good teachers sense as they proceed, as emphasised in a previous blog.[3] But tailoring your explanation according to the reaction of students is one thing, determining it according to a pre-test is another. In fact, the learning style hypothesis may impede good teaching by straightjacketing teaching according to a pre-determined format, rather than encouraging teachers to adapt to the needs of students in real time. Receptivity to the expressed needs of the learner seems preferable to following a script to which the learner is supposed to conform.

And why have I chosen this topic for the main News Blog article? Two reasons:

First, it shows how an idea may gain purchase in society with little empirical support, and we should be ever on our guard – the Guardian lived up to its name in this respect!

Second, because health workers are educators; we teach the next generation and we teach our peers. Also, patient communication has an undoubted educational component (see our previous main blog [4]). So we should keep abreast of general educational theory. Many CLAHRC WM projects have a strong educational dimension.

— Richard Lilford, CLAHRC WM Director

References:

  1. Hood B, Howard-Jones P, Laurillard D, et al. No Evidence to Back Idea of Learning Styles. The Guardian. 12 March 2017.
  2. Pashler H, McDaniel M, Rohrer D, Bjork R. Learning Styles: Concepts and Evidence. Psychol Sci Public Interest. 2008; 9(3): 105-19.
  3. Lilford RJ. Education Update. NIHR CLAHRC West Midlands News Blog. 2 September 2016.
  4. Lilford RJ. Doctor-Patient Communication in the NHS. NIHR CLAHRC West Midlands News Blog. 24 March 2017.

Can Thinking Make It So?

When we think of risk factors for mortality we properly think behaviours (e.g. smoking / obesity) or genetics (e.g. family history). What about psychological factors – can unhappiness increase your risk of risk of cancer? Well, Batty and colleagues [1] have tackled this problem as follows:

  1. They assembled 16 prospective cohort studies where behaviours and psychological state had been measured and in which participants were followed up to see if cancer developed.
  2. They obtained the raw data and obtained an individual patient meta-analysis.
  3. They adjusted for the usual things known to increase risk of cancer (obesity, smoking, etc).
  4. They calculated relative risk of cancer according to antecedent psychological state.

They found a positive correlation between psychological distress and risk of cancer. But causality might have run the other way – (occult) cancers may have been the cause of psychological distress, not the other way round. So:

  1. They ‘left censored’ the data, thereby widening the gap between the point in time where the psychological state was measured and the point where cancer supervened.

The association between psychological state and cancer death persisted, even when they were separated by many years. What is the explanation?

  1. Failure to fully control for all behaviours (although behaviour could be the mechanism through which the cancer risk is increased in people with depression, in which case they ‘over-controlled’).
  2. Reduced natural killer cell function.
  3. Increased steroid levels, which can apparently affect DNA repair in some way.
  4. Some mechanism yet to be discovered.

In any event, the findings are intriguing, for all that practical implications may be limited.

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Batty GD, Russ TC, Stamatakis E, Kivimäki M. Psychological distress in relation to site specific cancer mortality: pooling of unpublished data from 16 prospective cohort studies. BMJ. 2017; 356: j108.

Most Highly Cited Systematic Reviews in Service Delivery / Health Services Research

A few years ago CLAHRC WM affiliates Ola Uthman and Aileen Clarke published a paper on Citation Classics Among Systematic Reviews.[1] The top seven cited papers were all methodological articles and the leader of the pack is DerSimonian and Laird’s classic 1986 paper with 7,308 citations.[2] The most highly cited clinical article, weighing in at number 8, is a 2002 Lancet paper on statins by Colin Baigent and colleagues.[3] Thirteen of the top 100 cited systematic reviews dealt with health services / service delivery research or with a cognate psychological topic – these are listed in the Table.

18 Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients – A meta-analysis of prospective studies. JAMA. 1998; 279: 1200-5.
22 Barrick MR, Mount MK. The Big Five Personality Dimensions and Job-Performance – A Metaanalysis. Pers Psychol. 1991; 44: 1-26.
23 Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing Physician Performance – A Systematic Review of the Effect of Continuing Medical-Education Strategies. JAMA. 1995; 274: 700-5.
26 Grimshaw JM, Russell IT. Effect of Clinical Guidelines on Medical-Practice – A Systematic Review of Rigorous Evaluations. Lancet 1993, 342: 1317-22.
31 Armitage CJ, Conner M. Efficacy of the theory of planned behaviour: A meta-analytic review. Br J Soc Psychol. 2001, 40: 471-99.
45 Bero LA, Grilli R, Grimshaw JM, et al. Getting research findings into practice – Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ. 1998, 317: 465-8.
48 Oxman AD, Thomson MA, Davis DA, Haynes RB. No Magic Bullets – A Systematic Review of 102 Trials of Interventions to Improve Professional Practice. CMAJ. 1995, 153: 1423-31.
61 Sheppard BH, Hartwick J, Warshaw PR. The Theory of Reasoned Action – A Meta-Analysis of Past Research with Recommendations for Modifications and Future-Research. J Cons Res. 1988; 15: 325-43.
70 Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A Comparison of Results of Meta-analyses of Randomized Control Trials and Recommendations of Clinical Experts. Treatments for Myocardial Infarction. JAMA. 1992; 268: 240-8.
94 Claxton AJ, Cramer J, Pierce C. A systematic review of the associations between dose regimens and medication compliance. Clin Ther. 2001; 23: 1296-310.
96 Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes – A systematic review. JAMA. 1998; 280: 1339-46.
97 Deci EL, Koestner R, Ryan RM. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol Bull. 1999; 125: 627-68.
99 Kluger AN, DeNisi A. The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996; 119: 254-84.

The top cited HS&DR article in our series (18th overall) is the meta-analysis by Lazarou et al. on adverse drug events among hospitalised patients, published in JAMA back in 1998.

— Richard Lilford, CLAHRC WM Director

References:

  1. Uthman OA, Okwundu CI, Wiysonge CS, Young T, Clarke A. Citation Classics in Systematic Reviews and Meta-Analyses: Who Wrote the Top 100 Most Cited Articles? PLoS One. 2012; 8(10): e78517.
  2. DerSimonian R, & Laird N. Metaanalysis in Clinical-Trials. Control Clin Trials. 1986; 7: 177-88.
  3. Baigent C, Keech A, Kearney PM, et al. Efficacy and safety of cholesterol-lowering treatment: prospective meta-analysis of data from 90,056 participants in 14 randomised trials of statins. Lancet. 2005; 366: 1267-78.

Promoting hygienic weaning-food handling practices through a community based programme: protocol for a cluster-randomised controlled trial in rural Gambia.

Summary: Thirty villages were the unit of randomisation for this parallel clustered controlled trial in the Gambia, each with 20 randomly selected mothers with 6-24 month old children. A community-wide intervention was delivered by a team of four who visited each village during four intensive intervention activity days in 25 days, which involved performing arts, competitions and community mobilisation. The intervention used existing health systems and village/cultural structures. The primary outcome was the observed difference between the intervention and control in the mean proportion of all five key food-related behaviours (Table 1) versus all opportunities for performing the behaviours during the observation period (hereafter called 5-behaviours). Secondary outcomes included microbiological contamination of food and water for child’s consumption; the prevalence of diarrhoea and respiratory diseases; and mothers’ reporting of diarrhoea admission. Two random cross-sectional samples were taken to measure baseline characteristics and outcomes: one before randomisation, and the other 6 months post-intervention.

Trial registration: The trial was registered on the 17th October 2014 with the Pan African Clinical Trial Registry in South Africa with number PACTR201410000859336.

Keywords: cluster randomised controlled trial, diarrhoea, pneumonia, behaviour change, weaning-food, hygiene, food preparation, community intervention, performing arts, dramatic arts, motivational drives, scalability, Africa.

The full protocol is available to download here: Protocol – Manjang, et al. Promoting Hygienic Weaning-Food Handling Practices. 2017. (PDF viewer required).

Effects of Vitamin D Supplements

Bolland and colleagues have written a lovely summary of the evidence on the effects of vitamin D supplements, with or without calcium, on health.[1] Their careful and comprehensive systematic overview based on a large sample, and providing narrow confidence limits, finds that there is no evidence that vitamin D, with or without calcium, reduces the risk of fractures in elderly people with no known bone disease. It is, as expected, efficacious in people with established osteomalacia. Systematic reviews of lower quality or based on per protocol analyses, tend to find the more optimistic results, but the data, taken in the round, yield a null result. The reviewers find that additional research is unlikely to further clarify the issue, as an effect of more than a 10% reduction in fracture has been ‘excluded’ by the existing studies. From a Bayesian perspective, further data are unlikely to have much effect on credible limits. The studies do not find any evidence that calcium plus vitamin D have either harmful or beneficial effects on the other (non-skeletal) outcomes, such as cancer or heart disease. Perhaps this is an example of the horizon of science; science cannot prove a null result, merely exclude a positive or negative effect beyond certain limits. We will never know everything, but let’s just forget about the use of vitamin D and calcium as prophylaxis in healthy people as any benefit must be nugatory – less than 10% relative risk reduction, which equates to a very small absolute reduction.

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Bolland MJ, Leung W, Tai V, et al. Calcium intake and risk of fracture: systematic review. BMJ. 2015; 350: h4580.

 

On Foetal Growth Charts – the WHO May Have Adopted the Correct Policy for the Wrong Reason

The journal ‘Science’ reports a controversy over two studies of foetal growth across countries [1] – the first study showed very similar growth rates across eight countries (Brazil, Italy, Oman, UK, USA, China, India, Kenya).[2] They conclude that a common threshold should be used in countries to identify slow-growing foetuses. The second study looks only at the socio-economically advantaged populations across ten countries, ranging from Norway and Denmark to India and Egypt.[3] It finds markedly different rates across countries among socio-economically advantaged segments of the population. So that would suggest the use of country-specific thresholds.
I am not so sure – I question the assumption that the search for the growth-retarded foetus should be based on a fixed proportion of the foetal population – say the slowest growing 5%. The risk of stillbirth is higher in the countries with slower foetal growth (e.g. India and Egypt), than in those with higher growth rates (e.g. Norway and Denmark). So the cut-off threshold for foetal growth as a screening test should, logically, be set at a higher point in high-risk countries than in lower-risk countries. If it is set to identify the ‘bottom’ 5% in low-risk countries it should be set at, say, 10% in high-risk countries. This suggests that the WHO (which recommends a universal chart on the basis of the first study above) has the correct solution for the wrong reason. The universal chart will identify a higher proportion of still-births in the high-risk countries – just what one would want.

— Richard Lilford, CLAHRC WM Director

References:

  1. de Vrieze J. Big studies clash over fetal growth rates. Science. 2017; 355(6323): 336.
  2. Papageorghiou AT, Ohuma EO, Altman DG, et al. International standards for fetal growth based on serial ultrasound measurements: the Fetal Growth Longitudinal Study of the INTERGROWTH-21stProject. Lancet. 2014; 384: 869-79.
  3. Kiserud T, Piaggio G, Carroli G, et al. The World Health Organization Fetal Growth Charts: A Multinational Longitudinal Study of Ultrasound Biometric Measurements and Estimated Fetal Weight. PLoS Medicine. 2017.

Fine Dining and Fine Hygiene are Negatively Correlated

A recent study shows that restaurants rated highly in food guides are associated with a greater overall risk of foodborne gastrointestinal diseases outbreaks than your run-of-the-mill restaurant.[1] However, the ‘high-end’ restaurants also score more highly on the Food Agency Inspection visits. So why do the posh restaurants generate more GI diseases than their more mundane peers despite better hygiene in the restaurants with the best food? The high disease risk in highly rated restaurants probably comes from the nature of the food served (e.g. oysters) and cooking methods (e.g. low temperatures to produce chicken liver parfait). So the risk is real, but worth running!

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Kanagarajah S, Mook P, Crook P, Awofisayo-Okuyelu A, McCarthy N. Taste and Safety: Is the Exceptional Cuisine Offered by High End Restaurants Paralleled by High Standards of Food Safety? PLoS Curr Outbreaks. 2016.

Publishing Health Economic Models

It has increasingly become de rigueur – if not necessary – to publish the primary data collected as part of clinical trials and other research endeavours. In 2015 for example, the British Medical Journal stipulated that a pre-condition of publication of all clinical trials was the guarantee to make anonymised patient-level data available on reasonable request.[1] Data repositories, from which data can be requested such as the Yoda Project, and from which data can be directly downloaded such as Data Dryad provide a critical service for researchers wanting to make their data available and transparent. The UK Data Service also provides access to an extensive range of quantitative and, more recently, qualitative data from studies focusing on matters relating to society, economics and populations. Publishing data enables others to replicate and verify (or otherwise) original findings and, potentially, to answer additional research questions and add to knowledge in a particularly cost-effective manner.

At present, there is no requirement for health economic models to be published. The ISPOR-SMDM Good Research Practices Statement advocates publishing of sufficient information to meet their goals of transparency and validation.[2] In terms of transparency, the Statement notes that this should include sufficiently detailed documentation “to enable those with the necessary expertise and resources to reproduce the model”. The need to publish the model itself is specifically refuted, using the following justification: “Building a model can require a significant investment in time and money; if those who make such investments had to give their models away without restriction, the incentives and resources to build and maintain complex models could disappear”. This justification may be relatively hard to defend for “single-use” models that are not intended to be reused. Although the benefits of doing so are limited, publishing such models would still be useful if a decision-maker facing a different cost structure wanted to evaluate the cost-effectiveness of a specific intervention in their own context. The publication of any economic model would also allow for external validation which would likely be stronger than internal validation (which could be considered marking one’s own homework).

The most significant benefits of publication are most likely to arise from the publication of “general” or “multi-application” models because those seeking to adapt, expand or develop the original model would not have to build it from scratch, saving time and money (recognising this process would be facilitated by the publication of the technical documentation from the original model). Yet it is for these models that not publishing gives developers a competitive advantage in any further funding bids in which a similar model is required. This confers partial monopoly status in a world where winning grant income is becoming ever more critical. However, I like to believe most researchers also want to maximise the health and wellbeing of society: am aim rarely achieved by monopolies. The argument for publication gets stronger when society has paid (via taxation) for the development of the original model. It is also possible that the development team benefit from publication through increased citations and even the now much sought after impact. For example, the QRISK2 calculator used to predict cardiovascular risk is available online and its companion paper [3] has earned Julia Hippisley-Cox and colleagues almost 700 citations.

Some examples of published economic models exist, such as a costing model for selection processes for speciality training in the UK. While publication of more – if not all – economic models is not an unrealistic aim, it is also necessary to respect intellectual property rights. We welcome your views on whether existing good practice for transparency in health economic modelling should be extended to include the model itself.

— Celia Taylor, Associate Professor

References:

  1. Loder E, & Groves T. The BMJ requires data sharing on request for all trials. BMJ. 2015; 350: h2373.
  2. Eddy DM, Hollingworth W, Caro JJ, et al. Model transparency and validation: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force–7. Med Decis Making. 2012; 32(5): 733-43.
  3. Hippisley-Cox J, Coupland C, Vinogradova Y, et al. Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ. 2008; 336(7659): 1475-82.

Doctor-Patient Communication in the NHS

Andrew McDonald (former Chief Executive of Independent Parliamentary Standards Authority) was recently asked by the Marie Curie charity to examine the quality of doctor-patient communication in the NHS, as discussed on BBC Radio 4’s Today programme on 13 March 2017 (you can listen online). His report concluded that communication was woefully inadequate and that patients were not getting the clear and thorough counselling that they needed in order to understand their condition and make informed choices about options in their care. Patients need to understand what is likely to happen to them, and not all patients with the same condition will want to make the same choice(s). Indeed my own work [1] is part of a large body of research, which shows that better information leads to better knowledge, which in turn affects the choices that patients make. Evidence that the medical and caring professions do not communicate in an informative and compassionate way is therefore a matter of great concern.

However, there is a paradox – feedback from patients, that communication should lie at the heart of their care, has not gone unheard. For instance, current medical training is replete with “communication skills” instruction. Why then do patients still feel dissatisfied; why have matters not improved radically? My diagnosis is that good communication is not mainly a technical matter. Contrary to what many people think, the essence of good communication does not lie in avoiding jargon or following a set of techniques – a point often emphasised by my University of Birmingham colleague John Skelton. These technical matters should not be ignored – but they are not the nub of the problem.

In my view good communication requires effort, and poor communication reflects an unwillingness to make that effort; it is mostly a question of attitude. Good communication is like good teaching. A good communicator has to take time to listen and to tailor their responses to the needs of the individual patient. These needs may be expressed verbally or non-verbally, but either way a good communicator needs to be alive to them, and to respond in the appropriate way. Sometimes this will involve rephrasing an explanation, but in other cases the good communicator will respond to emotional cues. For example a sensitive doctor will notice if, in the course of a technical explanation, a patient looks upset – the good doctor will not ignore this cue, but will acknowledge the emotion, invite the patient to discuss his or her feelings, and be ready to deal with the flood of emotion that may result. The good doctor has to do emotional work, for example showing sympathy, not just in what is said, but also in how it is said. I am afraid to say that sometimes the busyness of the doctor is simply used as an excuse to avoid interactive engagements at a deeper emotional level. Yes, bringing feelings to the surface can be uncomfortable, but enduring the discomfort is part of professional life. In fact, recent research carried out by Gill Combes in CLAHRC WM showed that doctors are reticent in bringing psychological issues into the open.[2] Deliberately ignoring emotional clues and keeping things at a superficial level is deeply unsatisfying to patients. Glossing over feelings also impedes communication regarding more technical issues, as it is very hard for a person to assimilate medical information when they are feeling emotional, or nursing bruised feelings. In the long run such a technical approach to communication impoverishes a doctors professional life.

Doctors sometimes say that they should stick to the technical and that the often lengthy business of counselling should be carried out by other health professions, such as nurses. I have argued before that this is a blatant and unforgivable abrogation of responsibility; it vitiates values that lie (and always will lie) at the heart of good medical practice.[3] The huge responsibilities that doctors carry to make the right diagnosis and prescribe the correct treatment entail a psychological intimacy, which is almost unique to medical practice and which cannot easily be delegated. The purchase that a doctor has on a patient’s psyche should not be squandered. It is a kind of power, and like all power it may be wasted, misused or used to excellent effect.

The concept I have tried to explicate is that good communication is a function of ethical practice, professional behaviour and the medical ethos. It lies at the heart of the craft of medicine. If this point is accepted, it has an important corollary – the onus for teaching communication skills lies with medical practitioners rather than with psychologists or educationalists. Doctors must be the role models for other doctors. I was fortunate in my medical school in Johannesburg to be taught by professors of Oslerian ability who inspired me in the art of practice and the synthesis of technical skill and human compassion. Some people have a particular gift for communication with patients, but the rest of us must learn and copy, be honest with ourselves when we have fallen short, and always try to do better. The most important thing a medical school must do is to nourish and reinforce the attitudes that brought the students into medicine in the first place.

— Richard Lilford, CLAHRC WM Director

References:

  1. Wragg JA, Robinson EJ, Lilford RJ. Information presentation and decisions to enter clinical trials: a hypothetical trial of hormone replacement therapy. Soc Sci Med. 2000; 51(3): 453-62.
  2. Combes G, Allen K, Sein K, Girling A, Lilford R. Taking hospital treatments home: a mixed methods case study looking at the barriers and success factors for home dialysis treatment and the influence of a target on uptake rates. Implement Sci. 2015; 10: 148.
  3. Lilford RJ. Two Ideas of What It Is to be a Doctor. NIHR CLAHRC West Midlands News Blog. August 14, 2015.