Tag Archives: Education

UK Takes Over From the US as the Home of Trials of Educational Interventions

It is truly wonderful that trials are now so widely used to make causal inferences in education – long the preserve of theory over empirical evidence. I was delighted to see that England has now become, proportionally, the largest education laboratory in the world.[1] This rapid increase in high-quality empirical research in English schools builds on a £135m grant to establish the Education Endowment Foundation. This organisation has commissioned around a tenth of all RCTs ever carried out in education.[2] Already the results are influencing practice – £26m has been allocated to fund breakfast clubs, for example. Many ‘fads’ have been disproven, for example tailoring education according to individual learning styles, streaming pupils by ability,[3] or repeating a year of school.[4] Building on Hattie’s monumental work previously cited in your News Blog,[5] the Education Endowment Foundation has summarised the findings from 13,000 educational trials from around the world. Two-thirds of English teachers have turned to this evidence for guidance. This is a huge success for the values of the Enlightenment and an indication that we are moving to be, in Donald Campbell’s words, “an experimenting society.”

— Richard Lilford, CLAHRC WM Director

References:

  1. The Economist. England has become one of the world’s biggest education laboratories. The Economist. Mar 31, 2018.
  2. Education Endowment Foundation. The EEF at 5. 2016.
  3. Lilford RJ. Evidence-Based Education (or How Wrong the CLAHRC WM Director Was). NIHR CLAHRC West Midlands News Blog. 10 March 2017.
  4. Lilford RJ. Keeping a Child Back at School. NIHR CLAHRC West Midlands News Blog. 10 March 2017.
  5. Hattie J. Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. Oxon, UK: Routledge, 2009.
Advertisements

A Fascinating Trial of an Educational Intervention to Improve the Quality of Care in Rural India

The background to this interesting paper [1] (published in the journal Science) is the appalling standard of care provided for poor people in low-income countries. Even when a patient presents with straightforward features of a condition that is easy to diagnose, the correct treatment is given in less than half of all cases. Antibiotics are widely prescribed even for clearly non-infectious diseases, such as angina.

Doctors provide better care than informal providers, but the size of this difference is not large. Only in 51% of straightforward clinical cases did doctors give the right treatment in a study in rural India.[2] Their mean consultation lasted for less than two minutes. It is well established that doctors in the public sector often fail to turn up for their clinical assignments. Moreover, there are not enough doctors to go round and many people rely on informal providers.

So what should be done to remedy this unfortunate state of affairs? Should we train more doctors or upskill informal providers? Das and colleagues conducted a well-designed and executed RCT to find out.[1] Informal providers were randomised to receive an educational intervention or control. The educational intervention consisted of 72 sessions, delivered over nine months. The mean proportion of attendances at training sessions was 56%. Nevertheless, there was a clear improvement in performance among the informal providers who had to been allocated to the intervention group.

Interestingly, the main method of endpoint assessment was by standardised patients (see our previous News Blog). These are ‘mystery shoppers’ who provide the health care provider with a straightforward and fairly unequivocal symptom combination for conditions such as asthma, angina and diarrhoea. Interestingly, the educators providing the training sessions were not party to the testing scenarios that were to be presented to the trainees (and their controls). Use of unnecessary treatments (e.g. antibiotics for angina) was not affected by the intervention. The results obtained by standardised patients were ‘triangulated’ by means of direct observation of a subset of consultations. The trends were corroborated – a statistically significant improvement was again observed across intervention and control groups.

The authors also compared the informal providers (both groups) with a group of medically trained providers (they had not received any additional training). Training made good half of the performance gap between informal and medically qualified providers. Nevertheless, the latter provided correct case management in only about two-thirds of cases, and were more likely than the informal group to prescribe antibiotics where they were not indicated. Some methodological points:

  1. I was impressed that the researchers avoided ‘training to the test’, as mentioned above.
  2. They only used post-randomisation endpoints, whereas there is a case for pre- and post- measurements to allow measurement of differences in differences/controlling for baseline.
  3. There is no use of (or mention of) clustering within the data – it would be interesting to examine for differences in performance by clinical scenarios within and across trainees.

Nevertheless, this is an extremely interesting study from both the scientific and practical points of view. It would be much less expensive to train more informal providers than to train more doctors. Perhaps doctors should be used as trainers and quality managers, and to see difficult cases while a less highly trained / cognitively elite workforce is trained to manage the more quotidian cases – a proposal we have made for high-income countries.[3]

— Richard Lilford, CLAHRC WM Director

References:

  1. Das J, Chowdhury A, Hussam R, Banerjee AV. The impact of training informal health care providers in India: A randomized controlled trial. Science. 2016; 354: aaf7384.
  2. Das J, Holla A, Mohpal A, Muralidharan K. Quality and Accountability in Health Care Delivery: Audit-Study Evidence from Primary Care in India. Am Econ Rev. 2016; 106(12): 3765-99.
  3. Lilford RJ. The Future of Medicine. NIHR CLAHRC West Midlands News Blog. 23 October 2015.

Revisiting the Milgram Experiment

The CLAHRC WM Director has made three inaugural lectures. While preparing for the first of these he read an inaugural lecture made a few months before to glean information on procedure and form. He chanced upon an inaugural delivered by a newly appointed Professor of Psychology – one Tony Chapman. In his lecture, Chapman recounted Stanley Milgram’s famous (possibly infamous) experiments in which volunteers administered simulated electrical shocks to learners. The volunteers were urged to administer the electric shocks if the learner faltered. The learner, who was an actor, would then react to the simulated shock so that the volunteer would think the learner was recoiling from a real electric shock. The majority of volunteers complied with this harsh instruction. Conducted in the wake of World War Two atrocities, the experiments seemed to explain the behaviour of prison guards who perpetrated institutionalised cruelty on an unconscionable scale in the European and Eastern theatres.

Alas, it was all just a little too neat. Milgram’s archive has been made available to researchers and his findings are, at the very least, exaggerated.[1] A high proportion (above half) of volunteers suspected the ruse. Worse Milgram was selective in the results reported. It is, however, to Milgram’s credit that he maintained the archive and, moreover, recorded the many experiments he performed. In his archive Milgram makes an entry where he questions his own work – how much is science and how much theatre, he wonders. As to his conclusions – perhaps not so much wrong as not right. Certainly, much of human behaviour is contingent on the prevailing social environment, as discussed elsewhere in this News Blog.[2] [3] The education a person has had at mother’s knee, at school and at university will all form the character that is eventually carried into clinical practice. The aim should be to generate a high rectitude individual who will resist, indeed oppose, the forces of evil.

— Richard Lilford, CLAHRC WM Director

References:

  1. Perry G. The Shocking Truth of Stanley Milgram’s Obedience Experiments. New Scientist. 14 March 2018.
  2. Lilford RJ. A Culture of Quality: Join the Debate. NIHR CLAHRC West Midlands News Blog. 13 June 2014.
  3. Lilford RJ. Care That Is Not Just Unskilled but Abusive. NIHR CLAHRC West Midlands News Blog. 8 May 2015.

Education and Alzheimer’s – More Mendelian

We recently carried the summary of an article showing that genes associated with higher educational attainment were also associated with a lower risk of coronary heart disease.[1] The noble BMJ has published another interesting article on Mendelian randomisation.[2] By Jupiter, they make the same finding; genes associated with high educational attainment are associated with a lower risk: this time of Alzheimer’s disease. These genes are also associated with a high level of intelligence, and both intelligence and educational attainment are associated with modifiable risk factors. Thus, it is not clear that modifying behaviour, for example through staying longer in education, would improve outcome in people who do not have these favourable genes. The paper does suggest that genes associated with alcohol consumption are also protective against Alzheimer’s disease, again reinforcing a recent news blog article.[3] Interestingly, genes associated with favourable metabolic factors and normal blood pressure, are in no way protective against Alzheimer’s disease. Many public health people argue that avoiding cardiovascular risk will also reduce the risk of Alzheimer’s disease. I think it is high time to put a stop to this nonsense. It has been clear for some while that Alzheimer’s disease and cardiovascular disease are on different pathways. The well-known association between nicotine consumption and a reduced risk of Alzheimer’s disease was confirmed at the genetic level,[4] but disturbingly for me, genes predisposing to high coffee intake were associated with an increased risk of Alzheimer’s disease.

I am a great fan of genetic randomisation and this is another extremely interesting and well-written paper. One thing that genetic randomisation completely gets around is of the problem of reverse causality. It is one of the most powerful techniques to have come into epidemiology in the last 50 years and the originators, Gray and Wheatley,[5] deserve the Nobel prize.

— Richard Lilford, CLARHC WM Director

References:

  1. Lilford RJ. A Very Interesting Paper Using Mendelian Randomisation to Determine the Effect of Extra Years of Education on Heart Disease. NIHR CLAHRC West Midlands News Blog. 10 November 2017.
  2. Larsson SC, Traylor M, Malik R, Dichgans M, Burgess S, Markus HS, for the CoSTREAM Consortium on behalf of the International Genomics of Alzheimer’s Project. Modifiable pathways in Alzheimer’s disease: Mendelian randomisation analysis. BMJ. 2017; 359: j5375.
  3. Lilford RJ. So Where Are We Up to With Alcohol and Health? NIHR CLAHRC West Midlands News Blog. 12 January 2018.
  4. Oddo S, Caccamo A, Green KN, Liang K, Tran L, Chen Y, Leslie FM, LaFerla FM. Chronic nicotine administration exacerbates tau pathology in a transgenic model of Alzheimer’s disease. Proc Natl Acad Sci USA. 2005; 102(8): 3046-51.
  5. Gray R & Wheatley K. How to avoid bias when comparing bone marrow transplantation with chemotherapy. Bone Marrow Transplant. 1991; 7(s3): 9-12.

Education Before Surgery

Within developed countries the most frequently performed major surgery is of the upper abdomen. However, research has shown that between 10-50% of patients suffer a postoperative pulmonary complication (PPC), which is strongly associated with an increase in mortality and morbidity, as well as healthcare costs. Some studies have suggested that education and training in breathing exercises prior to the surgery could reduce the risk of developing PPC (75% reduction in relative risk, 20% reduction in absolute risk), but these studies may have been subject to methodological biases. Now researchers in Australia have conducted an international, multicentre, blinded, parallel group RCT to assess the efficacy of preoperative physiotherapy in reducing PPCs.[1] They enrolled 441 patients to receive an information booklet alone (control), or with an additional 30-minute physiotherapy education and breathing exercise training session. The intervention group saw a significant reduction in the incidence of PPC (including hospital-acquired pneumonia) within 14 days of the operation (adjusted hazard ratio 0.48, 95% CI 0.30-0.75). This amounted to an absolute risk reduction of 15% (95% CI 7-22%).

— Peter Chilton, CLAHRC WM Research Fellow

Reference:

  1. Boden I, Skinner EH, Browning L, Reeve J, Anderson L, Hill C, Robertson IK, Story D, Denehy L. Preoperative physiotherapy for the prevention of respiratory complications after upper abdominal surgery: pragmatic, double blinded, multicentre randomised controlled trial. 2018; 360: j5916.

Is it Possible to Teach Empathy?

News blog readers will know that I am fascinated by the question of whether it is possible to teach people to be kinder, more patient-centered, and to show more empathy. A recent meta-analysis of RCTs sheds important light on the critical issue of empathy training.[1] Unlike previous systematic reviews, this study included only experimental studies. Overall, 19 studies met the inclusion criteria for the meta-analysis.

One important issue concerns how the endpoint was measured. In 11 of the 19 included studies the outcome was an objective measure, while in the remainder the outcome was self-reported.

Overall, educational interventions produced a positive benefit that was statistically significant. When the authors made an adjustment for possible publication bias, the effect size was only slightly reduced, remaining highly significant statistically.

I expected to find that the effect size was greater for the self-reported outcomes than for objective outcomes. In fact, the effect size was larger and more highly significant for the objective measures of effect.

Some people classify empathy training in two forms: cognitive and effective, to cover the intellectual and emotional aspects of empathy. Others have questioned this dichotomy, arguing that the emotional and the cognitive parts have to interact to produce empathetic behaviour. As it turned out, all studies included a cognitive component.

This is a very interesting and important study. My main problem with the study is that they do not give a breakdown according to whether the objective measure was self-reported or objective. Also, the results do not tell us how enduring the effects were. I have argued before that one of the main criteria of good communication and compassionate care is the desire to achieve these projectors. The most important thing to instil is a deep-seated desire to do a better job. It would seem that training has a part to play in achieving this objective. However, sustained exposure to excellent role models is also critically important and a crucial part of the education of health professionals.

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Teding van Berkhout E & Malouff JM. The Efficacy of Empathy Training: A Meta-analysis of Randomized Controlled Trials. J Counsel Psychol. 2016; 63(1): 32-41.

A Very Interesting Paper Using Mendelian Randomisation to Determine the Effect of Extra Years of Education on Heart Disease

It turns out that there are a number of genes, all associated with aspects of neurodevelopment, that predict how many years a person will spend in formal education.[1] It is already very well established that more years of education are associated with large reductions in coronary heart disease (CHD) (mediated by behaviour such as lower calorie intake, less smoking, more exercise).[2] So the authors of a recent well-written and most interesting BMJ paper did the obvious thing.[3] [4] They related the (random) presence or absence of educational propensity genes to CHD. Bingo, they measured a large effect (the genes that predispose to larger durations of formal education associate with reduced CHD). Now, the thing with Mendelian randomisation is that the genotype must not be linked to the outcome (CHD in this case), other than through the putative explanatory variable (duration of education in this case). The authors are aware that it is quite possible that education genes are linked to the outcome (CHD), net of (any) effect on education. To deal with this possibility they perform sensitivity analyses. They examine the association of genetic variates associated with education and the behaviours that lead to CHD. If the effects on education and on CHD behaviours are similar across the genetic variates this suggests that the effect on CHD is through education and not through another variable. And so it was. They also looked to see whether genetic variants already known to be associated with CHD (genes for high cholesterol, etc.) were also associated with education. If the genes associated with education do not associate with these other risk factors, then that favours a cause and effect explanation. There was no association. However, such an association would only be expected if there was a ‘massive’ effect of ‘education genes’ that bypassed education.

This all falls short of proof. Since the educational genes lead to education through mental processes, it is reasonable to suppose that almost all genetic variates that affect education also affect behaviour. Thus, they would affect CHD, even if there was no extra education. The authors say that their conclusion is strongly supported by identical twin studies where one twin stayed longer in education than the other, but this too ignores the fact that these twins are different, for all that their inherited genotype is the same, and so these differences could be the cause of both increased education and decrease in the behaviours that lead to heart disease.

One more point ­– even if years of education really are causative, this might well apply only to people genetically predisposed to more education and may not apply among those not so predisposed – there may be an interaction between the genes that predisposes to education and response to that education. After all, why would one persist in the classroom if you were not predisposed to benefit from the experience? People not predisposed would find being coerced to do so most unpalatable, and such an approach could even have a perverse effect. This is an excellent article and is beautifully presented. But I am a little more sceptical than the authors. I would like to see a debate on the issues.

— Richard Lilford, CLAHRC WM Director

References:

  1. Okbay A, Beauchamp JP, Fontana MA, et al. Genome-wide association study identifies 74 loci associated with educational attainment. Nature. 2016; 533(7604): 539–42.
  2. Veronesi G, Ferrario MM, Kuulasmaa K, et al. Educational class inequalities in the incidence of coronary heart disease in EuropeHeart. 201635895865.
  3. Tillmann T, Vaucher J, Okbay A, et al. Education and coronary heart disease: Mendelian randomisation study. BMJ. 2017; 358: j3542.
  4. Richards JB & Evans DM. Back to School to Protect Against Coronary Heart Disease? BMJ. 2017; 358: j3849.

Class Lectures in Medical School – Nearly Obsolete?

The University of Vermont’s College of Medicine advertises “no lectures required.” And empirical enquires show that content heavy, PowerPoint loaded, lectures are ineffective. But a thoughtful article in the New England Journal of Medicine [1] suggests that the class lecture should change rather than go. In fact, the classroom is well suited to active learning, with students who have already assimilated the core material at their own pace through private study. The lecturer interacts with the students who sit around tables and are provided with opportunities to discuss issues in small groups as the need arises. I learned that this is called the ‘flipped-classroom’ approach. Such an approach resulted in better outcomes when compared to traditional problem-based learning approaches in a randomised trial.[2] So a little bit of this and a little bit of that. And there is still a place for a little theatre. As to problem-based learning as a method to propel a new topic – forget it. It is sub-optimal, as discussed in a previous News Blog.[3]

— Richard Lilford, CLAHRC WM Director

References:

  1. Schwartztein RM & Roberts DH. Saying Goodbye to Lectures in Medical School – Paradigm Shift or Passing Fad? N Engl J Med. 2017; 377(7): 605-7.
  2. Krupat E, Richards JB, Sullivan AM, Fleenor TJ Jr, Schwartzstein RM. Assessing the effectiveness of case-based collaborative learning via randomized controlled trial. Acad Med. 2016; 91: 723-9.
  3. Lilford RJ. Bring Back the University Lecture: More on Evidence-Based Teaching. NIHR CLAHRC West Midlands News Blog. 26 September 2016.

Diet and Socioeconomic Status

People looking to lose weight and/or get healthy try a wide variety of diets, from fad diets with highly specific restrictions on what can be eaten, to general healthy eating plans. One such nutritional recommendation is the Mediterranean diet, based on the “food patterns typical of Crete… Greece and southern Italy…”,[1] and entails consumption of high amounts of plant foods (fruit, vegetables, cereals, legumes, etc.) and olive oil, moderate amounts of dairy, fish and wine, and low amounts of poultry and red meat. A number of observational studies have shown associations between such a diet and lower incidences of cardiovascular disease (CVD) and associated mortality, cancer, neuro-degenerative disorders, and overall mortality. However, there is uncertainty whether such benefits differ across different socioeconomic groups.

Bonaccio et al. carried out a prospective analysis of nearly 19,000 Italians to see the effect of the Mediterranean diet on CVD.[2] While there was an overall reduction in CVD risk associated with adherence to the diet (HR=0.85, 95% CI 0.73-0.99), this was not seen across all socioeconomic groups – only in those who were educated to a postgraduate or higher level (HR=0.43, 0.25-0.72) and in those with a high (>€40,000) household income (HR=0.39, 0.23-0.66). Those with less education (HR=0.94, 0.78-1.14) and lower income (HR=1.01, 0.79-1.29) had no significant association. Why such a difference? Subgroup analysis of people with similar adherence to the diet showed that there were a number of differences in the diet of those with high compared to low education, and those with high compared to low income. These included consumption of organic vegetables (which would have higher antioxidants and lower levels of pesticides), monounsaturated fatty acids (found in avocado, nuts, olives, etc.), micronutrients, and whole-grain bread, as well as greater dietary diversity.

So perhaps it is more important to make sure the food you are eating is of high quality and varied, than just simple healthy eating. Of course, access to high quality food of high nutritional value is not easy for poor people.

— Peter Chilton, Research Fellow

References:

  1. Willett WC, Sacks F, Trichopoulou A, Drescher G, Ferro-Luzzi A, Helsing E, Trichopoulos D. Mediterranean diet pyramid: a cultural model for healthy eating. Am J Clin Nutr. 1995; 61(6): 1402S–6S.
  2. Bonaccio M, Di Castelnuovo A, Pounis G, et al. High adherence to the Mediterranean diet is associated with cardiovascular protection in higher but not in lower socioeconomic groups: prospective findings from the Moli-sani study. Int J Epidemiol. 2017.

Another Interesting Trial of an Educational Intervention – This Time Concerning Access

Young people from disadvantaged backgrounds are less likely to apply to elite universities, both in the UK and the US, than those from economically better-off backgrounds. This finding applies even after controlling for exam results prior to application – i.e. the GCSE results in England. So Sanders and co-authors from the Behavioural Insights Team and the English Department for Education did an inexpensive trial of an inexpensive intervention.[1] The outcomes were application to, and acceptance into, an elite university (defined as belonging to the Russell Group). The intervention consisted of a letter sent to students from disadvantaged backgrounds who were on track to attend an elite university given their GCSE grades. Eligible schools were randomised to control conditions or one of three interventions: to receive a letter written by a pseudonymous male student (Ben) at Bristol University on Department for Education note paper; to receive a similar letter from a female student (Rachel) at the same university; or to receive letters from both Ben and Rachel. Three hundred schools (clusters) and 11,104 students participated. It was then a simple matter to collect the outcomes from the agency that supervises the admission process (the Universities and Colleges Admissions Service, UCAS). Receipt of a letter was associated with a non-significant increase in applications, and eventual admission to, an elite university. The increase was greatest and statistically significant for students who received both letters – from 8.5% acceptance among controls, to 11.4% in the ‘double dose’ intervention group – an increase of 2.9 percentage points (or 34 percent relative risk). Certainly, these results add to growing evidence concerning aspirations in education – see recent News Blogs on keeping children back a year [2], streaming [3], and the Michelle Obama effect.[4]

— Richard Lilford, CLAHRC WM Director

References:

  1. Sanders M, Chande R, Selley E. Encouraging People into University. London: Department for Education; 2017.
  2. Lilford RJ. Keeping a Child Back at School. NIHR CLAHRC West Midlands News Blog. 10 March 2017.
  3. Lilford RJ. Evidence-Based Education (or How Wrong the CLAHRC WM Director was). NIHR CLAHRC West Midlands News Blog. 15 July 2016.
  4. Lilford RJ. More on Education. NIHR CLAHRC West Midlands News Blog. 16 September 2016.