Tag Archives: Education

Reducing Class Size

News blog readers know that from time to time I make a diversion into the territory of evidence-based education. On the way home from work recently, I listened to a discussion about the merits of reduced class size. One of the protagonists argued that reducing class size was very beneficial to learning outcomes. The other said that educational outcomes were hardly affected by class size. So I turned again to Hattie’s monumental work.[1] There was support for both positions from this well-studied intervention; the debate concerns the magnitude of the effect. The total number of students across the studies is about 1 million and the effect of reducing class size from about 25 to about 15 is about 0.15 of a standard deviation. This might sound like a nugatory effect (as argued by one of the debaters). However, a standard deviation of this magnitude represents about half a year of learning achievement. Remember, a standard deviation of only 0.3 represents a whole year and a standard deviation of 1.0 represents going on for three years of achievement, on average. Reducing class size is much less effective than many other interventions, but it still seems highly desirable. There is also an argument that teacher satisfaction and retention might be improved by smaller class sizes. However, when all is said and done, class-size is not nearly as important as teacher ability (which in full is not nearly as important as student ability, but that is a given for any particular class).

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Hattie J. Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. Oxon, UK: Routledge, 2009.

Wrong Medical Theories do Great Harm but Wrong Psychology Theories are More Insidious

Back in the 1950s, when I went from nothing to something, a certain Dr Spock bestrode the world of child rearing like a colossus. Babies, said Spock, should be put down to sleep in the prone position. Only years later did massive studies show that children are much less likely to experience ‘cot death’ or develop joint problems if they are placed supine – on their backs. Although I survived prone nursing to become a CLAHRC director, tens of thousands of children must have died thanks to Dr Spock’s ill-informed theory.

So, I was fascinated by an article in the Guardian newspaper, titled ‘No evidence to back the idea of learning styles’.[1] The article was signed by luminaries from the world of neuroscience, including Colin Blakemore (who I knew, and liked, when he was head of the MRC). I decided to retrieve the article on which the Guardian piece was mainly based – a review in ‘Psychological Science in the Public Interest’.[2]

The core idea is that people have clear preferences for how they prefer to receive information (e.g. pictorial vs. verbal) and that teaching is most effective if delivered according to the preferred style. This idea is widely accepted among psychologists and educationalists, and is advocated in many current textbooks. Numerous tests have been devised to diagnose a person’s learning style so that their instruction can be tailored accordingly. Certification programmes are offered, some costing thousands of dollars. A veritable industry has grown up around this theory. The idea belongs to a larger set of ideas, originating with Jung, called ‘type theories’; the notion that people fall into distinct groups or ‘types’, from which predictions can be made. The Myers-Briggs ‘type’ test is still deployed as part of management training and I have been subjected to this instrument, despite the fact that its validity as the basis for selection or training has not been confirmed in objective studies. People seem to cling to the idea that types are critically important. That types exist is not the issue of contention (males/females; extrovert/introvert), it is what they mean (learn in different ways; perform differently in meetings) that is disputed. In the case of learning styles the hypothesis of interest is that the style (which can be observed ex ante) meshes with a certain type of instruction (the benefit of which can be observed ex post). The meshing hypothesis holds that different modes of instruction are optimal for different types of person “because different modes of presentation exploit the specific perceptual and cognitive strengths of different individuals.” This hypothesis entails the assumption that people with a certain style (based, say on a diagnostic instrument or ‘tool’) will experience better educational outcomes when taught in one way (say, pictorial) than when taught in another way (say, verbal). It is precisely this (‘meshing’) hypothesis that the authors set out to test.

Note then that finding that people have different preferences does not confirm the hypothesis. Likewise, finding that different ability levels correlate with these preferences would not confirm the hypothesis. The hypothesis would be confirmed by finding that teaching method 1 is more effective than method 2 in type A people, while teaching method 2 is more effective than teaching method 1 in type B people.

The authors find, from the voluminous literature, only four studies that test the above hypothesis. One of these was of weak design. The three stronger studies provide null results. The weak study did find a style-by-treatment interaction, but only after “the outliers were excluded for unspecified reasons.”

Of course, the null results do not exclude the possibility of an effect, particularly a small effect, as the authors point out. To shed further light on the subject they explore related literatures. First they examine aptitude (rather than just learning style preference) to see whether there is an interaction between aptitude and pedagogic method. Here the literature goes right back to Cornbach in 1957. One particular hypothesis was that high aptitude students fare better in a less structured teaching format, while those with less aptitude fare better where the format is structured and explicit. Here the evidence is mixed, such that in about half of studies, less structure suits high ability students, while more structure suits less able students – one (reasonable) interpretation for the different results is that there may be certain contexts where aptitude/treatment interactions do occur and others where they do not. Another hypothesis concerns an aspect of personality called ‘locus of control’. It was hypothesised that an internal locus of control (people who incline to believe their destiny lies in their own hands) would mesh with an unstructured format of instruction and vice versa. Here the evidence, taken in the round, tends to confirm the hypothesis.

So, there is evidence (not definitive, but compelling) for an interaction between personality and aptitude and teaching method. There is no such evidence for learning style preference. This does not mean that some students will need an idea to be explained one way while others need it explained in a different way. This is something good teachers sense as they proceed, as emphasised in a previous blog.[3] But tailoring your explanation according to the reaction of students is one thing, determining it according to a pre-test is another. In fact, the learning style hypothesis may impede good teaching by straightjacketing teaching according to a pre-determined format, rather than encouraging teachers to adapt to the needs of students in real time. Receptivity to the expressed needs of the learner seems preferable to following a script to which the learner is supposed to conform.

And why have I chosen this topic for the main News Blog article? Two reasons:

First, it shows how an idea may gain purchase in society with little empirical support, and we should be ever on our guard – the Guardian lived up to its name in this respect!

Second, because health workers are educators; we teach the next generation and we teach our peers. Also, patient communication has an undoubted educational component (see our previous main blog [4]). So we should keep abreast of general educational theory. Many CLAHRC WM projects have a strong educational dimension.

— Richard Lilford, CLAHRC WM Director

References:

  1. Hood B, Howard-Jones P, Laurillard D, et al. No Evidence to Back Idea of Learning Styles. The Guardian. 12 March 2017.
  2. Pashler H, McDaniel M, Rohrer D, Bjork R. Learning Styles: Concepts and Evidence. Psychol Sci Public Interest. 2008; 9(3): 105-19.
  3. Lilford RJ. Education Update. NIHR CLAHRC West Midlands News Blog. 2 September 2016.
  4. Lilford RJ. Doctor-Patient Communication in the NHS. NIHR CLAHRC West Midlands News Blog. 24 March 2017.

Keeping a Child Back at School

We have often talked about Hattie’s work on evidence-based education.[1-4] Now we turn to retention (pages 97-99) – the act of keeping a child back and having them repeat a year of school on the grounds of poor performance.[5] Numerous papers, including a meta-analysis of 20 studies,[6] have shown retention to be associated with negative effects for the retained student. In subsequent years they have lower scores for a whole range of subjects: language, arts, reading, mathematics, work-study skills, and social studies, as well as social and emotional adjustment and behaviour, self-concept, and attitude to school. So a picture is starting to emerge – remember streaming is also unhelpful.[7] So any act that demeans a child or destroys her self-confidence is bad, while bright children are not held back by having less bright peers in the classroom. See also our News Blog on the Michelle Obama effect.[8]

— Richard Lilford, CLAHRC WM Director

References:

  1. Lilford RJ. Evidence-based Education. NIHR CLAHRC West Midlands News Blog. 24 February 2017.
  2. Lilford RJ. The School, the Teacher of the Pupil – Which is Most Important? NIHR CLAHRC West Midlands News Blog. 28 October 2016.
  3. Lilford RJ. Ask Not to Whether, But Why, Before the Bell Tolls. NIHR CLAHRC West Midlands News Blog. 29 July 2016.
  4. Lilford RJ. Education Update. NIHR CLAHRC West Midlands News Blog. 2 September 2016.
  5. Hattie J. Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. Oxon, UK: Routledge, 2009.
  6. Jimerson SR. Meta-analysis of Grade Retention Research: Implications for Practice in the 21st Century. School Psychol Rev. 2001; 30(3): 420-37.
  7. Lilford RJ. Evidence-Based Education (or How Wrong the CLAHRC WM Director Was). NIHR CLAHRC West Midlands News Blog. 15 July 2016.
  8. Lilford RJ. More on Education. NIHR CLAHRC West Midlands News Blog. 16 September 2016.

More on Brain Health in Young Children and Effect on Life Course

Brain health in early childhood is a recurring theme of your News Blog. Peter Chilton referred me to an interesting article in Nature Human Behaviour published at the end of last year.[1] This study was based on a prospective study of children in the South Island of New Zealand. The investigators wanted to determine the prognosis for the 20% of the population with the worst brain health indicators at age three. These indicators include single parent family; low socioeconomic group; poor self-control; and low IQ. Outcome variables covered a range of important economically burdensome outcomes, such as obesity, cigarette smoking, and crime. These variables were harvested from various databases where health and crime statistics are recorded. A 20% ‘segment’ of this young population could be defined which predicted 80% of crime, and similar high rates on other outcomes. This 20:80 ratio, called the Pareto ratio, is often encountered in social science – for example, wealth distributes itself roughly in this proportion across many societies (about 20% of people control 80% of wealth). The authors say that their study shows plenty of ‘headroom’ for preventive interventions. That is to say, society could achieve massive gains if health and social outcomes among the highest risk segment could be improved to average levels. We have discussed interventions, such as early childhood education, before.[2-4] Many studies show statistically significant and economically worthwhile results for such interventions, but the gains come nowhere near the theoretical headroom defined here. Likely this is because brain health at age three is only partly the result of remediable factors.

— Richard Lilford, CLAHRC WM Director

References:

  1. Caspi A, Houts RM, Belsky DW, Harrington H, Hogan S, Ramrakha S, Poulton R, Moffitt TE. Childhood forecasting of a small segment of the population with large economic burden. Nature Hum Behav. 2016; 1: 0005.
  2. Lilford RJ. Pregnancy before age 16 – dropping quite rapidly from a peak in 1997. NIHR CLAHRC West Midlands News Blog. February 10, 2017.
  3. Lilford RJ. If you want to reduce partner violence or teenage pregnancy, then teach algebra and history? NIHR CLAHRC West Midlands News Blog. December 9, 2016.
  4. Lilford RJ. Evidence-based education (or how wrong the CLAHRC WM Director was). NIHR CLAHRC West Midlands News Blog. July 15, 2016.

Evidence-Based Education

Another shibboleth bites the dust. Have you ever heard (dinner parties?) that girls do better in all girl classes? If so, this is correct – there is indeed such an association. But the observational studies do not compare like with like – the comparison groups are not equivalent since single sex schools tend to have more selective intakes than their controls. So we need within school comparisons or, even better, RCTs. There is one example of each in Hattie’s monumental study (pages 96-97),[1] – Marsh & Rowe [2] and Signorella, Frieze & Hershey.[3] In both cases there were – wait for it – no advantages for all girl classes on educational attainment or career choices, and in the former study, the point estimates were negative for the brightest girls. Likewise, boys did not perform differently in all-male vs. mixed sex classes.

— Richard Lilford, CLAHRC WM Director

References:

  1. Hattie J. Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. Oxon, UK: Routledge, 2009.
  2. Marsh HW, & Rowe KJ. The Effects of Single-Sex and Mixed-Sex Mathematics Classes Within a Coeducational School: A Reanalysis and Comment. Austral J Educ. 1996; 40(2): 147-62.
  3. Signorella ML, Frieze and Hershey. Single-Sex versus Mixed-Sex Classes and Gender Schemata in Children and Adolescents: A Longitudinal Comparison. Psychol Women Quart. 1996; 20: 599-607.

 

If You Want to Reduce Partner Violence or Teenage Pregnancy, Then Teach Algebra and History?

There is little doubt that highly educated men are less likely than poorly educated men to perpetrate violence against their partners,[1] and that highly educated women are less likely than poorly educated women to get pregnant in their teens.[2] But what is going on here – which way does causality run? Certainly, an educated man is likely to earn more than one less educated. More money means less stress, and since stress is a harbinger of partner violence, it is plausible that education leads to less violence through this mediating (intervening) variable. Alternatively, the kind of person who acquires education may be the sort of person who is less innately pre-disposed to violence than a person who does not acquire education. A person who seeks out education may have greater mental resources, such that a wider range of responses are available to him – and hence he is less likely to lash out. But could it be that education per se increases moral rectitude, even when the education is not targeted at moral behaviour? One can devise a theory for such an effect. Algebra, history and other ‘academic’ subjects exercise the capacity for abstract thought. Could the capacity spill over from the topic of instruction to influence behaviour more generally? Compassion, for example, is abstract – it requires the ability to imagine what another person is feeling. Teaching abstract academic subjects may spill over in to heightened sensitivity to the suffering of others. This hypothesis could be tested neurophysiologically – highly educated persons, on average, may manifest greater specific responses on functional neuro-imaging than those of similar IQ, but lower educational attainment, when confronted with a compassion-arousing event. The brain, after all, is a learning machine that is permanently altered by education. This might explain why sex education has a rather small effect on teenage pregnancy, but being educated is associated with a large effect. It is sometimes said that education refers to what is left when all the facts have been forgotten, or to quote BF Skinner more accurately, “Education is what survives when what has been learnt has been forgotten”?

— Richard Lilford, CLAHRC WM Director

References:

  1. Abramsky T, Watts CH, Garcia-Moreno C, et al. What factors are associated with recent intimate partner violence? findings from the WHO multi-country study on women’s health and domestic violence. BMC Public Health. 2011; 11: 109.
  2. Girma S & Paton D. Is education the best contraception: The case of teenage pregnancy in England? Soc Sci Med. 2015; 131: 1-9.

The School, the Teacher, or the Pupil – Which is Most Important?

News Blog readers will know that the CLAHRC WM Director takes a critical look at Hattie’s monumental overview of evidence-based education in most of our posts.[1-4] Today let’s look at a fundamental question Hattie addresses in the monograph – the question of how much of the variance in educational outcomes resides in the pupil, the teacher, or the school. Not surprisingly this has been looked at many times. It is a little like separating the components of variance in the outcome of surgery according to the patient, the surgeon, and the hospital. Hattie [5] quotes a meta-analysis by Marzano who found that the lion’s share (80%) of variance in achievement was accounted for by student level variables, the smallest share (7%) by school effects, and an intermediate share by teachers (13%). Of course, as we shall see in future blogs based on Hattie’s work, this is not the end of the story. For instance, student effects include the effect on the student of previous education – i.e. previous teachers are consolidated within the student component of variance. More important still, is the effect of parents / guardians, since this is also consolidated within students in cross-sectional studies. We will look at unravelling these factors in future news blogs. In the meantime, we can say that ineffectual teachers have a negative effect on achievement, no matter how good the school, and exceptional teachers can more than compensate for a rubbish school.

— Richard Lilford, CLAHRC WM Director

References:

  1. Lilford RJ. Evidence-based Education. (Or How Wrong the CLAHRC WM Director Was). NIHR CLAHRC West Midlands News Blog. 15 July 2016.
  2. Lilford RJ. Ask Not to Whether, But Why, Before the Bell Tolls. NIHR CLAHRC West Midlands News Blog. 29 July 2016.
  3. Lilford RJ. Education Update. NIHR CLAHRC West Midlands News Blog. 2 September 2016.
  4. Lilford RJ. Bring Back the University Lecture: More on Evidence-Based Teaching. NIHR CLAHRC West Midlands News Blog. 2 September 2016.
  5. Hattie J. The Applicability of Visible Learning to Higher Education. Scholarship of Teaching and Learning in Psychology. 2015; 1(1): 79-91.

More on Education

Michelle Obama visited an all-girls school in London back in 2009. She met with the pupils again in 2011 and 2012. The school is named after the first English woman doctor, Elizabeth Garrett Anderson. Simon Burgess has examined performance at the school over the years preceding and following these meetings.[1] He used the results of national examinations (the GCSE) and compared the school’s results with those from the rest of London’s schools. A sharp uptick in performance, which later returned to baseline, was seen in the ‘intervention’ school, but not in the controls. Burgess used a difference-in-difference type approach in a multivariate statistical analysis (though a synthetic control may have been even better, as discussed in a previous post). The ‘treatment effect’ was half a standard deviation, which would carry a student destined to achieve eight grade ‘B’s, to achieving a mix of ‘A*’s and ‘A’s. The paper is worth a read – it is really beautifully written and packs a powerful message regarding the beneficial effect of aspirations. The First Lady did not tell her listeners that getting good grades is easy. She said it’s hard, but ‘you can do it’. Most of the pupils at the school are not white and Michelle Obama would have been a great role model.

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Burgess S. Michelle Obama and an English school: the power of inspiration. 2016.

Bring Back the University Lecture: More on Evidence-Based Teaching

News Blog readers are now familiar with Hattie’s monumental work on evidence-based education [1] – an overview (meta-synthesis) of:

063-dcvi-more-on-evidence-based-teaching

To remind you, the huge proportion of the meta-analyses and studies (96%) show positive effects – maybe a Hawthorne effect of some sort. So an influence or intervention that produces an effect size of, say, only 0.2 of a standard deviation must be considered not particularly useful – it will be at the bottom end of a distribution in which nearly everything ‘works’.

In our last two posts [2] [3] we identified two factors that were, perhaps surprisingly, effete:

  1. Small class sizes.
  2. Problem-based learning.

I should have mentioned that there is no threshold class size – reducing from 200 to 60; 60 to 20; 20 to 8 all yield nugatory benefits. Moreover, and again perhaps surprisingly, the results of most studies are not very age-group dependent. You can see where I am going – abandoning the lecture in universities, in line with current fashion, should be questioned, especially given the cost-efficiency of the method. Important variables (have the students pre-prepared; does the lecturer stop and ask questions to assess understanding; do the students set time aside to reflect; does the lecturer assess herself; does she adapt herself to the type of class/group she is teaching) are all more important than the size of the class. A great lecturer is a scarce resource to be used wisely. Think TED talks.

— Richard Lilford, CLAHRC WM Director

References:

  1. Hattie J. The Applicability of Visible Learning to Higher Education. Scholarship of Teaching and Learning in Psychology. 2015; 1(1): 79-91.
  2. Lilford RJ. Evidence-Based Education (or how wrong the CLAHRC WM Director was). NIHR CLAHRC West Midlands News Blog. 15 July 2016.
  3. Lilford RJ. Ask to Not Whether, But Why, Before the Bell Tolls! NIHR CLAHRC West Midlands News Blog. 29 July 2016.

Education Update

As News Blog readers know the CLAHRC WM Director summarises an empirical finding from the experimental educational literature in each fortnightly post. In this issue our focus turns to university education [1] and to just one aspect of it – the perennial question of Problem-Based Learning (PBL). Nine meta-analyses have evaluated this method, and most constituent studies were carried out in universities rather than in schools. Most studies were carried out among medical students. It turns out that PBL is effete – the summary measure of effect is nugatory (0.08 of a standard deviation). It is one of the smallest effect sizes of any pedagogic method evaluated across the entire corpus of the experimental education literature. Moreover, it is actually harmful in some situations – namely those where PBL precedes learning the basic content. PBL is most likely to be effective where the intellectual scaffold has already been built and the student now has to learn to apply the new knowledge.

Consider a patient with pyrexia of unknown origin. Working back to the causes of a temperature when one does not know the causes does not create an intellectual scaffold from which forward reasoning can work. Rather start with the potential causes and then narrow them down as information accrues.

— Richard Lilford,

Reference:

  1. Hattie J. The applicability of visible learning to higher education. Scholarship Teaching Learning in Psychology. 2015; 1(1): 79-91.