CLAHRC WM is not just interested in health care since the methods we use are equally relevant to decision-makers in education, social services, industrial policy, criminology, and so on. We should all be learning from each other. In a previous blog I reported on the (mostly positive) results of the ‘Moving to Opportunity’ experiment in the USA, where families were given an opportunity to move from a deprived neighbourhood to a more salubrious one. So I was interested to spot an RCTs of vouchers that allowed children (over a wide age range) from government schools to attend private schools (also in the USA). The experiment was recent (last five years) and we have outcomes at one year only. Seventy percent of pupils allocated a voucher to attend a private school took up their offer; so both intention to treat and per protocol analyses are reported. The educational outcomes were lower in the intervention group, and were statistically significantly lower for mathematics. This negative effect was greater if the voucher was taken up than if it was not. The negative effect was greater if the child came from a school that was not rated as poor performing than if the previous school was rated satisfactory or good. The negative effect was greatest if the child was in elementary school, and non-significantly positive if they were already in high school.
What caused the negative effect on educational outcomes? Simply moving school does not seem to explain the results, since a proportion of control children moved school with little or no apparent effect. However, private schools provide less instructional time than government schools, especially in elementary school. Other studies have also noted negative effects of moving children to private school on educational outcomes in the short term. But it is far too early to declare the intervention a failure. There is a limit to how much an elementary school child can assimilate, and it is the long-term effects that are important. However, I was surprised by this result – educational interventions have a habit of producing results different to those intended. Full marks to the US Congress, which had the wisdom to evaluate its own policies. The UK Cabinet Office has published a document arguing for more RCTs of policy, and I expect to be able to report the results of further RCTs of educational interventions in the News Blog.
— Richard Lilford, CLAHRC WM Director
- Dynarski M, Rui N, Webber A, Gutmann B, Bachman M. Evaluation of the DC Opportunity Scholarship Program. Impacts After One Year. Alexandria, VA: Institute of Education Sciences, 2017.
- Haynes L, Service O, Goldacre B, Torgerson D. Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials. London: UK Cabinet Office, 2012.