Government vs. Private Schools

CLAHRC WM is not just interested in health care since the methods we use are equally relevant to decision-makers in education, social services, industrial policy, criminology, and so on. We should all be learning from each other. In a previous blog I reported on the (mostly positive) results of the ‘Moving to Opportunity’ experiment in the USA, where families were given an opportunity to move from a deprived neighbourhood to a more salubrious one. So I was interested to spot an RCTs of vouchers that allowed children (over a wide age range) from government schools to attend private schools (also in the USA).[1] The experiment was recent (last five years) and we have outcomes at one year only. Seventy percent of pupils allocated a voucher to attend a private school took up their offer; so both intention to treat and per protocol analyses are reported. The educational outcomes were lower in the intervention group, and were statistically significantly lower for mathematics. This negative effect was greater if the voucher was taken up than if it was not. The negative effect was greater if the child came from a school that was not rated as poor performing than if the previous school was rated satisfactory or good. The negative effect was greatest if the child was in elementary school, and non-significantly positive if they were already in high school.

What caused the negative effect on educational outcomes? Simply moving school does not seem to explain the results, since a proportion of control children moved school with little or no apparent effect. However, private schools provide less instructional time than government schools, especially in elementary school. Other studies have also noted negative effects of moving children to private school on educational outcomes in the short term. But it is far too early to declare the intervention a failure. There is a limit to how much an elementary school child can assimilate, and it is the long-term effects that are important. However, I was surprised by this result – educational interventions have a habit of producing results different to those intended. Full marks to the US Congress, which had the wisdom to evaluate its own policies. The UK Cabinet Office has published a document arguing for more RCTs of policy,[2] and I expect to be able to report the results of further RCTs of educational interventions in the News Blog.

— Richard Lilford, CLAHRC WM Director

References:

  1. Dynarski M, Rui N, Webber A, Gutmann B, Bachman M. Evaluation of the DC Opportunity Scholarship Program. Impacts After One Year. Alexandria, VA: Institute of Education Sciences, 2017.
  2. Haynes L, Service O, Goldacre B, Torgerson D. Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials. London: UK Cabinet Office, 2012.
Advertisements

2 thoughts on “Government vs. Private Schools”

  1. You are absolutely correct – educational interventions do need to be evaluated with the same rigour we use with health care interventions. We, at York Trials Unit, regularly conduct educational trials as well as health care trials an in our experience more of these seem to be having no effect or a negative effect on intended outcomes for example:
    RCT of a commonly used software package to teach nurses numeracy skills for application to clinical settings showed a negative effect (See Ainsworth et al, 2012 Educational Studies http://dx.doi.org/10.1080/03055698.2011.598668)

    Or what about using financial incentives to increase attendance to Adult Literacy classes – these led to a reduction in attendance (See Brooks et al, 2008 Oxford Review of Education http://dx.doi.org/10.1080/03054980701768741)

  2. Fascinating. Main lesson I take is that educationalists are still struggling to design and report trials properly. Unregistered. Searching for comparability of groups at baseline came across this;

    “Because of the lotteries, the students and families in the evaluation’s treatment and control groups were expected to have similar characteristics—ones that could be observed, such as age, gender, and income, and ones that could not be observed or were difficult to observe, such as motivation to succeed in school and desire to attend a private school. In fact, the characteristics of the treatment and control groups were quite similar. For example, average reading scores at the time of application were 573 for the treatment group and 570 for the control group—the difference was not statistically significant.”

    A nice description of the benefits of randomisation, but in an 84 page report I couldn’t find a table of baseline characteristics by group.

    My worry would be that educationalists tend to be hostile to voucher systems. If so, failure to pre-register primary outcomes, analysis plans etc. gives ample scope for data dredging to reinforce preconceived ideas.

    But shouldn’t carp. A great improvement on the zero RCTs ever conducted by the 30 full professors of education in Nottingham!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s