Tag Archives: Public

Patient and Public Involvement in Data Collection

Further to last fortnight’s News blog article [1] I have found a further study in which patients participated in data collection.[2] This paper, by and large, corroborates the procedural requirements for public and patient involvement in data collection that I had specified. For example, it was necessary for lay observers to undergo DBS checks; the ethics approval form had to include lay observers; and training had to be arranged for the lay observers. Recruitment of lay observers proved more difficult than anticipated. The lay observers had a positive experience and brought a different perspective to the research according to feedback. The extent to which observer perspective is a good thing is, however, contestable. Generally I think the role of the observer is to collect data for analysis, and not colour it with a ‘perspective’. The professional researchers on the project felt that having lay researchers involved increased their workloads. The thorny issues of payment and selection do not seem to have been fully discussed in this paper. Also not discussed was the idea that, in qualitative research, respondents may be less inhibited to disclose information to a lay observer. Let the debate continue!

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ. Patient and Public Involvement: Direct Involvement of Patient Representatives in Data Collection. NIHR CLAHRC West Midlands News Blog. 4 August 2017.
  2. Garfield S, Jheeta S, Jacklin A, Bischler A, Norton C, Franklin BD. Patient and public involvement in data collection for health services research: a descriptive study. Res Involve Engage. 2015; 1: 8.

Patient and Public Involvement: Direct Involvement of Patient Representatives in Data Collection

It is widely accepted that the public and patient voice should be heard loud and clear in the selection of studies, in the design of those studies, and in the interpretation and dissemination of the findings. But what about involvement of patient and the public in the collection of data? Before science became professionalised, all scientists could have been considered members of the public. Robert Hooke, for example, could have called himself architect, philosopher, physicist, chemist, or just Hooke. Today, the public are involved in data collection in many scientific enterprises. For example, householders frequently contribute data on bird populations, and Prof Brian Cox involved the public in the detection of new planets in his highly acclaimed television series. In medicine, patients have been involved in collecting data; for example patients with primary biliary cirrhosis were the data collectors in a randomised trial.[1] However, the topic of public and patient involvement in data collection is deceptively complex. This is because there are numerous procedural safeguards governing access to users of the health service and that restrict disbursement of the funds that are used to pay for research.

Let us consider first the issue of access to patients. It is not permissible to collect research data without undergoing certain procedural checks; in the UK it is necessary to be ratified by the Disclosure and Barring Service (DBS) and to have necessary permissions from the institutional authorities. You simply cannot walk onto a hospital ward and start handing out questionnaires or collecting blood samples.

Then there is the question of training. Before collecting data from patients it is necessary to be trained in how to do so, covering both salient ethical and scientific principles. Such training is not without its costs, which takes us to the next issue.

Researchers are paid for their work and, irrespective of whether the funds are publically or privately provided, access to payment is governed by fiduciary and equality/diversity legislation and guidelines. Access to scarce resources is usually governed by some sort of competitive selection process.

None of the above should be taken as an argument against patients and the public taking part in data collection. It does, however, mean that this needs to be a carefully managed process. Of course things are very much simpler if access to patients is not required. For example, conducting a literature survey would require only that the person doing it was technically competent and in many cases members of the public would already have all, or some, of the necessary skills. I would be very happy to collaborate with a retired professor of physics (if anyone wants to volunteer!). But that is not the point. The point is that procedural safeguards must be applied, and this entails management structures that can manage the process.

Research may be carried out by accessing members of the public who are not patients, or at least who are not accessed through the health services. As far as I know there are no particular restrictions on doing so, and I guess that such contact is governed by the common law covering issues such as privacy, battery, assault, and so on. The situation becomes different, however, if access is achieved through a health service organisation, or conducted on behalf of an institution, such as a university. Then presumably any member of the public wishing to collect data from other members of the public would fall under the governance arrangements of the relevant institution. The institution would have to ensure not only that the study was ethical, but that the data-collectors had the necessary skills and that funds were disbursed in accordance with the law. Institutions already deploy ‘freelance’ researchers, so I presume that the necessary procedural arrangements are already in place.

This analysis was stimulated by a discussion in the PPI committee of CLAHRC West Midlands, and represents merely my personal reflections based on first principles. It does not represent my final, settled position, let alone that of the CLAHRC WM, or any other institution. Rather it is an invitation for further comment and analysis.

— Richard Lilford, CLAHRC WM Director


  1. Browning J, Combes B, Mayo MJ. Long-term efficacy of sertraline as a treatment for cholestatic pruritus in patients with primary biliary cirrhosis. Am J Gastroenterol. 2003; 98: 2736-41.

‘Information is not knowledge’: Communication of Scientific Evidence and how it can help us make the right decisions

Every one of us is required to make many decisions: from small decisions, such as what shoes to wear with an outfit or whether to have a second slice of cake; to larger decisions, such as whether to apply for a new job or what school to send our children to. For decisions where the outcome can have a large impact we don’t want to play a game of ‘blind man’s buff’ and make a decision at random. We do our utmost to ensure that whatever decision we arrive at, it is the right one. We go through a process of getting hold of information from a variety of sources we trust and processing that knowledge to help us make up our minds. And in this digital age, we have access to more information than ever before.

When it comes to our health, we are often invited to be involved in making shared decisions about our own care as patients. Because it’s our health that’s at stake, this can bring pressures of not only making a decision but also making the right decision. Arriving at a wrong decision can have significant consequences, such as over- or under-medication or missing out from advances in medicine. But how do we know how to make those decisions and where do we get our information from? Before we start taking a new course of medication, for example, how can we find out if the drugs are safe and effective, and how can we find out the risks as well as the benefits?

The Academy of Medical Sciences produced a report, ‘Enhancing the use of scientific evidence to judge the potential benefits and harms of medicine’,[1] which examines what changes would be necessary to help patients make better-informed decisions about taking medication. It is often the case that there is robust scientific evidence that can be useful in helping patients and clinicians make the right choices. However, this information can be difficult to find, hard to understand, and cast adrift in a sea of poor-quality or misleading information. With so much information available, some of it conflicting – is it any surprise that in a Medical Information Survey, almost two-thirds of British adults would trust experiences of friends and family compared to data from clinical trials, which only 37% of British adults would trust?[2]

The report offers recommendations on how scientific evidence can be made available to enable people to weigh up the pros and cons of new medications and arrive at a decision they are comfortable with. These recommendations include: using NHS Choices as a ‘go to’ hub of clear, up-to-date information about medications, with information about benefits and risks that is easy to understand; improving the design, layout and content of patient information leaflets; giving patients longer appointment times so they can have more detailed discussions about medications with their GP; and a traffic-light system to be used by the media to endorse the reliability of scientific evidence.

This is all good news for anyone having to decide whether to start taking a new drug. I would welcome the facility of going to a well-designed website with clear information about the risks and benefits of taking particular drugs rather than my current approach of asking friends and family (most of whom aren’t medically trained), searching online, and reading drug information leaflets that primarily present long lists of side-effects.

Surely this call for clear, accessible information about scientific evidence is just as relevant to all areas of medical research, including applied health. Patients and the public have a right to know how scientific evidence underpinning important decisions in care is generated and to be able to understand that information. Not only do patients and the public also make decisions about aspects of their care, such as whether to give birth at home or in hospital, or whether to take a day off work to attend a health check, but they should also be able to find and understand evidence that explains why care is delivered in a particular way, such as why many GPs now use a telephone triage system before booking in-person appointments. Researchers, clinicians, patients and communicators of research all have a part to play.

In CLAHRC West Midlands, we’re trying to ‘do our bit’. We aim to make accessible a sound body of scientific knowledge through different information channels and our efforts include:

  • Involving patients and the public to write lay summaries of our research projects on our website so people can find out about the research we do.
  • Communication of research evidence in accessible formats, such as CLAHRC BITEs, which are reviewed by our Public Advisors.
  • Method Matters, a series aimed to give members of the public a better understanding of concepts in Applied Health Research.

The recommendations from the Academy of Medical Sciences can provide a useful starting point for further discussions on how we can communicate effectively in applied health research and ensure that scientific evidence, rather than media hype or incomplete or incorrect information, is the basis for decision-making.

— Magdalena Skrybant, CLAHRC WM PPIE Lead


  1. The Academy of Medical Sciences. Enhancing the use of scientific evidence to judge the potential benefits and harms of medicine. London: Academy of Medical Sciences; 2017.
  2. The Academy of Medical Sciences. Academy of Medical Sciences: Medical Information Survey. London: Academy of Medical Sciences; 2016

Government vs. Private Schools

CLAHRC WM is not just interested in health care since the methods we use are equally relevant to decision-makers in education, social services, industrial policy, criminology, and so on. We should all be learning from each other. In a previous blog I reported on the (mostly positive) results of the ‘Moving to Opportunity’ experiment in the USA, where families were given an opportunity to move from a deprived neighbourhood to a more salubrious one. So I was interested to spot an RCTs of vouchers that allowed children (over a wide age range) from government schools to attend private schools (also in the USA).[1] The experiment was recent (last five years) and we have outcomes at one year only. Seventy percent of pupils allocated a voucher to attend a private school took up their offer; so both intention to treat and per protocol analyses are reported. The educational outcomes were lower in the intervention group, and were statistically significantly lower for mathematics. This negative effect was greater if the voucher was taken up than if it was not. The negative effect was greater if the child came from a school that was not rated as poor performing than if the previous school was rated satisfactory or good. The negative effect was greatest if the child was in elementary school, and non-significantly positive if they were already in high school.

What caused the negative effect on educational outcomes? Simply moving school does not seem to explain the results, since a proportion of control children moved school with little or no apparent effect. However, private schools provide less instructional time than government schools, especially in elementary school. Other studies have also noted negative effects of moving children to private school on educational outcomes in the short term. But it is far too early to declare the intervention a failure. There is a limit to how much an elementary school child can assimilate, and it is the long-term effects that are important. However, I was surprised by this result – educational interventions have a habit of producing results different to those intended. Full marks to the US Congress, which had the wisdom to evaluate its own policies. The UK Cabinet Office has published a document arguing for more RCTs of policy,[2] and I expect to be able to report the results of further RCTs of educational interventions in the News Blog.

— Richard Lilford, CLAHRC WM Director


  1. Dynarski M, Rui N, Webber A, Gutmann B, Bachman M. Evaluation of the DC Opportunity Scholarship Program. Impacts After One Year. Alexandria, VA: Institute of Education Sciences, 2017.
  2. Haynes L, Service O, Goldacre B, Torgerson D. Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials. London: UK Cabinet Office, 2012.

Private Consultations More Effective than Public Provision in Rural India

Doing work across high-income countries (CLAHRC WM) and lower income countries (CLAHRC model for Africa) provides interesting opportunities to compare and contrast. For example, our work on user fees in Malawi [1] mirrors that in high-income countries [2] – in both settings, relatively small increments in out-of-pocket expenses results in a large decrease in demand and does so indiscriminately (the severity of disease among those who access services is not shifted towards more serious cases). However, the effect of private versus public provision of health care is rather more nuanced.

News Blog readers are likely aware of the famous RAND study in the US.[3] People were randomised to receive their health care on a fee-for-service basis (‘privately’) vs. on a block contract basis (as in a public service). The results showed that fee-for-service provision resulted in more services being provided (interpreted as over-servicing), but that patients were more satisfied clients, compared to those experiencing public provision. Clinical quality was no different. In contrast, a study from rural India [4] found that private provision results in markedly improved quality compared to public provision, albeit with a degree of over-servicing.

The Indian study used ‘standardised patients’ (SPs) to measure the quality of care during consultations covering three clinical scenarios – angina, asthma and the parent of a child with dysentery. The care SPs received was scored against an ideal standard. Private providers spent more time/effort collecting the data essential for making a correct diagnosis, and were more likely to give treatment appropriate to the condition. First, they compared private providers with public providers and found that the former spent 30% more time gathering information from the SPs than the public providers. Moreover, the private providers were more likely to be present when the patient turned up for a consultation. There was a positive correlation between the magnitude of fees charged by private providers and time spent eliciting symptoms and signs, and the probability that the correct treatment would be provided. However, the private providers are often not doctors, so this result could reflect different professional mix, at least in part. To address this point, a second study was done whereby the same set of doctors were presented with the same clinical cases – a ‘dual sample’. The results were even starker, with doctors spending twice as long with each patient when seen privately.

Why were these results from rural India so different from the RAND study? The authors suggest that taking a careful history and examination is part of the culture for US doctors, and that they had reach a kind of asymptote, such that context made little difference to this aspect of their behaviour. Put another way, there was little headroom for an incentive system to drive up quality of care. However, in low-income settings where public provision is poorly motivated and regulated, fee-for-service provision drives up quality. The same seems to apply to education, where private provision was found to be of higher quality than public provision in low-income settings – see previous News Blog.[5]

However, it should be acknowledged that none of the available alternatives in rural India were good ones. For example, the probability of receiving the correct diagnosis varied across the private and public provider, but never exceeded 15%, while the rate of correct treatment varied from 21% to about 50%. Doctors were more likely than other providers to provide the correct diagnosis. A great deal of treatment was inappropriate. CLAHRC West Midlands’ partner organisation in global health is conducting a study of service provision in slums with a view to devising affordable models of improving health care.[6]

— Richard Lilford, CLAHRC WM Director


  1. Watson SI, Wroe EB, Dunbar EL, et al. The impact of user fees on health services utilization and infectious disease diagnoses in Neno District, Malawi: a longitudinal, quasi-experimental study. BMC Health Serv Res. 2016; 16: 595.
  2. Carrin G & Hanvoravongchai P. Provider payments and patient charges as policy tools for cost-containment: How successful are they in high-income countries? Hum Resour Health. 2003; 1: 6.
  3. Brook RH, Ware JE, Rogers WH, et al. The effect of coinsurance on the health of adults. Results from the RAND Health Insurance Experiment. Santa Monica, CA: RAND Corporation, 1984.
  4. Das J, Holla A, Mohpal A, Muralidharan K. Quality and Accountability in Healthcare Delivery: Audit-Study Evidence from Primary Care in India . Am Econ Rev. 2016; 106(12): 3765-99.
  5. Lilford RJ. League Tables – Not Always Bad. NIHR CLAHRC West Midlands News Blog. 28 August 2015.
  6. Lilford RJ. Between Policy and Practice – the Importance of Health Service Research in Low- and Middle-Income Countries. NIHR CLAHRC West Midlands News Blog. 27 January 2017.

World’s First Randomised Trial of Public Involvement in Health Care and Policy Decisions

Evaluating the effects of public and patient involvement in priority setting in health care is notoriously tricky. However, two recent studies reported on a cluster RCT of enhanced public involvement among six health authorities in Canada.[1] [2] The results favoured the intervention, in that the priorities selected were different across the two groups and “Professionals’ choices moved toward indicators prioritized by the public (eg, access), and public representatives’ choices moved toward indicators prioritized by professionals (eg, self-care support).” A great strength of the study is the use of mixed methods advocated by CLAHRC WM – actually by all CLAHRCs. The public involvement was very carefully implemented with thorough induction of public representatives and moderation of the discussions, and the public representatives were selected because they were thought to have legitimacy and to represent different interests. These are landmark papers. In CLARHC WM what gets studied is what is implemented. The corollary is that there is no point in asking people what they want to research without also asking them what they think should be implemented. We therefore ensure that the members of the public who advise on the research are largely the same as those advising on the service.

— Richard Lilford, CLAHRC WM Director


  1. Boivin A, Lehoux P, Burgers J, Grol R. What Are the Key Ingredients for Effective Public Involvement in Health Care Improvement and Policy Decisions? A Randomized Trial Process Evaluation. Milbank Q. 2014. 92(2):319-50.
  2. Boivin A, Lehoux P, Lacombe R, Burgers J, Grol R. Involving patients in setting priorities for healthcare improvement: a cluster randomized trial. Implement Sci. 2014; 9:24.