Tag Archives: Patients

Patient and Public Involvement: Direct Involvement of Patient Representatives in Data Collection

It is widely accepted that the public and patient voice should be heard loud and clear in the selection of studies, in the design of those studies, and in the interpretation and dissemination of the findings. But what about involvement of patient and the public in the collection of data? Before science became professionalised, all scientists could have been considered members of the public. Robert Hooke, for example, could have called himself architect, philosopher, physicist, chemist, or just Hooke. Today, the public are involved in data collection in many scientific enterprises. For example, householders frequently contribute data on bird populations, and Prof Brian Cox involved the public in the detection of new planets in his highly acclaimed television series. In medicine, patients have been involved in collecting data; for example patients with primary biliary cirrhosis were the data collectors in a randomised trial.[1] However, the topic of public and patient involvement in data collection is deceptively complex. This is because there are numerous procedural safeguards governing access to users of the health service and that restrict disbursement of the funds that are used to pay for research.

Let us consider first the issue of access to patients. It is not permissible to collect research data without undergoing certain procedural checks; in the UK it is necessary to be ratified by the Disclosure and Barring Service (DBS) and to have necessary permissions from the institutional authorities. You simply cannot walk onto a hospital ward and start handing out questionnaires or collecting blood samples.

Then there is the question of training. Before collecting data from patients it is necessary to be trained in how to do so, covering both salient ethical and scientific principles. Such training is not without its costs, which takes us to the next issue.

Researchers are paid for their work and, irrespective of whether the funds are publically or privately provided, access to payment is governed by fiduciary and equality/diversity legislation and guidelines. Access to scarce resources is usually governed by some sort of competitive selection process.

None of the above should be taken as an argument against patients and the public taking part in data collection. It does, however, mean that this needs to be a carefully managed process. Of course things are very much simpler if access to patients is not required. For example, conducting a literature survey would require only that the person doing it was technically competent and in many cases members of the public would already have all, or some, of the necessary skills. I would be very happy to collaborate with a retired professor of physics (if anyone wants to volunteer!). But that is not the point. The point is that procedural safeguards must be applied, and this entails management structures that can manage the process.

Research may be carried out by accessing members of the public who are not patients, or at least who are not accessed through the health services. As far as I know there are no particular restrictions on doing so, and I guess that such contact is governed by the common law covering issues such as privacy, battery, assault, and so on. The situation becomes different, however, if access is achieved through a health service organisation, or conducted on behalf of an institution, such as a university. Then presumably any member of the public wishing to collect data from other members of the public would fall under the governance arrangements of the relevant institution. The institution would have to ensure not only that the study was ethical, but that the data-collectors had the necessary skills and that funds were disbursed in accordance with the law. Institutions already deploy ‘freelance’ researchers, so I presume that the necessary procedural arrangements are already in place.

This analysis was stimulated by a discussion in the PPI committee of CLAHRC West Midlands, and represents merely my personal reflections based on first principles. It does not represent my final, settled position, let alone that of the CLAHRC WM, or any other institution. Rather it is an invitation for further comment and analysis.

— Richard Lilford, CLAHRC WM Director


  1. Browning J, Combes B, Mayo MJ. Long-term efficacy of sertraline as a treatment for cholestatic pruritus in patients with primary biliary cirrhosis. Am J Gastroenterol. 2003; 98: 2736-41.

‘Information is not knowledge’: Communication of Scientific Evidence and how it can help us make the right decisions

Every one of us is required to make many decisions: from small decisions, such as what shoes to wear with an outfit or whether to have a second slice of cake; to larger decisions, such as whether to apply for a new job or what school to send our children to. For decisions where the outcome can have a large impact we don’t want to play a game of ‘blind man’s buff’ and make a decision at random. We do our utmost to ensure that whatever decision we arrive at, it is the right one. We go through a process of getting hold of information from a variety of sources we trust and processing that knowledge to help us make up our minds. And in this digital age, we have access to more information than ever before.

When it comes to our health, we are often invited to be involved in making shared decisions about our own care as patients. Because it’s our health that’s at stake, this can bring pressures of not only making a decision but also making the right decision. Arriving at a wrong decision can have significant consequences, such as over- or under-medication or missing out from advances in medicine. But how do we know how to make those decisions and where do we get our information from? Before we start taking a new course of medication, for example, how can we find out if the drugs are safe and effective, and how can we find out the risks as well as the benefits?

The Academy of Medical Sciences produced a report, ‘Enhancing the use of scientific evidence to judge the potential benefits and harms of medicine’,[1] which examines what changes would be necessary to help patients make better-informed decisions about taking medication. It is often the case that there is robust scientific evidence that can be useful in helping patients and clinicians make the right choices. However, this information can be difficult to find, hard to understand, and cast adrift in a sea of poor-quality or misleading information. With so much information available, some of it conflicting – is it any surprise that in a Medical Information Survey, almost two-thirds of British adults would trust experiences of friends and family compared to data from clinical trials, which only 37% of British adults would trust?[2]

The report offers recommendations on how scientific evidence can be made available to enable people to weigh up the pros and cons of new medications and arrive at a decision they are comfortable with. These recommendations include: using NHS Choices as a ‘go to’ hub of clear, up-to-date information about medications, with information about benefits and risks that is easy to understand; improving the design, layout and content of patient information leaflets; giving patients longer appointment times so they can have more detailed discussions about medications with their GP; and a traffic-light system to be used by the media to endorse the reliability of scientific evidence.

This is all good news for anyone having to decide whether to start taking a new drug. I would welcome the facility of going to a well-designed website with clear information about the risks and benefits of taking particular drugs rather than my current approach of asking friends and family (most of whom aren’t medically trained), searching online, and reading drug information leaflets that primarily present long lists of side-effects.

Surely this call for clear, accessible information about scientific evidence is just as relevant to all areas of medical research, including applied health. Patients and the public have a right to know how scientific evidence underpinning important decisions in care is generated and to be able to understand that information. Not only do patients and the public also make decisions about aspects of their care, such as whether to give birth at home or in hospital, or whether to take a day off work to attend a health check, but they should also be able to find and understand evidence that explains why care is delivered in a particular way, such as why many GPs now use a telephone triage system before booking in-person appointments. Researchers, clinicians, patients and communicators of research all have a part to play.

In CLAHRC West Midlands, we’re trying to ‘do our bit’. We aim to make accessible a sound body of scientific knowledge through different information channels and our efforts include:

  • Involving patients and the public to write lay summaries of our research projects on our website so people can find out about the research we do.
  • Communication of research evidence in accessible formats, such as CLAHRC BITEs, which are reviewed by our Public Advisors.
  • Method Matters, a series aimed to give members of the public a better understanding of concepts in Applied Health Research.

The recommendations from the Academy of Medical Sciences can provide a useful starting point for further discussions on how we can communicate effectively in applied health research and ensure that scientific evidence, rather than media hype or incomplete or incorrect information, is the basis for decision-making.

— Magdalena Skrybant, CLAHRC WM PPIE Lead


  1. The Academy of Medical Sciences. Enhancing the use of scientific evidence to judge the potential benefits and harms of medicine. London: Academy of Medical Sciences; 2017.
  2. The Academy of Medical Sciences. Academy of Medical Sciences: Medical Information Survey. London: Academy of Medical Sciences; 2016

Numbers and the Doctor/Patient Relationship

I have always been interested in communicating scientific information and probability. A paper co-authored by CLAHRC WM colleague Eivor Oborn [1] therefore caught my eye. The paper concerns numbers and their ‘performativity’, by which the authors mean how the numbers affect doctors, patients, and the interaction between doctors and patients. They use medical consultations in a Swedish rheumatology clinic to explore the issue, since this is a ‘data-rich’ environment. By this I mean charts are used to plot long-run numerical data relating to patient-reported outcomes, medical assessments, and laboratory data. The study shows that the numbers have high salience for patients who generally find graphical representation of long-run data useful. Doctors also find graphical display of trends useful in spotting threats to patient health. However, patients sometimes feel that the data on display take precedence over how they actually feel. That is to say, the doctor tends to focus on the numbers while the patient’s main symptom might not be captured in the numbers. Of course, there is no counterfactual, so how much of this dissatisfaction is caused by availability of numbers is uncertain. Also I felt that more could be said about the extent to which patients, and indeed doctors, really understand the meaning of the numbers they were seeing. Many people have poor numeracy skills and draw erroneous inferences from data. For instance, people tend to over-interpret improving trends following a run of high-values – the issue of regression to the mean, covered in the Method Matters section of a previous News Blog.

— Richard Lilford, CLAHRC WM Director


  1. Essén A & Oborn E. The performativity of numbers in illness management: The case of Swedish Rheumatology. Soc Sci Med. 2017; 184: 134-43.

Doctor-Patient Communication in the NHS

Andrew McDonald (former Chief Executive of Independent Parliamentary Standards Authority) was recently asked by the Marie Curie charity to examine the quality of doctor-patient communication in the NHS, as discussed on BBC Radio 4’s Today programme on 13 March 2017 (you can listen online). His report concluded that communication was woefully inadequate and that patients were not getting the clear and thorough counselling that they needed in order to understand their condition and make informed choices about options in their care. Patients need to understand what is likely to happen to them, and not all patients with the same condition will want to make the same choice(s). Indeed my own work [1] is part of a large body of research, which shows that better information leads to better knowledge, which in turn affects the choices that patients make. Evidence that the medical and caring professions do not communicate in an informative and compassionate way is therefore a matter of great concern.

However, there is a paradox – feedback from patients, that communication should lie at the heart of their care, has not gone unheard. For instance, current medical training is replete with “communication skills” instruction. Why then do patients still feel dissatisfied; why have matters not improved radically? My diagnosis is that good communication is not mainly a technical matter. Contrary to what many people think, the essence of good communication does not lie in avoiding jargon or following a set of techniques – a point often emphasised by my University of Birmingham colleague John Skelton. These technical matters should not be ignored – but they are not the nub of the problem.

In my view good communication requires effort, and poor communication reflects an unwillingness to make that effort; it is mostly a question of attitude. Good communication is like good teaching. A good communicator has to take time to listen and to tailor their responses to the needs of the individual patient. These needs may be expressed verbally or non-verbally, but either way a good communicator needs to be alive to them, and to respond in the appropriate way. Sometimes this will involve rephrasing an explanation, but in other cases the good communicator will respond to emotional cues. For example a sensitive doctor will notice if, in the course of a technical explanation, a patient looks upset – the good doctor will not ignore this cue, but will acknowledge the emotion, invite the patient to discuss his or her feelings, and be ready to deal with the flood of emotion that may result. The good doctor has to do emotional work, for example showing sympathy, not just in what is said, but also in how it is said. I am afraid to say that sometimes the busyness of the doctor is simply used as an excuse to avoid interactive engagements at a deeper emotional level. Yes, bringing feelings to the surface can be uncomfortable, but enduring the discomfort is part of professional life. In fact, recent research carried out by Gill Combes in CLAHRC WM showed that doctors are reticent in bringing psychological issues into the open.[2] Deliberately ignoring emotional clues and keeping things at a superficial level is deeply unsatisfying to patients. Glossing over feelings also impedes communication regarding more technical issues, as it is very hard for a person to assimilate medical information when they are feeling emotional, or nursing bruised feelings. In the long run such a technical approach to communication impoverishes a doctors professional life.

Doctors sometimes say that they should stick to the technical and that the often lengthy business of counselling should be carried out by other health professions, such as nurses. I have argued before that this is a blatant and unforgivable abrogation of responsibility; it vitiates values that lie (and always will lie) at the heart of good medical practice.[3] The huge responsibilities that doctors carry to make the right diagnosis and prescribe the correct treatment entail a psychological intimacy, which is almost unique to medical practice and which cannot easily be delegated. The purchase that a doctor has on a patient’s psyche should not be squandered. It is a kind of power, and like all power it may be wasted, misused or used to excellent effect.

The concept I have tried to explicate is that good communication is a function of ethical practice, professional behaviour and the medical ethos. It lies at the heart of the craft of medicine. If this point is accepted, it has an important corollary – the onus for teaching communication skills lies with medical practitioners rather than with psychologists or educationalists. Doctors must be the role models for other doctors. I was fortunate in my medical school in Johannesburg to be taught by professors of Oslerian ability who inspired me in the art of practice and the synthesis of technical skill and human compassion. Some people have a particular gift for communication with patients, but the rest of us must learn and copy, be honest with ourselves when we have fallen short, and always try to do better. The most important thing a medical school must do is to nourish and reinforce the attitudes that brought the students into medicine in the first place.

— Richard Lilford, CLAHRC WM Director


  1. Wragg JA, Robinson EJ, Lilford RJ. Information presentation and decisions to enter clinical trials: a hypothetical trial of hormone replacement therapy. Soc Sci Med. 2000; 51(3): 453-62.
  2. Combes G, Allen K, Sein K, Girling A, Lilford R. Taking hospital treatments home: a mixed methods case study looking at the barriers and success factors for home dialysis treatment and the influence of a target on uptake rates. Implement Sci. 2015; 10: 148.
  3. Lilford RJ. Two Ideas of What It Is to be a Doctor. NIHR CLAHRC West Midlands News Blog. August 14, 2015.

Evaluating Interventions to Improve the Integration of Care (Among Multiple Providers and Across Multiple Sites)

Typically healthcare improvement programmes have been institution specific examining, for example hospitals, general practices or care homes. While such solipsistic quality improvement initiatives obviously have their place, they also have severe limitations for the patient of today who typically has many complex conditions and whose care is therefore fragmented across many different care providers working in different places. Such patients perceive, and are sometimes the victims of, gaps in the system. Recent attention has therefore turned to approaches to close these gaps, and I am leading an NIHR programme development grant specifically for this purpose (Improving clinical decisions and teamwork for patients with multimorbidity in primary care through multidisciplinary education and facilitation). There are many different approaches to closing these gaps in care and the Nobel Prize winner Elinor Ostrom has featured previously in this News Blog for her seminal work on barriers and facilitators to institution collaboration [1]; while my colleague, CLAHRC WM Deputy Director Graeme Currie, has approached this issue from a management science perspective.

The problem for a researcher is to measure the effectiveness of initiatives to improve care across centres. This is not natural territory for cluster RCTs since it would be necessary to randomise whole ‘health economies’ rather than just organisations such as hospitals or general practices. Furthermore, many of the outcomes that might be observed in such studies, such as standardised mortality rates, are notoriously insensitive to change.[2] The ESTHER Project in Sweden is famous for closing gaps in care across the hospital/community nexus.[3] The evaluation, however, consists of little more than stakeholder interviews where people seem to recite the perceived wisdom of the day as evidence of effectiveness. While I think it is eminently plausible that the intervention was effective, and while the statements made during the qualitative interviews may have a certain verisimilitude, this all seems very weak evidence of effectiveness. It lacks any quantification, such as could be used in a health economic model. Is there a halfway house between a cluster RCT with hard outputs like mortality on the one hand, and ‘how was it for you?’ research on the other?

While it is not easy to come up with a measurement system, there is one person who perceives the entire pathway and that is the patient. The patient is really the only person who can provide an assessment of care quality across multiple providers. There are many patient measures. Some relate to outcome, for instance health and social care related quality of life (EQ-5DL, ASCOT SCT4 and OPOQL-brief [4]). Such measures should be used in service delivery studies, but may be insensitive to change, as stated above. It is therefore important to measure patient perception of the quality of their care. However, such measurements tend to either be non-specific (e.g. LTC-6 [5]) or look at only one aspect of care, such as continuity (PPCMC),[6] treatment burden [7] or person contentedness.[8] We propose a single quality of integrated care tool incorporating dimensions that have been shown to be important to patients and are collaborating with PenCLAHRC who are working on such a tool. Constructs that should be considered include conflicting information from different caregivers; contradicting forms of treatment (such as one clinician countermanding a prescription from another caregiver); duplication or redundancy of advice and information; satisfaction with care overall and with duration of contacts. We suspect that most patients would prefer fewer, more in-depth, contacts to a larger number of rushed contacts.

It might also be possible to design more imaginative qualitative research that goes beyond simply asking questions and uses method to elicit some of their deeper feelings, by prompting their memory. One such method is photo-voice where patients are asked to take photos in various points in their care, and use these as a basis for discussion. We have used such naturalistic settings in our CLAHRC.[9] Such methods could be harnessed in the co-design of services where patients / carers are not just asked how they perceive services, but are actively involved in designing solutions.

Salient quantitative measurements as may be obtained from NHS data systems. Hospital admission and readmission rates should be measured in studies of system-wide change. An effective intervention would result in more satisfied patients with lower rates of hospital admission. What about quantifying physical health? Adverse events in general and mortality in particular have poor sensitivity, such that signal, even after risk adjustment, would only emerge from noise in an extremely large study, or in a very high-risk client group – see ‘More on Integrated Care’ in this News Blog. Adverse events and death can be consolidated into generic health measurements (QALYs/DALYs), but, again, these are insensitive for reasons given above. Evaluating methods to improve the integration of care may be an ‘inconvenient truth scenario’ [10] where it is necessary to rely on process measures and other proxies for clinical / welfare out. Since our CLAHRC is actively exploring the evaluation of service interventions to improve integration of care, we would be very interested to hear from others and explore approaches to evaluating care across care boundaries.

— Richard Lilford, CLAHRC WM Director


  1. Ostrom E. Beyond Markets and States: Polycentric Governance of Complex Economic Systems. Am Econ Rev. 2010; 100(3): 641-72.
  2. Girling AJ, Hofer TP, Wu J, et al. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study. BMJ Qual Saf. 2012; 21(12): 1052-6.
  3. Institute for Healthcare Improvement. Improving Patient Flow: The Esther Project in Sweden. Boston, MA: Institute for Healthcare Improvement, 2011.
  4. Bowling A, Hankins M, Windle G, Bilotta C, Grant R. A short measure of quality of life in older age: the performance of the brief Older People’s Quality of Life questionnaire (OPQOL-brief). Arch Gerontol Geriatr. 2013; 56: 181-7.
  5. Glasgow RE, Wagner EH, Schaefer J, Mahoney LD, Reid RJ, Greene SM. Development and validation of the Patient Assessment of Chronic Illness Care (PACIC). Med Care. 2005; 43(5): 436-44.
  6. Haggerty JL, Robergr D, Freeman GK, Beaulieu C, Breton M. Validation of a generic measure of continuity of care: When patients encounter several clinicians. Ann Fam Med. 2012; 10: 443-51.
  7. Tran VT, Harrington M, Montori VM, Barnes C, Wicks P, Ravaud P. Adaptation and validation of the Treatment Burden Questionnaire (TBQ) in English using an internet platform. BMC Medicine. 2014; 12: 109.
  8. Mercer SW, Scottish Executive. Care Measure. Scottish Executive 2004
  9. Redwood S, Gale N, Greenfield S. ‘You give us rangoli, we give you talk’ – Using an art-based activity to elicit data from a seldom heard group. BMC Medl Res Methodol. 2012; 12: 7.
  10. Lilford RJ. Integrated Care. NIHR CLAHRC West Midlands News Blog. 19 June 2015.

A Disappointing Article

All that glitters in the fabled New England Journal of Medicine is not gold. A recent article by Dale and colleagues is a masterclass in producing pleasing-sounding statements, and truisms that go precisely nowhere, but impress the undiscerning reader.[1] They write an article in favour of using quality metrics to improve care. Then they show that process measures may focus attention on things that can be counted at the expense of more important things that cannot. So they say we should count “what’s important to patients”. Then they point out that the signal to noise ratio will not emerge in most cases where outcomes are used – patients value not dying from cancer, but you can never judge your clinician’s performance in screening by cancer death rates. They advocate a ‘balanced mixture’ of measures and advertise their own. But they do not say or prove that they have the right balance. And they admit that using payment to change behaviour is effete. But they say it is a good idea. The whole thing is a muddle. Truth is, no one knows how to use metrics in performance management. But we advocate for task-based (clinical process) measures to ensure that the essentials are in place. We think outcome measures are a poor idea except for patient satisfaction and maybe outcomes of a very small number of highly technical procedures.[2]

— Richard Lilford, CLAHRC WM Director


  1. Dale CR, Myint M, Compton-Phillips AL. Counting Better – the Limits and Future of Quality-Based Compensation. New Engl J Med. 2016; 375(7): 609-11.
  2. Lilford RJ. Risk Adjusted Outcomes – Again! NIHR CLAHRC West Midlands News Blog. 24 April 2015.

What is a GP for? – The End of the Fixed Consultation Period?

General Practitioners throughout the UK provide ten-minute consultation slots by default. An increasing portion of patients are seen for chronic disease and increasingly routine consultation are carried out by practice nurses. Telephone consultations are routine in many practices. The idea is that the GP should see the more complex cases. In theory this skill substitution should liberate time allowing GPs to provide larger default slots to patients. However, this has not happened, likely because demand has increased. Given that a further expansion in the GP cadre is not going to happen any time soon, should this state of affairs be accepted? It seems to me that the status quo cannot be endured – at least not without concerted effort to change it.

Consider first the patient. Can the needs of the patient really be met in such a short consultation unless they are very straightforward? And if they are straightforward, then is a highly trained and expensive professional needed? Dealing with depression, managing refractory blood pressure in a diabetic patient who is also complaining of a sore foot, counselling a patient who is considering whether or not to have a third round of chemotherapy – these are not ten minute tasks, but they are the stuff of primary care.

If you cannot provide compassionate care in five minutes, then how can you make a diagnosis? You need to record the symptoms, ask the patient to undress, elicit signs, and think about what it all means. Twenty-two percent of cancers present as emergencies,[1] and many of these patients have been seen by GPs before presentation.[2]

And what about the provider? I am consumer-orientated by philosophy, but at some point the provider interest becomes the consumer interest. Imagine consulting all day long, five days a week, for 45 years, at six patients per hour! Contrast that with an “old-fashioned” doctor; deliver a baby, consult in rooms, set a fracture, few more consultations, home visits.

Back in the 1950s the great Michael Balint spotted the problem. He advocated fewer consultations in greater detail for non-psychotic mental illness.[3] [4] So the idea is not new, and the challenge now is to find a way to make best use of advanced medical skills. Lots of things that seemed inviolate have been changed by human agency – the closing of the great mental asylums to be replaced by community care, for example. So let’s model how the service could look based on real patient lists. Then let’s simulate different methods to change the pattern of care, invoking plentiful skill substitution and perhaps greater reliance on technology and self-help. Then we could pilot it and finally roll it out, seeking buy-in from professional bodies. In fact, reverse that – let the professional bodies make the running while researchers in organisations such as CLAHRCs provide scientific ballast.

— Richard Lilford, CLAHRC WM Director


  1. National Cancer Intelligence Network. Routes to Diagnosis 2006-13. 2015.
  2. Kidney E, Berkman L, Macherianakis A, et al. Preliminary results of a feasibility study of the use of information technology for identification of suspected colorectal cancer in primary care: the CREDIBLE study. Br J Cancer. 2015; 112(s1): s70-6.
  3. Balint M. The Doctor, his Patient, and the Illness. Lancet. 1955; 265(6866): 683-8.
  4. Balint M. The Doctor, his Patient, and the Illness. 2nd Edinburgh: Churchill Livingstone, 2000.

Assessing Patient-Centred Care Through Direct Observation of Clinical Encounters

Ironically the most private and sensitive part of medical care is also one of the most accessible to measurement – yes doctor-patient psychological interaction (communication is too shallow a word) can be observed directly and does not suffer from poor signal to noise ratios since it differs so much between doctors and across contexts. It can be calibrated on a continuous or ordinal scale, and thus tends to be measured with high precision. The CLAHRC WM Director does not know how reliable it is, but it is undoubtedly hugely important. Stojan and colleagues [1] write a very nice paper about this subject and remind us of the chilling finding that doctors tend to adopt a different style for different ethnic groups. It is important that we have this mirror in place. There is no asymptote to improved interaction with patients, and we should feel good about ourselves because we have faced up to our imperfections and are trying to improve, rather than bad because we are not perfect in this – by far the most difficult – aspect of professional practice.

— Richard Lilford, CLAHRC WM Director


  1. Stojan JN, Clay MA, Lypson ML. Assessing patient-centred care through direct observation of clinical encounters. BMJ Qual Saf. 2016; 25: 135-7.

World’s First Randomised Trial of Public Involvement in Health Care and Policy Decisions

Evaluating the effects of public and patient involvement in priority setting in health care is notoriously tricky. However, two recent studies reported on a cluster RCT of enhanced public involvement among six health authorities in Canada.[1] [2] The results favoured the intervention, in that the priorities selected were different across the two groups and “Professionals’ choices moved toward indicators prioritized by the public (eg, access), and public representatives’ choices moved toward indicators prioritized by professionals (eg, self-care support).” A great strength of the study is the use of mixed methods advocated by CLAHRC WM – actually by all CLAHRCs. The public involvement was very carefully implemented with thorough induction of public representatives and moderation of the discussions, and the public representatives were selected because they were thought to have legitimacy and to represent different interests. These are landmark papers. In CLARHC WM what gets studied is what is implemented. The corollary is that there is no point in asking people what they want to research without also asking them what they think should be implemented. We therefore ensure that the members of the public who advise on the research are largely the same as those advising on the service.

— Richard Lilford, CLAHRC WM Director


  1. Boivin A, Lehoux P, Burgers J, Grol R. What Are the Key Ingredients for Effective Public Involvement in Health Care Improvement and Policy Decisions? A Randomized Trial Process Evaluation. Milbank Q. 2014. 92(2):319-50.
  2. Boivin A, Lehoux P, Lacombe R, Burgers J, Grol R. Involving patients in setting priorities for healthcare improvement: a cluster randomized trial. Implement Sci. 2014; 9:24.