Tag Archives: Hospitals

A Heretical Suggestion!

The locus of health care is moving increasingly towards the community. In high-income countries (HIC) the greatest burden of health falls to frail elderly people with multiple chronic diseases. Hospital is often bad news for such people, both from a psychological and physical point of view.[1] There are good arguments for avoidance of admissions, and for increasing care provision in the community in HICs. In low- and middle-income countries (LMICs) there are also good arguments for community care. The WHO estimates that over three-quarters of all care could be most appropriately delivered in the community. The Declaration of Alma Ata and the Bamako Initiative from the United Nations both support the development of community care for LMICs. In this News Blog I wish to probe this subject more deeply. I will argue that community care is entirely appropriate for preventive outreach care. However, I will argue that we should re-examine the case for promoting community over hospital settings for demand-led care, especially in deprived urban environment.

My re-examination of this subject came about as a result of recent tours of eight slums within Nigeria, Kenya, Pakistan and Bangladesh. While all of these areas have a strong need for supply-side preventive care in the community, I have come to question the wisdom of trying to develop demand-led care within slum localities. My misgivings are based on a number of personal observations and from a reading of the relevant literature.

On site observations suggest that local residents prefer to use hospital facilities, even when this is less convenient than a more accessible community clinic. Some, but not all, slums are reasonably well supplied by local clinics. These clinics are usually staffed by medical officers or nurses rather than doctors. In many cases they have been provided by NGOs. I have observed that these clinics do not have many clients. When I draw attention to this, I am often told that this is because I have come at a quiet period. However, when I probe more deeply, I learn that the outpatients departments of nearby hospitals receive the bulk of the patients. Certainly that is my impression on visits to hospitals in LMICs where outpatient departments ‘heave’ with patients. This finding triangulates with work that colleagues and I have carried out in India under MRC sponsorship.

Not only do local residents seem to prefer hospital-based outpatient’s care, but my reading of the literature suggests that they are right to do so. Working with colleagues, I am carrying out a review of the quality of care provided in local settings in LMICs. The literature shows that such care is almost universally of a low standard, irrespective of whether the provider is private or public. Care given by doctors is generally better than that given by non-medical personnel, but even so is of a poor standard when delivered in the community. The quality of care across both settings is a topic of enquiry in the NIHR Unit on Health Service Provision in Slums that I direct. However, I suspect that the hospital will come out on top.

The corollary of the above, rather preliminary findings, is that we should be cautious about wholesale, and perhaps ideologically-driven, policies to deliver demand-based healthcare coverage in community settings  in low-income urban environments. Pending further research I hypothesise that it may be better to improve access and quality in hospital settings, at least in the first instance. Before taking fixed positions on these important issues we need to understand more about access to healthcare at the demand-side, the quality of such healthcare, and the most-cost effective approaches to driving up the quality of health care.

Please note that all of the above remarks apply to healthcare at the demand-side. That is to say, where a person has sought care for a perceived health problem. We fully support outreach primary preventive services to ensure vaccination, detect malnutrition, and ensure that people stick to their HIV and other treatment regimes.

Box A. Section VI of the Declaration of Alma-Ata

Primary health care is essential health care based on practical, scientifically sound and socially acceptable methods and technology made universally accessible to individuals and families in the community [emphasis added] through their full participation and at a cost that the community and country can afford to maintain at every stage of their development in the spirit of selfreliance and self-determination. It forms an integral part both of the country’s health system, of which it is the central function and main focus, and of the overall social and economic development of the community. It is the first level of contact of individuals, the family and community with the national health system bringing health care as close as possible to where people live and work, and constitutes the first element of a continuing health care process.”

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ. Intensive Care Harmful in Elderly Patients. NIHR CLAHRC West Midlands News Blog. 7 December 2017.

Patient’s experience of hospital care at weekends

The “weekend effect”, whereby patients admitted to hospitals during weekends appear to be associated with higher mortality compared with patients who are admitted during weekdays, has received substantial attention from the health service community and the general public alike.[1] Evidence of the weekend effect was used to support the introduction of ‘7-day Service’ policy and associated changes to junior doctor’s contracting arrangement by the NHS,[2-4] which have further propelled debates surrounding the nature and causes of the weekend effect.

Members of the CLAHRC West Midlands are closely involved in the HiSLAC project,[5] which is an NIHR HS&DR Programme funded project led by Professor Julian Bion (University of Birmingham) to evaluate the impact of introducing 7-day consultant-led acute medical services. We are undertaking a systematic review of the weekend effect as part of the project,[6] and one of our challenges is to catch up with the rapidly growing literature fuelled by the public and political attention. Despite that hundreds of papers on this topic have been published, there has been a distinct gap in the academic literature – most of the published papers focus on comparing hospital mortality rates between weekends and weekdays, but virtually no study have compared quantitatively the experience and satisfaction of patients between weekends and weekdays. This was the case until we found a study recently published by Chris Graham of the Picker Institute, who has unique access to data not in the public domain, i.e. the dates of admission to hospital given by the respondents.[7]

This interesting study examined data from two nationwide surveys of acute hospitals in 2014 in England: the A&E department patient survey (with 39,320 respondents representing a 34% response rate) and the adult inpatient survey (with 59,083 respondents representing a 47% response rate). Patients admitted at weekends were less likely to respond compared to those admitted during weekdays, but this was accounted for by patient and admission characteristics (e.g. age groups). Contrary to the inference that would be made on care quality based on hospital mortality rates, respondents attending hospital A&E department during weekends actually reported better experiences with regard to ‘doctors and nurses’ and ‘care and treatment’ compared with those attending during weekdays. Patients who were admitted to hospital through A&E during weekends also rated information given to them in the A&E more favourably. No other significant differences in the reported patient experiences were observed between weekend and weekday A&E visits and hospital admissions. [7]

As always, some cautions are needed when interpreting these intriguing findings. First, as the author acknowledged, patients who died following the A&E visits/admissions were excluded from the surveys, and therefore their experiences were not captured. Second, although potential differences in case mix including age, sex, urgency of admission (elective or not), requirement of a proxy for completing the surveys and presence of long-term conditions were taken into account in the aforementioned findings, the statistical adjustment did not include important factors such as main diagnosis and disease severity which could confound patient experience. Readers may doubt whether these factors could overturn the finding. In that case the mechanisms by which weekend admission may lead to improved satisfaction Is unclear. It is possible that patients have different expectations in terms of hospital care that they receive by day of the week and consequently may rate the same level of care differently. The findings from this study are certainly a very valuable addition to the growing literature that starts to unfold the complexity behind the weekend effect, and are a further testament that measuring care quality based on mortality rates alone is unreliable and certainly insufficient, a point that has long been highlighted by the Director of the CLAHRC West Midlands and other colleagues.[8] [9] Our HiSLAC project continues to collect and examine qualitative,[10] quantitative,[5] [6] and economic [11] evidence related to this topic, so watch the space!

— Yen-Fu Chen, Principal Research Fellow


  1. Lilford RJ, Chen YF. The ubiquitous weekend effect: moving past proving it exists to clarifying what causes it. BMJ Qual Saf 2015;24(8):480-2.
  2. House of Commons. Oral answers to questions: Health. 2015. House of Commons, London.
  3. McKee M. The weekend effect: now you see it, now you don’t. BMJ 2016;353:i2750.
  4. NHS England. Seven day hospital services: the clinical case. 2017.
  5. Bion J, Aldridge CP, Girling A, et al. Two-epoch cross-sectional case record review protocol comparing quality of care of hospital emergency admissions at weekends versus weekdays. BMJ Open 2017;7:e018747.
  6. Chen YF, Boyal A, Sutton E, et al. The magnitude and mechanisms of the weekend effect in hospital admissions: A protocol for a mixed methods review incorporating a systematic review and framework synthesis. Systems Review 2016;5:84.
  7. Graham C. People’s experiences of hospital care on the weekend: secondary analysis of data from two national patient surveys. BMJ Qual Saf 2017;29:29.
  8. Girling AJ, Hofer TP, Wu J, et al. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study. BMJ Qual Saf 2012;21(12):1052-56.
  9. Lilford R, Pronovost P. Using hospital mortality rates to judge hospital performance: a bad idea that just won’t go away. BMJ 2010;340:c2016.
  10. Tarrant C, Sutton E, Angell E, Aldridge CP, Boyal A, Bion J. The ‘weekend effect’ in acute medicine: a protocol for a team-based ethnography of weekend care for medical patients in acute hospital settings. BMJ Open 2017;7: e016755.
  11. Watson SI, Chen YF, Bion JF, Aldridge CP, Girling A, Lilford RJ. Protocol for the health economic evaluation of increasing the weekend specialist to patient ratio in hospitals in England. BMJ Open 2018:In press.

Reducing Radiation Risk from Hospital Scans

Even though it is something carried out in hospitals hundreds of times a day, X-rays and CT (computed topography) scans are procedures that expose the patient to radiation. Yes, the radiation dosage for the majority of scans carried out is very little when compared to every day exposure; for example an X-ray of the arm is 0.001 mSv (millisievert), a dental X-ray is 0.005 mSv, a chest X-ray is 0.020 mSv – in comparison the average background radiation received over one day is 0.010 mSV, while someone flying across the continental USA would receive 0.040 mSV. However, other scans are higher, a mammogram is 0.400 mSv (equivalent of 40 days worth of exposure in one dose), while a head CT scan gives a dose of 2 mSv (equivalent to ~7 months) and a chest CT scan 7 mSv (equivalent to ~20 months) (see the below image from Randall Munroe for more examples).

Although the cells in our body are able to repair and restore DNA damage resulting from radiation, the greater the dose received in one go, and the greater received in the long-term, the more likely it is that damage won’t be repaired correctly. Thus, we should aim to reduce patients’ exposure to radiation where possible. A recent paper by Kitchen and colleagues may have an answer by using phase-contrast x-ray imaging.[1] Because soft tissue has similar X-ray absorption properties to bone, which results in poor image contrast the radiation dosage has to be increased in standard scans. This new technique combines CT scans with phase retrieval and an algorithm to define edges, densities, etc. and results in a reduction in dosage by a factor of 300 fold (with the potential for a reduction factor in the tens of thousands), while still retaining equivalent image quality. Although the study only tested this in an animal model it is an important first step.

092 DCvi - radiation

— Peter Chilton, Research Fellow


  1. Kitchen MJ, Buckley GA, Gureyev TE, et al. CT dose reduction factors in the thousands using X-ray phase contrast. Sci Rep. 2017; 7: 15953.

Fair is Fair: Preventing the Misuse of Visiting Hours to Reduce Inequities

The experience of healthcare as a social activity feels very different when viewed from the perspectives of the patient, their relatives, or the healthcare staff. The patient is the centre of attention, but profoundly dependent; the relatives are independent, but unempowered and on foreign ground; and the staff are on home territory and authoritative. These unequal relationships come into sharp focus in the emotionally charged context of critical illness and the Intensive Care Unit. Which of us would not want our family to be near to us and supported by the staff in such a situation? And yet surveys repeatedly show that there is wide variation between countries in national policies, that restrictive visiting is common in practice, and that there is wide variation between ICUs in how those policies are applied.[1-3] Why should this be so?

When patients are asked, they express a strong preference to be visited by their relatives.[4] Involvement of relatives in their loved one’s care has been linked to improved outcomes in a number of conditions, including stroke.[5] [6] However, nursing staff attitudes to visiting [7] reveal concerns about the additional workload involved in caring for and communicating with relatives, and that their presence by the bedside might impede delivery of care, adversely affect infection control, or result in exhaustion of family members. Deeper enquiry might well reveal a lack of empathy and professional confidence: anxiety about being constantly observed by family members, or that lapses in care might result in criticism.

Netzer and Iwashyna take a social justice perspective to argue that this is wrong, and that ICUs should implement current national best practice guidance by making open visiting for families the default,[8] thereby avoiding selection bias in permitting or restricting access. The authors argue that excluding families from their relative’s care can impact negatively on both the patient and relative. The visiting hours offered to relatives may be misaligned with their working hours, creating a further obstacle for those with less flexibility and support from their employer, especially in a society where zero hour contracts are more common.

Moreover, staff discretion to vary these restrictions creates opportunities for conscious or unconscious selection bias. The authors describe a personal experience in which visiting hours reinforced the racial inequalities seen in US healthcare. Such biases might also affect other minorities such as same-sex couples, or transgender communities. Training in equality and diversity organised by NHS Trusts might minimise conscious bias, but the fact remains that while restricted visiting is the default, discretion increases the opportunity for social discrimination.

In considering an open visiting policy, attention must be paid to the potential negatives this may pose. Organisations will be conscious of staff limitations and resources, and the potential for abusive/disruptive family members. Ethnic minority or migrant families bring with them different cultural norms and behaviours which may impact adversely on the family members of indigenous patients. Implementation of open visiting would need to include contingencies to cope with such events as they occur, an example being training staff to have the necessary skills and behaviours to deal with such situations. We are working on this as part of the HS&DR-funded PEARL Project (Patient Experience And Reflective Learning), which also includes interventions designed to maximise empathy.[9] Ultimately, the level of involvement of relatives in their family members’ care should be a decision made by the patient and the family and supported by professionally confident and compassionate staff.

— Olivia Brookes, PEARL Project Manager;
— Prof Julian Bion, PEARL Chief Investigator, Professor of Intensive Care Medicine


  1. Liu V, Read JL, Scruth E, Cheng E. Visitation policies and practices in US ICUs. Crit Care. 2013; 17(2):R71.
  2. Giannini A, Miccinesi G, Leoncino S. Visiting policies in Italian intensive care units: a nationwide survey. Intensive Care Med. 2008; 34(7):1256-62.
  3. Greisen G, Mirante N, Haumont D, Pierrat V, Pallás-Alonso CR, Warren I, Smit BJ, Westrup B, Sizun J, Maraschini A, Cuttini M; ESF Network. Parents, siblings and grandparents in the Neonatal Intensive Care Unit. A survey of policies in eight European countries. Acta Paediatr. 2009 Nov;98(11):1744-50.
  4. Wu C, Melnikow J, Dinh T, Holmes JF, Gaona SD, Bottyan T, Paterniti D, Nishijima DK. Patient Admission Preferences and Perceptions. West J Emerg Med. 2015; 16(5):707-14.
  5. Inouye SK, Bogardus ST, Jr., Charpentier PA, et al. A multicomponent intervention to prevent delirium in hospitalized older patients. New Engl J Med. 1999; 340(9):669-76.
  6. Tsouna-Hadjis E, Vemmos KN, Zakopoulos N, Stamatelopoulos S. First-stroke recovery process: the role of family social support. Arch Phys Med Rehabil. 2000; 81(7): 881-7.
  7. Berti D, Ferdinande P, Moons P. Beliefs and attitudes of intensive care nurses toward visits and open visiting policy. Intensive Care Med. 2007; 33(6): 1060-5.
  8. Netzer G, Iwashyna TJ. Fair is Fair: Preventing the Misuse of Visiting Hours to Reduce Inequities. Ann Am Thorac Soc. 2017.
  9. Teding van Berkhout E, Malouff JM. The efficacy of empathy training: A meta-analysis of randomized controlled trials. J Couns Psychol. 2016; 63(1):32-41.

Measuring the Quality of Health Care in Low-Income Settings

Measuring the quality of health care in High-Income Countries (HIC) is deceptively difficult, as shown by work carried out by many research groups, including CLAHRC WM.[1-5] However, a large amount of information is collected routinely by health care facilities in HICs. This data includes outcome data, such as Standardised Mortality Rates (SMRs), death rates from ’causes amenable to health care’, readmission rates, morbidity rates (such as pressure damage), and patient satisfaction, along with process data, such as waiting times, prescribing errors, and antibiotic use. There is controversy over many of these endpoints, and some are much better barometers of safety than others. While incident reporting systems provide a very poor basis for epidemiological studies (that is not their purpose), case-note review provides arguably the best and most widely used method for formal study of care quality – at least in hospitals.[3] [6] [7] Measuring safety in primary care is inhibited by the less comprehensive case-notes found in primary care settings as compared to hospital case-notes. Nevertheless, increasing amounts of process information is now available from general practices, particularly in countries (such as the UK) that collect this information routinely in electronic systems. It is possible, for example, to measure rates of statin prescriptions for people with high cardiovascular risk, and anticoagulants for people with ventricular fibrillation, as our CLAHRC has shown.[8] [9] HICs also conduct frequent audits of specific aspects of care – essentially by asking clinicians to fill in detailed pro formas for patients in various categories. For instance, National Audits in the UK have been carried out into all patients experiencing a myocardial infarction.[10] Direct observation of care has been used most often to understand barriers and facilitators to good practice, rather than to measure quality / safety in a quantitative way. However, routine data collection systems provide a measure of patient satisfaction with care – in the UK people who were admitted to hospital are surveyed on a regular basis [11] and general practices are required to arrange for anonymous patient feedback every year.[12] Mystery shoppers (simulated patients) have also been used from time to time, albeit not as a comparative epidemiological tool.[13]

This picture is very different in Low- and Middle-Income Countries (LMIC) and, again, it is yet more difficult to assess quality of out of hospital care than of hospital care.[14] Even in hospitals routine mortality data may not be available, let alone process data. An exception is the network of paediatric centres established in Kenya by Prof Michael English.[15] Occasionally large scale bespoke studies are carried out in LMICs – for example, a recent study in which CLAHRC WM participated, measured 30 day post-operative mortality rates in over 60 hospitals across low-, middle- and high-income countries.[16]

The quality and outcomes of care in community settings in LMICs is a woefully understudied area. We are attempting to correct this ‘dearth’ of information in a study in nine slums spread across four African and Asian countries. One of the largest obstacles to such a study is the very fragmented nature of health care provision in community settings in LMICs – a finding confirmed by a recent Lancet commission.[17] There are no routine data collection systems, and even deaths are not registered routinely. Where to start?

In this blog post I lay out a framework for measurement of quality from largely isolated providers, many of whom are unregulated, in a system where there is no routine system of data and no archive of case-notes. In such a constrained situation I can think of three (non-exclusive) types of study:

  1. Direct observation of the facilities where care is provided without actually observing care or its effects. Such observation is limited to some of the basic building blocks of a health care system – what services are present (e.g. number of pharmacies per 1,000 population) and availability (how often the pharmacy is open; how often a doctor / nurse / medical officer is available for consultation in a clinic). Such a ‘mapping’ exercise does not capture all care provided – e.g. it will miss hospital care and municipal / hospital-based outreach care, such as vaccination provided by Community Health Workers. It will also miss any IT based care using apps or online consultations.
  2. Direct observation of the care process by external observers. Researchers can observe care from close up, for example during consultations. Such observations can cover the humanity of care (which could be scored) and/or technical quality (which again could be scored against explicit standards and/or in a holistic (implicit) basis).[6] [7] An explicit standard would have to be based mainly on ‘if-then’ rules – e.g. if a patient complained of weight loss, excessive thirst, or recurrent boils, did the clinicians test their urine for sugar; if the patient complained of persistent productive cough and night sweats was a test for TB arranged? Implicit standards suffer from low reliability (high inter-observer variation).[18] Moreover, community providers in LMICs are arguably likely to be resistant to what they might perceive as an intrusive or even threatening form of observation. Those who permitted such scrutiny are unlikely to constitute a random sample. More vicarious observations – say of the length of consultations – would have some value, but might still be seen as intrusive. Provided some providers would permit direct observation, their results may represent an ‘upper bound’ on performance.
  3. Quality as assessed through the eyes of the patient / members of the public. Given the limitations of independent observation, the lack of anamnestic records of clinical encounters in the form of case-notes, absence of routine data, and likely limitations on access by independent direct observers, most information may need to be collected from patients themselves, or as we discuss, people masquerading as patients (simulated patients / mystery shoppers). The following types of data collection methods can be considered:
    1. Questions directed at members of the public regarding preventive services. So, households could be asked about vaccinations, surveillance (say for malnutrition), and their knowledge of screening services offered on a routine basis. This is likely to provide a fairly accurate measure of the quality of preventive services (provided the sampling strategy was carefully designed to yield a representative sample). This method could also provide information on advice and care provided through IT resources. This is a situation where some anamnestic data collection would be possible (with the permission of the respondent) since it would be possible to scroll back through the electronic ‘record’.
    2. Opinion surveys / debriefing following consultations. This method offers a viable alternative to observation of consultations and would be less expensive (though still not inexpensive). Information on the kindness / humanity of services could be easily obtained and quantified, along with ease of access to ambulatory and emergency care.[19] Measuring clinical quality would again rely on observations against a gold standard,[20] but given the large number of possible clinical scenarios standardising quality assessment would be tricky. However, a coarse-grained assessment would be possible and, given the low quality levels reported anecdotally, failure to achieve a high degree of standardisation might not vitiate collection of important information. Such a method might provide insights into the relative merits and demerits of traditional vs. modern health care, private vs. public, etc., provided that these differences were large.
    3. Simulated patients offering standardised clinical scenarios. This is arguably the optimal method of technical quality assessment in settings where case-notes are perfunctory or not available. Again, consultations could be scored for humanity of care and clinical/ technical competence, and again explicit and/or implicit standards could be used. However, we do not believe it would be ethical to use this method without obtaining assent from providers. There are some examples of successful use of the methods in LMICs.[21] [22] However, if my premise is accepted that providers must assent to use of simulated patients, then it is necessary to first establish trust between providers and academic teams, and this takes time. Again, there is a high probability that only the better providers will provide assent, in which case observations would likely represent ‘upper bounds’ on quality.

In conclusion, I think that the basic tools of quality assessment, in the current situation where direct observation and/or simulated patients are not acceptable, is a combination of:

  1. Direct observation of facilities that exist, along with ease of access to them, and
  2. Debriefing of people who have recently used the health facilities, or who might have received preventive services that are not based in these facilities.

We do not think that the above mentioned shortcomings of these methods is a reason to eschew assessment of service quality in community settings (such as slums) in LMICs – after all, one of the most powerful levers to improvement is quantitative evidence of current care quality.[23] [24] The perfect should not be the enemy of the good. Moreover, if the anecdotes I have heard regarding care quality (providers who hand out only three types of pill – red, yellow and blue; doctors and nurses who do not turn up for work; prescription of antibiotics for clearly non-infectious conditions) are even partly true, then these methods would be more than sufficient to document standards and compare them across types of provider and different settings.

— Richard Lilford, CLAHRC WM Director


  1. Brown C, Hofer T, Johal A, Thomson R, Nicholl J, Franklin BD, Lilford RJ. An epistemology of patient safety research: a framework for study design and interpretation. Part 1. Conceptualising and developing interventions. Qual Saf Health Care. 2008; 17(3): 158-62.
  2. Brown C, Hofer T, Johal A, Thomson R, Nicholl J, Franklin BD, Lilford RJ. An epistemology of patient safety research: a framework for study design and interpretation. Part 2. Study design. Qual Saf Health Care. 2008; 17(3): 163-9.
  3. Brown C, Hofer T, Johal A, Thomson R, Nicholl J, Franklin BD, Lilford RJ. An epistemology of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement. Qual Saf Health Care. 2008; 17(3): 170-7.
  4. Brown C, Hofer T, Johal A, Thomson R, Nicholl J, Franklin BD, Lilford RJ. An epistemology of patient safety research: a framework for study design and interpretation. Part 4. One size does not fit all. Qual Saf Health Care. 2008; 17(3): 178-81.
  5. Brown C, Lilford R. Evaluating service delivery interventions to enhance patient safety. BMJ. 2008; 337: a2764.
  6. Benning A, Ghaleb M, Suokas A, Dixon-Woods M, Dawson J, Barber N, et al. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ. 2011; 342: d195.
  7. Benning A, Dixon-Woods M, Nwulu U, Ghaleb M, Dawson J, Barber N, et al. Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase. BMJ. 2011; 342: d199.
  8. Finnikin S, Ryan R, Marshall T. Cohort study investigating the relationship between cholesterol, cardiovascular risk score and the prescribing of statins in UK primary care: study protocol. BMJ Open. 2016; 6(11): e013120.
  9. Adderley N, Ryan R, Marshall T. The role of contraindications in prescribing anticoagulants to patients with atrial fibrillation: a cross-sectional analysis of primary care data in the UK. Br J Gen Pract. 2017. [ePub].
  10. Herrett E, Smeeth L, Walker L, Weston C, on behalf of the MINAP Academic Group. The Myocardial Ischaemia National Audit Project (MINAP). Heart. 2010; 96: 1264-7.
  11. Care Quality Commission. Adult inpatient survey 2016. Newcastle-upon-Tyne, UK: Care Quality Commission, 2017.
  12. Ipsos MORI. GP Patient Survey. National Report. July 2017 Publication. London: NHS England, 2017.
  13. Grant C, Nicholas R, Moore L, Sailsbury C. An observational study comparing quality of care in walk-in centres with general practice and NHS Direct using standardised patients. BMJ. 2002; 324: 1556.
  14. Nolte E & McKee M. Measuring and evaluating performance. In: Smith RD & Hanson K (eds). Health Systems in Low- and Middle-Income Countries: An economic and policy perspective. Oxford: Oxford University Press; 2011.
  15. Tuti T, Bitok M, Malla L, Paton C, Muinga N, Gathara D, et al. Improving documentation of clinical care within a clinical information network: an essential initial step in efforts to understand and improve care in Kenyan hospitals. BMJ Global Health. 2016; 1(1): e000028.
  16. Global Surg Collaborative. Mortality of emergency abdominal surgery in high-, middle- and low-income countries. Br J Surg. 2016; 103(8): 971-88.
  17. McPake B, Hanson K. Managing the public-private mix to achieve universal health coverage. Lancet. 2016; 388: 622-30.
  18. Lilford R, Edwards A, Girling A, Hofer T, Di Tanna GL, Petty J, Nicholl J. Inter-rater reliability of case-note audit: a systematic review. J Health Serv Res Policy. 2007; 12(3): 173-80.
  19. Schoen C, Osborn R, Huynh PT, Doty M, Davis K, Zapert K, Peugh J. Primary Care and Health System Performance: Adults’ Experiences in Five Countries. Health Aff. 2004.
  20. Kruk ME & Freedman LP. Assessing health system performance in developing countries: A review of the literature. Health Policy. 2008; 85: 263-76.
  21. Smith F. Private local pharmacies in low- and middle-income countries: a review of interventions to enhance their role in public health. Trop Med Int Health. 2009; 14(3): 362-72.
  22. Satyanarayana S, Kwan A, Daniels B, Subbaramn R, McDowell A, Bergkvist S, et al. Use of standardised patients to assess antibiotic dispensing for tuberculosis by pharmacies in urban India: a cross-sectional study. Lancet Infect Dis. 2016; 16(11): 1261-8.
  23. Kudzma E C. Florence Nightingale and healthcare reform. Nurs Sci Q. 2006; 19(1): 61-4.
  24. Donabedian A. The end results of health care: Ernest Codman’s contribution to quality assessment and beyond. Milbank Q. 1989; 67(2): 233-56.

Patient Involvement in Patient Safety: Null Result from a High Quality Study

Most patient safety evaluations are simple before and after / time series improvement studies. So it is always refreshing to find a study with contemporaneous controls. Lawton and her colleagues report a nice cluster randomized trial covering 33 hospital wards in five hospitals.[1] They evaluate a well-known patient safety intervention based on the idea of giving patients a more active role in monitoring safety on their ward.

The trial produced a null result, but some of the measures of safety were in the right direction and there was a correlation between the enthusiasm/fidelity with which the intervention was implemented and measures of safety.

Safety is hard to measure (as the authors state), and improvement often builds on a number of small incremental changes. So, it would be very nice to see this intervention replicated, possibly with measures to generate greater commitment from ward staff.
Here is the problem with patient safety research; on the one hand the subject of patient safety is full of hubristic claims made on the basis of insufficient (weak) evidence. On the other hand, high quality studies, such as the one reported here, often fail to find an effect. In many cases, as in the study reported here, there are reasons to suspect a type 2 error (false negative result). Beware also the rising tide – the phenomenon that arises where a trial occurs in the context of a strong secular trend – this trend ‘swallows up’ the headroom for a marginal intervention effect.[2] What is to be done? First, do not declare defeat too early. Second, be prepared to either carry out larger studies or replication studies that can be combined in a meta-analysis. Third, make multiple measurements across a causal chain [3] and synthesise this disparate data using Bayesian networks.[4] Fourth, further to the Bayesian approach, do not dichotomise results on the standard frequentist statistical convention into null and positive. It is stupid to classify a p-value of 0.06 as null if other evidence supports an effect, or to classify a p-value of 0.04 as positive if other data point the opposite way. Knowledge of complex areas, such as service interventions to improve safety, should take account of patterns in the data and information external to the index study. Bayesian networks provide a framework for such an analysis.[4] [5]

— Richard Lilford, CLAHRC WM Director


  1. Lawton R, O’Hara JK, Sheard L, et al. Can patient involvement improve patient safety? A cluster randomised control trial of the Patient Reporting and Action for a Safe Environment (PRASE) intervention. BMJ Qual Saf. 2017; 26: 622-31.
  2. Chen YF, Hemming K, Stevens AJ, Lilford RJ. Secular trends and evaluation of complex interventions: the rising tide phenomenon. BMJ Qual Saf. 2016; 25: 303-10.
  3. Lilford RJ, Chilton PJ, Hemming K, Girling AJ, Taylor CA, Barach P. Evaluating policy and service interventions: framework to guide selection and interpretation of study end points. BMJ. 2010; 341: c4413.
  4. Watson SI & Lilford RJ. Essay 1: Integrating multiple sources of evidence: a Bayesian perspective. In: Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Southampton (UK): NIHR Journals Library, 2016.
  5. Lilford RJ, Girling AJ, Sheikh, et al. Protocol for evaluation of the cost-effectiveness of ePrescribing systems and candidate prototype for other related health information technologies. BMC Health Serv Res. 2014; 14: 314.

Declining Readmission Rates – Are They Associated with Increased Mortality?

I have always been a bit nihilistic about reducing readmission rates to hospitals.[1][2] However, I may have been overly pessimistic. A new study confirms that it is possible to reduce readmission rates by imposing financial incentives.[3] Importantly, this does not seem to have caused an increase in mortality – as might occur if hospitals were biased against re-admitting sick patients in order to avoid a financial penalty. “False null result” (type two error), do I hear you ask? Probably not, since the data are based on nearly seven million admissions. In fact, 30 day mortality rates were slightly lower among hospitals that reduced readmission rates.

— Richard Lilford, CLAHRC WM Director


  1. Lilford RJ. If Not Preventable Deaths, Then What About Preventable Admissions? NIHR CLAHRC West Midlands News Blog. 6 May 2016.
  2. Lilford RJ. Unintended Consequences of Pay-For-Performance Based on Readmissions. NIHR CLAHRC West Midlands News Blog. 13 January 2017.
  3. Joynt KE, & Maddox TM. Readmissions Have Declined, and Mortality Has Not Increased. The Importance of Evaluating Unintended Consequences. JAMA. 2017; 318(3): 243-4.

Predicting Readmissions on the Basis of a Well-Known Risk of Readmission Score

A recent NIHR CLAHRC West Midlands study examined a score based on co-morbidities, hospital use before the index admission, length of stay, and rate of admission – the LACE score.[1] The findings broadly corroborate the score and previous evidence – high scores are statistically associated with risk of readmission, but predictive accuracy is low and hardly likely to improve on clinical assessment; no doctor would use such a test to identify patients. This is an inpatient study based on over 90,000 admissions. We do not want every clinical action to be codified in a score – it is a waste of time. Moreover, most readmissions are caused by a new problem.[2] So a more sensible way forward, from my point of view, would be a general index of risk of deterioration to cover patients at all points in their journey. Would the ‘frailty index’ [3] [4] serve this purpose perfectly well?

— Richard Lilford, CLAHRC WM Director


  1. Damery S, Combes G. Evaluating the predictive strength of the LACE index in identifying patients at high risk of hospital readmission following an inpatient episode: a retrospective cohort study. BMJ Open. 2017; 7: e016921.
  2. Lilford RJ. Unintended Consequences of Pay-for-Performance Based on Readmissions. NIHR CLAHRC West Midlands News Blog. 13 January 2017.
  3. Lilford RJ. Future Trends in NHS. NIHR CLAHRC West Midlands News Blog.
  4. Clegg A, Bates C, Young J, et al. Development and validation of an electronic frailty index using routine primary care electronic health record data. Age Ageing. 2016.

Introducing Hospital IT systems – Two Cautionary Tales

The beneficial effects of mature IT systems, such as at the Brigham and Women’s Hospital,[1] Intermountain Health Care,[2] and University Hospitals Birmingham NHS Foundation Trust,[3] have been well documented. But what happens when a commercial system is popped into a busy NHS general hospital? Lots of problems according to two detailed qualitative studies from Edinburgh.[4] [5] Cresswell and colleagues document problems with both stand-alone ePrescribing systems and with multi-modular systems.[4] The former drive staff crazy with multiple log-ins and duplicate data entry. Nor does their frustration lessen with time. Neither system types (stand-alone or multi-modular) presented a comprehensive overview of the patient record. This has obvious implications for patient safety. How is a doctor expected to detect a pattern in the data if they are not presented in a coherent format? In their second paper the authors examine how staff cope with the above problems.[5] To enable them to complete their tasks ‘workarounds’ were deployed. These workarounds frequently involved recourse to paper intermediaries. Staff often became overloaded with work and often did not have the necessary clinical information at their fingertips. Some workarounds were sanctioned by the organisation, others not. What do I make of these disturbing, but thorough, pieces of research? I would say four things:

  1. Move slowly and carefully when introducing IT and never, never go for heroic ‘big bang’ solutions.
  2. Employ lots of IT specialists who can adapt systems to people – do not try to go the other way round and eschew ‘business process engineering’, the risks of which are too high – be incremental.
  3. If you do not put the doctors in charge, make sure that they feel as if they are. More seriously – take your people with you.
  4. Forget integrating primary and secondary care, and social care and community nurses, and meals on wheels and whatever else. Leave that hubristic task to your hapless successor and introduce a patient held booklet made of paper – that’s WISDAM.[6]

— Richard Lilford, CLAHRC WM Director


  1. Weissman JS, Vogeli C, Fischer M, Ferris T, Kaushal R, Blumenthal B. E-prescribing Impact on Patient Safety, Use and Cost. Rockville, MD: Agency for Healthcare Research and Quality. 2007.
  2. Bohmer RMJ, Edmondson AC, Feldman L. Intermountain Health Care. Harvard Business School Case 603-066. 2002
  3. Coleman JJ, Hodson J, Brooks HL, Rosser D. Missed medication doses in hospitalised patients: a descriptive account of quality improvement measures and time series analysis. Int J Qual Health Care. 2013; 25(5): 564-72.
  4. Cresswell KM, Mozaffar H, Lee L, Williams R, Sheikh A. Safety risks associated with the lack of integration and interfacing of hospital health information technologies: a qualitative study of hospital electronic prescribing systems in England. BMJ Qual Saf. 2017; 26: 530-41.
  5. Cresswell KM, Mozaffar H, Lee L, Williams R, Sheikh A. W. Workarounds to hospital electronic prescribing systems: a qualitative study in English hospitals. BMJ Qual Saf. 2017; 26: 542-51.
  6. Lilford RJ. The WISDAM* of Rupert Fawdry. NIHR CLAHRC West Midlands News Blog. 5 September 2014.

Length of Hospital Stay

The average length of hospital stay for patients has ‘plummeted’ over the last thirty years, from 10 days in 1983 to 5 days in 2013.[1] However, the proportion of patients discharged to a nursing facility has quadrupled over this same period.[2] So, from the point of view of the patient, the stay away from home has not changed as much as it might be inferred from an uncritical analysis of inpatient stays. So, how have home-to-home times changed? This was assessed by Barnett et al.[3] on the basis of Medicare administration claims for 82 million hospitalisations over the years 2004 to 2011 inclusive.

Yes, the mean length of hospital stay declined (from 6.3 to 5.7 days), but the mean length of stay in post-acute care facilities increased from 4.8 to 6 days. Total home-to-home time increased from 11.1 to 11.7 days. This is not necessarily a bad thing, but it must be taken into account in assessing costs and benefits of care. The risk of iatrogenic harm and costs are lower in nursing facilities than hospitals. However, the article cited here does not consider the possibility that these risks and costs are not lower for the group of people in nursing facilities who would otherwise be cared for in hospital.

— Richard Lilford, CLAHRC WM Director


  1. Centers for Medicare and Medicaid Services. CMS program statistics: 2013 Medicare Utilization Section. 2017.
  2. Tian W (AHRQ). An All-Payer View of Hospital Discharge to Postacute Care, 2013. Rockville, MD: Agency for Healthcare Research and Quality; 2016.
  3. Barnett ML, Grabowski DC, Mehrotra A. Home-to-Home Time – Measuring What Matters to Patients and Payers. N Engl J Med. 2017; 377: 4-6.