Extraordinary claims require extraordinary evidence

The subject of last issue’s quiz was the results of a study from The Tufts Center for the Study of Drug Development regarding the new estimates of the costs of developing a new drug. As is rightly stated the estimate was $2.6 billion. This study is an update of the original study by DiMasi and colleagues,[1] whose finding that the costs (in 2000 USD) of drug development were close to $1 billion, has achieved near canonical status. However, considerable doubt has been thrown on these claims, and the criticisms of the original study should be applied to this new research. Light and Warburton’s critique [2] [3] drew on a number of points: the lack of comparability and reliability about the original survey data as well as the lack of transparency (as the data were not made publicly available); there was a clear interest for pharmaceutical companies to overstate their costs in survey responses; neither the firms nor the drugs considered were random samples; the only drugs considered in the study were “self-originated new chemical entities” (NCEs) whose costs of development are many times higher than acquired or licensed-in NCEs, new formulations, combinations, or administrations of existing drugs, and yet only comprise around 22% of new drug approvals; government subsidies were not deducted; and, there was no adjustment for tax deductions and credits, to name but a few.

Articles in major journals based on industry sponsored research are three to four times more likely to report results favourable to the sponsors than articles with independent funding.[4] [5] Considerable variation therefore exists in estimates of the costs of drug development. Light and Warburton have estimated the median figure to be roughly a tenth of the original DiMasi estimate.[6] While this may seem (perhaps implausibly) low it certainly suggests we need to take industry sponsored research that affects health policy with a healthy dose of skepticism.

— Sam Watson, University of Warwick

References:

  1. DiMasi JA, Hansen RW, Grabowski HG. The price of innovation: new estimates of drug development costs. J Health Econ. 2003; 22(2): 151-85.
  2. Light DW & Warburton RN. Extraordinary claims require extraordinary evidence. J Health Econ. 2005; 24(5): 1030-3.
  3. Light DW & Warburton RN. Setting the record straight in the reply by DiMasi, Hansen and Grabowski. J Health Econ. 2005; 24(5): 1045-8.
  4. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research. A systematic reviewJAMA. 2003; 289(4): 454-65.
  5. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic reviewBMJ. 2003; 326: 1167.
  6. Light DW & Warburton R. Demythologizing the high costs of pharmaceutical research. BioSocieties. 2011; 6: 34-50.
Advertisements

Should Researchers Double as Policy Activists?

The UK, from whence this News Blog emanates, is preparing for a general election – one that is exercising psephologists because, like other European countries, the UK is experiencing a fracturing of the vote, with smaller parties encroaching on “traditional” major parties. This News Blog will stand on the sidelines. We cannot be silent on how to reach objectives, of course, and will continue to report on the relevant evidence. However, the political implications will not be discussed. This is not because the CLAHRC WM Director is apolitical. On the contrary, he is a member of a political party for which he has canvassed in years gone by. Instead, the CLARHC WM Director’s ordinance is based on the ideas that firstly, politics should be kept out of the day job (unless politics is the day job), and secondly, it is quite possible to separate the political persona from the work persona, as the Director had to do in his civil service days.

But, what about policy rather than politics? Here we deal with rather more specific issues: caps on carbon emissions, badger culling, promoting contraception in African countries. What should scientists do in these cases? Should they be dispassionate collectors of evidence, or should they double as both evidence collectors and activists for the relevant causes? Climate change scientists could lobby for more stringent controls on emissions; those who show that badger culling has minimised effects on TB control [1] could join protests; and those who find that lack of contraception is the cause of unwanted pregnancy could agitate against puritanical governments, such as that in Uganda.

The issue of scientists as “activists” was discussed at the ‘Global Symposium on Health Systems Research’ conference, Cape Town, in October 2014. The majority – in fact all – who commented supported the idea of scientists as activists. But there was little critical reflection on the potential dangers of activism and how they might be avoided. The CLAHRC WM Director is circumspect when it comes to this idea of scientist activism.

First, scientists have no particular expertise when it comes to the values on which policy may turn. Nobel prize winners sometimes succumb to the temptation to appoint themselves as an oracle, and soon become a public nuisance, as pointed out by Paul Nurse, himself a laureate. Scientific evidence is about how to reach an objective, but which objectives to pursue – that is a question of preferences/values which lie in the realm of philosophy or religion rather than science. Churchill seems to have spotted this distinction when he said “scientists should be on tap, but not on top.”[2]

Second, a reason for scientist activism is the passion scientists have for the cause. But this is precisely where the danger lies. The CLAHRC WM Director is no climate change denier, but he is worried by the frequent gerrymandering of scientific results in this field, where some seem to think the end justifies the means. Activism has the power to corrupt.[3]

There is a counter-argument to the Director’s detached position. This argument holds that scientific results are inevitably socially constructed, and so, rather than pretend that science is neutral/objective, it is better to declare one’s allegiance and let the reader judge. This is not the place for a comprehensive demolition of this kind of constructivist argument – the writer is an unreformed child of the Enlightenment, holding that we should strive to be objective in the collection and analysis of evidence, and hence that we should adhere to scientific rules whose purpose is the avoidance of error.

The CLAHRC WM Director does not go so far as to say that a scientist should not express an opinion. The moral insights of scientists may be no better than those of the population as a whole, but there is no reason to think they are any worse. The crucial point for scientists is to separate their lives as dispassionate empiricists from their lives as advocates. They should be crystal clear, both in their own minds and in their pronouncements, when they have toggled from scientist to citizen advocate. And, of course, there are times when a decision really does turn only on the science – where values don’t enter the equation because nearly everyone has the same preferences. So, in the case of Thabo Mbeki’s failure to accept that HIV is the cause of AIDS, it was quite right for Jimmy Volmink, and other courageous South African scientists, to “speak truth unto power.”

— Richard Lilford, CLAHRC WM Director

References:

  1. Jenkins HE, Woodroffe R, Donnelly CA. The Duration of the Effects of Repeated Widespread Badger Culling on Cattle Tuberculosis Following the Cessation of Culling. PLoS ONE. 2010. 5(2): e9090.
  2. Quoted in: Churchill RS. Twenty-One Years. London: Weidenfeld & Nicolson. 1965. p.127.
  3. Lomborg B. The Skeptical Environmentalist: Measuring the Real State of the World. Cambridge: Cambridge University Press. 2001.

Je suis Raif: bloggers, freedom of speech, doctors and torture

Recent events have drawn attention to three related issues of interest to this part-time blogger with an interest in health care. One shows the high value that Western democracies place on freedom of speech. Another illustrates a contrasting value system and, in a curious way, highlights the complicity of doctors in torture. Could these events on the world’s stage have any resonance within the microcosm of the NHS?

The attack on the French satirical magazine Charlie Hebdo provoked a robust defence of freedom of speech from nations across the world. Even the Saudi Arabian ambassador took part in a solidarity march in France. Which is ironic – back in Saudi Arabia, Raif Badawi faces 50 lashes a week for 20 weeks outside of al-Jafali mosque in Jeddah. His crime was to write a blog criticising religious authorities in Saudi Arabia. The right to criticise is not universally valued and speaking out can be construed as disloyalty.

Fortunately, the latest instalment of his sentence has been postponed because the prison doctor decided that his wounds had not sufficiently healed.[1] Medical intervention has gained Raif Badawi a week’s respite, but it also raises the question of medical complicity in torture. Interestingly this phenomenon is not confined to Saudi Arabia. Medical complicity in legally sanctioned torture has also occurred in Western democracies. How can this happen?

Over a decade ago in the BMJ I predicted that doctors in Guantanamo Bay risked becoming accessories to torture.[2] Three doctors responded to my letter, vying each other in outrage at my suggestion. Anyone with the tiniest knowledge of social psychology would have made the same prediction. Individuals’ behaviour is constrained by the norms of the organisation. In his famous 1971 Stanford Prison Experiment,[3] Zimbardo demonstrated that given the power over others, some individuals will behave sadistically, while others feel unable to intervene. This was of course exactly what happened in Guantanamo.[4] It had previously happened in Abu Ghraib. It will happen again in similar circumstances unless specific measures are put in place to create an organisational culture where this is unacceptable. This means leadership, clear standards of behaviour, external oversight, and open reporting of lapses in standards of behaviour.

Are there any lessons for health services? Freedom of speech is important. The NHS does not flog whistle-blowers in public places, but they often suffer punishment.[5] [6] Organisational culture shapes behaviour, even among health professionals. Without leadership, transparency and external oversight, any organisation risks slipping into poor practice. Cartoonists, bloggers and whistle-blowers may make us feel uncomfortable. But this is all the more reason not to silence them.

— Tom Marshall, Deputy Director CLAHRC WM, Prevention and Detection of Diseases

References:

  1. Tran M. Saudi blogger Raif Badawi’s case referred to supreme court, says his wife. The Guardian. 2015-01-16. [Online].
  2. Marshall T. Doctors in Guantanamo Bay are at risk of being accessories to torture. BMJ. 2002. 324: 235.
  3. Haney C, Banks C, Zimbardo P. A Study of Prisoners and Guards in a Simulated Prison. Washington, D.C.: Office of Naval Research. 1973.
  4. Nye J. U.S. military doctors broke oath to design new torture techniques to be used at Guantanamo Bay. Daily Mail. 2013-11-04. [Online].
  5. Public Concern at Work. Whistleblowing the inside story – Main report. 2013. [Online].
  6. BBC News. Worcestershire hospitals ban paramedic Stuart Gardner. BBC News. 2015-01-16. [Online].

Speaking Truth Unto Power*

* The original quote, “Speak truth to power”, is thought to have been coined by Bayard Rustin in 1942 and referred to the role of religious groups.[1]

Science plays an increasing role in modern society, identifying problems and then developing and testing solutions – climate change being a prime example. Individual scientists often make discoveries, but professional/scientific societies are arguably more persuasive in influencing practice and policy.[2] The Nigerian Academy of Sciences orchestrated the country’s successful programme to contain the Ebola outbreak according to Andrew Green, writing in the Lancet.[3] He gives further examples of successful policy interventions prompted by African Academies. The CLAHRC WM Director was reminded of the iconic advice, often attributed to Germaine Greer, “Don’t complain, organise.”

— Richard Lilford, CLAHRC WM Director

References:

  1. Green J. The Origin of the Phrase “Speaking Truth to Power”. [Online].
  2. Wilson B, Thornton JG, Hewison J, Lilford RJ, Watt I, Braunholtz D, Robinson M. The Leeds University Maternity Audit Project. Int J Qual Health Care. 2002; 14(3): 175-81.
  3. Green A. African science academy initiative reflects on progress. Lancet. 2014; 384: 1836.

Exercise and Health

Readers of the News Blog will have seen previous articles on exercise and health.[1] [2] It is more dangerous to have diabetes or to smoke than to be a coach potato, but there are many more coach potatoes than people living with diabetes or smokers, so overall, more people lose their lives from inactivity than from either the diabetes or the tobacco. It also turns out that the marginal gains of more exercise decrease as total exercise increases. Totally sedentary people (about 25% of the US population) have by far the highest risk. This is the basis for an interesting review and editorial in the Annals of Internal Medicine, advocating policy focusing on the totally sedentary,[3] [4] as well as a news article in the BMJ.[5]

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Lilford R. Anti-obesity interventions. Dec 12 2014. [Online].
  2. Lilford R. Two provocative papers on diet and health. Dec 12 2014. [Online].
  3. Biswas A, Oh PI, Faulkner GE, Bajaj RR, Silver MA, Mitchell MS, et al. Sedentary time and its association with risk for disease incidence, mortality and hospitalization in adults: a systematic review and meta-analysis. Ann Intern Med. 2015;162; 123-32.
  4. Lynch BM, Owen N. Too much sitting and chronic disease risk: steps to move the science forward. Ann Intern Med. 2015;162: 146-7
  5. Mayor S. Prolonged sitting increases risk of serious illness and death regardless of exercise, study finds. BMJ. 2015; 350: h306.

Feeling Young is Good News if You Are Feeling Old!

People who feel older than their chronological age have a considerably higher chance of dying within ten years than age-matched counterparts who feel younger, according to a recent prospective population-based study.[1] This is despite the authors controlling for baseline medical conditions and excluding deaths in the first year to mitigate reverse causality. This is yet another study relating general attitude to health.

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Rippon I, & Steptoe A. Feeling Old vs Being Old: Associations Between Self-perceived Age and Mortality. JAMA Intern Med. 2014. [ePub].

Evaluating Service Interventions

CLAHRCs were invented to align research practice with service change. As a result of such alignment, here in the West Midlands we have been able to evaluate service interventions with the scope and timescale of the service imperative, including:

  • Peer support for mothers at high social risk to see whether there is an improvement in mother/child bonding – individual person RCT.[1]
  • Rearranging mental health services to reduce delay in treatment for schizophrenia in adolescents – multi-centre before and after study.[2]
  • Case-finding for cardiovascular risk in Primary Care to assess uptake of services – cluster step-wedge RCT.[3]
  • Educational intervention to improve attitudes to mental health in schools – cluster parallel RCT.[4]

A brilliant example of an evaluation driven by service change is the Oregon Health Insurance experiment. Expansion of medical coverage was implemented by lottery, inadvertently generating an RCT. This was used to evaluate the effects of improving access to health care. The political and logistical issues behind the trial are discussed by Allen et al.[5] and the result will be summarised in a forthcoming blog.

— Richard Lilford, CLAHRC WM Director

References:

  1. Kenyon S, Jolly K, Hemming K, Ingram L, Gale N, Dann S-A, Chambers J, MacArthur C. Evaluation of Lay Support in Pregnant women with Social risk (ELSIPS): a randomised controlled trial. BMC Pregnancy Childbirth. 2012; 12: 11.
  2. Birchwood M, Bryan S, Jones-Morris N, Kaambwa B, Lester H, Richards J, Rogers H, Sirvastava N, Tzemou E. EDEN: Evaluating the development and impact of Early Intervention Services (EISs) in the West Midlands. NIHR Service Delivery & Organisation. HS&DR 08/1304/042. 2007.
  3. Marshall T, Caley M, Hemming K, Gill P, Gale N, Kolly K. Mixed methods evaluation of targeted case finding for cardiovascular disease prevention using a stepped wedge cluster RCT. BMC Public Health. 2012; 12: 908.
  4. Chisholm KE, Patterson P, Torgerson C, Turner E, Birchwood M. A randomised controlled feasibility trial for an educational school-based mental health intervention: study protocol. BMC Psychiatry. 2012; 12: 23.
  5. Allen H, Baicker K, Finkelstein A, Taubman S, Wright BJ, Oregon Health Study Group. What the Oregon health study can tell us about expanding Medicaid. Health Aff. 2010; 29(8): 1498-1506.

Improving Hospital Care: Not easy when budgets are pressed

In the last News Blog the idea of hospital culture was discussed. This has generated a lively debate, as you can see in the correspondence section of this News Blog. The responses are broadly supportive of the idea that hospital culture is over-emphasised. One of the problems is that culture is a rather slippery concept; clever, well-informed people can fall out over its meaning, just as the early Christian theologians would argue bitterly over whether Christ had two natures, one human and one divine, or one nature, both human and divine. So let’s move on to something less nebulous – money.

The rich world is emerging uncertainly from a nasty depression. The dependency ratio is widening as fertility rates fall below the optimum threshold discussed in a previous blog. Money is in short supply relative to need. Inevitably, the consequences fall unevenly across health facilities, such as hospitals. However hospitals are reimbursed, whether by insurance payments, capitation payments or block-grants, there are likely to be winners and losers in relative terms. In addition, hospitals have differing estate costs, economies to scale and so on. At any moment in time, some may have healthy financial reserves, while others are accumulating debt. What can managers do to improve financial health while also improving (or at least not damaging) patient care?

It is possible to achieve efficiencies in health care. Operations research and ‘lean’ processes can reduce waste – increasing use of day-case surgery for example.[1] However, much of the low hanging fruit may have already been plucked. The spectacular gains that can be achieved in many industries by ‘re-arranging’ work processes are not easily replicated in labour intensive industries, such as health. Also, quality control processes, such as computerised decision support,[2] and feedback and audit,[3] reduce the costs of having to care for patients with adverse events. They are frequently cost-effective and sometimes save health service costs overall by reducing the need to care for patients who have suffered from adverse events.[4] [5] However, few of these measures are cost-releasing in services where demand exceeds supply and many require up-front expenditure.

The largest call on money is the wage bill – it accounts for around two thirds of hospital expenditure.[6] It is hard to collect valid information on staff-patient ratios and care quality, but such evidence as there is suggests that quality of care is sensitive to nurse-patient [7] and doctor-patient [8] ratios. This is hardly counter-intuitive.

Managers may try ‘skill-substitution’ – essentially allocating tasks traditionally carried out by doctors to less expensive staff. Clinical officers have provided front-line diagnosis and clinical care for nearly 100 years in Africa, and a cadre of physicians’ assistants is being created in richer countries, especially the USA. However, the scale of substitution will be limited by numbers for at least a decade, by which time the demand for physicians and their assistants will likely have increased.

The corollary is that managers’ hands are largely tied – their ability to improve their financial position is limited – at any moment in time their budgets are largely or totally committed before they receive the money. Small differences in the financial health of organisations can make large differences at the margin, given the scale of the operation over many years.

Hospitals with a financial advantage at the margin would have somewhat lower ‘production pressures’ in serving their local population. They would be able to innovate occasionally, increase their quality assurance efforts a little, and develop new services from time to time. Gradually they would develop a brand and become more interesting places to work. They would find it easier to attract staff of the highest calibre, all the way up to the Chief Executive. In short, they would generate a virtuous cycle.

If hospital culture receives too much attention, money is afforded too little. The hapless ex-Chief Executive of the Mid-Staffordshire hospital, which was the subject of a series of enquires for providing sub-standard care, was accused of putting “money before patients” by not employing sufficient nurses. One has to ask the question about the financial viability of certain types of hospital. It is fashionable to blame poor care on poor management, but it may be a myth that managers make much difference in a hospital. It suits the chief executives of successful hospitals to perpetuate this myth, and applicants for chief executive jobs must feed the myth to convince the appointment panel of their miraculous powers.

A systematic search across many countries of factors associated with successful services is needed. Throw culture in if you wish, but don’t forget the money. And for anyone serving on a hospital board, be leery of the slogan “put money before patients”; they may be synonymous.

— Richard Lilford, CLAHRC WM Director

References:

  1. Cima RR, Brown MJ, Hebl JR, et al. Use of Lean and Six Sigma Methodology to Improve Operating Room Efficiency in a High-Volume Tertiary-Care Academic Medical Center. J Am Coll Surg. 2011. 213(1): 83-92.
  2. Nuckols TK. Smith-Spangler C, Morton SC, Asch SM, Patel VM, Anderson LJ, Deichsel EL, Shekelle PG. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev. 2014; 3: 56.
  3. Cochrane Effective Practice and Organisation of Care Group. EPOC Website. 2014. [Online].
  4. Yao GL, Novielli N, Manaseki-Holland S, et al. Evaluation of a predevelopment service delivery intervention: an application to improve clinical handovers. BMJ Qual Saf. 2012; 21(s1): i29-38.
  5. Hoonhout LH, de Bruijne MC, Wagner C, et al. Direct medical costs of adverse events in Dutch hospitals. BMC Health Serv Res. 2009; 9: 27.
  6. Lafond S, Arora S, Charlesworth A, McKeon A. Into the red? The state of the NHS’ finances. London: Nuffield Trust. 2014
  7. Shekelle PG. Chapter 34. Effect of Nurse-to-Patient Staffing Ratios on Patient Morbidity and Mortality. In: Shekelle PG, et al. Making Health Care Safer II: An Updated Critical Analysis of the Evidence for Patient Safety Practices. Comparative Effectiveness Review No. 211. Rockville, MD: AHRQ; 2013. p. 372-84.
  8. Pronovost PJ, Angus DC, Dorman T, Robinson KA, Dremisizov TT, Young TL. Physician Staffing Patterns and Clinical Outcomes in Critically Ill Patients. A Systematic Review. JAMA. 2002; 288: 2151-62.

International comparison of trial results

Nearly two decades ago colleagues from the Cochrane Complementary and Alternative Medicine (CAM) Field found that a high proportion of trials originating from Eastern Asia or Eastern Europe tended to report positive results – nearly 100% in some cases.[1] This was not only for acupuncture trials but also for trials in other topics. More recently, a team led by Professor John Ioannidis, a renowned epidemiologist, conducted a meta-epidemiological study (a methodological study that examines and analyses data obtained from many systematic reviews/meta-analyses) and showed that treatment effects reported in trials conducted in less developed countries are generally larger compared with those reported in trials undertaken in more developed countries.[2] Many factors could have contributed to these observations, for example, publication bias, reporting bias, rigour of scientific conduct, difference in patient populations and disease characteristics, and genuine difference in intervention efficacy. While it is almost certain that the observation was not attributed to genuine difference in intervention efficacy alone, teasing out the influence of various factors is not an easy task. Lately, colleagues from CLAHRC WM have compared results from cardiovascular trials conducted in Europe with those conducted in North America, and did not find a convincing difference between them.[3] Perhaps the more interesting findings will come from the comparison between trials from Europe/America and those from Asia. The results? The paper is currently in press, so watch this space!

— Yen-Fu Chen, Senior Research Fellow

References:

  1. Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Controlled Clin Trials. 1998; 19(2): 159-66.
  2. Panagiotou OA, Contopoulos-Ioannidis DG, Ioannidis, JPA, Rehnborg CF. Comparative effect sizes in randomised trials from less developed and more developed countries: meta-epidemiological assessment. BMJ. 2013; 346: f707.
  3. Bowater RJ, Hartley LC, Lilford RJ. Are cardiovascular trial results systematically different between North America and Europe? A study based on intra-meta-analysis comparisons. Arch Cardiovasc Dis. 2015; 108(1):23-38.

Alteplase for Stroke

Readers of this News Blog will probably have seen a post on thrombolysis with alteplase for acute ischaemic stroke. A meta-analysis of individual data from 6,756 patients and nine RCTs has now reported.[1] It concludes:

Irrespective of age or stroke severity, and despite an increased risk of fatal intracranial haemorrhage during the first few days after treatment, alteplase significantly improves the overall odds of a good stroke outcome when delivered within 4·5 h of stroke onset, with earlier treatment associated with bigger proportional benefits.”

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Emberson J, Lees KR, Lyden P, et al. Effect of treatment delay, age, and stroke severity on the effects of intravenous thrombolysis with alteplase for acute ischaemic stroke: a meta-analysis of individual patient data from randomised trials. Lancet. 2014; 384(9958): 1929-35.