Tag Archives: Data

An Intriguing Suggestion to Link Trial Data to Routine Data

When extrapolating from trial data to a particular context, it is important to compare the trial population to the target population. Given sufficient data, it is possible to examine treatment effect across important subgroups of patients. Then the trial results can be related to a specific sub-group, say with less severe disease than the average in the trial. One problem is that trial data are collected with greater diligence than routine data. Hence a suggestion to link trial data to routine data collected on the same patients. That way one can compare subgroups of trial and non-trial patients recorded in a broadly similar (i.e. routine) way.[1] This strikes me as a half-way house to the day when (most) trial data are collected by routine systems, and trials are essentially nested within routine data-collection systems.

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Najafzadeh M, Schneeweiss S. From Trial to Target Populations – Calibrating Real-World Data. N Engl J Med. 2017; 376: 1203-4.

Ethics of Using Other Researcher’s Data

It is good practice to make data collected in research projects available to others. A recent editorial in the New England Journal of Medicine [1] says that just using such data is ‘parasitical’, and suggest that researchers who use archived data should collaborate with the researchers who collected the original data. The CLAHRC WM Director disagrees. While there may be times when collaboration with the originators of the data is a good idea, it should not be expected or required. The original researchers are ‘invested’, and in many occasions it is in the public and scientific interest for the new investigations to maintain independence. It is best that data ‘gifted’ to the research community should be just that – a gift. Also, the original researchers might have lost interest, retired or died. Reinhart and Rogoff’s magisterial database of economic data is a case in point.[2] Independent re-analysis of the data they had collected and magnanimously made available to other researchers sometimes produced conclusions different to the original.

— Richard Lilford, CLAHRC WM Director

References:

  1. Longo DL, & Drazen JM. Data Sharing. New Engl J Med. 2016; 374: 276-7.
  2. Reinhart CM, & Rogoff KS. Growth in a Time of Debt. Am Econ Rev. 2010; 100(2): 573-8.

I Agree with Fiona

Dr Fiona Godlee, Editor-in-Chief of the BMJ, recently published a piece arguing that ‘data transparency is the only way‘.[1] This News Blog has featured a number of posts where many large RCTs have left a matter in contention – deworming children, clot busters for stroke, and vitamin A prophylaxis in children. When this happens, a dispute typically opens up about nuances in the handling of the data that might have introduced bias; bias so small that it is only material when the trials are large and hence the confidence limits narrow. The right policy is to stick the anonymised data in the public domain so that everyone can have a go at it. What is not okay, is to assume that one lot have the moral high ground – not industry, nor academics, nor editors, nor even CLAHRC Directors!

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Godlee F. Data transparency is the only way. BMJ. 2016; 352: i1261.

Another Study of Studies: Effectiveness using Routinely Collected Health Data vs. RCT Data

News Blog readers will be familiar with previous meta-epidemiological studies comparing effectiveness of the same treatment when evaluated using Routinely Collected Data (RCD) vs. prospective experiments (RCTs).[1] [2] Here is another such study from John Ioannidis, the world’s number one clinical epidemiologist – masterful.[3]
The RCD studies all:

  1. Preceded their complimentary RCTs
  2. Used prediction score modelling, and
  3. Had mortality as their main outcome.

Sixteen primary routine database studies were complimented by a mean of just over two subsequent RCTs examining the same clinical question. The findings here are not as sanguine regarding database studies as those cited in previous posts. The direction of effect was different in five (30%) of the 16 studies; confidence intervals in nine (59%) of the database studies did not include the observed effect in complimentary RCTs and, where they differed, the database studies tended to over-estimate treatment effects relative to the RCT estimate by a substantial 30%. This re-informs the perceived wisdom – experimental studies are the gold standard and database studies should not supplant them.

— Richard Lilford, CLAHRC WM Director

Reference:

  1. Lilford RJ. Yet Again RCTs Fail to Confirm Promising Effects of Dietary Factors in Observational Studies. CLAHRC WM News Blog. 25 September 2015.
  2. Lilford RJ. Very Different Results from RCT and Observational Studies? CLAHRC WM News Blog. 25 September 2015.
  3. Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JPA. Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ. 2016; 352: i493.

Big Data in Health Improvement

Big data – which the CLAHRC WM Director takes to be large datasets (terabytes or more) that have been collected from more than one source – have great potential use in health improvement according to a recent article.[1] However, the article provides few examples, and those that are mentioned are parochial, relating how an individual hospital improved performance on the basis of local audit. Hardly big data. A better example would come from improvements in diabetes care in Scotland based on the National data-linkage system.[2] The CLAHRC WM Director suspects there are many others and asks readers to send examples.

— Richard Lilford, CLAHRC WM Director

References:

  1. Wyber R, Vaillancourt S, Perry W, Mannava P, Folaranmi T, Celi LA. Big data in global health: improving health in low- and middle-income countries. Bull World Health Organ. 2015; 93(3): 203-8.
  2. Cunningham S, McAlpine R, Leese R, et al. Using web technology to support population-based diabetes care. J Diabetes Sci Technol. 2011; 5(3): 523-34.