Are Mathematically or Logically Contradictory Statements in a Manuscript a Sign that the Whole Study is Unreliable?

Cole, et al.[1] found that logical discrepancies in a manuscript are associated with unreliable studies. So unreliable that they had to be retracted. In fact discrepancy counts are 2.7-fold higher in retracted reports than in unretracted reports, according to this blinded case-controlled study. However, the prevalence of retraction is extremely low, so the probability that a study with lots of discrepancies warrants retraction is also very low – remember posterior odds = prior odds x likelihood ratio. In short it is a lousy diagnostic (or even screening) test. This study reminded the CLAHRC WM Director of a study conducted in our precursor CLAHRC pilot [2] where it was found that doctors who make minor errors are more likely to make serious errors but where, again, statistical significance did not entail clinical significance.

— Richard Lilford, CLAHRC WM Director

References:

  1. Cole GD, Nowbar AN, Mielewczik M, et al. Frequency of discrepancies in retracted clinical trial reports versus unretracted reports: blinded case-control study. BMJ. 2015; 351: h4708.
  2. Coleman JJ, Hemming K, Nightingale PG, et al. Can an electronic prescribing system detect doctors who are more likely to make a serious prescribing error? J R Soc Med. 2011; 104(5): 208-18.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s