The All Trials campaign is very fond of quoting the statistic that only half of all clinical trials have ever been published. That statistic is not based on good evidence, as I have explained at some length previously.
Now, if they are just sending the odd tweet or writing the odd blogpost with dodgy statistics, that is perhaps not the most important thing in the whole world, as the wonderful XKCD pointed out some time ago:
But when they are using dodgy statistics for fundraising purposes, that is an entirely different matter. On their USA fundraising page, they prominently quote the evidence-free statistic about half of clinical trials not having been published.
Giving people misleading information when you are trying to get money from them is a serious matter. I am not a lawyer, but my understanding is that the definition of fraud is not dissimilar to that.
The All Trials fundraising page allows comments to be posted, so I posted a comment questioning their “half of all clinical trials unpublished” statistic. Here is a screenshot of the comments section of the page after I posted my comment, in case you want to see what I wrote:
Now, if the All Trials campaign genuinely believed their “half of all trials unpublished” statistic to be correct, they could have engaged with my comment. They could have explained why they thought they were right and I was wrong. Perhaps they thought there was an important piece of evidence that I had overlooked. Perhaps they thought there was a logical flaw in my arguments.
But no, they didn’t engage. They just deleted the comment within hours of my posting it. That is the stuff of homeopaths and anti-vaccinationists. It is not the way that those committed to transparency and honesty in science behave.
I am struggling to think of any reasonable explanation for this behaviour other than that they know their “half of all clinical trials unpublished” statistic to be on shaky ground and simply do not wish anyone to draw attention to it. That, in my book, is dishonest.
This is such a shame. The stated aim of the All Trials campaign is entirely honourable. They say that their aim is for all clinical trials to be published. This is undoubtedly important. All reasonable people would agree that to do a clinical trial and keep the results secret is unethical. I do not see why they need to spoil the campaign by using exactly the sort of intellectual dishonesty themselves that they are campaigning against.
http://www.alltrials.net/find-out-more/faq/
They try and define the ‘around’ half in their faq.
Yes they do, and I explain why they are wrong in the blogpost I link to in the first paragraph above.
Yea, that is a pretty reasonable justification on their evidence behind the soundbite. It is very uncool to delete that comment, as they could have simply linked to that FAQ, which seems to address the issues you had discussed. If you had to choose a percentage for a soundbite, what would you pick given your analysis?
It doesn’t really address the issues I have discussed. My zombie statistics blogpost is a response to their claims, not vice versa.
Personally, I wouldn’t choose a percentage for a soundbite. I’d rather have something based on evidence than make something up because it sounds good. But if I really had to, I would guess that somewhere in the region of 70-80% of trials relevant to the drugs doctors prescribe today are published.
Perhaps it would be better if they presented the data in a Bayesian framework, so we’d end up with something more representative of the various samples and a credibility interval.
It looks like they touch on the issue of older trials, which is what you are saying brings down the percentage of unpublished trials.