Monthly Archives: March 2016

Uncategorised

Asking for evidence from All Trials

Leave a comment

I’ve written before about how the statistic “50% of all clinical trials are unpublished”, much beloved of the All Trials campaign, is simply not evidence based.

The charity Sense About Science, who run the All Trials campaign, also run a rather splendid website called “ask for evidence“, which encourages people to ask for evidence when people make dodgy claims.

So I used Sense About Science’s website to ask for evidence from Sense About Science for their claim that 50% of all trials are unpublished.

To their credit, they responded promptly, and pointed me to this article, which they claimed provided the evidence behind their claim.

So how well does it support their claim?

Interestingly, the very first sentence states that they don’t really have evidence. The first paragraph of the document reads as follows:

“We may never know the answer to this question. In some ways, maybe it doesn’t matter. Even one clinical trial left unreported is unacceptable.”

So if they don’t know, why are they making such a confident claim?

As an aside, they are of course spot on in the rest of that paragraph. Even if the proportion of unpublished trials is substantially less than 50%, if it’s greater than zero it’s still too high.

They go on to further emphasise the point that they really don’t know what proportion of trials are unpublished:

“It is clearly not a statistic, and we wouldn’t advocate trying to roll up the results of all the studies listed below to produce something spuriously precise.”

They then go on to explain the complexity of estimating the proportion of unpublished trials. It certainly is complex, and they give a good explanation of why. It’s not a bad document, and even includes some studies showing much higher rates of disclosure that they don’t admit to on their main website article.

But if they understand, as they clearly do, that the claim that half of all trials are unpublished is spuriously precise and that it would be wrong to claim that, why do they do so anyway?

Not only do they make the claim very confidently on their own website,  the claim also often appears in the very many articles that their PR machine churns out. These articles state it as fact, and do not acknowledge the problems that they describe in their background document. You will never see those articles citing the most recent research showing greater than 90% disclosure rates.

This still seems dishonest to me.

Uncategorised

The Dianthus blog

Leave a comment

Those of you who have known me for some time will know that I used to blog on the website of my old company, Dianthus Medical.

Well, Dianthus Medical is no more, but I have preserved the blog for posterity. You can find it again at its original home, dianthus.co.uk. If by some remote chance you happen to have any links to any of the old blogposts, they should work again now. All the blogposts have their original URLs.

The Dianthus blog no longer accepts comments, but if you have an urgent need to leave a comment on anything posted there, you are welcome to leave a comment on this page.

Economics, Politics

Sugar tax

11 Comments

One of the most newsworthy features of yesterday’s budget was the announcement that the UK will introduce a tax on sugary drinks.

There is reason to think this may have primarily been done as a dead cat move, to draw attention away from the fact the the Chancellor is missing all his deficit reduction targets and cutting disability benefits (though apparently he can still afford tax cuts for higher rate tax payers).

But what effect might a tax on sugary drinks have?

Obviously it will reduce consumption of sugary drinks:  it’s economics 101 that when the price of something goes up, consumption falls. But that by itself is not interesting or useful. The question is what effect will that have on health and well-being?

The only honest answer to that is we don’t know, as few countries have tried such a tax, and we do not have good data on what the effects have been in countries that have.

For millionaires such as George Osborne and Jamie Oliver, the tax is unlikely to make much difference. Sugary drinks are such a tiny part of their expenditure, they will probably not notice.

But what about those at the other end of the income scale? While George Osborne may not realise this, there are some people for whom the weekly grocery shop is a significant proportion of their total expenditure. For such people, taxing sugary drinks may well have a noticeable effect.

For a family who currently spends money on sugary drinks, 3 outcomes are possible.

The first possibility is that they continue to buy the same quantity of sugary drinks as before (or sufficiently close to the same quantity that their total expenditure still rises). They will then be worse off, as they will have less money to spend on other things. This is bad in itself, but also one of the strongest determinants of ill health is poverty, so taking money away from people is unlikely to make them healthier.

The second possibility is that they reduce their consumption of sugary drinks by an amount roughly equivalent to the increased price. They will then be no better or worse off in terms of the money left in their pocket after the weekly grocery shopping, but they will be worse off in welfare terms, as they will have less of something that they value (sugary drinks). We know that they value sugary drinks, because if they didn’t, they wouldn’t buy them in the first place.

Proponents of the sugar tax will argue that they will be better off in health terms, as sugary drinks are bad for you, and they are now consuming less of them. Well, maybe. But that really needs a great big [citation needed]. This would be a relatively modest decrease in sugary drink consumption, and personally I would be surprised if it made much difference to health. There is certainly no good evidence that it would have benefits on health, and given that you are harming people by depriving them of something they value, I think it is up to proponents of the sugar tax to come up with evidence that the benefits outweigh those harms. It seems rather simplistic to suppose that obesity, diabetes, and the other things the the sugar tax is supposed to benefit are primarily a function of sugary drink consumption, when there are so many other aspects of diet, and of course exercise, which the sugar tax will not affect.

The third possibility is that they reduce their consumption by more than the amount of the price increase. They will now have more money in their pocket at the end of the weekly grocery shop. Perhaps they will spend that money on vegan tofu health drinks and gym membership, and be healthier as a result, as the supporters of the sugar tax seem to believe. Or maybe they’ll spend it on cigarettes and boiled sweets. We simply don’t know, as there are no data to show what happens here. The supposed health benefits of the sugar tax are at this stage entirely hypothetical.

But whatever they spend it on, they would have preferred to spend it on sugary drinks, so we are again making them worse off in terms of the things that they value.

All these considerations are trivial for people on high incomes. They may not be for people on low incomes. What seems certain is that the costs of the sugar tax will fall disproportionately on the poor.

You may think that’s a good idea. George Osborne obviously does. But personally, I’m not a fan of regressive taxation.

Clinical trials, Publications

Are a fifth of drug trials really designed for marketing purposes?

1 Comment

A paper by Barbour et al was published in the journal Trials a few weeks ago making the claim that “a fifth of drug trials published in the highest impact general medical journals in 2011 had features that were suggestive of being designed for marketing purposes”.

That would be bad if it were true. Clinical trials are supposed to help to advance medical science and learn things about drugs or other interventions that we didn’t know before. They are not supposed to be simply designed to help promote the use of the drug. According to an editorial by Sox and Rennie, marketing trials are not really about testing hypotheses, but “to get physicians in the habit of prescribing a new drug.”

Marketing trials are undoubtedly unethical in my opinion, and the question of how common they are is an important one.

Well, according to Barbour et al, 21% of trials in high impact medical journals were designed for marketing purposes. So how did they come up with that figure?

That, unfortunately, is where the paper starts to go downhill. They chose a set of criteria which they believed were associated with marketing trials. Those criteria were:

“1) a high level of involvement of the product manufacturer in study design 2) data analysis, 3) and reporting of the study, 4) recruitment of small numbers of patients from numerous study sites for a common disease when they could have been recruited without difficulty from fewer sites, 5) misleading abstracts that do not report clinically relevant findings, and 6) conclusions that focus on secondary end-points and surrogate markers”

Those criteria appear to be somewhat arbitrary. Although Barbour et al give 4 citations to back up those criteria, none of the papers cited provides any data to validate those criteria.

A sample of 194 papers from 6 top medical journals were then assessed against those criteria by 6 raters (or sometimes 5, as raters who were journal editors didn’t assess papers that came from their own journal), and each rater rated each paper as “no”, “maybe”, or “yes” for how likely it was to be a marketing trial. Trials rated “yes” by 4 or more raters were considered to be marketing trials, and trials with fewer than 4 “yes” ratings could also be considered marketing trials if there were no more than 3 “no” ratings and a subsequent consensus discussion decided they should be classified as marketing trials.

The characteristics of marketing trials were then compared with other trials. Not surprisingly, the characteristics described above were more common in the trials characterised as marketing trials. Given that that’s how the “marketing” trials were defined, that outcome was completely predictable. This is a perfectly circular argument. Though to be fair to the authors, they do acknowledge the circularity of their argument in the discussion.

One of the first questions that came to my mind was how well the 6 raters agreed. Unfortunately, no measure of inter-rater agreement is presented in the paper.

Happily, the authors get top marks for their commitment to transparency here. When I emailed to ask for their raw data so that I could calculate the inter-rater agreement myself, the raw data was sent promptly. If only all authors were so co-operative.

So, how well did the authors agree? Not very well, it turns out. The kappa coefficient for agreement among the raters was a mere 0.36 (kappa values vary between 0 and 1, where 0 is no better than guessing and 1 is perfect agreement, with values above about 0.7 generally considered to be acceptable agreement). This does not suggest that the determination of what counted as a marketing trial was obvious.

To look at this another way, of the 41 trials characterised as marketing trials, only 4 of those trials were rated “yes” by all raters, and only 9 were rated “yes” by all but one. This really doesn’t suggest that the authors could agree on what constituted a marketing trial.

So what about those 4 trials rated “yes” by all reviewers? Let’s take a look at them and see if the conclusion that they were primarily for marketing purposes stacks up.

The first paper is a report of 2 phase III trials of linaclotide for chronic constipation. This appears to have been an important component of the clinical trial data leading to licensing of rifamixin for IBS, as the trials are mentioned in the press release where the FDA describes the licensing of the drug. So the main purpose of the study seems to have been to get the drug licensed. And in contrast to point 6) in the criteria for determining a marketing study, the conclusions were based squarely on the primary endpoint. As for point 5), obviously the FDA thought the findings were clinically relevant as they were prepared to grant the drug a license on the back of them.

The second is a report of 2 phase III trials of rifamixin for patients with irritable bowel syndrome. Again, the FDA press release shows that the main purpose of the studies was for getting the drug licensed.  And again, the conclusions were based on the primary endpoint and were clearly considered clinically relevant by the FDA.

The third paper reports a comparative trial of tiotropium versus salmeterol for the prevention of exacerbations of COPD. Tiotropium was already licensed when this trial was done so this trial was not for the purposes of original licensing, but it does appear that it was important in subsequent changes to the licensing, as it is specifically referred to in the prescribing information.  Again, the conclusions focussed on the primary outcome measure, which was prevention of exacerbations: certainly a clinically important outcome in COPD.

The fourth paper was also done after the drug was originally licensed, which in this case was eplerenone. The study looked at overall mortality in patients with heart failure. Again, the study is specifically referenced in the prescribing information, and again, the study’s main conclusions are based on the primary outcome measure. In this case, the primary outcome measure was overall mortality. How much more clinically relevant do you want it to be?

Those 4 studies are the ones with the strongest evidence of being designed for marketing purposes. I haven’t looked at any of the others, but I think it’s fair to say that there is really no reason to think that those 4 were designed primarily for marketing.

Of course in one sense, you could argue that they are all marketing studies. You cannot market a drug until it is licensed. So doing studies with the aim of getting a drug licensed (or its licensed indications extended) could be regarded as for marketing purposes. But I’m pretty sure that’s not what most people would understand by the term.

So unfortunately, I think Barbour et al have not told us anything useful about how common marketing studies are.

I suspect they are quite rare. I have worked in clinical research for about 20 years, and have worked on many trials in that time. I have never worked on a study that I would consider to be designed mainly for marketing. All the trials I have worked on have had a genuine scientific question behind them.

This is not to deny, of course. that marketing trials exist. Barbour et al refer to some well documented examples in their paper. Also, in my experience as a research ethics committee member, I have certainly seen studies that seem to serve little scientific purpose and the accusation of being designed mainly for marketing would be reasonable.

Again, they are rare: certainly nothing like 1 in 5. I have been an ethics committee member for 13 years, and typically review about 50 or so studies per year. The number of studies I have suspected of being marketing studies in that time could be counted on the fingers of one hand. If it had been up to me, I would have not given those studies ethical approval, though other members of my ethics committee do not share my views on the ethics of marketing trials, so I was outvoted and the trials were approved.

So although Barbour et al ask an important question, it does not seem to me that they have answered it. Still, by being willing to share their raw data, they have participated fully in the scientific process. Publishing something and letting others scrutinise your results is how science is supposed to be done, and for that they deserve credit.

 

 

 

Cancer, Economics, Healthcare

Solving the economics of personalised medicine

1 Comment

It’s a well known fact that many drugs for many diseases don’t work very well in in many patients. If we could identify in advance which patients will benefit from a drug and which won’t, then drugs could be prescribed in a much more targeted manner. That is actually a lot harder to do than it sounds, but it’s an active area of research, and I am confident that over the coming years and decades medical research will make much progress in that direction.

This is the world of personalised medicine.

Although giving people targeted drugs that are likely to be of substantial benefit to them has obvious advantages, there is one major disadvantage. Personalised medicine simply does not fit the economic model that has evolved for the pharmaceutical industry.

Developing new drugs is expensive. It’s really expensive. Coming up with a precise figure for the cost of developing a new drug is controversial, but some reasonable estimates run into billions of dollars.

The economic model of the pharmaceutical industry is based on the idea of a “blockbuster” drug. You develop a drug like Prozac, Losec, or Lipitor that can be used in millions of patients, and the huge costs of that development can be recouped by the  huge sales of the drug.

But what if you are developing drugs based on personalised medicine for narrowly defined populations?  Perhaps you have developed a drug for patients with a specific variant of a rare cancer, and it is fantastically effective in those patients, but there may be only a few hundred patients worldwide who could benefit. There is no way you’re going to be able to recoup the costs of a billion dollars or more of development by selling the drug to a few hundred patients, without charging sums of money that are crazily unaffordable to each patient.

Although the era of personalised medicine is still very much in its infancy, we have already seen this effect at work with drugs like Kadcyla, which works for only a specific subtype of breast cancer patients, but at £90,000 a pop has been deemed too expensive to fund in the NHS. What happens when even more targeted drugs are developed?

I was discussing this question yesterday evening over a nice bottle of Chilean viognier with Chris Winchester. I think between us we may have come up with a cunning plan.

Our idea is as follows. If a drug is being developed for a suitably narrow patient population that it could be reasonably considered a “personalised medicine”, different licensing rules would apply. You would no longer have to obtain such a convincing body of evidence of efficacy and safety before licensing. You would need some evidence, of course, but the bar would be set much lower. Perhaps some convincing laboratory studies followed by some small clinical trials that could be done much more cheaply than the typical phase III trials that enrol hundreds of patients and cost many millions to run.

At that stage, you would not get a traditional drug license that would allow you to market the drug in the normal way. The license would be provisional, with some conditions attached.

So far, this idea is not new. The EMA has already started a pilot project of “adaptive licensing“, which is designed very much in this spirit.

But here comes the cunning bit.

Under our plan, the drug would be licensed to be marketed as a mixture of the active drug and placebo. Some packs of the drug would contain the active drug, and some would contain placebo. Neither the prescriber nor the patient would know whether they have actually received the drug. Obviously patients would need to be told about this and would then have the choice to take part or not. But I don’t think this is worse than the current situation, where at that stage the drug would not be licensed at all, so patients would either have to find a clinical trial (where they may still get placebo) or not get the drug at all.

In effect, every patient who uses the drug during the period of conditional licensing would be taking part in a randomised, double-blind, placebo-controlled trial.  Prescribers would be required to collect data on patient outcomes, which, along with a code number on the medication pack, could then be fed back to the manufacturer and analysed. The manufacturer would know from the code number whether the patient received the drug or placebo.

Once sufficient numbers of patients had been treated, then the manufacturer could run the analysis and the provisional license could be converted to a full license if the results show good efficacy and safety, or revoked if they don’t.

This wouldn’t work in all cases. There will be times when other drugs are available but would not be compatible with the new drug. You could not then ethically put patients in a position where a drug is available but they get no drug at all. But in cases where no effective treatment is available, or the new drug can be used in addition to standard treatments, use of a placebo in this way is perfectly acceptable from an ethical point of view.

Obviously even when placebo treatment is a reasonable option, there would be logistical challenges with this approach (for example, making sure that the same patient gets the same drug when their first pack of medicine runs out). I don’t pretend it would be easy. But I believe it may be preferable to a system in which the pharmaceutical industry has to abandon working on personalised medicine because it has become unaffordable.