What’s a drug company gonna do?

I feel sorry for the big pharmaceutical companies. They are picked on by pretty much everyone that is not on their payroll, and even some that are. Take Pfizer and GSK and other drug companies for example, accused of deliberately hiding results from clinical trials that had results showing their antidepressants weren’t what they’re cracked up to be. Most of us, if we had to show the world the results of our personal clinical and social trials would do the same: Do do the same.

But, yes, they should be held to a higher standard than I am. In part because they claim a high ethical ground and spend millions advertising their integrity, and in part because they are developing products for use by people who may suffer or die as a result of using products that are not as advertised. And in part because they are abusing the trust of the patients who volunteer for the studies on the promise that the results obtained will be useful to science and to other patients. Pretty good arguments.

So what happens. Why are they hiding results?

The companies will claim that it is hard to publish studies with mixed results, where the results are unclear. They are right. Journals, particularly the higher profile journals, are less interested in publishing a study if it does not show a positive result. The reason is that most of these studies have relatively small sample sizes (as was the case in the recent publication - about 153 patients per trial). Failure to show a statistically significant benefit does not preclude the fact that one exists if a larger trial were done and certainly doesn’t indicate that the drug being tested has no effect. Thus the value of the information obtained from a particular trial of small sample size showing no statistical positive effect is minimal. (It is only later, when small studies accumulate, that pooling the results of these trials reveals the true efficacy or inefficacy of a drug.)

This, however, is a weak company argument. There is nothing that stops the company from publishing the results on its own web site. There are also many journals available that have the space to publish the results of smaller trials. Indeed, PLoSONE, is an excellent journal that aims to publish all research, no matter what the result or statistical signifigance.

It is also true that the FDA has copies of the complete results of these unpublished trials but is prohibited by covenants set by government from publishing trials submitted for a drug that subsequently was not approved: The information remains proprietary. The FDA ought to seek approval to make these trials publicly available on its web pages. And the pharmaceutical company lobby group - the Pharmaceutical Research and Manufacturers of America - ought to lobby government to make this regulatory change instead of lobbying to keep it in place.

But the company problems are deeper than non-publication of results that might hinder the marketing and sales of products. As this recent study showed, many companies alter the results when it comes to publication. They do this mainly by selecting study end-points that put the products in a more favourable light. In the 51 reported studies where the results were published, 11 of them (22%) had the results altered and the published results were different than the results show to the FDA. (The FDA had the drug as having no statistically significant effect, the published study showed a positive effect.)

How can this happen? Other than falsification of the data, the main reason is that all studies have multiple possible outcomes of interest. In studies of antidepressants, we may be interested in changes of mood of study patients, or suicide ideation, or attempts at suicide and so on. Yet each randomized clinical trial must declare, before the start of the study, at the time of construction of the protocol, a primary outcome of interest. The trial is designed around this primary outcome. Now in any trial there may be interest in other outcomes. For example, in a study is designed to look at suicide ideation as a primary outcome, the result may show no statistically significant difference. Yet, on another outcome, say a particular scale of patient mood, patients randomly assigned to the active compound, may have higher (better) scores and these may be statistically significant. Nonetheless, this is considered a post-hoc analysis and could easily have occurred by chance. By the rules of science it could have been a chance finding and the result can not be claimed as proof that the drug improves mood.

That major pharmaceutical companies are cheating. They are unethical. They are violating the promise made to study subjects that their participation in trials will benefit other patients and the enterprise of science. They need to be condemned, without reserve. Journals also, need to be chastised for lax editorial oversight in the publication of clinical trials that report non-primary outcomes as primary outcomes, thus misleading their readers and patients. Journals and reviewers have only to request study protocols to validate that the research submitted precisely matches the study hypotheses.

Will these companies ever get it right? Surely the will, I hope.

In a future blog I’ll get into what needs to happen within companies if they are too get it right.

Reference:
Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and influence on apparent efficacy. N Engl J Med 2008;358:252-60