US FDA sets standards for medical journals and peer review

Photo Credit vaneska~tHOmz's
In seeking to define standards for ‘good reprint practices’ by pharmaceutical companies seeking to provide copies of journal articles to doctors the FDA has had to grapple with the quality of articles and of journals. That they have failed is hardly a surprise, but that they even tried is quite astonishing given their intimate knowledge of the hundreds (if not thousands) of flimsy and misleading research reports submitted to the agency from these same companies seeking drug approvals, most of which are published in these same journals.

The FDA has a legislated role not only to approve new pharmaceuticals, but to do so quickly. The agency is under pressure from all sides except the ranks of the cautious and skeptical to approve drugs and new usages with minimal delay and minimum of assurance they are effective and safe. Once approved, pharmaceutical companies market these drugs to physicians and patients for use.

Pharmaceutical companies (and some physicians and sick patients) are anxious to see if drugs approved for one indication - say epilepsy in adults - might just work for other health problems - say Parkinson’s disease or among groups of individuals (such as children and pregnant women) where there are often few clinical trials.

These ‘off-label’ (non-FDA approved) uses are the target of this FDA Guidance for Industry on Good Reprint Practices for the Distribution of Journal Articles.... The guidance on good reprint practices is a set of suggestions that pharmaceutical companies should use when they are promoting drugs for unapproved uses. The FDA claims that:

“public health may be advanced by healthcare professionals’ receipt of medical journal articles ... on unapproved or new uses of approved or cleared medical products that are truthful and not misleading.”

This is ludicrous. That pharmaceutical companies would select journal articles that are “truthful and not misleading” is irrational from the companies point of view - they will select articles that support use of the product, that emphasize benefit and minimize harm - and naive or blind from the FDA point of view.

There is a wealth of evidence that the published literature looks like a shipwreck of clinical trials that have been shown to be wrong. First of all it is clear even from the perspective of the FDA that published studies sponsored by pharmaceutical companies show a terrific bias towards benefit and against harm. Studies showing benefit of a product are published, those showing no benefit or harm are not. The astonishing thing is that the FDA is intimately aware of this sham evidence base, for it has the only complete record of all clinical trials done to explore new uses of existing products.

For example of 74 clinical trials of antidepressants (involving 12 drugs for which companies sought approval 31% were never published. Published clinical trials show that the drugs are on average 31% better than the comparison drugs or placebos; however, when all the evidence is examined - published and unpublished - the 12 drugs on average showed no benefit over the comparison group.<1> Ludicrous is not too extreme an adjective to describe a belief that pharmaceutical companies will show doctors the complete evidence, on anything.

In its disconnect from reality the FDA goes further by listing a set of criteria that pharmaceutical companies can use (are not required to use) when choosing articles to flog to doctors and their patients. Under the rubric “Good Reprint Practices” we find that:

The Journal should:
1. Have an editorial board that uses experts.
2. Have an editor and board independent of the journal owners
3. Have a policy of full disclosure of conflict of interest or biases.
4. Be peer reviewed

The Journal article should not:
1. Be false or misleading
2. Be a drug company funded special supplement.
3. Have been withdrawn by the journal
4. (Promote a product) that poses a significant risk

I know of few journals that would; a) not have these policies in place; b) publish articles known to be false or misleading and; c) not know and understand that they regularly violate them all, despite their best intentions.

Even well funded journals like the New England Journal of Medicine and JAMA get duped and make mistakes. And they know that they are publishing but part of the evidence, mostly the part showing benefit.

The vast vast majority of journals, however, are small, employ a part-time editor and have but limited time and resources to provide effective oversight of the material they are publishing. They can't guarantee to effectively sort the excellent science from the sloppy, and even less to detect purposeful deceit by funding sponsors with embedded conflicts of interest. Yet, the FDA criteria render virtually all medical journals eligible for cherry picking of articles by company marketing departments and peddling of it to practicing physicians and their patients.

This is not an effective or safe way to improve the health of the public. It is a an effective and now legally protected way ("the FDA said we should") for pharmaceutical and device manufacturing companies to expand markets for unapproved uses.

Further there is no need to promote selected research publications on specific drugs to physicians. Although individual practitioners ought to be able to recognize the conflict of interest of the pharmaceutical company salesperson, they don’t. The majority continue to accept visits and advice form various peddlers. The average practitioner (and patient) has little training, experience or frankly interest in reading and understanding research articles.

In fact, individual practitioners should be encouraged to not read research articles involving randomized trials of a single drug, for there is a reasonable probability that any one study will be flawed or subsequently disproved by accumulating further evidence. Average practitioners are encouraged to stick to guidelines which while themselves are still susceptible to bias and influence by pharmaceutical companies are at least one-half degree of freedom removed from Wall Street.

The FDA criteria for choosing a journal and article do not reflect the very flawed reality of current medical publishing nor of the sad state of market driven medical research, nor of the abilities, availabilities and interests of practitioners to read an understand. The FDA does not have to cave to pressure from Wall Street to let pharmaceutical companies market directly to individual practitioners and their patients by flogging journal articles. Ethical pharmaceutical companies should act responsibly and urge the FDA to drop this guidance.

Reference

Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008;358:252-60.

Diabetes cured? - Rosiglitazone trials raise new questions.

What your doctor is reading, or should be.

Today, diabetes in adults is common. Yet, when Banting, Best and others in 1922 showed the effects of a crude extract of pancreatic tissue on the blood sugar of a 14 year old boy the disease was uncommon. <1> Only those who had extremely high blood sugars, usually discovered in childhood with what we would now call Type I diabetes, were diagnosed with diabetes. They were ill, had ketosis (acid in their blood), were extremely ill and usually did not survive long beyond adolescence. Insulin changed that dramatically.

Type II diabetes was obviously around, but was not diagnosed by physicians because it was asymptomatic unless the blood sugar levels were exceedingly high, in which case the patient had loss of sugar in the urine which was accompanied by water and produced symptoms of thirst and frequent urination, day and night. Thus, in the 1960’s and 70’s it was uncommon for physicians to diagnose diabetes unless the patient symptoms, in which case, either insulin or one of the oral agents could be prescribed - in addition to diet and weight loss, neither of which was effective (then and today).

Type II diabetes became a recognized disease when it was observed that individuals with high blood sugars had a higher risk of micro-vascular complications (peripheral small arterial disease in the legs, leading eventually to ulcers and amputations, and microvascular effects on the kidney and retina and associated renal failure and loss of vision) as well as macrovascular complications (atherosclerosis of larger arteries and associated heart attacks and strokes).

Epidemiologic studies revealed a clear association between high blood sugars and these bad outcomes. But it is a jump in logic to assume that the culprit is the blood sugar. Perhaps the blood sugar increases are due to some other cause, which is also causing disease in small and large arteries? In this counter explanation, the blood sugar is but a sign of the disease, not the cause. Reducing blood sugar may not prevent the bad outcomes.

The interest in the new oral agents, for the treatment of asymptomatic increases in blood sugar, and aggressive marketing by manufacturer’s of these agents, along with endorsements by then nascent patient advocacy groups such as national diabetes associations, led to an epidemic of Type II diabetes, later fueled by increasingly sedentary lifestyles and a glut of obesity.

What was needed to unravel this possible paradox - that we were diagnosing and treating a disease that did not exist (treating a blood test not a disease) was a randomized clinical trial tabulating outcomes that really mattered to people - fewer amputations, less renal and visual failure, fewer strokes and heart attacks and perhaps longer life.

Readers will understand the importance of RCTs for this type of question. We understand that such RCTs will be difficult to complete because the outcomes are relatively rare and occur but years later; thus the need for very large numbers of study subjects with Type II diabetes followed for long periods of time. Expensive trials, long durations, prone to missing information and loss to follow-up of many trial subjects makes this type of research unwieldy, unfriendly and open to variable interpretations, all of which has happened. .

There have been but a few such trials. Earliest was the University Group Diabetes Study about 30 years ago. The results were unclear and the trial is still being interpreted. Nonetheless the results form the basis of current therapeutic recommendations.

The other important outcome trial of type II diabetes is the United Kingdom Prospective Diabetes Study (UKPDS), involving over 4,000 patients randomized to receive an oral anti-diabetic agent and followed on average for about 10 years. <2> That study showed improved outcomes, but the trial was messy (patients often took multiple agents over this time frame including insulin, many were lost to follow-up etc.). The results are still discussed.

Since then, as newer oral pharmaceuticals were developed to lower blood sugar or increase insulin sensitivity, regulatory agencies have required RCT evidence only that they work: that they do lower blood sugar. Ignoring patient meaningful outcomes, only small RCTs of short duration are needed.

Recently, however, pharmaceutical companies have conducted larger RCTs to determine if use resulted in fewer serious complications among patients with Type II diabetes. Undoubtedly, the motivation for doing these trials is to develop data that would give their product a competitive edge in an increasingly crowded market of new compounds.

The ADOPT trial (A Diabtes Outcome Prevention Trial), <3>The DREAM trial (Diabetes Reduction Assessment with Rampril and Rosiglitazone Medication), <4> and the RECORD trial (Rosiglitazone Evalutaed for Cardiac Outcomes and Regulation of Glycemia in Diabetes) <5> trials were all sponsored by pharmaceutical companies anxious to show that their particular drugs were better than cheap generic versions of older drugs (sulphonyl ureas and metformin).

All 3 trials showed no benefit. And all 3 trials strongly suggest that the new medications are not only of no benefit, but are harmful to patients with Type II diabetes.

Suffice to look at Rosiglitazone.

In an extraordinary piece of research, Nisan and Wolski <6> found 26 reports of small clinical trials with rosiglitazone. Summarizing these in a meta-analysis they showed that patients randomized to receive this drug, often along with metformin or a sulfonyl urea were at a statistically significant higher risk (43%) of heart attacks and/or death than the control groups receiving just sulfonyl urea or metformin. [Relative risk 1.43, 95% CI 1.03 to 1.98]

In an effort to refute the Nisan Wolski paper, GalaxoSmithKlein hurriedly released an unplanned interim analysis of their ongoing trial (RECORD) that, unfortunately for the pharmaceutical company, not only failed to refute the previous results but rather confirmed them; although the company interpretation disparaged the seriousness of the harms (heart attacks, strokes and death).<4>

Pharmaceutical company sponsors generally attribute the unexpected bad-for-sales results (for their products) to unforeseen difficulties in carrying out the trials (for example lower than expected frequencies of outcome events) and, curiously, tend to attribute the unfavourable results - even if statistically significant, to chance.

But even a casual observer would interpret these studies as showing no benefit form these newer compounds. And with the potential for harm, cautious physicians and patients would conclude that is best to avoid prescribing and taking them.

Interpretation

These results are striking and deserve our attention. The evidence is clear that even within the company sponsored RCT, patients taking rosglitazone along with another agent suffered more heart attacks, strokes and/or death than those taking a single older agent such as metformin or a sulphonylurea.

These results are applicable to all drugs of this class - thiazolidinediones

Implications

For patients

Patients with type II diabetes should not be taking thiazolidinediones. Those receiving these compounds should be switched to sulphonylureas and or metformin or insulin.

Patients with type II diabetes, who are asymptomatic and have blood sugars lower than their renal threshold (no sugar in the urine), should be encouraged to lose weight and exercise. Often this will bring their blood sugars into more normal ranges.

Current guidelines <7> are less cautious than I am and continue to recommend aggressive lowering of blood sugar even in asymptomatic patients. There is some evidence from randomized trials of insulin and from animal experiments that maintaining blood sugars in near normal ranges is associated with reductions in micro-vascular disease and thus ought to reduce the risk of peripheral vascular disease, renal failure and loss of vision. Thus theoretically, a lower blood sugar is better. But there is little clinical evidence to support the guideline recommendations which are all financed by pharmaceutical companies with deeply vested interests.

For society

The epidemic of type II diabetes is the result of the hard work and clever execution of projects hatched by marketing divisions of pharmaceutical companies eager to expand markets for their products. The fact that all trials of treatments of type II diabetes have shown that treatment can cause harm (or might in the UKPDS trial), makes it imperative that we once again consider a large scale simple trial of these oral agents, and measure the simple but devastating outcomes that count for patients; amputations, renal failure, blindness, myocardial infarctions and stokes and deaths.

In the interim, patients with abnormal blood sugars as their only ‘abnormality’ might be best to avoid drugs.

References used - links to those that are free

1. Banting FG, Best CH, Collip JB, Campbell WR, Fletcher AA. Pancreatic extracts in the treatment of diabetes mellitus. Preliminary report. CMAJ 1922;12:141-6

2, UK Prospective Diabetes Study (UKPDS) Group. Effect of intensive blood-glucose control with metformin on complications
in overweight patients with type 2 diabetes (UKPDS 34). UK Prospective Diabetes Study (UDPKS) Group

3, Kahn SE, Haffner SM, Heise MA, et al. Glycemic durability of rosiglitazone, metformin, or glyburide monotherapy. N Engl J Med 2006;355:2427-43. [Erratum, N Engl J Med 2007;356:1387-8.]

4. Gerstein HC, Yusuf S, Bosch J, et al. Effect of rosiglitazone on the frequency of diabetes in patients with impaired glucose toleranceor impaired fasting glucose: a randomised controlled trial. Lancet 2006;368:1096-105. [Erratum, Lancet 2006;368:1770.]

5. Home PD, Pocock SJ, Beck-Nielsen H, et al. Rosiglitazone Evaluated for Cardiac Outcomes and Regulation of Glycaemia in Diabetes (RECORD) Study: interim findings on cardiovascular hospitalizations and deaths. N Engl J Med 2007;357.
DOI: 10.1056/NEJMoa073394.

6. Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular disease. N Engl J Med 2007;356:2457-2471. [Erratum, N Engl J Med 2007;357:100.]
online free

7. Nathan, D. M., Buse, J. B., Davidson, M. B., Ferrannini, E., Holman, R. R., Sherwin, R., Zinman, B. (2008). Management of Hyperglycemia in Type 2 Diabetes: A Consensus 8. Algorithm for the Initiation and Adjustment of Therapy: Update regarding thiazolidinediones: a consensus statement from the American Diabetes Association and the European Association for the Study of Diabetes . Diabetes Care 31: 173-175
online free

Chronic Knee pain from osteoarthritis and what to do about it - Topical ibuprophen.

About 1 in 3 adults over the age of 50 have chronic pains in the knees. Most are caused by osteoarthritis, essentially a wearing out of the joint, most likely due to age, prior injuries, obesity or other overuse (work or weight bearing), and in part to one’s genes. If you want to know if you have osteoarthritis a good source is the US National Institutes of Health. They have a patient friendly guide that is free. (Handout on Health Osteoarthritis) Figure is in the handout and shows the main joints that are affected by osteoarthritis.

Most people with more severe forms of pain, especially pain that limits walking, will try anti-inflammatory drugs like aspirin or ibuprofen. These drugs, collectively called NSAIDs (non-steroidal anti-inflammatory drugs), however, can cause side effects, the most serious of which is bleeding in the stomach. Minor side effects are also reported such as indigestion (gastritis or abdominal pains) and a few others such as increases in blood pressure or decreases in kidney function. They should be used with care in patients with pre-existing hypertension, heart, kidney or liver disease and in patients with known gastric bleeding and or taking drugs that might make bleeding easier (anti-coagulants, steroids).

Recently topical ibuprofen has become available.

This study is an RCT of 282 patients selected from general practices in the UK. They represent about 1% of the patients in these practices who were identified as possibly having osteoarthritis. Of these, 85% completed the study.

Primary Outcome Measure: Knee pain at 12 months in the topical and oral groups.

Result: There was no difference between the two groups in pain severity, joint stiffness or mobility.


Interpretation of study:

Topical and oral ibuprofen provided equivalent relief for osteoarthritis of the knees. The study was not designed to show that topical ibuprofen was less likely than oral ibuprophen to cause side effects, especially gastrointestinal bleeding, nor that other side effects were less or more frequent.

The study was not blinded at the patient level: subjectivity may have played a part in this result. Given the low statistical power obtained in this carefully planned study it is difficult to imagine that any other study could do much better. The we may never know the answer to the study question - are topical NSAIDs as good and safer and better tolerated than oral NSAIDs? It is likely however that oral ibuprophen would provide a larger effective dose, resulting in better pain relief and a higher incidence of side effects.

Implications:

For the patient

The rational patient could try topical ibuprofen and expect to find equivalent symptom relief to his or her oral pill: And judge for themselves if this occurred and was worth the additional cost of the topical drug.

The authors say that symptom relief generally did not change over the study, suggesting, perhaps that neither route of administration works - that a placebo or simple acetaminophen (Tylenol, Paracetamol), or no drug treatment might do just as well, and have with fewer side effects.

For Society

Given the higher costs (currently) of topical Ibuprofen, the equivalence of oral and topical ibuprophen for symptom relief and the paucity of data on serious adverse effects, health insurance plans should consider topical ibuprophen to be cost-ineffective compared to oral generic products and therefore not covered. Database studies of large numbers of users might be used to detect rate differences in serious adverse events and if this were so, then the topical agents might be cost-effective.

References:

Underwood M, Ashby D, Cross P et al. Advice to use topical or oral ibuprofen for chronic knee pain in older people: randomized controlled trial and patient preference study. BMJ 2008:336:138-42 doi:10.1136/bmj.39399.656331.25

Ovarian Cancer

What your doctor is reading - or should be
January 28, 2008
Ovarian Cancer
Cancer of the ovaries occurs in about 1 in 50 women over the age of 50 or so. While not that common, the disease is troublesome because it is difficult to diagnosis early and most forms of ovarian cancer are highly malignant, spreading quickly. Thus there has been an interest in detecting the causes of ovarian cancer and perhaps preventing it.

January 28, 2008

Most of these studies are case-control designs. Here women with diagnosed ovarian cancer are interviewed about possible risk factors. Their answers are compared with answers to the same questions by women without ovarian cancer. These types of studies provide the most of the evidence we have, but are fraught with difficulties: For example, people with a particular condition may remember past events they think might have caused their illness more frequently than people without the condition or illness. A bias (recall bias) results, distorting the true picture and creating a false association and a misleading risk factor.

In this week’s issue of The Lancet evidence is summarized from 48 different case-control and prospective studies that included questions about past or present use of oral contraceptives. The evidence is coherent and consistent across all studies and in the two research designs (case control and prospective): Women who reported using oral contraceptives had a lower risk of developing ovarian cancer. The risk ratio was 0.42 (about a 60% reduction in risk) for women who had reported using oral contraceptives for 15 years or more. Of great interest, the protective effect of taking oral contraceptives early in life, persists for years after a woman has stopped taking them. A podcast by one of the authors and a Lancet editor is available free.

Implications

For an individual woman:

As ovarian cancer is relatively rare, the chances of benefit for any one woman are small. For example if 250 women took oral contraceptives for 10 years, 1 case of ovarian cancer would be prevented.

There are hazards of oral contraceptives - mainly deep vein thrombosis (clots in the veins in the legs and abdomen) which may lead to pulmonary emboli and death, but these events are extremely rare especially with the newer low estrogen dose contraceptive pills. Oral contraceptive use has also been linked, but not strongly, with other cancers, particularly breast and cervical cancer - although the latter is most likely confounded (misleading) because we now know that it is caused by several papilloma viruses, themselves associated with more frequent sex (and now potentially preventable with the new vaccines).

And there are benefits. Oral contraceptives are the most effective form of birth control and prevention of unwanted pregnancy and even more unwanted abortions, (both of which carry substantially higher health risks than oral contraceptives).


For society:

The widespread use of oral contraceptives in high and middle-income countries means that millions of women are taking them. Thus even if the benefits of risk reduction for a single woman are small, for all women, as many as 300,000 cases of ovarian cancer will be prevented by use of oral contraceptives. This is an argument for making oral contraceptives more easily available throughout the world. Indeed, The Lancet editors have called for making the pill available over the counter, without a physicians prescription.

Cautions

This research was carefully done by experienced epidemiologists. Nonetheless, the fundamental problem with cancer prevention research on humans, is that we have to rely on non-experimental information. Thus, it remains possible that some other factor, unknown to the women or to the researchers, may have led women who were unlikely to ever get ovarian cancer to be more likely to use oral contraceptives, and women, more likely to get ovarian cancer, less likely to use them. We have no way of knowing if this bias exists.

Warnings

Women with previous deep vein thrombi, migraines, heart disease or liver disease should probably not take oral contraceptives, or at least should discuss this with their physicians.

http://en.wikipedia.org/wiki/Ovarian_cancer
http://www.nlm.nih.gov/medlineplus/ovariancancer.html
http://podcast.thelancet.com/audio/lancet/2008/9609_26january.mp3

So many small clinical trials, so little information

January 25, 2008
Editors get a lot of manuscripts reporting relatively small clinical trials, in the range of a few hundred study subjects. Such studies can never evaluate clinically meaningful outcomes because like death or hospitalization because these events are rare and require large trials in order to collect enough events for statistical testing. The smallish trials usually measure indirect outcomes, such as serum cholesterol or a change on some indicator of well being such as mood or depression. While helpful, these kinds of measure are not particularly useful to patients and physicians have to take on faith or biochemistry that there is an underlying chain of causality that will in the end yield some tangible benefit that can be experienced by the patient.

Many of these small trials are never published. Individually they convey virtually no information that can be translated into patient care. At best, if enough of them accumulate they can be combined in meta-analyses and then perhaps some estimate of treatment efficacy can be determined. But this is a long term hope and the result always remains uncertain.

It is perhaps ironic that Orwell’s 1984 was the year that Yusuf and others published their cogent arguments for doing large clinical trials of common serious illnesses. They correctly argued that discovery of a small benefit or risk (say 20% improvement) would carry enormous advantage to a large number of patients and to society. Small trials could never detect this level of benefit. Further, they pointed out that doing small trials on common illnesses and conditions would be very unlikely to yield much information about clinically important events, unless the effect was very large and even then, if the treatment was good enough to produce a big benefit, the benefit would have been obvious to the average clinician without a clinical trial. So why do them?

Yet small trials continue to be published. Yesterday, while trawling the clinical trial registry www.clinicaltrials.gov I found 1,788 studies related to the keyword ‘depression’. Limiting this search to ongoing trials (those still recruiting study subjects), those that were investigator classified as phase III or phase lV studies (not the smaller very early Phase I and II assessments for safety and efficacy that are sometimes used to determine if a ‘larger’ trial should go ahead) and those that were registered after January 1, 2005, showed 331 trials.



There were only 10 trials that had more than 1,000 subjects (and many of these were only secondarily looking at my condition of interest - depression - being studies of patients with cancer on different types of chemotherapy, for example). Most (almost half) of the trials for depression had fewer than 100 subjects. Over 9,000 patients were involved in these very small trials, all of which are unlikely to provide any meaningful information and may in fact prove harmful with false leads.

Now readers may argue that depression and other psychiatric diagnoses are difficult to study and that would be true. But this ought to support the Yusuf position that the study of interventions for depression, a common chronic disease, ought to be carried on very large patient groups looking for interventions that have some modest effect, say a 20-25% reduction in suicide or suicide attempts. There is also growing concern that the active compounds being evaluated may not only not be helpful, but may be in some way a cause of suicides. At present there are 160 ongoing trials involving almost 9,000 patients, each with fewer than 100 study subjects. When one adds in to the equation study subject drop-out, non-compliance, observer and measurement error and so on there is virtually no probability that these studies will yield useful results.

I did not examine or try to judge the adequacy of trial designs, objectives and premises, although this might be interesting given that a large proportion (73%) of these designated Phase III and IV trials are sponsored by industry.

My list of trials is obviously a potpourri. Had I limited the search to Major Depression as a diagnostic category, I would have found fewer trials and a different pattern. Also, mine was a desktop exercise and would need to be more rigoursly double checked for errors and validated. Nonetheless, the graph is a reasonable summary of ongoing clinical trials for an important clinical condition and global public health problem.

I suspect that choosing any other condition in the trial registry would yield similar distributions of study size. It would be interesting to do some of this work more formally. I wonder also, if the registry could incorporate some measure of robustness of the trials that are registered. For example, including the elements of trial design recommended in the Consort Statement www.consort-statement.org would be relatively easy for registrants to complete, (they ought to have it completed before taking their studies for ethical and institutional review in any case) and would allow users of the registry (patients, physicians, researches) to judge at a glance the quality of the trial.

Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Statistics in Medicine 1984;3:409-20

Why do pharmaceutical companies keep screwing up?

Perhaps it’s because they are trying too hard. Or lying to hard. But probably it’s because it is really difficult to bring new compounds to market, costs a lot of shareholder money and is very high risk for failure. It must be similar to designing a new commercial aircraft, or even bringing out a new car model that might or might not be market successful. The difference between a pharmaceutical and a vehicle, however, is not only one of size, it is of efficacy and safety. The auto industry knows before hand that they can design a safe product that actually works. Their only concern is whether there are enough buyers. For the pharmaceutical industry, there are no guarantees at the design stage that the product will work, nor that it will be safe.

The statins are a wonderful example of this difference and of the financial pressures on big pharmaceutical companies. Thus Merck embarked in a joint venture with Schering-Plough on a clinical trial to demonstrate that when their two products are combined into a single tablet called Vytorin, that treatment would be significantly superior to simvastatin alone, Merck’s main cholesterol lowering agent which was about to come off patent and thus be reproduced by generic drug manufacturers at a tiny fraction of the price Merck was able to get when it had the patent-monopoly on the product.

The idea to combine the two compounds made biological sense in that they acted on different sites in the cholesterol metabolic pathways. The Vytorin Enhance trial began in June 2002, aimed to recruit 720 subjects with primary hypercholesterolemia, and after 24 months of treatment to show that the cholesterol plaques in their carotids had statistically significantly shrunk. The results were eagerly anticipated by physicians, patients and shareholders. Although the study end-date was April 2006, by November 2007, the results had not been made public. Merck/Schering-Plough issued a statement to explain the delay:

“The independent panel recommended focusing the primary endpoint to the common carotid artery to expedite the reporting of the study findings.  Merck/Schering-Plough now anticipates that these results of the ENHANCE study will be presented at the American College of Cardiology meeting in March 2008.”

This is a curious statement because the study had only one primary endpoint: carotid artery intima-media thickness per subject over 24 months, comparing baseline reading with the endpoint reading (trial registration, www.clinicaltrials.gov)
Why did the head investigator, Dr. Enrico P. Vetri, a Schering-Plough employee, need to “re-focus”? There was only one focal point. It also does not escape notice that as long as the study was not published, revenue from sales of Vytorin would remain unaffected.

The answer, as it appears in a statement by the two companies on January 14, 2008, (now coming on 2 years after the projected end date of the trial) is that Vytorin doesn’t work. It won’t fly. Not only that, but the data strongly suggests that the combination product is harmful. Subjects in the trial who took simvastatin alone had cleaner carotid arteries than those who took Vytorin.

Now, in fairness to the companies, the result must have been a surprise and they must have wondered if somewhere in the analysis of the carotid artery results a data-entry problem might have occurred, perhaps that the patient data files got mixed up. (Nonetheless, in their November 17 statement, they should have been clear that this was not a question of re-focusing on end points, but something far more serious. Patients taking both drugs did worse. And this is not just that they felt worse, their carotid arteries carrying blood to their brains got more clogged up!

Pharmaceutical company corporate behaviour, we are assured, is changing. Merck claims to "put patient safety first". Yet, this kind of press statement along with the fact that it appeared 16 months after the trial end date belies that claim. The Orwellian language of the press releases must have been a deliberate choice of words by the marketing department, pressured as it always is by Finance. Obfuscation, truthiness - if not lies.

The companies have taken pains to point out that the results show only a tendency towards clogging of the carotid arteries by their product, there is no proof - i.e. statistical significant proof. Sure, the result might have arisen by chance. But the difference in intimal carotid artery plaque was 0.111 mm in the Vytorin patients and only 0.0053 in the simvasatin group. Thus plaques grew almost twice as big in the Vytorin group (91% bigger).

The Vytorin trial also demonstrates that although huge amounts of money are spent annually by patients and governments on statins, there is precious little evidence that they do anything more than lower blood cholesterol. Most folks would not give a damn about some molecule or other in their blood were it not for the fact that they believe that the blood level of their molecule is highly correlated with their atherosclerosis and of their chances of having a heart attack or a stroke. What’s measured in trials however, is 2 steps away from what is of benefit to patients - cholesterol, plaques, strokes. In fact there isn’t much evidence that lowering cholesterol reduces rates of heart attacks and strokes and this study suggests that the the combination product Vytorin may be harmful.

The same is true for most pharmaceuticals. We evaluate whether they work or not, not on the basis of a test drive or flight - will this thing actually fly - but on a set of theoretical arguments that it ought to fly. This is equivalent to Boeing saying “Hey, the engine started it ought to go up!”

In a way it is surprising that Merck and Schering-Plough agreed to go ahead with this trial. They could have proposed a trial that had the soft intermediate end points like levels of cholesterol. Indeed, of the 18 ongoing trials of Vytorin, all but one have intermediate metabolic endpoints - usually levels of LDL-c. Only one large trial - to be completed in 2011, will look at what really matters to patients and doctors - rates of heart attacks and strokes. I wonder now if it is ethical to continue further studies of Vytorin? At a minimum one would have to warn study subjects that it looks like Vytorin doesn’t prevent the clogging of their carotid (and presumably coronary) arteries and there’s a pretty good chance that it may make them worse.

Vytorin is not the only example of a drug trial going wrong. I believe that Merck got into trouble with Vioxx because they made a judgement error in terminating the collection of data on drug adverse events before the termination of the collection period for data on good outcomes. They looked for good outcomes for a longer time than they looked for bad outcomes. The extra deaths that occurred in the termination intervals did not materially effect the estimates of risk for Vioxx, but it sure looked bad for the company - that they had deliberately made this decision once they had looked at the data, or at least had seen it coming. It turns out, really, that Vioxx is no worse than the other drugs in this class - they all carry a risk of cardiovascular disease (and their benefit is no greater than less toxic drugs).

Why not just come clean? Pharmaceutical companies know that not all the drugs they design will fly. Hanging on to losers and trying to bend clinical trial results to hoodwink the FDA and physicians is not good enough. They need to set their own internal standards higher than is required and much higher than they are now. This will undoubtedly increase the cost of bringing new compounds to market, but it will preserve their reputations and long run viability.

To come clean will mean that the research divisions must be separated from the finance and marketing divisions not by a Chinese wall, but by a cement one. For Merck and Schering-Plough to choose an employee as the Principal Investigator is a mistake of Titanic proportions. Not that Dr. Vetri is not capable: It is that he could never be seen as independent. But picking a principal investigator is almost beside the point. The entire research enterprise must be in an intellectual vault. Researchers should be rewarded with attractive positions, but those positions and salaries must receive no monetary incentives related to the launch of a successful product. And they must have complete independence in choosing designing the research protocol.

The Finance departments must re-evaluate upward their risk assessments for new compounds and new studies that are proposed. With a truly independent research division it is likely that more studies will fail to show enough efficacy and safety to permit a test flight of the product on the open market. This is a cost of doing business in this high-risk environment, but it is a risk that must be factored into company bottom lines. Such a policy ought to re-assure shareholders and decrease the downside risk of endless collective action lawsuits.

What’s a drug company gonna do?

I feel sorry for the big pharmaceutical companies. They are picked on by pretty much everyone that is not on their payroll, and even some that are. Take Pfizer and GSK and other drug companies for example, accused of deliberately hiding results from clinical trials that had results showing their antidepressants weren’t what they’re cracked up to be. Most of us, if we had to show the world the results of our personal clinical and social trials would do the same: Do do the same.

But, yes, they should be held to a higher standard than I am. In part because they claim a high ethical ground and spend millions advertising their integrity, and in part because they are developing products for use by people who may suffer or die as a result of using products that are not as advertised. And in part because they are abusing the trust of the patients who volunteer for the studies on the promise that the results obtained will be useful to science and to other patients. Pretty good arguments.

So what happens. Why are they hiding results?

The companies will claim that it is hard to publish studies with mixed results, where the results are unclear. They are right. Journals, particularly the higher profile journals, are less interested in publishing a study if it does not show a positive result. The reason is that most of these studies have relatively small sample sizes (as was the case in the recent publication - about 153 patients per trial). Failure to show a statistically significant benefit does not preclude the fact that one exists if a larger trial were done and certainly doesn’t indicate that the drug being tested has no effect. Thus the value of the information obtained from a particular trial of small sample size showing no statistical positive effect is minimal. (It is only later, when small studies accumulate, that pooling the results of these trials reveals the true efficacy or inefficacy of a drug.)

This, however, is a weak company argument. There is nothing that stops the company from publishing the results on its own web site. There are also many journals available that have the space to publish the results of smaller trials. Indeed, PLoSONE, is an excellent journal that aims to publish all research, no matter what the result or statistical signifigance.

It is also true that the FDA has copies of the complete results of these unpublished trials but is prohibited by covenants set by government from publishing trials submitted for a drug that subsequently was not approved: The information remains proprietary. The FDA ought to seek approval to make these trials publicly available on its web pages. And the pharmaceutical company lobby group - the Pharmaceutical Research and Manufacturers of America - ought to lobby government to make this regulatory change instead of lobbying to keep it in place.

But the company problems are deeper than non-publication of results that might hinder the marketing and sales of products. As this recent study showed, many companies alter the results when it comes to publication. They do this mainly by selecting study end-points that put the products in a more favourable light. In the 51 reported studies where the results were published, 11 of them (22%) had the results altered and the published results were different than the results show to the FDA. (The FDA had the drug as having no statistically significant effect, the published study showed a positive effect.)

How can this happen? Other than falsification of the data, the main reason is that all studies have multiple possible outcomes of interest. In studies of antidepressants, we may be interested in changes of mood of study patients, or suicide ideation, or attempts at suicide and so on. Yet each randomized clinical trial must declare, before the start of the study, at the time of construction of the protocol, a primary outcome of interest. The trial is designed around this primary outcome. Now in any trial there may be interest in other outcomes. For example, in a study is designed to look at suicide ideation as a primary outcome, the result may show no statistically significant difference. Yet, on another outcome, say a particular scale of patient mood, patients randomly assigned to the active compound, may have higher (better) scores and these may be statistically significant. Nonetheless, this is considered a post-hoc analysis and could easily have occurred by chance. By the rules of science it could have been a chance finding and the result can not be claimed as proof that the drug improves mood.

That major pharmaceutical companies are cheating. They are unethical. They are violating the promise made to study subjects that their participation in trials will benefit other patients and the enterprise of science. They need to be condemned, without reserve. Journals also, need to be chastised for lax editorial oversight in the publication of clinical trials that report non-primary outcomes as primary outcomes, thus misleading their readers and patients. Journals and reviewers have only to request study protocols to validate that the research submitted precisely matches the study hypotheses.

Will these companies ever get it right? Surely the will, I hope.

In a future blog I’ll get into what needs to happen within companies if they are too get it right.

Reference:
Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and influence on apparent efficacy. N Engl J Med 2008;358:252-60