Editors get a lot of manuscripts reporting relatively small clinical trials, in the range of a few hundred study subjects. Such studies can never evaluate clinically meaningful outcomes because like death or hospitalization because these events are rare and require large trials in order to collect enough events for statistical testing. The smallish trials usually measure indirect outcomes, such as serum cholesterol or a change on some indicator of well being such as mood or depression. While helpful, these kinds of measure are not particularly useful to patients and physicians have to take on faith or biochemistry that there is an underlying chain of causality that will in the end yield some tangible benefit that can be experienced by the patient.
Many of these small trials are never published. Individually they convey virtually no information that can be translated into patient care. At best, if enough of them accumulate they can be combined in meta-analyses and then perhaps some estimate of treatment efficacy can be determined. But this is a long term hope and the result always remains uncertain.
It is perhaps ironic that Orwell’s 1984 was the year that Yusuf and others published their cogent arguments for doing large clinical trials of common serious illnesses. They correctly argued that discovery of a small benefit or risk (say 20% improvement) would carry enormous advantage to a large number of patients and to society. Small trials could never detect this level of benefit. Further, they pointed out that doing small trials on common illnesses and conditions would be very unlikely to yield much information about clinically important events, unless the effect was very large and even then, if the treatment was good enough to produce a big benefit, the benefit would have been obvious to the average clinician without a clinical trial. So why do them?
Yet small trials continue to be published. Yesterday, while trawling the clinical trial registry www.clinicaltrials.gov I found 1,788 studies related to the keyword ‘depression’. Limiting this search to ongoing trials (those still recruiting study subjects), those that were investigator classified as phase III or phase lV studies (not the smaller very early Phase I and II assessments for safety and efficacy that are sometimes used to determine if a ‘larger’ trial should go ahead) and those that were registered after January 1, 2005, showed 331 trials.
There were only 10 trials that had more than 1,000 subjects (and many of these were only secondarily looking at my condition of interest - depression - being studies of patients with cancer on different types of chemotherapy, for example). Most (almost half) of the trials for depression had fewer than 100 subjects. Over 9,000 patients were involved in these very small trials, all of which are unlikely to provide any meaningful information and may in fact prove harmful with false leads.
Now readers may argue that depression and other psychiatric diagnoses are difficult to study and that would be true. But this ought to support the Yusuf position that the study of interventions for depression, a common chronic disease, ought to be carried on very large patient groups looking for interventions that have some modest effect, say a 20-25% reduction in suicide or suicide attempts. There is also growing concern that the active compounds being evaluated may not only not be helpful, but may be in some way a cause of suicides. At present there are 160 ongoing trials involving almost 9,000 patients, each with fewer than 100 study subjects. When one adds in to the equation study subject drop-out, non-compliance, observer and measurement error and so on there is virtually no probability that these studies will yield useful results.
I did not examine or try to judge the adequacy of trial designs, objectives and premises, although this might be interesting given that a large proportion (73%) of these designated Phase III and IV trials are sponsored by industry.
My list of trials is obviously a potpourri. Had I limited the search to Major Depression as a diagnostic category, I would have found fewer trials and a different pattern. Also, mine was a desktop exercise and would need to be more rigoursly double checked for errors and validated. Nonetheless, the graph is a reasonable summary of ongoing clinical trials for an important clinical condition and global public health problem.
I suspect that choosing any other condition in the trial registry would yield similar distributions of study size. It would be interesting to do some of this work more formally. I wonder also, if the registry could incorporate some measure of robustness of the trials that are registered. For example, including the elements of trial design recommended in the Consort Statement www.consort-statement.org would be relatively easy for registrants to complete, (they ought to have it completed before taking their studies for ethical and institutional review in any case) and would allow users of the registry (patients, physicians, researches) to judge at a glance the quality of the trial.
Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Statistics in Medicine 1984;3:409-20
No comments:
Post a Comment