Why randomized controlled trials fail but needn't: a new series is launched =========================================================================== * David L. Sackett * John Hoey Although introduced decades earlier, randomized controlled trials (RCTs) came to medical notice only 50 years ago. The rapid recognition of their usefulness in promoting health and limiting harm has led to their adoption by every health discipline (some more enthusiastically than others), and they now play a dominant role in medical discourse, both in the halls of science and at the bedside. By 1988 more than 5000 reports of clinical trials were being published each year, and that figure rose to more than 12 000 a decade later. Currently, more than 200 clinical trials are being published every week. This growth has major implications for patients, their clinicians and the agencies responsible for funding RCTs. Ever larger numbers of clinicians are being asked to invite ever larger numbers of patients to enter RCTs, and both groups want to be certain that patient care will not be compromised and that participation will lead to valid, credible, important advances in the treatment or prevention of disease. Moreover, the time lag between the publication of individual trials and their systematic review by groups such as the Cochrane Collaboration means that patients and clinicians will often have to make sense out of growing numbers of individual reports. Finally, as government after government fails to provide their nonprofit independent granting agencies with sufficient resources to meet society's growing demand for high-quality evidence,1 the drug industry, with its inescapable competition between health and profit, pays and calls the tune for more and more RCTs.2 Some RCTs are flawed. Often these flaws are detectable only after the trial is complete, but sometimes they are apparent at the outset. How can the alert clinician and patient avoid being misled by faulty RCT results and, more important, avoid participating in faulty RCTs at their inception? Beginning in this issue (page 1311) *CMAJ* will publish an occasional series of essays on why RCTs fail but needn't. Their intentions are 3. First, they intend to help clinicians advise the patient who is eligible for, or has been invited to join, an RCT by showing both of them how to determine whether the proposed trial is likely to generate results that are valid, clinically useful and of sufficient value to the patient that they ought to consider joining it. Second, these essays intend to help clinicians and patients decide whether the report of an RCT is true enough and clinically sensible enough for its results to be offered by clinicians and requested by patients. The series' third objective, narrower but deeper, is to help clinicians at any stage of their training and experience who are contemplating becoming co- or principal investigators in an RCT and want to evade the avoidable errors. There are plenty of excellent textbooks on how to conduct RCTs the right way.[3–7] The *CMAJ* series is based on the premise that it is at least as instructive, and certainly more stimulating and fun, to use RCTs that were done the wrong way as the starting point for learning. The series' author (D.L.S.), involved to varying degrees in over 200 RCTs over the last 40 years, claims pre-eminence in knowing ways to fail in the design, conduct, analysis and interpretation of RCTs. Each essay begins with a vignette, taken from the author's career at the interface between clinical medicine and clinical research methods, and shows how an RCT (almost) failed. After reporting the effect of this failure on the RCT's conclusions, the essay tracks the failure back to its root cause. Next, some specific principles are described that, had they been acted upon, would have prevented the failure. Wherever possible these principles will be based on empirical methodological research, such as that being gathered and systematically reviewed by the Cochrane Empirical Methodological Studies Working Group. Finally, these principles are converted into precise preventive strategies to avoid similar failures in future RCTs. Throughout, the emphasis will be on clarity for front-line clinicians; arithmetic will be minimized and statistical formulas avoided like the plague. The preventive strategies that appear in this series make no claim of exclusivity; sometimes there will be other ways to rescue (at least in theory) the RCTs in question. The strategies presented, however, have actually been applied by the author and thus have worked at least once. Occasionally these proposals will disagree with established views on how RCTs should be designed and executed, and often they will challenge both the assumptions and the behaviours of contemporary trialists, ethics committees and trial monitors. Readers who disagree, have better ideas or simply feel peevish are encouraged to take it out on the author in the Letters column. Competing interests: Dr. Sackett has been wined, dined, supported, transported, and paid to speak by countless pharmaceutical firms for over 40 years, beginning with 2 research fellowships and interest-free loans that allowed him to stay to finish medical school. Dozens of his randomized trials have been supported in part, but never in whole, by pharmaceutical firms, who never received or analysed primary data and never had veto power over any reports, presentations or publications of the results. He has twice worked as a paid consultant to advise pharmaceutical firms whether their products caused lethal side-effects (on both occasions he told them Yes). He has testified as an unpaid expert witness for a stroke victim who successfully sued a manufacturer of oral contraceptives, and he has been paid by a second to develop "levels of evidence" for determining the causation of adverse drug reactions. His wife inherited and sold stock in a pharmaceutical company. While head of a division of medicine he enforced the banning of drug-detail personnel from clinical teaching units (despite the threat of withdrawal of drug-industry funding for resident research projects). He received the Pharmaceutical Manufacturers' Association of Canada Medal of Honour (and cash) for "contributions to medical science in Canada" for the decade 1984-1994. No competing interests declared for Dr. Hoey. ## Acknowledgments Holiday reviews have become an annual tradition at CMAJ. Underneath their heavy parkas or layers of mosquito repellent, Canadian physicians have shown that they have a thoughtful soul and a quirky sense of fun. MEDICINE IN CANADA. We encourage you to submit reflective essays on difficult decisions you've had to make as a physician, adventures (big or small) in your professional life and defining moments in your career. We're also looking for descriptions of medical events that could only happen in Canada. CANADA IN MEDICINE. This year we hope to hear from Canadians who have practised medicine abroad. Tell us about the unforeseen challenges and the unexpected joys of working far from home. How did your experiences change your perception of what it means to be a Canadian physician? YOU'VE GOT TO BE KIDDING. As in the past 2 years (click on Back Issues at [www.cma.ca/cmaj](http://www.cma.ca/cmaj) and go to the December issues for 1998 and 1999) we plan to leaven the mix with some humour. Show us your inner child - or better yet, the collective inner child of your clinic or hospital department. We're looking for papers that take a playful poke at sober scientific thought by postulating the preposterous and proving the unprovable. To discuss an idea for the Holiday Review issue, contact Editor-in-Chief Dr. John Hoey, tel 800 663-7336 x2118; hoeyj{at}cma.ca. Articles should be no more than 1200 words, preferably accompanied by illustrations. Submissions received by Oct. 1, 2000, are more likely to be published. Send submissions to: Dr. John Hoey; CMAJ; 1867 Alta Vista Dr.; Ottawa ON K1G 3Y6 ## Footnotes * See related article page 1311 Correspondence to: Dr. David L. Sackett, Trout Research & Education Centre at Irish Lake, RR 1, Markdale ON N0C 1H0; [sackett@bmts.com](http://sackett@bmts.com) This commentary was posted early on *eCMAJ* (Apr. 13, 2000). ## References 1. 1. Sackett DL. Time to put the Canadian Institute for Health Research on trial [editorial]. CMAJ 1999;161(11):1414-5. Available: [www.cma.ca/cmaj/vol-161/issue-11/1414.htm](http://www.cma.ca/cmaj/vol-161/issue-11/1414.htm) [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTYxLzExLzE0MTQiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTYyLzkvMTMwMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 2. 2. Hailey D: Scientific harassment by pharmaceutical companies: time to stop. CMAJ 2000;162(2):212-3. Available: [www.cma.ca/cmaj/vol-162/issue-2/0212.htm](http://www.cma.ca/cmaj/vol-162/issue-2/0212.htm) [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNjIvMi8yMTIiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTYyLzkvMTMwMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 3. 3. Schwartz D, Flamant R, Lellouch J. *Clinical trials* [translated by Healy MJR]. Toronto: Academic Press; 1980. 4. 4. Friedman LM, Furberg CD, DeMets DL. *Fundamentals of clinical trials.* London: John Wright; 1981. 5. 5. Shapiro SH, Louis TA, editors. *Clinical trials: issues and answers.* New York: Marcel Dekker; 1983. 6. 6. Pocock SJ. *Clinical trials: a practical approach.* Toronto: John Wiley & Sons; 1983. 7. 7. Jadad A. *Randomised controlled trials.* London: BMJ Books; 1998.