What kind of randomized trials do we need?
==========================================

* Merrick Zwarenstein
* Shaun Treweek

In 1967, Daniel Schwartz and Joseph Lellouch, 2 French statisticians, and their British colleague and translator Michael Healy wrote “[M]ost therapeutic trials are inadequately formulated, and this from the earliest stages of their conception.”

The seminal paper 1 from which this dramatic assertion is drawn is reprinted in the May 2009 issue of the *Journal of Clinical Epidemiology* as part of a joint focus with *CMAJ* on making randomized controlled trials (RCTs) more useful.

Schwartz and Lellouch argued that there are 2 kinds of randomized trials embodying radically different attitudes to evaluation of treatment, which they named “pragmatic” and “explanatory.” They go on to say that these 2 attitudes require different approaches to the design of a randomized trial. The pragmatic attitude seeks to directly inform real-world decisions among alternative treatments. Schwartz and Lellouch show that this purpose is satisfied in trials that select typical participants, settings and comparator care to widen real-world applicability. In contrast, the explanatory attitude seeks to understand a biological process by testing the hypothesis that the specified biological response is explained by exposure to a particular treatment. Tight restrictions on participants, treatment, control and setting maximize the contrast with the control group and increase the ability to test this kind of hypothesis.

Their assertion of inadequate formulation relates to the mismatch between the use we make of most trials (which is to inform decisions on therapy) and the design of these trials (which generally takes the opposite form, best suited to testing causal hypotheses). This mismatch between the clinical context in which clinicians must make decisions and the clinical context of the randomized trials that they must use for evidence means that health professionals (and, writ large, health care funders) are left without direct evidence upon which to base most of the patient care decisions (and funding decisions) that each must make. Since information from an explanatory trial is unlikely to inform a pragmatic question, nor vice versa, Schwartz and Lellouch proposed that investigators should explicitly specify the purpose of their trial and design it to match that purpose.

There are few trials whose purpose and design choices match. The only review of this subject 2 identified fewer than 100 pragmatic designed randomized trials, of the quarter million or so RCTs listed by the US National Library of Medicine, which suggests that existing RCTs are mostly explanatory in design and thus not directly applicable to choosing between treatment options. This is ironic since the very first published randomized trial was pragmatic in purpose and in many of its design choices. It showed clear benefits for patients receiving streptomycin and usual care (bed rest) over the control group receiving only usual care. 3 The decline of tuberculosis in high-income countries is thus due in part to the pragmatic trial.

Why so few pragmatic trials? Because of the size of the market, US pharmaceutical licensing regulations are the principal stimulus for the conduct of RCTs of treatments and the main influence on their design. The requirement that pharmaceutical manufacturers demonstrate efficacy of their products was first legislated in the 1962 Kefauver–Harris amendments to the US Federal Food, Drug, and Cosmetic Act, passed in the wake of the thalidomide tragedy, from which the United States had been largely spared by caution on the part of the US Food and Drug Administration (FDA). 4 Perhaps because of their focus on safety, these licensing regulations 5 devote most of their attention to preparatory studies in animals, safety issues in humans and documentation. The little guidance there is argues against trials with a pragmatic attitude: “One problem [with active-control trials] is that there are numerous ways of conducting a study that can obscure differences between treatments, such as poor diagnostic criteria, poor methods of measurement, poor compliance, medication errors, or poor training of observers. As a general statement, carelessness of all kinds will tend to obscure differences between treatments. Where the objective of a study is to show a difference, investigators have powerful stimuli toward assuring study excellence.” 6 Much of what the FDA labels “careless” or “poor” is typical in usual care. The FDA thus equates pragmatic design choices aimed at increasing applicability with carelessness and poor study design, which results in trials that lack the attributes needed to directly support decisions about the real-world usefulness of a treatment.

FDA regulations and guidance are influential, but 2 other factors cannot be ignored: after spending many millions of dollars on development of a therapeutic drug or device, no corporation wants to give such an investment less than an ideal setting for displaying its benefits, hence the emphasis on starkly contrasting placebo control groups — often placebo controlled, enhanced adherence and highly selected patient centres and clinicians. The shared desire of the FDA and industry for conducting trials under ideal conditions with strong contrasts is reinforced by the strong preference of the US National Institutes of Health for trials that elucidate clear physiologic hypotheses. This triangle of actors contributes to the flood of stringently conducted and internally valid randomized trials, of doubtful applicability to most patients, most settings and most clinicians.

There is disquiet about the remoteness from real-world decision-making of regulatory randomized trials 7 among third-party funders such as Medicare in the United States, who are concerned that the potentially lower real-world benefits of a treatment might be outweighed by potentially higher risks, which would leave decisions on use and funding unclear. As a consequence, there is a rising interest in the design of trials that would avoid these misleading design attributes and provide, as Schwartz and Lellouch long ago suggested, direct decision-making information to those who must choose whether or not to prescribe, use, pay for or promote particular treatments.

Several papers take up this critically important theme. All appear in the May 2009 issue of the *Journal of Clinical Epidemiology*. The article by Thorpe and colleagues, 8 which also appears in this issue of *CMAJ* (page [E47](http://www.cmaj.ca/lookup/volpage/180/E47?iss=10)), offers us a first draft of a means for classifying design choices as to their degree of pragmatism, important because this is not an all-or-none phenomenon. Karanicolas and colleagues 9,10 argue that a trial has a fundamental “point of view” that changes the relationship between the decision-making purpose of a trial and its design. Oxman and colleagues 11,12 argue the contrary, namely that patients and clinicians benefit most from having more pragmatic evidence on which to base their decisions. Finally, Maclure, 13 whose article also appears in this issue of *CMAJ* (page 1001), points to the close match between decision-makers’ preferences for directly applicable evidence and the pragmatic attitude favoured by Schwartz and Lellouch. Some of these authors have also recently published an extension to the CONSORT (Consolidated Standards of Reporting Trials) statement for pragmatic trials (Table 1), 14 with recommendations on reporting of trials whose aim is to inform decisions. This CONSORT extension encourages authors to include in their trial publications information to help readers judge the applicability of the results to their own settings, the better to decide whether or not to implement the tested intervention (Table 1). 14 

View this table:
[Table1](http://www.cmaj.ca/content/180/10/998/T1)

**Table 1:** Key differences between trials with explanatory and pragmatic attitudes*

The expansion of interest in the design of pragmatic randomized trials to support decision-making is surely now much needed. In Canada, for example, the regulatory authority (Health Canada) is currently considering a progressive licensing model for therapeutic agents. 15 Foremost in its consideration is speeding the process of licensing while maintaining high standards for the evaluation of safety and efficacy. Health Canada might consider revisiting its regulations and encouraging the use of pragmatic designs to support its decision-making. It might be argued that this would lengthen an already prolonged licensing process, reducing the ability of inventors to profit from their discoveries. There is a counter argument: if the initial randomized trial of a treatment were pragmatic in purpose and design, funders of the treatment could immediately use that information to make decisions on usefulness in their setting and patient group, eliminating what are today entirely separate and sequential processes for regulatory approval and formulary inclusion. Combining the trials that collect information for regulatory and for formulary inclusion processes could simplify both and make more transparent the reasons for decisions on reimbursement. And since large public funders demanding information for decision-making are the world’s most important markets for new pharmaceuticals, this could lengthen the period of profitable patent protection, rather than shorten it.

Schwartz and Lellouch ended their paper with a devastating conclusion: “Most trials done hitherto have adopted the explanatory approach without question; the pragmatic approach would often have been more justifiable.” Forty years and hundreds of thousands of randomized trials later, this remains true. It is time to shift our design choices so that they match our usual purpose in conducting a trial, most often to directly inform the decisions of real-world patients, clinicians and third-party funders.

See related article by Thorpe and colleagues, page [E47](http://www.cmaj.ca/lookup/volpage/180/E47?iss=10), and commentary by Maclure and colleagues, page [1001](http://www.cmaj.ca/lookup/volpage/180/1001?iss=10)

## Footnotes

*   Published at [www.cmaj.ca](http://www.cmaj.ca) on Apr. 16, 2009. This article was published simultaneously in the May 2009 issue of the *Journal of Clinical Epidemiology* ([www.jclinepi.com](http://www.jclinepi.com)).
    
    **Competing interests:** None declared.
    
    **Contributors:** Both of the authors were involved in the preparation of the manuscript and approved the final version submitted for publication.

## REFERENCES

1.  1. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. *J Chronic Dis* 1967;20:637–48. [Reprinted in J Clin Epidemiol 2009;62:499–505.]
    
    

2.  2. Vallvé C. A critical review of the pragmatic clinical trial [Spanish]. *Med Clin (Barc)* 2003;27:384–8.
    
    

3.  3. Fox W, Hill A. Streptomycin treatment of pulmonary tuberculosis. BMJ 1948;2:769–82.
    
    [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czozOiJibWoiO3M6NToicmVzaWQiO3M6MTA6IjIvNDU4Mi83NjkiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTgwLzEwLzk5OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

4.  4. *A brief history of the Center for Drug Evaluation and Research*. Rockville (MD): Center for Drug Evaluation and Research, US Food and Drug Administration; 1997. p. 32. Available: [www.fda.gov/cder/about/history/Page32.htm](http://www.fda.gov/cder/about/history/Page32.htm) (accessed 2009 Feb. 12).
    
    

5.  5. Investigational new drug application. *Code of federal regulations*. Title 21, Pt 312 (revised 2008 Apr. 1). Available: [www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRsearch.cfm?CFRPart=312](http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRsearch.cfm?CFRPart=312) (accessed 2009 Feb. 12).
    
    

6.  6. Drug study designs. In:*Drugs and biologics. Guidance for institutional review boards and clinical investigators: 1998 update* [information sheet]. Rockville (MD): US Food and Drug Administration; 1998. Available: [www.fda.gov/oc/ohrt/irbs/drugsbiologics.html#study](http://www.fda.gov/oc/ohrt/irbs/drugsbiologics.html#study) (accessed 2009 Feb. 12).
    
    

7.  7. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003;290:1624–32.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.290.12.1624&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=14506122&link_type=MED&atom=%2Fcmaj%2F180%2F10%2F998.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000185461300028&link_type=ISI) 

8.  8. Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers. *J Clin Epidemiol* 2009;62:464–75. [Also in CMAJ 2009;180:E47–E57.]
    
    [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTgwLzEwL0U0NyI7czo0OiJhdG9tIjtzOjIxOiIvY21hai8xODAvMTAvOTk4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

9.  9. Karanicolas PJ, Montori VM, Deveraux PJ, et al. A new “Mechanistic-Practical” framework for designing and interpreting randomized trials. J Clin Epidemiol 2009;62:479–84.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2008.02.009&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=18468856&link_type=MED&atom=%2Fcmaj%2F180%2F10%2F998.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000265424400004&link_type=ISI) 

10. 10. Karanicolas PJ, Montori VM, Devereaux PJ, et al. The practicalists’ response. J Clin Epidemiol 2009;62:489–94.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2008.08.013&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19348974&link_type=MED&atom=%2Fcmaj%2F180%2F10%2F998.atom) 

11. 11. Oxman AD, Lombard C, Treweek S, et al. Why we will remain pragmatists: four problems with the impractical mechanistic framework and a better solution. J Clin Epidemiol 2009;62:485–8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2008.08.015&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19348973&link_type=MED&atom=%2Fcmaj%2F180%2F10%2F998.atom) 

12. 12. Oxman AD, Lombard C, Treweek S, et al. A pragmatic resolution. J Clin Epidemiol 2009;62:495–8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2008.08.014&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19348975&link_type=MED&atom=%2Fcmaj%2F180%2F10%2F998.atom) 

13. 13. Maclure M. Explaining pragmatic trials to pragmatic policymakers. *J Clin Epidemiol* 2009;62:476–8. [Also in CMAJ 2009;180:1001–3.]
    
    [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTgwLzEwLzEwMDEiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTgwLzEwLzk5OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

14. 14. Zwarenstein M, Treweek S, Gagnier J, et al.; CONSORT and Pragmatic Trials in Healthcare (Practihc) groups. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ 2008;337:a2390.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzMzcvbm92MTFfMi9hMjM5MCI7czo0OiJhdG9tIjtzOjIxOiIvY21hai8xODAvMTAvOTk4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

15. 15. *Progressive licensing model*. Ottawa (ON): Health Canada; 2007. Available: [www.hc-sc.gc.ca/dhp-mps/homologation-licensing/model/index-eng.php](http://www.hc-sc.gc.ca/dhp-mps/homologation-licensing/model/index-eng.php) (accessed 2009 Feb. 12).