Observational studies form an important part of the medical evidence base, particularly for assessing rare adverse events and long-term effectiveness of medications and devices. 1 However, observational studies, like interventional studies (clinical trials), are subject to publication bias and reporting bias. 2–4 Registration of clinical trials is a widely recognized tool for facilitating complete public reporting. 5 Registration of observational studies has received less attention, although interest is growing. 6–8 Because existing registries (e.g., ClinicalTrials.gov) accommodate observational studies, and the rationale and benefits of registration are similar, we ask the scientific community and other stakeholders to consider the systematic, prospective registration of observational studies.
Why register observational studies?
Much of the rationale for the prospective registration of clinical trials 9 applies to the registration of observational studies (Table 1). 7 For example, observational studies in which researchers acquire data directly from human participants entail ethical obligations to participants, even though such research generally involves less risk than interventional studies. 10 These obligations include oversight by ethical review boards, informed consent, and public release of the study findings to advance biomedical knowledge. As with clinical trials, incomplete reporting of observational studies has been documented. 3 Some researchers suggest that observational studies are also at increased risk for publication bias or other types of bias, including misrepresentation of prespecified analyses or phenotype definitions. 2,4 Such biases are a concern because they undermine the validity of observational studies, which are an important component of the medical evidence base in areas of public health, such as detection of rare adverse events. 1,16
Observational studies of medications and devices are playing a more visible role at the United States Food and Drug Administration (FDA) (Table 2). For instance, the FDA posted an “Early communication about an ongoing safety review” 18 in response to a published observational study associating abacavir and didanosine with an increased risk of cardiovascular and cerebrovascular events. 19 The authors of the study noted that, although a randomized controlled trial is necessary to show a causal association, such a trial design is “unlikely to be feasible,” given that it would require more than 10 000 participants to be followed for at least two years. In addition, there may be ethical concerns in conducting a randomized controlled trial if harms are expected. Given that this and other associations between marketed products and possible harms are likely to be investigated further using observational studies, a registry containing summary protocol information would allow researchers to track such studies from initiation to completion. Such a tool could be useful to researchers who are evaluating the current evidence, considering initiating similar studies, identifying gaps in research or seeking collaborators. 15 Similarly, a database of summary results could improve access to information about published and unpublished observational study analyses (whether prespecified or post hoc), thereby mitigating publication bias and incomplete reporting of results.
Given these potential benefits, observational studies are already being registered for various reasons, and there is increasing attention given to this practice. For example, the state of Maine requires registration and reporting of results of postmarketing observational studies of medications and biological products marketed in Maine 20 and the corporate policies of some drug manufacturers address disclosure of observational studies. 21 Recently, an international workshop was held on the topic, 6 and several medical journals published editorials on the registration of observational studies. 7,8,22 Some have suggested that ethics review boards should require prospective registration of any study involving human participants, 12 whereas others have proposed that observational studies under the mandate of the FDA should be subject to the same requirements for registration and reporting of results as those for clinical trials. 17,23 The European Medicines Agency recently issued a work plan to create a registry of post-authorization safety studies that would include observational studies, 24 and a draft report by the Agency for Healthcare Research and Quality considered the utility of creating a “registry of patient registries.” 25 With growing interest in this topic, an understanding of current practices in registration is useful.
Where can observational studies be registered?
An increasing number of clinical trial registries exist and, despite their names, some include observational studies. For example, ClinicalTrials.gov (http://clinicaltrials.gov), established by the US National Library of Medicine on behalf of the US National Institutes of Health, is the largest publicly accessible registry, with over 87 000 ongoing and completed interventional and observational studies as of March 2010. There are 10 registries in the World Health Organization’s Registry Network, which range in size from 25 to more than 8400 registered studies as of March 2010. Of these registries, five explicitly state that observational studies can be registered (Appendix 1, available at www.cmaj.ca/cgi/content/full/cmaj.092252/DC1). A structured search tool for identifying observational studies is provided at ClinicalTrials.gov, where such studies represent 17% (n = 14 595) of all registrations, and at the Australian New Zealand Clinical Trials Registry, where 5% (n = 214) of registered studies are observational.
This cursory review of international registries suggests that the basic technical infrastructure for the registration of observational studies already exists. Closer examination is required to determine whether such registries are meeting the needs of users — from the perspective of both sponsors entering data and researchers (among others) using the display and retrieving studies. The remainder of this commentary will use the experience of ClinicalTrials.gov to characterize some current practices in the registration of observational studies and identify potential issues.
Registration of observational studies at ClinicalTrials.gov
ClinicalTrials.gov has accommodated the registration of observational studies since its launch in February 2000. It standardizes key attributes of study protocols using an established set of mandatory and voluntary data elements (http://prsinfo.clinicaltrials.gov/definitions.html). Sponsors must designate a study as observational or interventional using a mandatory data element that indicates type of study before they complete other design-specific data elements. Observational studies are found on the public site by selecting “Observational Studies” from the “Study Type” drop-down menu on the advanced search form at ClinicalTrials.gov (http://clinicaltrials.gov/ct2/search/advanced).
In October 2007, modifications were made to the design-specific data elements used for registering observational studies. These changes were strongly influenced by protocol-related items in the statement Strengthening the Reporting of Observational Studies in Epidemiology (STROBE). 14 The STROBE initiative aims to improve the reporting of observational studies by specifying a core set of items to include in a publication that would allow a reader to follow “what was planned, done, and found.” 14 Consistent with the aims of the STROBE initiative, registration facilitates prospective documentation of key prespecified study attributes. In September 2008, Clinical-Trials.gov added a database of summary results that allows for reporting of many of the results-related items identified in the STROBE initiative, such as information on study participants (e.g., numbers of participants starting and completing the study, baseline demographic characteristics), outcome measures (including appropriate statistical analyses) and adverse events, if relevant.
After the design-specific data elements for observational studies were updated, prospective cohort studies (n = 2617) comprised the largest segment of observational studies registered in ClinicalTrials.gov between November 2007 and March 2010 (n = 7735) (Appendix 2, available at www.cmaj.ca/cgi/content/full/cmaj.092252/DC1). Trends in observational studies registered in ClinicalTrials.gov from February 2000 to March 2010 include the following:
Registrations and results
The average number of observational studies registered per year has increased consistently with overall increases in registered studies. Historically, observational studies have represented about 15% of all studies registered in any given year. At the time of writing, they represented 17% (n = 14 595) of all registrations of studies (n = 87 530) and 5% (n = 85) of all entries of results (n = 1578) in ClinicalTrials.gov.
Geographic distribution
Study locations are distributed worldwide, with over 50% in North America, 20% in Europe and 13% in Asia (Appendix 3, available at www.cmaj.ca/cgi/content/full/cmaj.092252/DC1).
Type of sponsorship
Most registered observational studies (85%) are funded by non-industry sources (24% by the National Institutes of Health and other US government agencies and 61% by academic and nonprofit organizations); the remaining 15% are sponsored by industry. In comparison, 64% of registered interventional studies are funded by non-industry sources (10% by the US government and 54% by academic and nonprofit organizations), and 36% are sponsored by industry (see Appendix 4, available at www.cmaj.ca/cgi/content/full/cmaj.092252/DC1).
Diseases and conditions studied
The distributions of diseases and conditions evaluated among registered observational and interventional studies were generally similar across categories of conditions as derived from the National Library of Medicine Medical Subject Headings (MeSH) controlled vocabulary (Appendix 5, available at www.cmaj.ca/cgi/content/full/cmaj.092252/DC1).
These data provide a snapshot of observational studies carried out around the world during a 10-year period. However, our findings are not representative of all observational studies conducted during that period. In addition, the accuracy of our analysis is limited by the quality and completeness of information provided by registrants.
Gaps in knowledge
Based on our experience, we believe that the existing model for and system of registration of clinical trials can be applied to observational studies. Although ClinicalTrials.gov is able to accommodate a range of observational study designs, those that involve collection of primary, prospective data may fit this model more easily because such studies are most similar to interventional studies. In each type of study, data are generally collected prospectively from well-defined groups of individuals (e.g., exposed and nonexposed) according to a protocol. As previously noted, prospective cohort studies were the most frequently registered in ClinicalTrials.gov (Appendix 2). Whether and how other types of observational studies (e.g., retrospective data collections, secondary data analyses and evaluations of data from patient registries) should be represented in a study registry requires further consideration. For example, patient registries are established to collect a defined set of standardized variables about specific patient populations, but they may or may not have specific protocols with prespecified plans for analysis. Although hundreds of patient registries have been successfully registered (e.g., http://clinicaltrials.gov/search/term=registry), when they should be registered and which attributes should be represented could be further elucidated. The Agency for Healthcare Research and Quality is currently considering these issues. 25
Other challenges related to the registration of observational studies include difficulties in delineating individual studies from a broad plan for data analysis, and whether and how sub-studies and secondary studies or analyses using the same prospective data set should be registered. In addition, data elements and definitions related to data collection for study variables, 14 as well as the specific structure for reporting results of observational studies, may require further consideration. For example, although the ClinicalTrials.gov registry currently collects information on outcome measures, it may also be useful to prespecify other variables (e.g., baseline characteristics) that will be used to characterize a cohort or that will be included in an analysis. 26
Some have questioned the practicality of registering observational studies because of the difficulty of documenting all hypotheses and have expressed concern that prespecification of all hypotheses will inhibit exploratory research. 13,26 Other approaches to publicly documenting original study design and variables already exist, such as the publication of cohort profiles in the International Journal of Epidemiology. 27 Registration of observational studies may be complementary to such publications, given that the registry record is a standard format, is broadly accessible and can be updated over time and given that links or cross-references between the registry and publication could be established.
Next steps: Is it time to register observational studies?
There is interest in the prospect of registering observational studies, as reviewed earlier and as shown by the number of such studies registered at ClinicalTrials.gov. We believe that making both observational and interventional studies available in a single registry will provide researchers and others with a more comprehensive view of the growing evidence base. Based on the clinical trial experience, there is a pre-existing (but evolving) model and infrastructure for registration, and study sponsors are already registering observational studies on a voluntary basis. Given that much of the ethical and scientific rationale for registration of interventional studies appears to apply to prospective observational studies, we believe that further discussion among stakeholders (e.g., researchers, regulatory authorities and the public) is warranted. Such discussion is necessary to address specific issues regarding implementation, including assessing and defining common data elements that would meet the needs of the stakeholders. In the future, the data elements for registration of observational studies in ClinicalTrials.gov could be modified to better meet these needs.
-
Clinical trial registries are established tools for improving access to information on trials and for addressing publication bias and reporting bias.
-
Much of the ethical and scientific rationale for registering clinical trials also applies to observational studies.
-
The existing infrastructure for trial registration is being used for observational studies, which make up 17% of the studies registered in ClinicalTrials.gov.
-
Further discussion is necessary to assess the scope and specific implementation-related issues of systematic registration of observational studies.
Key points
Footnotes
-
Previously published at www.cmaj.ca
This article has been peer reviewed.
Competing interests: Rebecca Williams, Tony Tse and Deborah Zarin are employed by ClinicalTrials.gov, the registry and results database of the National Library of Medicine, National Institutes of Health, USA, where Rebecca Williams is Assistant Director, Tony Tse is Program Analyst and Deborah Zarin is Director. No competing interests declared for William Harlan.
Contributors: All of the authors were involved in the conception of the article. Rebecca Williams was involved in the acquisition of the data, and all of the authors were involved in the analysis of the data. William Harlan drafted the article. All of the authors critically revised the manuscript for important intellectual content and approved the final version submitted for publication.
Funding: Financial support for this article was received from the Intramural Research Program of the National Institutes of Health, National Library of Medicine.
Acknowledgements: The authors thank Annice Bergeris, Sarah Kornmeier and Rachel Machta for their assistance in data collection and summarization.