Accelerated publication versus usual publication in 2 leading medical journals
==============================================================================

* William A. Ghali
* Jacques Cornuz
* Finlay A. McAlister
* Jean-Blaise Wasserfallen
* P.J. Devereaux
* C. David Naylor

## Abstract

**Background:** A number of medical journals have developed policies for accelerated publication of articles judged by the authors, the editors or the peer reviewers to be of special importance. However, the validity of these judgements is unknown. We therefore compared the importance of articles published on a “fast track” with those published in the usual way.

**Methods:** We identified 12 “case” articles — 6 articles from the *New England Journal of Medicine* that were prereleased on the journal's Web site before publication in print and 6 “fast-tracked” articles from *The Lancet*. We then identified 12 “control” articles matched to the case articles according to journal, disease or procedure of focus, theme area and year of publication. Forty-two general internists rated the articles, using 10-point scales, on dimensions addressing the articles' importance, ease of applicability and impact on health outcomes.

**Results:** For each dimension, the mean score for the case articles was significantly higher than the mean score for the control articles: importance to clinical practice 7.6 v. 7.1 respectively (*p* = 0.001), importance from a public health perspective 6.5 v. 6.0 (*p* < 0.001), contribution to advancement of medical knowledge 6.2 v. 5.8 (*p* < 0.001), ease of applicability in practice 7.0 v. 6.5 (*p* < 0.001), potential impact on health outcomes 6.5 v. 5.9 (*p* < 0.001). Despite these general findings, in 5 of the 12 matched pairs of articles the control article had a higher mean score than the case article across all the dimensions.

**Interpretation:** The accelerated publication practices of 2 leading medical journals targeted articles that, on average, had slightly higher importance scores than similar articles published in the usual way. However, our finding of higher importance scores for control articles in 5 of the 12 matched pairs shows that current journal practices for selecting articles for expedited publication are inconsistent.

A number of medical journals have developed policies for accelerated publication of articles describing findings that are judged by the authors, the editors or the peer reviewers to be particularly important and deserving of rapid dissemination. In the case of the *New England Journal of Medicine*, accepted articles that have “immediate clinical implications”1,2 are occasionally prereleased on the journal's Web site ([www.nejm.org](http://www.nejm.org)) before their official publication date, and an early press release is issued to the media. A recent high-profile example of a prereleased article is that of the RALES study that assessed the efficacy of spironolactone for congestive heart failure.2,3

*The Lancet,*4*,*5*,*6 the *British Medical Journal* 7 and *CMAJ* 8 are other medical journals that have adopted mechanisms for occasionally accelerating the peer review and printing process for articles judged to present especially important research findings needing urgent dissemination. In these journals the expedited process is referred to as “fast-track” publication. A number of other high-profile journals, including the *Journal of the American Medical Association*,9 *Science*10* and* Nature,11 have also adopted mechanisms for expedited publication, with *Nature* recently prereleasing on its Web site 2 articles on the molecular biology of anthrax infections.12,13

Despite the existence, and increasing profile, of these journal publication policies, no study has formally assessed the importance, methodological quality and general visibility of articles published in an accelerated manner relative to articles published in the usual manner. In this article we address these questions by asking a group of physicians to rate the importance, quality and visibility of a selection of prereleased articles from the *New England Journal of Medicine* and fast-tracked articles from *The Lancet.*

## Methods

We used the general framework of a case–control study to address these research questions. We used “case” to refer to articles that were either prereleased on the journal's Web site or fast-tracked and “control” to refer to articles published in the usual manner.

### Identification of articles

For case articles from the *New England Journal of Medicine*, we wrote to the journal's editorial office in late September 1999 to obtain a full list of articles that had been prereleased online since the journal established its Web site. Among these 11 articles were 2 editorials,14,15 which were excluded because we sought to study only original research articles. One research article16 was excluded because it was prereleased online only after a news agency broke the journal's news embargo. Of the remaining 8 articles 3, on new treatments for cervical cancer,17,18,19 were prereleased together. To avoid burdening study participants with 3 case articles on the same theme, we randomly selected only 1 of these17 for inclusion in our study. We were thus left with 6 case articles from the *New England Journal of Medicine.*

For case studies from *The Lancet*, we selected 6 fast-tracked articles published between 1997 and 1999 in order to have the same number of case articles from both journals. We randomly selected individual issues from a listing of weekly issues of *The Lancet* published since 1996 and then scanned each issue for fast-tracked articles. If the issue contained no fast-tracked articles another issue was selected. If 1 fast-tracked article was published in the selected issue, the article was included in our study. When more than 1 fast-tracked article was published in an issue, we wrote the citations of each article on folded pieces of paper and randomly selected only 1 article per issue.

For control articles, one of us (W.A.G.) searched MEDLINE to identify 12 articles matched to the 12 case articles by journal of publication, condition or treatment of focus, main theme of the article (e.g., therapy, prevention, adverse effects) and, if possible, calendar year of publication. In most instances, only 1 candidate article would meet all of these criteria. When 2 or more potential control articles were identified, we randomly selected 1 for study. If no matching articles were found in the same year of publication, we broadened the fourth matching criterion to allow articles published within ±1 calendar year. Although there are statistical advantages to selecting more than 1 control per case in case–control studies, we limited the number of control articles to 1 per case because we wanted to limit the total number of articles to be reviewed and because we frequently found only 1 control article that matched the case article well.

### Rating scale

We developed an importance rating instrument that incorporated many of the dimensions addressed by Lawrence and associates20 in their recent work assessing the importance of published articles. Our instrument asked study participants (described in the next section) to rate articles on a 10-point scale that addressed each of the following 6 dimensions: importance to their own clinical practice (i.e., relevance); importance to clinical practice in general (allowing for the possibility that some respondents would not care for patients with the condition in question); importance from a local, national or international public health perspective; importance to the general advancement of our collective medical knowledge (i.e., knowledge context); ease with which the new information described in the article can be applied in daily practice (i.e., ease of applicability); and the impact that the new information described in the article is likely to have on the health outcomes of those affected by, or at risk for, the disease or condition addressed by the article. To assess the articles' general “visibility,” the instrument also asked respondents to indicate if they either had heard of the article in question or had read it.

Potential responses for the first 4 dimensions ranged from “unimportant” (0) to “extremely important” (10). For the question on ease of applicability, potential responses ranged from “very difficult” (0) to “very easy” (10). For the question on impact on health outcomes, potential responses ranged from “no impact” (0) to “huge impact” (10).

### Rating process

General internists affiliated with 4 academic institutions (University of Calgary, University of Alberta, Dalhousie University and the University of Lausanne) were invited to participate in this study as raters of article importance. We considered general internists to have a broad enough perspective to rate the importance of the wide variety of articles that we compiled. A total of 76 general internists were sent an email with a brief message inviting them to participate in a study “assessing the importance of articles recently published in the *New England Journal of Medicine* and *The Lancet*”; 42 agreed to participate. A priori, we projected a need for 43 participants to allow 90% power to detect a difference in scores of 1 point, assuming a standard deviation of 2 points. With 42 participants, we ended up having more than 90% power to detect 1-point differences because the standard deviations of scores were considerably less than 2 points.

Questionnaires and articles (24 in total) were mailed to the general internists who volunteered to participate in the study. Pairs of articles were presented to respondents consecutively in a pile, with the order of case and control articles alternating between pairs (i.e., case before control in one pair, control before case in the next). Aside from being aware that this study assessed the importance of published articles, the participants were blinded to the study's focus. Successful blinding was verified when we collected completed questionnaires: none of the respondents questioned knew that the study was assessing the issue of expedited publication.

As an ancillary assessment of each article's importance, we used the Science Citation Index to determine citation counts for each of the 24 articles. These citation counts were performed in December 1999.

### Methodological quality

Two methodological reviewers (F.A.M. and J.B.W.) blinded to the study's objectives rated the methodological quality of the selected articles using a quality scale described by Downs and Black.21 The scale has established reliability and criterion validity.21 We chose this quality scale because it can be applied to some types of observational studies as well as to clinical trials. As originally described, the scale yields scores ranging from 0 to 31. We modified one of the scale's items, a complex 5-point question on study power, to a simpler 1-point item. For our study, therefore, potential scores ranged from 0 to 27. We were able to apply the scale to 9 of the 12 pairs of articles; the scale could not be applied to 3 pairs because one or both of the articles were case reports or case series, study types to which the scale is not applicable.

### Analysis

We used a paired *t*-test to calculate the statistical significance of differences between mean scores for case and control articles. Our analyses assessed the differences in mean scores for each dimension, averaged across articles, as well as the differences in mean scores for each article, averaged across dimensions. Parametric statistical tests were appropriate because scores tended to be normally distributed.

## Results

Table 1 lists the titles and key matching elements of the pairs of articles.3,17,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43 In most instances the case and matched control articles addressed very similar topics. However, in 2 instances the absence of other articles on the theme addressed by the case article necessitated a broadening of search themes to identify a matching control article. In the first instance (pair B) the case article from *The Lancet* was a randomized controlled trial of ultrasonographic surveillance versus early surgery for abdominal aortic aneurysms.24 Because of the absence of other *Lancet* articles on aortic aneurysms or aortic surgery, the search theme was broadened to vascular surgery in general. With this broader search, we identified a control article describing a randomized controlled trial of a vascular surgical procedure, transjugular-intrahepatic-portosystemic shunt, versus endoscopy plus propranolol for the prevention of variceal bleeding.25 In the second instance (pair J) the case article from the *New England Journal of Medicine* was a randomized controlled trial of neostigmine for the treatment of colonic pseudo-obstruction.39 We were unable to find a matching article on either bowel obstruction or neostigmine, so we had to broaden the search theme to capture all colorectal diseases. This broader search allowed us to identify an article describing a randomized controlled trial of the treatment of chronic anal fissure.40

View this table:
[Table1](http://www.cmaj.ca/content/166/9/1137/T1)

Table 1. 

Table 2 presents the characteristics of the general internists who rated the articles. A majority of the respondents had served as manuscript reviewers in the year before the study, with 14% having done so at least 5 times. Aside from being general internists, 55% of the respondents declared focused interest or special expertise in a variety of clinical and academic areas, such as hypertension, clinical pharmacology, venous thromboembolic disease, clinical epidemiology and medical education.

View this table:
[Table2](http://www.cmaj.ca/content/166/9/1137/T2)

Table 2. 

### Importance ratings

Table 3 presents the mean scores across all case and control articles for each of the dimensions assessed. For every dimension the case articles had a significantly higher mean score than did the control articles. The responding physicians often felt that the case and control articles were not of great importance to their personal practices (mean scores 3.3 and 2.9 respectively), but they did acknowledge the importance of the case and control articles to clinical practice in general (mean scores 7.6 and 7.1 respectively).

View this table:
[Table3](http://www.cmaj.ca/content/166/9/1137/T3)

Table 3. 

Table 4 presents the mean scores across all 6 dimensions for each of the case–control pairs of articles studied. This table therefore presents the mean rating for importance, applicability and impact on health outcomes combined for each article studied. Despite the fact that the mean scores were generally higher for the case articles than for the control articles in each dimension (Table 3), there were 5 instances (pairs A, C, D, H and K) in which the mean scores across dimensions were higher for the control article than for the case article (Table 4). The difference in favour of the control article was statistically significant in 1 instance (pair K, *p* = 0.004). Scores were higher for the case article in the remaining 7 pairs, and the difference in favour of the case article was statistically significant for 5 of these (pairs B, E, F, I and L, *p* < 0.05).

View this table:
[Table4](http://www.cmaj.ca/content/166/9/1137/T4)

Table 4. 

There was a greater tendency for the case articles than the control articles to have been previously heard of or read by the respondents. On average, the respondents indicated that they had previously heard of 4.4 (standard deviation [SD] 1.6) of the 12 case articles compared with 3.5 (SD 2.0) of the 12 control articles (*p* = 0.002). They indicated that they had previously read 2.9 (SD 1.5) of the case articles compared with 1.9 (SD 1.3) of the control articles (*p* < 0.001).

### Citation counts

As of December 1999 the case and control articles had been cited in the medical literature a mean of 29.0 (SD 52.1) and 32.8 (SD 53.3) times respectively. However, the mean number of months from publication to the time of citation was not equivalent for the case and control articles (12.8 months v. 17.2 months). Taking time since publication into account, we calculated the mean number of citations per month (for each article and then averaged across articles) to be 1.8 for the case articles and 1.5 for the control articles (*p* = 0.44). For articles published in or before December 1998, the mean number of citations per month was 2.1 and 1.9 respectively (*p* = 0.84).

### Methodological quality

The studies' methodological quality scores were equivalent for the 9 pairs of articles to which the 27-point rating scale could be applied. Reviewer 1 (F.A.M.) assigned mean scores of 18.9 (SD 4.6) and 20.8 (SD 3.5) for the case and control articles respectively (*p* = 0.34). Reviewer 2 (J.B.W.) assigned mean scores of 20.2 (SD 6.1) and 20.2 (SD 5.3) respectively (*p* = 1.0). Despite demonstrated reliability of the scale developed by Downs and Black,21 and general agreement between the 2 reviewers in the global finding of no differences in methodological quality between the 2 groups of articles, we found that the quality scoring system was only modestly reproducible for individual articles, with a Pearson correlation coefficient between reviewers of only 0.43 (*p* = 0.13).

## Interpretation

Our findings indicate that the *New England Journal of Medicine* and *The Lancet* were successful in selecting articles for accelerated publication that were, on average, more important and applicable than the articles published in the usual manner. However, the differences were modest. Our results also demonstrated 5 instances in which the matched control article was considered to be of similar or greater importance than the expedited article.

An optimistic interpretation would be that current publication practices are supported by the modest differences in favour of articles given accelerated publication or dissemination. We suggest, in contrast, that our findings call into question current publication practices. Journals in some instances are not expediting the publication or release of important articles, and in other instances are selecting relatively less important articles for expedited publication. In this regard, general internists assigned relatively low mean scores across multiple dimensions (Table 4) for pairs C, D, E and K. Furthermore, the differences in mean scores for the case articles versus the control articles were less than 1 point for all dimensions (Table 3). Perhaps not surprisingly, we did not find any differences between the case and control articles in average study quality or citation counts, but we did find evidence of higher visibility for the prereleased and fast-tracked articles, which may or may not be causatively linked to the expedited publication process.

What might explain our findings? The policies for expedited publication of the *New England Journal of Medicine*1*,*2* and* The Lancet5,6 do not appear to involve the systematic screening of all articles as potential candidates. Rather, the process appears to be activated only when the authors, editors or peer reviewers specifically request that accelerated publication be considered on the basis of their subjective assessments of importance, which may or may not be explicitly defined. Perhaps recognizing this reality, the editors of this journal recently acknowledged, in an editorial introducing fast-tracking at *CMAJ*,8 that they were uncertain whether they could “reliably discern which [article] merits acceleration, or which is likely to be genuinely important in the long run.” When considered together with our findings, such expressions of uncertainty ought to incite journal editors to consider developing a unified approach to expedited publication, akin to the consensus positions that have recently been adopted on issues of authorship,44 reporting of clinical trials45 and industry sponsorship.46

One potential limitation of our study is that we asked general internists to rate the importance of the articles. Ratings might have been different had we asked subspecialists to rate the importance of articles in their fields (e.g., obstetricians for the articles on amniocentesis32 and chorionic villus sampling33). We decided to use internists because we recognized that it would be extremely difficult to involve different groups of subspecialists for each subject area represented by these articles. Furthermore, general internists have a broad enough perspective to rate the importance of the wide variety of articles compiled for this study. A second potential limitation of this study is that the clinimetric properties of the scale used by the physicians are not established, since the scale was developed specifically for this study. However, we found evidence for the scale's criterion validity in our data: there was a relatively strong positive correlation between articles' importance scores and their monthly citation frequencies (Pearson *r* = 0.55, *p* = 0.006). A third limitation is that there were 2 case articles for which we were unable to find control articles with close subject matches (pairs B and J). However, when we excluded these 2 pairs, we found that the results were essentially unchanged. Although the assessment of methodological quality was not the primary focus of our study, a fourth limitation is that the methodological quality scale that we used proved to be suboptimal in the reproducibility of quality scores assigned to individual articles. However, the 2 reviewers nonetheless agreed in their finding of no significant differences in methodological quality between case and control articles. Lastly, we acknowledge that the selection of the control articles was performed by only 1 of us (W.A.G.) and may thus not be replicable. However, we would argue that, even if this person had embarked on a targeted, nonrandom, and even biased, search for important control articles, our study would nonetheless prove an important point: articles published in the usual way are occasionally judged to be more important than prereleased or fast-tracked articles from the same journal.

Despite these acknowledged limitations, our study does generate useful information. Our results lead us to conclude that policies for expedited publication are, on average, targeting important articles and may be contributing to the visibility of research findings. However, journals now need to find ways to consistently rate the importance of every submitted article so that all important articles can be objectively considered for accelerated peer review and publiation. We hope that this study will stimulate dialogue among the editors of journals that offer accelerated publication.

𝛃 See related articles pages [1151](http://www.cmaj.ca/lookup/volpage/166/1151) and [1153](http://www.cmaj.ca/lookup/volpage/166/1153)

## Footnotes

*   *This article has been peer reviewed.* 
    
    *Contributors:* Dr. Ghali designed the study, recruited subjects (Calgary), compiled and analyzed data, and wrote the manuscript. Dr. Cornuz contributed to the study design, reviewed study materials, recruited subjects (Lausanne), assisted in compilation of data and provided critical comments on the manuscript. Dr. McAlister conducted detailed methodological quality reviews of selected articles, recruited subjects (Edmonton) and provided critical comments on the manuscript. Dr. Wasserfallen conducted detailed methodological quality reviews of selected articles and provided critical comments on the manuscript. Dr. Devereaux arranged for compilation of article citation counts, recruited subjects (Halifax) and provided critical comments on the manuscript. Dr. Naylor contributed to the study design and provided critical comments on the manuscript.
    
    *Acknowledgements:* We thank the 42 physicians who volunteered their time to rate the articles for this study: University of Calgary — M. Mintz, S. Ernst, J. Schaefer, J. Walsh, R. Dear, T. Pedersen, R. Hull, N. Khan, J. Mellor, N. Campbell and E. MacKay; University of Alberta — S. Majumdar, T.K. Lee, R. Padwal, G. Hrynchyshyn, B. Wirzba, N. Gibson, N. Kassam and B. Fisher; Dalhousie University — B. O'Brien, S. Workman, T. Dean and A. Borzecki; University of Lausanne — J.C. Luthi, L. Portmann, G. Pralong, F. Pralong, O. Lamy, J.B. Daeppen, A. Broccard, J. Ruiz, G. Waeber, P. Vollenweider, V. Mooser, M. Schapira, T. Buclin, D. Graf, J.M. Meier, A. Berger, P.A. Bart, P. Staeger and D. Beer.
    
    Dr. Ghali was hosted for a 6-month sabbatical leave by the Department of Medicine and the Institute for Social and Preventive Medicine, University of Lausanne, Lausanne, Switzerland. Drs. Ghali and McAlister are supported by Population Health Investigator Awards from the Alberta Heritage Foundation for Medical Research. Dr. Ghali is also supported by a Government of Canada Research Chair. Dr. Devereaux is supported by a Heart and Stroke Foundation of Canada/ Canadian Institutes of Health Research Fellowship Award.
    
    *Competing interests:* None declared.

## References

1.  1. Angell M, Kassirer JP. The Ingelfinger Rule revisited. N Engl J Med 1991; 325:1371-3.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199111073251910&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=1669838&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1991GN47000010&link_type=ISI) 

2.  2. Kassirer JP, Angell M. Prepublication release of journal articles. N Engl J Med 1997;337:1762-3.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199712113372409&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9392703&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997YK65300009&link_type=ISI) 

3.  3. Pitt B, Zannad F, Remme WJ, Cody R, Castaigne A, Perez A, et al, for the Randomized Aldactone Evaluation Study Investigators. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med 1999;341:709-17.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199909023411001&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10471456&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000082291900001&link_type=ISI) 

4.  4. Ghali WA, Cornuz J. Early uptake of research findings after fast-track publication. Lancet 2000;355:579-80.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10683036&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000085487400057&link_type=ISI) 

5.  5. McNamee D, Horton R. Fast-track to publication in *The Lancet*. Lancet 1997; 349:970.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(05)62891-7&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9132517&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 

6.  6. McNamee D. Fast-track publication at *The Lancet*. Lancet 1998;351:542.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(05)78553-6&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9492771&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 

7.  7. Goldbeck-Wood S, Robinson R. *BMJ* introduces a fast track system for papers. BMJ 1999;318:620.
    
    [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTgvNzE4NC82MjAiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTY2LzkvMTEzNy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

8.  8. With all due haste: fast tracking at *CMAJ* [editorial].CMAJ 2001;164 (8):1117. Available: [www.cmaj.ca/cgi/content/full/164/8/1117](http://www.cmaj.ca/cgi/content/full/164/8/1117)
    
    [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTY0LzgvMTExNyI7czo0OiJhdG9tIjtzOjIxOiIvY21hai8xNjYvOS8xMTM3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

9.  9. Winker MA, Fontanarosa PB. JAMA-EXPRESS: Rapid peer review and publication. JAMA 1999;281:1754-5.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.281.18.1754&link_type=DOI) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000080161200035&link_type=ISI) 

10. 10. Science Express. Available: [www.sciencemag.org/feature/express/introduction.shl](http://www.sciencemag.org/feature/express/introduction.shl) (accessed 2002 Mar 28).
    
    

11. 11. Advance online publication at *Nature*. Available: [www.nature.com/nature/journal/vaop/ncurrent/about.html](http://www.nature.com/nature/journal/vaop/ncurrent/about.html) (accessed 2002 Mar 28).
    
    

12. 12. Pannifer AD, Wong TY, Schwarzenbacher R, Renatus M, Petosa C, Bienkowska J, et al. Crystal structure of the anthrax lethal factor. Nature 2001; 414:229-33 [prepublished 2001 Oct 23].
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/n35101998&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=11700563&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000172029100053&link_type=ISI) 

13. 13. Bradley KA, Mogridge J, Mourez M, Collier RJ, Young JAT. Identification of the cellular receptor for anthrax toxin. Nature 2001;414:225-9 [prepublished 2001 Oct 23].
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1038/n35101999&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=11700562&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000172029100052&link_type=ISI) 

14. 14. Thomas GM. Improved treatment for cervical cancer — concurrent chemotherapy and radiotherapy. N Engl J Med 1999;340:1198-200.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199904153401509&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10202172&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000079687700009&link_type=ISI) 

15. 15. Weber KT. Aldosterone and spironolactone in heart failure. N Engl J Med 1999;341:753-5.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199909023411009&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10471464&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000082291900009&link_type=ISI) 

16. 16. Estacio RO, Jeffers BW, Hiatt WR, Biggerstaff SL, Gifford N, Schrier RW. The effect of nisoldipine as compared with enalapril on cardiovascular outcomes in patients with non-insulin-dependent diabetes and hypertension. N Engl J Med 1998;338:645-52.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199803053381003&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9486993&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000072299700003&link_type=ISI) 

17. 17. Rose PG, Bundy BN, Watkins EB, Thigpen JT, Deppe G, Maiman MA, et al. Concurrent cisplatin-based radiotherapy and chemotherapy for locally advanced cervical cancer. N Engl J Med 1999;340:1144-53.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199904153401502&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10202165&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000079687700002&link_type=ISI) 

18. 18. Morris M, Eifel PJ, Lu J, Grigsby PW, Levenback C, Stevens RE, et al. Pelvic radiation with concurrent chemotherapy compared with pelvic and para-aortic radiation for high-risk cervical cancer. N Engl J Med 1999;340:1137-43.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199904153401501&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10202164&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000079687700001&link_type=ISI) 

19. 19. Keys HM, Bundy BN, Stehman FB, Muderspach LI, Chafe WE, Suggs CL III, et al. Cisplatin, radiation, and adjuvant hysterectomy compared with radiation and adjuvant hysterectomy for bulky stage IB cervical carcinoma. N Engl J Med 1999;340:1154-61.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199904153401503&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10202166&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000079687700003&link_type=ISI) 

20. 20. Lawrence VA, Richardson WS, Henderson M, Wathen P, Williams JW, Mulrow CD. The best evidence from ACP Journal Club for general internal medicine, 1999 [editorial].* ACP J Club* 1999;131:A13-6.
    
    

21. 21. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomized and non-randomized studies of health care interventions. J Epidemiol Community Health 1998;52:377-84.
    
    [Abstract](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiamVjaCI7czo1OiJyZXNpZCI7czo4OiI1Mi82LzM3NyI7czo0OiJhdG9tIjtzOjIxOiIvY21hai8xNjYvOS8xMTM3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

22. 22. Reijers MH, Weverling GJ, Jurriaans S, Wit FW, Weigel HM, Ten Kate RW, et al. Maintenance therapy after quadruple induction therapy in HIV-1 infected individuals: Amsterdam Duration of Antiretroviral Medication (ADAM) study. Lancet 1998;352:185-90.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(98)06193-5&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9683207&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000074859800011&link_type=ISI) 

23. 23. Cameron DW, Heath-Chiozzi M, Danner S, Cohen C, Kravcik S, Maurath C, et al, for the Advanced HIV Disease Ritonavir Study Group. Randomised placebo-controlled trial of ritonavir in advanced HIV-1 disease. Lancet 1998; 351:543-9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(97)04161-5&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9492772&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000072231900008&link_type=ISI) 

24. 24. UK Small Aneurysm Trial Participants. Mortality results for randomised controlled trial of early elective surgery or ultrasonographic surveillance for small abdominal aortic aneurysms. Lancet 1998;352:1649-55.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(98)10137-X&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9853436&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000077110300008&link_type=ISI) 

25. 25. Rossle M, Deibert P, Haag K, Ochs A, Olschewski M, Siegerstetter V, et al. Randomised trial of transjugular-intrahepatic-portosystemic shunt versus endoscopy plus propranolol for prevention of variceal rebleeding. Lancet 1997; 349:1043-9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(96)08189-5&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9107241&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997WU00200007&link_type=ISI) 

26. 26. Turnstall-Pedoe H, Kuulasmaa K, Mahonen M, Tolonen H, Ruokokoski E, Amouyel P, for the WHO MONICA Project. Contribution of trends in survival and coronary-event rates to changes in coronary heart disease mortality: 10-year results from 37 WHO MONICA Project populations. Lancet 1999; 353:1547-57.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(99)04021-0&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10334252&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000080278300009&link_type=ISI) 

27. 27. Davey Smith G, Neaton JD, Wentworth D, Stamler R, Stamler J, for the MRFIT Research Group. Mortality differences between black and white men in the USA: contribution of income and other risk factors among men screened for the MRFIT. Lancet 1998;351:934-9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(05)60603-4&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9734939&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000072770900010&link_type=ISI) 

28. 28. Meda HA, Diallo B, Buchet JP, Lison D, Barennes H, Ouangre A, et al. Epidemic of fatal encephalopathy in preschool children in Burkina Faso and consumption of unripe ackee (*Blighia sapida*) fruit. Lancet 1999;353:536-40.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(99)01088-0&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10028981&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000078670200010&link_type=ISI) 

29. 29. Krishna S, Taylor AM, Supanaranond W, Pukrittayakamee S, ter Kuile F, Tawfiq KM, et al. Thiamine deficiency and malaria in adults from southeast Asia. Lancet 1999;353:546-9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(98)06316-8&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10028983&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000078670200012&link_type=ISI) 

30. 30. Nashan B, Moore R, Amlot P, Schmidt AG, Abeywickrama K, Soulillou JP, for the CHIB 201 International Study Group. Randomised trial of basiliximab versus placebo for control of acute cellular rejection in renal allograft recipients. Lancet 1997;350:1193-8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(97)09278-7&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9652559&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997YD23800008&link_type=ISI) 

31. 31. Dantal J, Hourmant M, Cantarovich D, Giral M, Blancho G, Dreno B, et al. Effect of long-term immunosuppression in kidney-graft recipients on cancer incidence: randomised comparison of two cyclosporin regimens. Lancet 1998;351:623-8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(97)08496-1&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9500317&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000072364200007&link_type=ISI) 

32. 32. Canadian Early and Mid-trimester Amniocentesis Trial (CEMAT) Group. Randomised trial to assess safety and fetal outcome of early and midtrimester amniocentesis. Lancet 1998;351:242-7.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(97)12346-7&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9457093&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000071702200009&link_type=ISI) 

33. 33. Sundberg K, Bang J, Smidt-Jensen S, Brocks V, Lundsteen C, Parner J, et al. Randomised study of risk of fetal loss related to early amniocentesis versus chorionic villus sampling. Lancet 1997;350:697-703.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(97)02449-5&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9291904&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997XU97700010&link_type=ISI) 

34. 34. International Perinatal HIV Group. The mode of delivery and the risk of vertical transmission of human immunodeficiency virus type 1: a meta-analysis of 15 prospective cohort studies. N Engl J Med 1999;340:977-87.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199904013401301&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10099139&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000079489800001&link_type=ISI) 

35. 35. Wade NA, Birkhead GS, Warren BL, Charbonneau TT, French PT, Wang L, et al. Abbreviated regimens of zidovudine prophylaxis and perinatal transmission of the human immunodeficiency virus. N Engl J Med 1998;339:1409-14.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199811123392001&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9811915&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000077051700001&link_type=ISI) 

36. 36. Connolly HM, Crary JL, McGoon MD, Hensrud DD, Edwards BS, Edwards WD, et al. Valvular heart disease associated with fenfluramine-phenteramine. N Engl J Med 1997;337:581-8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199708283370901&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9271479&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997XT39400001&link_type=ISI) 

37. 37. Benhaim L, Moride Y, Brenot F, et al. for the International Primary Pulmonary Hypertension Study Group. Appetite-suppressant drugs and the risk of primary pulmonary hypertension. N Engl J Med 1996;335:609-16.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199608293350901&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=8692238&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1996VD42500001&link_type=ISI) 

38. 38. Turrisi AT, Kim K, Blum R, et al. Twice-daily compared with once-daily thoracic radiotherapy in limited small-cell lung cancer treated concurrently with cisplatin and etoposide. N Engl J Med 1999;340:265-71.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199901283400403&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9920950&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000078318700003&link_type=ISI) 

39. 39. Ponec RJ, Saunders MD, Kimmey MB. Neostigmine for the treatment of acute colonic pseudo-obstruction. N Engl J Med 1999;341:137-41.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199907153410301&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10403850&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000081413800001&link_type=ISI) 

40. 40. Brisinda G, Maria G, Bentivoglio AR, Cassetta E, Gui D, Albanese A. A comparison of injections of botulinum toxin and topical nitroglycerin ointment for the treatment of chronic anal fissure. N Engl J Med 1999;341:65-9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199907083410201&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10395629&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000081316200001&link_type=ISI) 

41. 41. Hecht FM, Grant RM, Petropoulos CJ, Dillon B, Chesney MA, Tian H, et al. Sexual transmission of an HIV-1 variant resistant to multiple reverse-transcriptase and protease inhibitors. N Engl J Med 1998;339:307-11.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199807303390504&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9682043&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000075031400004&link_type=ISI) 

42. 42. Zhang H, Dornadula G, Beumont M, Livornese L Jr, Van Uitert B, Henning K, et al. Human immunodeficiency virus type 1 in the semen of men receiving highly active antiretroviral therapy. N Engl J Med 1998;339:1803-9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199812173392502&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9854115&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000077545800002&link_type=ISI) 

43. 43. Torp-Pedersen C, Moller M, Bloch-Thomsen PE, Kober L, Sandoe E, Egstrup K, et al. for the Danish Investigations of Arrhythmia and Mortality on Dofetilide Study Group. Dofetilide in patients with congestive heart failure and left ventricular dysfunction. N Engl J Med 1999;341:857-65.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM199909163411201&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10486417&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000082569100001&link_type=ISI) 

44. 44. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals. Available: [www.icmje.org](http://www.icmje.org). (accessed 2001 Nov 1)
    
    

45. 45. Moher D, Schulze KF, Altman DG, for the CONSORT Group. The CONSORT Statement: revised recommendations for improving the quality of reports of parallel group randomized trials. JAMA 2001;285:1987-91.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.285.15.1987&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=11308435&link_type=MED&atom=%2Fcmaj%2F166%2F9%2F1137.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000167995900027&link_type=ISI) 

46. 46. Davidoff F, DeAngelis CD, Drazen JM, Nicholls MG, Hoey J, Højgaard L, et al. Sponsorship, authorship and accountability. CMAJ 2001;165(6):786-8. Available: [www.cmaj.ca/cgi/content/full/165/6/786](http://www.cmaj.ca/cgi/content/full/165/6/786)
    
    [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNjUvNi83ODYiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTY2LzkvMTEzNy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=)