So what has the Cochrane Collaboration ever done for us? A report card on the first 10 years
============================================================================================

* Jeremy Grimshaw

*   © 2004 Canadian Medical Association or its licensors

The Cochrane Collaboration is a unique, worldwide not-for-profit organization that aims to help people make well-informed decisions about health care by preparing, maintaining, and promoting the accessibility of, systematic reviews of the effects of health care interventions.1 A decade ago, C. David Naylor referred to the Collaboration as the health services equivalent of the Human Genome Project.2 The present commentary reflects on progress to date and on remaining challenges.

The core business of the Cochrane Collaboration is to produce systematic reviews that are regularly updated and published quarterly (with other electronic resources) in the Cochrane Library. There were 2074 systematic reviews in the latest edition of the Cochrane Library (2004, issue 3). The coverage of Cochrane reviews is continually improving, and reviews now exist relevant to the top 10 causes of disability in both developed and developing countries. It has been demonstrated that Cochrane reviews are of comparable or better quality and are updated more often than reviews published in print journals.3 In addition, the Cochrane Library includes the Cochrane Central Register of Controlled Trials (CENTRAL), with over 415 000 references to completed and ongoing randomized controlled trials; the Cochrane Methodology Register, with over 5600 references to methodologic papers; and 3 other databases of systematic reviews, health technology assessment reports and economic evaluations.

These reviews are authored by about 7000 volunteers worldwide (researchers, health care professionals and consumers) and are supported by a range of Cochrane entities. This is the health services research equivalent of “distributed computing,”4 by which computationally intense activities are managed by sharing workload across networked computers to provide computing power equivalent to the fastest commercial supercomputers. The Cochrane Collaboration builds on the enthusiasm of individuals to undertake reviews, often within their own resources, using a standardized approach; this has led to the dramatic increase in Cochrane reviews with relatively little external funding (see Fig. 1). Cochrane authors make a commitment to update their reviews as new evidence becomes available and as comments and criticisms from users of the Cochrane Library are received. The Collaboration has promoted the involvement of consumers throughout the conduct of reviews, and reviews include consumer synopses written in lay language.

![Figure1](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/171/7/747/F1.medium.gif)

[Figure1](http://www.cmaj.ca/content/171/7/747/F1)

**Fig. 1: Number of reviews in the Cochrane Database of Systematic Reviews.** 

Through national licences, the Cochrane Library is available free of charge to users in Australia, Denmark, England, Finland, Ireland, Norway, Northern Ireland, South Afri ca, Spain and Wales. For users in countries classified by the World Bank as low- or middle-income the Cochrane Library is available through HINARI (the Health InterNetwork Access to Research Initiative). The Cochrane Library is published by John Wiley & Sons on CD-ROM and the Internet; on average, someone accesses this Web site every 37 seconds. Additional coverage is provided worldwide through partnerships with information aggregators such as Ovid Technologies and Silver Platter Information, which facilitate broader dissemination; for example, Ovid Technologies has over 5000 institutional customers worldwide with more than 2.5 million individual users.

Cochrane reviews have been used to develop a number of “knowledge translation” products for professionals, patients and policy-makers.5 Increasingly, these products are being labelled as having “Cochrane inside” (similar to the “Intel inside” label on personal computers). For example, Cochrane reviews provide the evidence base for the Reproductive Health Library, a WHO-sponsored electronic publication containing Cochrane reviews, guidelines and commentaries that is distributed free to over 34 000 professionals and health care organizations in low- and middle-income countries in both English and Spanish versions.6 Cochrane reviews are also an important source of evidence for Clinical Evidence, which is distributed free to over 50 000 professionals within the UK National Health Service, to professionals in the United States (BMJ Publishing, courtesy of the Unit Health Foundation) and via the Internet to over 100 low- and middle-income countries. All of these products are regularly updated using data from Cochrane reviews.

In Canada, Cochrane reviews are being used to support health technology reports prepared by the Coordinating Council of Health Technology Assessment and the work of the Therapeutics Initiative. They are consulted by the peer review panels of the Canadian Institutes of Health Research and are used in the preparation of clinical practice guidelines, such as those by the Canadian Paediatric Society.5

The work of the Cochrane Collaboration has also led to substantial methodologic advances. For example, an early product from the Collaboration led to the development of highly sensitive search strategies for randomized controlled trials.7 Cochrane reviews provided the data that demonstrated that failure to conceal randomization allocation was associated with bias.8The volume of Cochrane reviews produced using common methods within a standardized format9 ensures that the Cochrane Library is the best single resource for methodologic research and for developing the science of meta-epidemiology.

## Challenges for the future

Considerable challenges remain for the Cochrane Collaboration. The distributed-computing model results in interest-driven as opposed to priority-driven reviews, leading to the criticism that the Collaboration is not sufficiently responsive to the immediate needs of policy-makers. Cochrane entities have responded to this criticism by engaging policy-makers in dialogue to identify priorities10 and by encouraging reviewers to addressing these identified needs. However, the distributed-computing model makes it difficult for the Collaboration to be more responsive without substantial investment. Within the UK, the NHS Research and Development Programme has recognized this difficulty by commissioning priority-driven reviews from the broad research community, many of which are subsequently converted into Cochrane reviews.

Infrastructure funding for the Collaboration remains problematic in some parts of the world. Although governments in Australia, Denmark, England, Finland, Ireland, Norway, South Africa and Wales have provided funding, many others have yet to do so. On a parochial note, it remains disappointing that all but one of the entities based in Canada lack stable infrastructure funding despite the fact that they were responsible for 16.3% of reviews published in the second issue of 2004 (and that, apart from residents of Saskatchewan, Canadians do not yet have access to the Cochrane Library through a government-sponsored licence). The one Canadian entity with secure infrastructure funding (from the US National Institute of Child Health and Human Development) is the Cochrane Neonatal Group based at McMaster University; this is the second most productive review group within the Collaboration.11 The total budget required for a Canadian national licence and infrastructure support for Canadian Cochrane entities is about $4 million per year (less than 12 cents per citizen).

The Cochrane Collaboration continues to promote international collaboration: currently, about 9% of Cochrane reviewers are from low- and middle-income countries. This is a considerable achievement, but further efforts are needed to ensure wider international involvement.

Admittedly, Cochrane reviews are not particularly user-friendly at present. A key question for the Collaboration is whether it should concentrate on developing a more accessible interface itself, or work with other “knowledge packagers” to develop derivative products specific to the needs of different target groups, or try both approaches.

Although Cochrane reviews appear to be of better quality than other reviews, areas for improvement remain.3 Cochrane reviews have not adequately assessed the potential harms of health care interventions. Maintaining the enthusiasm of reviewers to update their reviews is an increasing challenge as the number of reviews increase.

## So far, so good

The Cochrane Collaboration is a unique organization that has made substantial progress within the last decade in undertaking reviews of health care interventions and making them available to a wide range of users throughout the world. It has contributed to the development of methods for randomized controlled trials and systematic reviews. There are many ways that individuals can contribute to this global effort. Getting involved is easy; interested parties can visit the “How to get involved section” on the Cochrane Web site at [www.cochrane.org](http://www.cochrane.org); alternatively, Canadians can contact the Canadian Cochrane Centre (cochranemcmaster.ca). The Cochrane Collaboration has much to celebrate at its 12th Colloquium, which will be held in Ottawa Oct. 2–6. I encourage readers to join us there.

𝛃 See related article page [721](http://www.cmaj.ca/lookup/volpage/171/721)

## Footnotes

*   *Acknowledgements:* I thank Claire Allen, Kathie Clark, Martin Eccles, Ian Graham, Sally Green, Roberto Grilli, Metin Gulmezoglu, Jini Hetherington, Terry Klassen, Alain Mayhew, David Moher, Arne Ohlsson, Nick Royle, Jack Sinclair, Peter Tugwell, Merrick Zwarenstein for their helpful comments on early drafts of the paper.
    
    *Competing interests:* Jeremy Grimshaw remains unashamedly enthusiastic about the Cochrane Collaboration after 10 years of involvement. He is the Co-ordinating Editor of the Cochrane Effective Practice and Organisation of Care (EPOC) review group and the author of several Cochrane reviews. His life would be made considerably easier if EPOC acquired stable infrastructure funding.
    
    ***Correspondence to:**** Dr. Jeremy Grimshaw, Director, Clinical Epidemiology Program, Ottawa Health Research Institute, Civic Campus, 1053 Carling Ave., Administration Building, Rm. 2-017, Ottawa ON K1Y 4E9; fax 613 761-5402;* jgrimshawohri.ca

## References

1.  1. Bero L, Rennie D. The Cochrane Collaboration. Preparing, maintaining, and disseminating systematic reviews of the effects of health care. JAMA 1995; 274: 1935-8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.1995.03530240045039&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=8568988&link_type=MED&atom=%2Fcmaj%2F171%2F7%2F747.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1995TL16900036&link_type=ISI) 

2.  2. Naylor CD. Grey zones of clinical practice: some limits to evidence-based medicine. Lancet 1995;345:840-2.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(95)92969-X&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=7898234&link_type=MED&atom=%2Fcmaj%2F171%2F7%2F747.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1995QQ19500014&link_type=ISI) 

3.  3. Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, et al. Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA 1998;280:278-80.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.280.3.278&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9676681&link_type=MED&atom=%2Fcmaj%2F171%2F7%2F747.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000074706000026&link_type=ISI) 

4.  4. DC Central. Available: [http://library.thinkquest.org/C007645/english/0-welcome.htm](http://library.thinkquest.org/C007645/english/0-welcome.htm) (accessed 2004 Aug. 19).
    
    

5.  5. The Cochrane Collaboration. *The dissemination of Cochrane evidence. An inventory of resources that use Cochrane reviews*. May 2004. Available: [www.cochrane.org/reviews/impact/dissemination.htm](http://www.cochrane.org/reviews/impact/dissemination.htm) (accessed 2004 Aug. 19).
    
    

6.  6. The WHO Reproductive Health Library. Avaialble: [www.update-software.com/rhl/](http://www.update-software.com/rhl/) (accessed 2004 Aug. 19).
    
    

7.  7. Lefebvre C, Clarke MJ. Identifying randomised trials. In: Eggar M, Davey Smith G, Altman DG, editors. *Systematic reviews in health care. Meta-analysis in context*. London (UK): BMJ Publishing; 2001.
    
    

8.  8. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408-12.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.1995.03520290060030&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=7823387&link_type=MED&atom=%2Fcmaj%2F171%2F7%2F747.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1995QD20300026&link_type=ISI) 

9.  9. The Cochrane Collaboration. *The Cochrane reviewers' handbook*. Available: [www.cochrane.org/resources/handbook/index.htm](http://www.cochrane.org/resources/handbook/index.htm) (accessed 2004 Aug. 19).
    
    

10. 10. The Cochrane Collaboration. Cochrane Health Promotion and Public Health Field. Priority review topics in health promotion and public health. Available: [www.vichealth.vic.gov.au/cochrane/activities/priorities.htm](http://www.vichealth.vic.gov.au/cochrane/activities/priorities.htm) (accessed 2004 Aug. 19).
    
    

11. 11. Sinclair JC, Haughton DE, Bracken MB, Horbar JD, Soll RF. Cochrane neonatal systematic reviews: a survey of the evidence for neonatal therapies. Clin Perinatol 2003;30:285-304.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0095-5108(03)00025-3&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=12875355&link_type=MED&atom=%2Fcmaj%2F171%2F7%2F747.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000183946100007&link_type=ISI)