Abstract
Background: Gaps continue to exist between research-based evidence and clinical practice. We surveyed health care providers in 10 low- and middle-income countries about their use of research-based evidence and examined factors that may facilitate or impede such use.
Methods: We surveyed 1499 health care providers practising in one of four areas relevant to the Millennium Development Goals (prevention of malaria, care of women seeking contraception, care of children with diarrhea and care of patients with tuberculosis) in each of China, Ghana, India, Iran, Kazakhstan, Laos, Mexico, Pakistan, Senegal and Tanzania.
Results: The proportion of respondents who reported that research was likely to change their clinical practice if performed and published in their own country (84.6% and 86.0% respectively) was higher than the proportion who reported the same about research and publications from their region (66.4% and 63.1%) or from high-income countries (55.8% and 55.5%). Respondents who were most likely to report that the use of research-based evidence led to changes in their practice included those who reported using clinical practice guidelines in paper format (odds ratio [OR] 1.54, 95% confidence interval [CI] 1.03–2.28), using scientific journals from their own country in paper format (OR 1.70, 95% CI 1.26–2.28), viewing the quality of research performed in their country as above average or excellent (OR 1.93, 95% CI 1.16–3.22); trusting systematic reviews of randomized controlled trials (OR 1.59, 95% CI 1.08–2.35); and having easy access to the Internet (OR 1.90, 95% CI 1.19–3.02).
Interpretation: Locally conducted or published research has played an important role in changing the professional practice of health care providers surveyed in low- and middle-income countries. Increased investments in local research, or at least in locally adapted publications of research-based evidence from other settings, are therefore needed. Although access to the Internet was viewed as a significant factor in whether research-based evidence led to concrete changes in practice, few respondents reported having easy access to the Internet. Therefore, efforts to improve Internet access in clinical settings need to be accelerated.
Gaps continue to exist between evidence generated by clinical research and practice. 1 Efforts to improve access to health information in low- and middle-income countries 2 and a greater knowledge of how to support the use of research evidence in clinical practice have made little difference. The health consequences of these gaps can be particularly profound when highly effective interventions exist. For example, in the 42 countries in which 90% of the deaths involving children worldwide occurred in 2000, nearly 2.2 million deaths among those under five years of age could have been prevented through the universal use of oral rehydration therapy in those with diarrhea and the use of insecticide-treated materials to prevent malaria. 3
We conducted this study to examine the use of research-based evidence in defined clinical areas in a sample of health care providers in 10 low- and middle-income countries. We also examined factors that may facilitate or impede such use.
Methods
Study participants
Our survey was part of a larger project that sought to explore factors that explain whether and how producers and users of research — health care providers and policy-makers — support the use of, or use, research-based evidence for decision-making. We surveyed health care providers in 10 low- and middle-income countries (China, Ghana, India, Iran, Kazakhstan, Laos, Mexico, Pakistan, Senegal and Tanzania) who were practising in one of four clinical areas relevant to the Millennium Development Goals: prevention of malaria (Ghana, Laos, Senegal and Tanzania), care of women seeking contraception (China, Kazakhstan, Laos and Mexico), care of children with diarrhea (Ghana, India, Pakistan and Senegal) and care of patients with tuberculosis (China, India, Iran and Mexico). Within each area, a particular emphasis was placed on an intervention that was supported by strong evidence from international and local research: insecticide-treated materials to prevent malaria; intrauterine devices for family planning; oral rehydration therapy to prevent dehydration in children with diarrhea; and the DOTS strategy (directly observed treatment, short course) to control tuberculosis. In a related research article, we describe the findings from a survey of researchers in these countries who were involved in the production of research on one of these health topics about their engagement in activities to bridge the gaps between research, policy and practice. 4
We purposively sampled countries to achieve breadth in levels of economic development and health care systems, political systems and geographic locations. In addition, all country teams had to have demonstrated (through collaborations with the World Health Organization [WHO]) a strong interest in bridging the gaps between research, policy and practice and in further developing the capacity to evaluate such efforts. The selected countries differ in their health status indicators, rates of coverage for the interventions under study, and access to the Internet or computers with a CD-ROM (Appendix 1, available at www.cmaj.ca/cgi/content/full/cmaj.081165/DC1).
Resource constraints prevented the survey of a fully representative sample of health care providers at all study sites. The country teams sought to survey at least 100 providers for each clinical area examined in each country. The sampling frame in each country was developed with the use of lists of health care providers identified by country investigators. Study participants were selected with the use of random sampling processes in all countries except Tanzania, where a purposive approach was used to sample district medical officers, and Kazakhstan, where all gynecologists in the city of Almaty were surveyed.
Development of the questionnaire
A self-administered questionnaire was developed from existing sources. 5–13 Its framework was drawn from one described by the Canadian Health Services Research Foundation for capacities to bridge the gap between research and action. 14 Emphasis was placed on measures of health care providers’ ability to acquire and apply research. Areas covered in the questionnaire included individual and practice characteristics; views related to improving clinical practice (views on research performed or published in particular places, views on particular types of research that can inform prevention or treatment decisions, and views on issues deemed important to improve practice); networking activities; awareness of, access to and use of research evidence; and changes in clinical practice attributed to particular sources of research evidence used.
Administration of the survey
Health care providers were sampled between October 2004 and December 2005. Detailed information about the samples and the development, reliability and validation of the data-collection instrument are provided elsewhere. 15 The questionnaire was administered with the use of a drop-off and pick-up approach in all countries except Iran, Mexico and Senegal, where a mix of drop-off and pick-up and in-person administration was used. We used several approaches to increase the response rate: personalized letters, follow-up of contacts, and provision of a set of World Health Organization publications as an incentive. 16
Statistical analysis
In addition to presenting detailed descriptive statistics, we performed logistic regression analyses to explore associations between reports by health care providers that research-based evidence has led to concrete changes in their professional practice (using a binary transformation of the influence component of a measure of research use 5) and (a) their use of particular sources of evidence; (b) their views and activities related to improving their practice; and (c) individual and practice characteristics. Specifically, we examined the potential impact of their use of clinical practice guidelines, systematic reviews from the Cochrane Library, research articles from scientific journals published in their own country and in high-income countries, and summaries of articles, reports and reviews from public and not-for-profit health organizations.
We chose the Cochrane Library because it is the most comprehensive source of systematic reviews. Systematic reviews can help health care providers navigate the medical literature by summarizing large bodies of evidence and explaining differences between studies. 17,18 We differentiated between scientific journals from the providers’ own countries and those from high-income countries because physicians in a survey by Page and colleagues 8 reported that local research and publications were more likely than research and publications from elsewhere to change their clinical practice. We distinguished between full reports and summaries of articles, reports and reviews from public and not-for-profit health organizations because summaries can facilitate communication. 19 We examined the level of trust in systematic reviews of randomized controlled trials because such reviews have informed clinical practice in the four areas under study. 20–29 We looked specifically at networking activities between health care providers and researchers because interactions have been found to be associated with increased use of research-based evidence. 30,31
For missing values, we used multiple imputation, whereby each missing value was replaced by 10 different estimates. Standard errors were adjusted according to Rubin’s rules. 32 We excluded observations when the dependent variable was missing. All models were estimated with the use of Stata/SE 9.2 for Macintosh with robust variances. 33
Results
In each country, approximately 95 health care providers per clinical area completed the questionnaire, for a total sample of 1499 respondents. The overall response rate was high (92%). The majority of respondents were general practitioners (68.9%) and devoted two-thirds (67.3%) of their time to clinical practice. Few had a master’s or doctorate degree (8.0%). Most practised in a government-run facility (89.9%), in either a hospital or community health centre (94.0%). Few practised in a rural area (15.3%) (Table 1).
Most of the respondents received clinical training since completing their last degree, although few received training to improve their capacity to acquire, assess or adapt research evidence. Less than one-fifth of the respondents said that they (or someone who could act on their behalf) had easy access to the Internet (18.1%).
The proportion of respondents who reported that research was likely to change their clinical practice if performed and published in their own country (84.6% and 86.0% respectively) was higher than the proportion who reported the same about research and publications from their region (66.4% and 63.1%) or from high-income countries (55.8% and 55.5%) (Table 2). The proportion who reported that research performed in their own country is of “above average” or “excellent” applicability to their local setting (63.5%) was higher than the proportion who reported the same about research from their region (43.6%) or from high-income countries (37.9%). On the other hand, the proportion of respondents who reported that research performed in high-income countries is of “above average” or “excellent” quality (79.5%) was higher than the proportion who felt the same about research performed in their own country (51.0%) or region (50.6%). These findings were generally consistent across the four clinical areas and the 10 countries. The proportion of respondents who reported trusting systematic reviews of randomized controlled trials somewhat or completely (54.8% overall) varied little across the four clinical areas (from 51.7% to 58.6%). These values were substantially lower than those for several other types of evidence, including the respondents’ practical experience (85.5% overall) and expert opinion and advice (82.9% overall).
Among the networking activities engaged in to improve clinical practice or quality of working life, working with peers to exchange ideas, experiences and best practices was the most common (reported by 72.9%); the next most common was working with patient groups (56.5%) (Table 2). Interactions with researchers or research groups (30.4%) and with representatives of for-profit organizations (e.g., pharmaceutical companies) (20.9%) were the least common networking activities.
Most of the respondents stated that the availability of higher quality (85.4%) and more locally applicable research (85.3%) and of more training (89.2%) were important or very important issues that would help to improve their work. They cited many other issues with similar frequency (Table 2).
About half of the respondents reported that they had used the following sources in paper format over the 12 months before the survey: clinical practice guidelines (55.6%), scientific journals from their own country (55.0%), and articles, reports and reviews from public and not-for-profit health organizations (52.4%) (Table 3). About one-fifth (22.2%) reported having read scientific journals from high-income countries in paper format, and almost two-fifths (38.1%) reported being aware of but not having access to journals from high-income countries. On the whole, respondents’ awareness of, access to and use of online and paper sources of information were similar except for the use of journals from high-income countries: more respondents reported having read journals from high-income countries in an online or electronic format than in a paper format (32.1% v. 22.2%). The use of information not available in paper format tended to be less frequent than the online use of information available in both formats. Few respondents who had access to the Internet or to a computer with a CD-ROM reported having used the Cochrane Library (15.9%).
The only changes in practice reported by more than one-third of the respondents as having been made in response to their use of particular types of research-based evidence were changes to their approach to treating a clinical condition and, less often, changes to or development of a local clinical practice guideline (Table 4). For example, almost half (44.5%) of those who had used clinical practice guidelines in paper format in the 12 months before the survey reported that such use led to changes in their approach to treating a clinical condition.
The likelihood that the use of research-based evidence led to concrete changes in the respondents’ professional practice was significantly increased among those who reported using clinical practice guidelines in paper format (odds ratio [OR] 1.54, 95% confidence interval [CI] 1.03–2.28), using scientific journals from their own country in paper format (OR 1.70, 95% CI 1.26–2.28), viewing research performed in their own country as being of above average or excellent quality (OR 1.93, 95% CI 1.16–3.22) and trusting systematic reviews of randomized controlled trials (OR 1.59, 95% CI 1.08–2.35) (Table 5).
Among the 758 respondents who had access to the Internet or a personal computer with a CD-ROM, use of the online or electronic sources of research-based evidence examined in our survey did not have a significant influence on their professional practice (Table 5). In this group of respondents, concrete changes in practice were significantly more likely among those who viewed the quality of research performed in their own country as above average or excellent (OR 1.66, 95% CI 1.14–2.41) and among those who engaged in networking activities that involved working with researchers or research groups to improve clinical practice or the quality of working life (OR 1.77, 95% CI 1.11–2.82).
The impact of individual and practice characteristics on the odds of research-based evidence leading to concrete changes in professional practice was generally similar between those who used paper formats of information and those who used online or electronic formats (Table 5). Having received training in acquiring systematic reviews through the Cochrane Library and in critically appraising systematic reviews, having easy access to the Internet, higher age and (for those with access to the Internet or a personal computer with a CD-ROM) being based in a facility with a nongovernmental organization as the operating authority increased the likelihood of reporting that the use of research-based evidence has led to concrete changes in the respondents’ professional practice. Practising in an urban setting and (for those who reported using research-based evidence in paper format) being a specialist physician decreased those odds.
Interpretation
In our survey of health care providers in 10 low- and middle-income countries, we found that those who reported using clinical practice guidelines in paper format or scientific journals from their own country in paper format had significantly increased odds of reporting that the use of research-based evidence has led to concrete changes in their professional practice. Also, as was found by Page and colleagues in a sample of physicians from five developing countries, 8 the proportion of health care providers who reported that research performed and published in their own country was likely to change their clinical practice was higher than the proportion who reported the same about research and publications from their region or from high-income countries. Given the salience and influence of locally conducted or published research, we believe there is a need for increased investments in local research, or at least in locally adapted publications of research-based evidence from other settings.
We found that the proportion of health care providers who reported having easy access to the Internet was low. Given that such access was a significant factor in whether research-based evidence led to concrete changes in clinical practice, we believe that efforts to increase Internet access in clinical settings need to be renewed. In the interim, access to paper-based clinical practice guidelines and other sources of information on which many providers currently rely should not be diminished. These two approaches could be accompanied by more proactive strategies for supporting the use of research-based evidence. If they are undertaken on a sufficiently large scale and evaluated rigorously, robust conclusions could be drawn about their impact on clinical practice.
Others have shown that health care providers in high-income countries and in one low-income country had much better access to printed sources of research-based evidence than to online or electronic sources. 11,34–36 We found that about half of the respondents in our study reported having used the following sources of evidence in paper formats over the 12 months before the survey: clinical practice guidelines, scientific journals from their own country, and articles, reports and reviews from public and not-for-profit health organizations. Previous studies have found that scientific journals are among the least useful sources of information for health care providers in practice, 34,36,37 yet they have been previously reported by health care providers in high-income countries as being an important influence on their clinical practice. 6,11,38 Printed educational materials, when compared with no intervention, were found to improve process outcomes slightly, but not patient outcomes. 39
Limitations
Our study has three limitations worth noting. First, as is the case with most research that relies on self-reported questionnaires, social desirability bias (when respondents provide socially desirable answers) cannot be ruled out. Responses may represent either true beliefs and behaviours or perceptions about what respondents thought we wanted to hear, or a combination of both. However, because of the positive nature of most questions asked, and the low reported frequencies of some behaviours that are believed to be beneficial (e.g., use of the Cochrane Library, trust in systematic reviews of randomized controlled trials), we are reasonably confident that reported data were not overly inflated upward. In addition, self-reports of current behaviours can tell us where the most room for improvement is, regardless of whether responses tap into actual behaviours or a social desirability bias. Biased responses are presumably based on providers’ awareness and knowledge of what they think we want them to say, with the added constraint that in some domains they may be concerned that they could be questioned further. Second, linguistic or cultural differences may have affected respondents’ interpretation of select questions. Third, the small samples of health care providers surveyed in our study cannot be assumed to be representative of all health care providers practising in the four clinical areas in the 10 countries. As such, future surveys such as this one should be conducted using representative samples in order to enhance generalizability.
Conclusion
Our findings indicate that locally conducted or published research plays an important role in changing the professional practice of health care providers surveyed in low- and middle-income countries. Increased investments in local research, or at least in locally adapted publications of research-based evidence from other settings, are thus needed. Access to the Internet was viewed as a significant factor in whether research-based evidence led to concrete changes in clinical practice; however, few health care providers reported having easy access to the Internet. Therefore, efforts to improve Internet access in clinical settings need to be accelerated. In the interim, access to paper-based clinical practice guidelines and other sources of information on which many providers currently rely should not be diminished.
Footnotes
-
Previously published at www.cmaj.ca
See related research article by Lavis and colleagues
This article has been peer reviewed.
Competing interests: None declared.
Contributors: Emmanuel Guindon contributed substantially to the study concept and design and to the acquisition, analysis and interpretation of data; he drafted and revised the article critically for important intellectual content. John Lavis, Francisco Becerra-Posada, Hossein Malek-Afzali, Guang Shi, C. Ashok K. Yesudian and Steven Hoffman contributed substantially to the study concept and design, the acquisition of data, or the analysis and interpretation of data and revised the article critically for important intellectual content. All of the authors gave final approval of the version to be published.
Funding: The Alliance for Health Policy and Systems Research funded part of all phases of the project. McMaster University and the World Health Organization provided substantial in-kind donations of staff time and other resources. The Global Development Network funded an early phase of the project. Emmanuel Guindon is supported by a Canada Graduate Scholarship from the Social Sciences and Humanities Research Council of Canada. John Lavis receives salary support as the Canada Research Chair in Knowledge Transfer and Exchange. The views expressed in this paper are those of the authors and do not represent the views of the funding organizations.
Members of the Research to Policy and Practice Study Team: Writing group: G. Emmanuel Guindon and John N. Lavis (Canada); Francisco Becerra-Posada (Mexico); Hossein Malek-Afzali (Iran); Guang Shi (China); C. Ashok K. Yesudian (India); and Steven J. Hoffman (Canada). Other members: David Cameron (Canada); Tinglin Qiu (China); Eric J.A. Osei and Kudjoe Dovlo (Ghana); Prema Ramachandran (India); Masoumeh Dejman, Katayoun Falahat, Monir Baradaran, Elham Habibi, Hoshmand Kohanzad, Mahshid Nasehi and Salek Salek (Iran); Aikan A. Akanov, Botagoz S. Turdaliyeva, Nurgul K. Hamzina, Kazbek A. Tulebayev, Tatiana I. Clazhneva and Zhamilya Battakova (Kazakhstan); Boungnong Boupha, Sengchanh Kounnavong and Latsamy Siengsounthone (Lao People’s Democratic Republic); Leticia Alfaro Ramos and Israel Mejia (Mexico); Tasleem Akhtar and M. Mubashir A. Khan (Pakistan); Mintou Fall Sidibe, Awa Sidibe and Djiby Ndiaye (Senegal); Godwin D. Ndossi and Julius Massaga (Tanzania); and Ritu Sadana and Tikki Pang (World Health Organization)
Acknowledgements: The authors thank the technical experts who provided support to one or more phases of the study, the researchers who shared their questionnaires, and the participants in the project workshop held in Geneva to discuss the data-collection process, interim findings, and potential implications for dissemination and next steps. The authors also thank Andrew Kennedy and Carol D’Souza for providing scientific input in one or more phases of the study.