Canada's programs to prevent mental health problems in children: the research–practice gap
============================================================================================

* John D. McLennan
* Harriet L. MacMillan
* Ellen Jamieson

*   © 2004 Canadian Medical Association or its licensors

The increased number of prevention activities directed at reducing child mental health problems in Canada is welcome. However, practitioners and policy-makers should reflect on 2 questions that were posed 20 years ago by Michael Rutter: “How much do we really know about prevention in the field of psychosocial disorders of childhood?” and “Is it necessarily the helpful enterprise that it is usually portrayed to be?”1

In this commentary we highlight examples of the mismatch between what is known and what is practised in the prevention of child mental health problems in Canada, identify some of the underlying factors that may contribute to the gap between research and practice, and make recommendations for strengthening the link between research and practice.

A number of preventive interventions have been shown in replicated randomized controlled trials (RCTs) to be effective in reducing child psychosocial problems. We briefly outline 3 exemplary prevention programs in the US and contrast these with a set of programs used in Canada.

***The Nurse–Family Partnership***. This is one of the most rigorously evaluated early-years interventions. It is an intensive US program of pre- and postnatal home visits by nurses to at-risk, first-time mothers. In 3 different RCTs the program has demonstrated multiple positive maternal and child outcomes in the short and long term. These include reductions in child abuse and neglect, in antisocial behaviour in youth and in behavioural problems related to the use of drugs and alcohol among adolescents.2,3,4 An equivalent program has not been implemented in Canada. A number of Canadian programs use the Healthy Families America model, in which home visitation is provided by trained lay staff. The home visitation component of the Healthy Babies, Healthy Children Program in Ontario, perhaps the largest in Canada, relies heavily on lay home visitors complemented by nursing visits and supervision. Unfortunately, it received no controlled evaluation prior to its province-wide dissemination.

The Healthy Families America program has shown limited effectiveness in RCTs, despite promising results under weaker research designs.5 Similarly, Olds and colleagues' recent RCT has shown that lay home visitors produced few significant improvements compared with a control group, while several positive differences were found in participants who received visits by nurses.4 Despite this, positive results of the Nurse–Family Partnership are often used to justify a variety of other home visitation programs. This is akin to suggesting that if one antibiotic is shown to be effective, another type within a related class can be assumed to work — regardless of duration, dosage or underlying mechanism.

***The Incredible Years Program.*** Community-based group parenting programs are another important strategy aimed at the early years of child development. The Incredible Years Program has been shown to improve parenting practices, parent–child interactions and child behaviour through multiple, independently replicated RCTs.6,7,8 Nevertheless, there are few offerings of this program in Canada. However, several other parenting programs are actively promoted; for example, Nobody's Perfect has been widely disseminated throughout Canada. According to Health Canada, “Nobody's Perfect has proven to be an effective program for parents who have limited access to sources of help and information.”9 In fact, the evidence for program effectiveness is inconclusive. The most rigorous evaluation we have found is a pre/post design with no control group or standardized measures.10

***The LifeSkills Training Program.*** For older children, this is arguably the most effective school-based substance abuse prevention program. It has been shown through replicated RCTs to reduce alcohol use, smoking rates and multiple substance abuse among young people.11,12,13 Despite the rigour of the evidence, we are unaware of any implementations of this program in Canada. However, many communities in Canada offer the Drug Abuse Resistance Education program (DARE), the most widely disseminated program of its kind in the United States.14 Unfortunately, there are serious concerns about the effectiveness of the DARE program, in which effect sizes for the reduction of substance abuse hover around zero.15

These examples illustrate the research–practice gap that has developed in Canada's prevention programs for child mental health problems. Although many factors are likely to have contributed to this gap, one of the most important of these is probably the strong desire to implement programs that are not resource-intensive. Yet most proven prevention programs are intensive and are provided over a relatively long period. As well, additional reinforcement may be required for already intensive programs (e.g., booster sessions, additional components, supports to reduce barriers to participation). These critical aspects increase costs and a need for long-term commitment.

Watering down effective approaches by using interveners with less training, reducing the duration and intensity of interventions, minimally investing in outreach and failing to provide support to maintain participation reduces costs. Some policy-makers appear to believe that “a little is better than nothing,” when in fact “a little” may result in no impact and no long-term savings. In contrast, more intensive and expensive evidence-based programs such as the Nurse– Family Partnership may result in long-term cost savings.16

Second, the fact that psychosocial prevention programs can do harm is rarely considered. A meta-analysis of randomized trials of juvenile awareness programs, such as “Scared Straight,” that aim to reduce juvenile delinquency by having at-risk youth meet with convicted felons, has found an increased rate of re-offending in those exposed to the intervention.17 Similar results have been found in other programs for at-risk youth.18,19 The failure to recognize this potential may decrease the pressure for rigorous evaluation.

Third, some agencies target psychosocial problems for which there are no proven-effective interventions. As one example among many, no prevention programs have been shown to reduce suicide rates among youth.20

### Recommendations

1.  Systematic efforts to search the empirical literature are needed to identify effective programs before launching a prevention effort. Although this would appear to be self-evident, present community practice suggests that this approach is not routine. University–community collaborations may be of help. The development of a technical assistance program for community-based organizations is another possibility.21

2.  New evaluations of effective programs may be required in some circumstances, if the original model is substantially altered or if the program is applied to a population that substantially differs from the one for which it was proven effective. It should be noted that all 3 programs highlighted earlier have demonstrated effectiveness in minority populations.2,13,22

3.  Innovation is needed where there are no evidence-based programs. However, rigorous evaluation should be a mandatory part of the effort. There are precedents for combining community-developed programs with high-quality evaluations.23 In addition, dissemination of a novel intervention should not occur until there is a clear indication of whether the program provides more benefit than harm.

4.  Current programs that have not been properly evaluated should not be disseminated. Although it may be too disruptive to discontinue all such programs until they are properly evaluated, at a minimum, further dissemination should be halted and specific investment in adequate evaluation undertaken. Programs found to have minimal positive impact need not be automatically abandoned. Enhancement and re-evaluation is an alternative approach. This is being attempted with some DARE programs. Such an approach may help reduce resistance to rigorous evaluation.

5.  Strategic planning around prevention programming is urgently needed. The present haphazard approach to psychosocial prevention planning for children is inadequate. Effective prevention programs will likely require substantial resources for implementation, the intervention, rigorous evaluation and replicated evaluation. Substantial leadership and extensive partnerships across multiple government levels, universities and communities are required. The National Cross-Site Evaluation of High-Risk Youth Programs in the United States may provide part of the model for such a partnership.23 This government-sponsored effort required rigorous evaluation of outcomes of a variety of university and community-agency developed models for reducing substance abuse.

Although gains have clearly been made in developing a handful of evidence-based programs, few strides appear to have been made in Canada in reaping the benefits of this empirical work. There is promise in the prevention “enterprise” but much work is needed before it will be realized in Canada.

## Footnotes

*   *Contributors:* All authors contributed to the conceptual development and drafting of this article and approved the final version.
    
    *Acknowledgements:* Dr. McLennan is supported by the Alberta Heritage Foundation for Medical Research and the Canadian Institutes of Health Research (CIHR). Dr. MacMillan is supported by the Wyeth Canada Inc. CIHR Clinical Research Chair in Women's Mental Health. We thank the CIHR Institutes of Gender and Health; Aging; Human Development, Child and Youth Health; Neurosciences, Mental Health and Addiction; and Population and Public Health, in addition to the Offord Centre for Child Studies for support.
    
    *Competing interests:* None declared.
    
    ***Correspondence to:**** Dr. Harriet MacMillan, Patterson Bldg., 2nd Flr., Chedoke Campus, McMaster University, 1200 Main St. W, Hamilton ON L8N 3Z5;* macmilnhmcmaster.ca

## References

1.  1. Rutter M. Prevention of children's psychosocial disorders: myth and substance. Pediatrics 1982;70:883-94.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6ODoiNzAvNi84ODMiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTcxLzkvMTA2OS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

2.  2. Kitzman H, Olds DL, Henderson CR Jr, Hanks C, Cole R, Tatelbaum R, et al. Effect of prenatal and infancy home visitation by nurses on pregnancy outcomes, childhood injuries, and repeated childbearing: a randomized controlled trial. JAMA 1997;278(8):644-52.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.1997.03550080054039&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9272896&link_type=MED&atom=%2Fcmaj%2F171%2F9%2F1069.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997XR54700037&link_type=ISI) 

3.  3. Olds D, Eckenrode J, Henderson CR Jr, Kitzman H, Powers J, Cole R, et al. Long-term effects of home visitation on maternal life course and child abuse and neglect: fifteen-year follow-up of a randomized trial. JAMA 1997;278:637-43.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.1997.03550080047038&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9272895&link_type=MED&atom=%2Fcmaj%2F171%2F9%2F1069.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997XR54700036&link_type=ISI) 

4.  4. Olds DL, Robinson J, O'Brien R, Luckey DW, Pettitt LM, Henderson CR Jr, et al. Home visiting by paraprofessionals and by nurses: a randomized, controlled trial. Pediatrics 2002;110:486-96.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6OToiMTEwLzMvNDg2IjtzOjQ6ImF0b20iO3M6MjE6Ii9jbWFqLzE3MS85LzEwNjkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

5.  5. Daro DA, Harding KA. Healthy Families America: using research to enhance practice. Future Child 1999;9:152-78.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.2307/1602726&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10414015&link_type=MED&atom=%2Fcmaj%2F171%2F9%2F1069.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000081126600008&link_type=ISI) 

6.  6. Scott S, Spender Q, Doolan M, Jacobs B, Aspland H. Multicentre controlled trial of parenting groups for childhood antisocial behaviour in clinical practice. Br Med J 2001;323:194-8.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjMvNzMwNi8xOTQiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTcxLzkvMTA2OS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

7.  7. Taylor T, Schmidt F, Pepler D, Hodgins C. A comparison of eclectic treatment with Webster-Stratton's Parents and Children Series in a children's mental health center: a randomized controlled trial. Behav Ther 1998;29:221-40.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0005-7894(98)80004-X&link_type=DOI) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000074136300004&link_type=ISI) 

8.  8. Webster-Stratton C, Hancock H. Training for parents of young children with conduct problems: content, methods, and therapeutic processes. In: Briesmeister J, Schaefer C, editors. *Handbook of parent training: parents as co-therapists for children's behavior problems*. 2nd ed. New York: John Wiley and Sons; 1998. p. 98-152.
    
    

9.  9. Health Canada. Family and parenting: Nobody's Perfect, 2002. Available: [www.hc-sc.gc.ca/dca-dea/family\_famille/nobody\_e.html](http://www.hc-sc.gc.ca/dca-dea/family_famille/nobody_e.html) (accessed 2004 Sep 21).
    
    

10. 10. VanderPlaat M; Martell Consulting Services Ltd. Nobody's Perfect: process and impact evaluation report. National Health and Welfare, Department of Health in New Brunswick, Nova Scotia, Prince Edward Island, Newfoundland and Labrador; 1989.
    
    

11. 11. Botvin GJ, Baker E, Dusenbury L, Tortu S, Botvin EM. Preventing adolescent drug abuse through a multimodal cognitive-behavioral approach: results of a 3-year study. *J Consult Clin* Psychol 1990;58:437-46.
    
    

12. 12. Botvin GJ, Baker E, Dusenbury L, Botvin EM, Diaz T. Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. JAMA 1995; 273:1106-12.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.1995.03520380042033&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=7707598&link_type=MED&atom=%2Fcmaj%2F171%2F9%2F1069.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1995QR05900029&link_type=ISI) 

13. 13. Botvin GJ, Griffin KW, Diaz T, Ifill-Williams M. Drug abuse prevention among minority adolescents: posttest and one-year follow-up of a school-based preventive intervention. Prev Sci 2001;2:1-13.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1023/A:1010025311161&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=11519371&link_type=MED&atom=%2Fcmaj%2F171%2F9%2F1069.atom) 

14. 14. Ringwalt CL, Ennett ST, Vincus A, Thorne J, Rohrbach LA, Simons-Rudolph A. The prevalence of effective substance use prevention curricula in US middle schools. Prev Sci 2002;3:257-65.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1023/A:1020872424136&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=12458764&link_type=MED&atom=%2Fcmaj%2F171%2F9%2F1069.atom) 

15. 15. West SL, O'Neal KK. Project D.A.R.E. outcome effectiveness revisited. Am J Public Health 2004;94:1027-9.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=15249310&link_type=MED&atom=%2Fcmaj%2F171%2F9%2F1069.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000221717700031&link_type=ISI) 

16. 16. Karoly L, Greenwood P, Everingham S, Houbé J, Kilburn MR, Rydell CP, et al. Investing in Our Children: what we know and don't know about the costs and benefits of early childhood interventions. Santa Monica (CA): RAND; 1998.
    
    

17. 17. Petrosino A, Turpin-Petrosino C, Buehler J. “Scared Straight” and other juvenile awareness programs for preventing juvenile delinquency (Cochrane Methodology Review). In: *The Cochrane Library*. Issue 4. Chichester (UK): John Wiley and Sons; 2003.
    
    

18. 18. Dishion TJ, McCord J, Poulin F. When interventions harm: peer groups and problem behavior. Am Psychol 1999;54:755-64.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1037/0003-066X.54.9.755&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10510665&link_type=MED&atom=%2Fcmaj%2F171%2F9%2F1069.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000082604300002&link_type=ISI) 

19. 19. Mahoney J, Stattin H, Magnusson D. Youth recreation centre participation and criminal offending: a 20-year longitudinal study of Swedish boys. Int J Behav Dev 2001;25:509-20.
    
    

20. 20. Shaffer D, Pfeffer C; and the Work Group on Quality Issues. Practice parameter for the assessment and treatment of children and adolescents with suicidal behavior. *J Am Acad Child Adolesc* Psychiatry 2001;40:24S-51S.
    
    

21. 21. Mitchell RE, Florin P, Stevenson JF. Supporting community-based prevention and health promotion initiatives: developing effective technical assistance systems. Health Educ Behav 2002;29:620-39.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BoZWIiO3M6NToicmVzaWQiO3M6ODoiMjkvNS82MjAiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTcxLzkvMTA2OS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

22. 22. Reid MJ, Webster-Stratton C, Beauchaine TP. Parent training in head start: a comparison of program response among African American, Asian American, Caucasian, and Hispanic mothers. Prev Sci 2001;2:209-27.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1023/A:1013618309070&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=11833925&link_type=MED&atom=%2Fcmaj%2F171%2F9%2F1069.atom) 

23. 23. SAMHSA The National Cross-Site Evaluation of High-Risk Youth Programs. Center for Substance Abuse Prevention. 2002. Available: [www.health.org/govpubs/fo36/overview.asp](http://www.health.org/govpubs/fo36/overview.asp) (accessed 2004 Sep 21).