Validity of utilization review tools
====================================

* Peter Dodek
* Barbara Trerise
* C. Brian Warriner

Norman Kalant and colleagues conclude that utilization review tools are not valid to assess appropriateness of setting.1 We argue that this conclusion is not supported for several reasons. First, and most importantly, acute care review tools assume that subacute care and acute care are separate, discrete levels. With only minor exceptions, in Canada subacute care is normally and appropriately delivered within the acute care setting. One report referred to by the authors specifically makes this point in terms of the structure of the Ontario health system.2 This oversight alone is likely to account for a significant proportion of the mismatch between the review tools for acute care (without including subacute care) and the opinion of a panel of experts.

The authors base their conclusions on a simple kappa statistic. However, they have not adjusted for the nonindependent nature of the days of stay of the 75 patients reviewed. Our work has demonstrated a 30% correlation between the appropriateness of 1 day of stay and appropriateness of the subsequent day (unpublished data). Nonindependence of observations may amplify disagreement as measured by the kappa statistic.

Finally, the authors claim that no previous studies have validated these tools by comparison with implicit review by a panel of physicians. In fact 2 published studies used physician panels to demonstrate validity.3,4

## References

1.  1. Kalant N, Berlinguet M, Diodati JG, Dragatakis L, Marcotte F. How valid are utilization review tools in assessing appropriate use of acute care beds? CMAJ 2000;162(13):1809-13.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTYyLzEzLzE4MDkiO3M6NDoiYXRvbSI7czoyNDoiL2NtYWovMTYzLzEwLzEyMzguMi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

2.  2. Utilization Steering Committee, Joint Policy and Planning Committee. Non-acute hospitalization project - final report. Toronto: Ontario Ministry of Health; 1997. Reference document RD6-3.
    
    

3.  3. Strumwasser I, Paranjpe NV, Ronis DL, Share D, Sell LJ. Reliability and validity of utilization review criteria. Med Care 1990;28:95-109.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1097/00005650-199002000-00001&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=2299897&link_type=MED&atom=%2Fcmaj%2F163%2F10%2F1238.2.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1990CN67200001&link_type=ISI) 

4.  4. Inglis AL, Coast J, Gray SF, Peters TJ, Frankel S. Appropriateness of hospital utilization. Med Care 1995;9:952-7.