The importance of study design in the assessment of nonnutritive sweeteners and cardiometabolic health
======================================================================================================

* John L. Sievenpiper
* Tauseef A. Khan
* Vanessa Ha
* Effie Viguiliouk
* Rodney Auyeung

Although we applaud Azad and colleagues for a comprehensive review of nonnutritive sweeteners (NNS) and cardiometabolic health,1 we are concerned that important methodological considerations were overlooked.

In interpreting the pooled analyses of randomized controlled trials (RCTs), the authors did not account for the nature of the comparator.

One would expect that the intended effect of NNS would differ depending on the amount of energy that is available to be displaced from that comparator. Because the comparator in the included RCTs was water (4 of 7),2–5 placebo (2 of 7)6,7 or a matched weight loss diet (1 of 7),8 the design did not allow for energy displacement by NNS. Therefore, the NNS in these comparisons would not be expected to result in any meaningful weight loss or attendant cardiometabolic benefits, an expectation supported by the lack of effect observed in the pooled estimates for body weight, body mass index, insulin resistance and glycosylated hemoglobin.1

On the other hand, if one focuses on those RCTs in which NNS were used to displace excess energy from sugar-sweetened beverages as the comparator, then the results do favour the intended benefits of NNS. The RCT conducted by Maersk and colleagues3 was the only eligible RCT to include a direct comparison with sugar-sweetened beverages, a comparison Azad and colleagues1 elected not to use in their pooled analyses. Although NNS in displacing excess energy from sugar-sweetened beverages did not show a significant reduction in body weight change, this comparison did show significant reductions in liver fat, systolic blood pressure, triglycerides and total cholesterol.3

Therefore, the conclusion by Azad and colleagues1 that the evidence does not support the intended benefits seems unjustified.

The authors also accorded too much weight to prospective cohort studies in their conclusions. Although prospective cohort studies offer the best protection against bias among observational studies, based on their long longitudinal follow-up, ability to adjust for multiple confounding factors and ascertainment of clinical outcomes rather than surrogate biomarkers, residual confounding in these studies cannot be ruled out. In particular, it is well documented that prospective cohort studies of NNS are at a high risk of reverse causality, because higher consumers of NNS may choose these products because they are at a higher risk for adverse cardiometabolic outcomes and not the other way around.9 The prospective design of these studies does not necessarily limit this type of confounding, despite the authors’ claims to the contrary.

The evidence presented by the authors is also subject to other important limitations including publication bias (diabetes); inconsistency (diabetes and hypertension); indirectness (hypertension, stroke, coronary heart disease and cardiovascular disease); and imprecision in the pooled estimates (diabetes, hypertension, stroke and coronary heart disease).

These uncertainties suggest that the current estimates from prospective cohort studies are not trustworthy and new research is likely to have an important impact on their direction, magnitude and precision.

Taken together, the available evidence supports the intended benefits of NNS as being similar to that of other interventions to reduce energy intake, such as water. Although prospective cohort studies have shown an adverse association with incident hypertension, stroke and cardiovascular disease, these associations are discordant with higher-quality evidence from RCTs and are at high risk of reverse causality, among other important limitations. To improve the estimates, we agree that there is a need for more long-term RCTs of interventions using NNS to reduce excess energy from sugars, especially from sugar-sweetened beverages.

## Footnotes

*   **Competing interests:** Tauseef Khan has received research support from the Canadian Institutes of Health Research (CIHR) and an unrestricted travel grant from Bee Maid Honey Ltd. Vanessa Ha has received research support from the World Health Organization (WHO). She has received a travel award and doctoral scholarship from CIHR. Effie Viguiliouk and Rodney Auyeung do not declare any competing interests. John Sievenpiper has received research support from the CIHR, Canadian Diabetes Association (CDA), PSI Foundation, Banting and Best Diabetes Centre (BBDC), Canadian Nutrition Society (CNS), American Society for Nutrition (ASN), Calorie Control Council, International Nut and Dried Fruit Council Foundation, National Dried Fruit Trade Association, The Tate and Lyle Nutritional Research Fund at the University of Toronto, and The Glycemic Control and Cardiovascular Disease in Type 2 Diabetes Fund at the University of Toronto (a fund established by the Alberta Pulse Growers). He has received speaker fees and/or honoraria from the CDA, CNS, Dr. Pepper Snapple Group, Dairy Farmers of Canada, Nutrition Foundation of Italy, C3 Collaborating for Health, Sprim Brasil, White-Wave Foods, Rippe Lifestyle, mdBriefcase, Alberta Milk, FoodMinds LLC, Memac Ogilvy & Mather LLC, PepsiCo, The Ginger Network LLC, International Sweeteners Association, and Pulse Canada. He has ad hoc consulting arrangements with Winston & Strawn LLP, Perkins Coie LLP, and Tate & Lyle. He is a member of the European Fruit Juice Association Scientific Expert Panel. He is on the Clinical Practice Guidelines Expert Committees of the CDA, European Association for the Study of Diabetes (EASD) and Canadian Cardiovascular Society (CCS), as well as an expert writing panel of the ASN. He serves as an unpaid scientific advisor for the Food, Nutrition, and Safety Program and the Technical Committee on Carbohydrates of the International Life Science Institute North America. He is a member of the International Carbohydrate Quality Consortium, Executive Board Member of the Diabetes and Nutrition Study Group of the EASD, and Director of the Toronto 3D Knowledge Synthesis and Clinical Trials foundation. His wife is an employee of Unilever Canada. No other competing interests were declared.

## References

1.  Azad MB, Abou-Setta AM, Chauhan BF, et al. Nonnutritive sweeteners and cardiometabolic health: a systematic review and meta-analysis of randomized controlled trials and prospective cohort studies. CMAJ 2017;189:E929–39.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTg5LzI4L0U5MjkiO3M6NDoiYXRvbSI7czoyMzoiL2NtYWovMTg5LzQ2L0UxNDI0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

2.  Tate DF, Turner-McGrievy G, Lyons E, et al. Replacing caloric beverages with water or diet beverages for weight loss in adults: main results of the Choose Healthy Options Consciously Everyday (CHOICE) randomized clinical trial. Am J Clin Nutr 2012;95:555–63.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpjbiI7czo1OiJyZXNpZCI7czo4OiI5NS8zLzU1NSI7czo0OiJhdG9tIjtzOjIzOiIvY21hai8xODkvNDYvRTE0MjQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

3.  Maersk M, Belza A, Stødkilde-Jorgensen H, et al. Sucrose-sweetened beverages increase fat storage in the liver, muscle, and visceral fat depot: a 6-mo randomized intervention study. Am J Clin Nutr 2012;95:283–9.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpjbiI7czo1OiJyZXNpZCI7czo4OiI5NS8yLzI4MyI7czo0OiJhdG9tIjtzOjIzOiIvY21hai8xODkvNDYvRTE0MjQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

4.  Madjd A, Taylor MA, Delavari A, et al. Effects on weight loss in adults of replacing diet beverages with water during a hypoenergetic diet: a randomized, 24-wk clinical trial. Am J Clin Nutr 2015; 102: 1305–12.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpjbiI7czo1OiJyZXNpZCI7czoxMDoiMTAyLzYvMTMwNSI7czo0OiJhdG9tIjtzOjIzOiIvY21hai8xODkvNDYvRTE0MjQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

5.  Peters JC, Beck J, Cardel M, et al. The effects of water and non-nutritive sweetened beverages on weight loss and weight maintenance: a randomized clinical trial. Obesity (Silver Spring) 2016; 24:2 97–304.
    
    

6.  Hsieh MH, Chan P, Sue YM, et al. Efficacy and tolerability of oral stevioside in patients with mild essential hypertension: a two-year, randomized, placebo-controlled study. Clin Ther 2003;25:2797–808.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0149-2918(03)80334-X&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=14693305&link_type=MED&atom=%2Fcmaj%2F189%2F46%2FE1424.atom) 

7.  Ferri LA, Alves-Do-Prado W, Yamada SS, et al. Investigation of the antihypertensive efect of oral crude stevioside in patients with mild essential hypertension. Phytother Res 2006;20:732–6.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1002/ptr.1944&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=16775813&link_type=MED&atom=%2Fcmaj%2F189%2F46%2FE1424.atom) 

8.  Blackburn GL, Kanders BS, Lavin PT, et al. The effect of aspartame as part of a multidisciplinary weight-control program on short- and long-term control of body weight. Am J Clin Nutr 1997;65: 409–18.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpjbiI7czo1OiJyZXNpZCI7czo4OiI2NS8yLzQwOSI7czo0OiJhdG9tIjtzOjIzOiIvY21hai8xODkvNDYvRTE0MjQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

9.  Scientific report of the 2015 Dietary Guidelines Committee Advisory Committee. Washington: US Department of Health and Human Services and US Department of Agriculture; 2015. Available: [https://health.gov/dietaryguidelines/2015-scientific-report/PDFs/Scientific-Report-of-the-2015-Guidelines-Advisory-Committee.pdf](https://health.gov/dietaryguidelines/2015-scientific-report/PDFs/Scientific-Report-of-the-2015-Guidelines-Advisory-Committee.pdf) (accessed 2017 Oct. 4).