RE: Response to Cimolai: Quality of systematic reviews and COVID-19
References
1. Wei SQ, Bilodeau-Bertrand M, Liu S, Auger N. The impact of COVID-19 on pregnancy outcomes: a systematic review and meta-analysis.CMAJ 2021;10.1503/cmaj.202604.
2. Deeks JJ, Higgins JPT, Altman DG. Analysing data and undertaking meta-analyses. Cochrane Handbook for Systematic Reviews of Interventions.Cochrane 2021.
3. Ang XL, Chonkar SP, Chua MSQ, Sulaiman S, Lee JCS. Problems with early systematic reviews: the case of coronavirus disease 2019 (COVID-19) in pregnancy. Matern Child Health J 2021; 25:38-41.
4. Allotey J, Stallings E, Bonet M, Yap M, Chatterjee S, Kew T, et al. Clinical manifestations, risk factors, and maternal and perinatal outcomes of coronavirus disease 2019 in pregnancy: living systematic review and meta-analysis. BMJ 2020;370:m3320.
5. Chmielewska B, Barratt I, Townsend R, Kalafat E, van der Meulen J, Gurol-Urganci I, et al. Effects of the COVID-19 pandemic on maternal and perinatal outcomes: a systematic review and meta-analysis. Lancet Glob Health 2021 Mar 31.
We thank Dr. Cimolai for commenting on our meta-analysis of COVID-19 in pregnancy.(1) Dr. Cimolai raised a number of concerns with the meta-analysis, including the risk of ‘integrating cumulative disperse data’, ‘heterogeneity in largely unrandomized or especially observational studies’, and ‘confounding variables in diversely accumulated studies’.
The methodology used in this systematic review followed standardized criteria meant to mitigate the problems raised by Dr. Cimolai. The entire purpose of meta-analysis is to integrate disperse data in order to provide more precise effect estimates and settle controversies between studies.(2) The current systematic review followed PRISMA guidelines and provides an accurate picture of the existing literature on COVID-19 in pregnancy. While it is true that Ang et al urged caution in early literature reviews of COVID-19,(3) the limitations raised by Ang et al pertained to case reports and case series and do not apply to the current meta-analysis of cohort and case-control studies.
Dr. Cimolai expressed concern that heterogeneity in unrandomized observational studies is a risk to analysis. Heterogeneity in meta-analysis refers to the variation in outcomes between studies and is not related to the lack of randomization.(2) The explicit advantage of meta-analysis is its ability to quantify and assess heterogeneity between studies. Without meta-analysis, it is not possible to know if heterogeneity even exists. In our study, there was no heterogeneity for outcomes such as preeclampsia and stillbirth. While heterogeneity was greater for some outcomes, sensitivity analyses indicated that there was little effect on the overall interpretation of studies. The claim that ‘amalgamating such studies attracts another level of ambiguity’ is therefore inaccurate.
We acknowledge that confounding remains an important problem in the current literature on COVID-19. As discussed in our paper, the existing data did not allow us to adjust associations for confounders. Recent systematic reviews in BMJ and Lancet were also limited by this problem.(4,5) However, many of the issues raised by Dr. Cimolai pertain to misclassification, not confounding. For instance, use of clinical versus laboratory-confirmed diagnoses results in misclassification of COVID-19 status. Nonstandard definitions of preeclampsia and preterm birth result in outcome misclassification. This distinction is important because misclassification frequently attenuates results, whereas confounders have a less easily predictable effect.
An enormous number of studies on COVID-19 are published daily and meta-analysis is currently the best available scientific tool to synthesize the ever-growing literature. Limitations in the current body of evidence should not be a deterrent, but rather an incentive for continuous updates to systematic literature reviews for timely decision-making. Meta-analysis will be essential in global efforts to tackle the pandemic.