In response to the sponsor’s (Pfizer’s) comments1 about our analysis,2 we reiterate the statement in the safety review by the US Food and Drug Administration (FDA): “The serious adverse event data suggest that varenicline may increase the risk of cardiac events, both ischemic and arrhythmic, particularly over longer treatment periods.”3 Because the sponsor never conducted adequately powered trials to determine whether varenicline increases the risk of cardiovascular events, a meta-analysis was conducted using available data.
It is true that the numbers of events in our analysis were small. However, that is because most of the trials were small and therefore underpowered to determine drug safety. Nevertheless, in spite of the small numbers, the difference in treatment groups was statistically significant. When statistically significant differences exist, the number of events is less relevant. A small number of events requires a larger difference to reach statistical significance.
We adhered to the intention-to-treat (ITT) analysis and counted events throughout the scheduled duration of each trial. This is in accordance with FDA regulations and established and generally accepted scientific principles. Athough pharmaceutical companies might prefer to analyse data by treatment level, such an analysis would allow exclusion of events occurring in randomized patients. This practice to make unfavourable findings appear more favourable is subject to potentially serious bias. In addition, by adhering to the ITT principles, noncompliance becomes a nonissue. Thus, the higher drop-out rate in the placebo group is irrelevant.
Our statistical analysis dealt with the trials with no events. According to the Cochrane handbook, “The standard practice in meta-analysis of odds ratios and risk ratios is to exclude studies from the meta-analysis where there are no events in both arms. This is because such studies do not provide any indication of either the direction or magnitude of the relative treatment effect.”4 Sensitivity analyses with the fixed Mantel–Haenszel approach showed similar results.
We wanted to analyse the serious adverse event data by time after randomization. This was not possible because most trials failed to report time to the event. If the harmful effects of varenicline are limited to the period of active treatment, the data from the post-treatment phase would dilute these effects. In addition, any reduction in cardiovascular risk attributed to varenicline would also lead to an underestimate of the on-treatment harm. Post-hoc adjudication of investigator-reported cardiovascular events can only lead to reduction in events and statistical power.
It would be unfortunate if the FDA’s proposed review of the safety of varenicline3 were to be handled by the sponsor or its paid consultants. The scientific literature is replete with examples in which such conflicts of interest have impeded independent and fair analysis of drug safety.