Abstract
Background: Acute kidney injury after cardiac surgery is associated with adverse in-hospital and long-term outcomes. Novel risk factors for acute kidney injury have been identified, but it is unknown whether their incorporation into risk models substantially improves prediction of postoperative acute kidney injury requiring renal replacement therapy.
Methods: We developed and validated a risk prediction model for acute kidney injury requiring renal replacement therapy within 14 days after cardiac surgery. We used demographic, and preoperative clinical and laboratory data from 2 independent cohorts of adults who underwent cardiac surgery (excluding transplantation) between Jan. 1, 2004, and Mar. 31, 2009. We developed the risk prediction model using multivariable logistic regression and compared it with existing models based on the C statistic, Hosmer–Lemeshow goodness-of-fit test and Net Reclassification Improvement index.
Results: We identified 8 independent predictors of acute kidney injury requiring renal replacement therapy in the derivation model (adjusted odds ratio, 95% confidence interval [CI]): congestive heart failure (3.03, 2.00–4.58), Canadian Cardiovascular Society angina class III or higher (1.66, 1.15–2.40), diabetes mellitus (1.61, 1.12–2.31), baseline estimated glomerular filtration rate (0.96, 0.95–0.97), increasing hemoglobin concentration (0.85, 0.77–0.93), proteinuria (1.65, 1.07–2.54), coronary artery bypass graft (CABG) plus valve surgery (v. CABG only, 1.25, 0.64–2.43), other cardiac procedure (v. CABG only, 3.11, 2.12–4.58) and emergent status for surgery booking (4.63, 2.61–8.21). The 8-variable risk prediction model had excellent performance characteristics in the validation cohort (C statistic 0.83, 95% CI 0.79–0.86). The net reclassification improvement with the prediction model was 13.9% (p < 0.001) compared with the best existing risk prediction model (Cleveland Clinic Score).
Interpretation: We have developed and validated a practical and accurate risk prediction model for acute kidney injury requiring renal replacement therapy after cardiac surgery based on routinely available preoperative clinical and laboratory data. The prediction model can be easily applied at the bedside and provides a simple and interpretable estimation of risk.
Acute kidney injury is one of the most serious complications of cardiac surgery. The development of acute kidney injury requiring renal replacement therapy is associated with increased duration of mechanical ventilation,1 prolonged stay in intensive care and in hospital,2 and high short- and long-term mortality.3
Although clinical and perioperative risk factors for severe acute kidney injury have been identified, accurate preoperative risk stratification remains challenging. Several renal risk-scoring systems exist;4–8 however, there are a number of limitations to their adoption, including the number and complexity of variables required, poor discrimination among patients at highest risk and the frequent inclusion of perioperative data (which precludes use of the scoring systems as preoperative risk stratification tools).
Several new predictors of postoperative acute kidney injury have recently been identified. Proteinuria has been recognized as a powerful independent predictor in both surgical and nonsurgical settings in addition to a reduced estimated glomerular filtration rate.9,10 Anemia has also long been known to be a risk factor for acute kidney injury, but it has not been incorporated into previous risk scores for patients undergoing cardiac surgery.11,12 In addition, no risk prediction tools for acute kidney injury have been evaluated for their association with long-term outcomes of acute kidney injury, including 1-year mortality, and risk of end-stage renal disease, all of which are highly relevant to patients and their caregivers.
We derived and validated a practical risk prediction model for acute kidney injury requiring renal replacement therapy after cardiac surgery based on routinely available preoperative clinical and laboratory variables.
Methods
Data sources
We obtained all clinical variables from the Alberta Provincial Project for Outcome Assessment in Coronary Heart Disease (APPROACH), a prospective data-collection initiative that has been collecting detailed clinical and treatment information on all patients who undergo cardiac catheterization in the province of Alberta since 1995.13 When information about comorbid conditions was unavailable from APPROACH, we obtained it from hospital and claims data (3-yr look back) using validated algorithms (see Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151447/-/DC1).14 We obtained laboratory data from the Alberta Kidney Disease Network, which maintains a data set of all creatinine, proteinuria and hemoglobin values measured in the province. With the exception of serum creatinine, all laboratory values were assessed within 6 months before surgery; if more than 1 value was available, we used the value closest to the surgery date.
Derivation cohort
The derivation cohort included all adults (age ≥ 18 yr) in Alberta who underwent cardiac surgery (excluding transplantation) in Calgary between Jan. 1, 2004, and Mar. 31, 2009. To be eligible, patients had to have had at least 1 preoperative serum creatinine measurement within 3 months before hospital admission and at least 1 measurement during the hospital stay to assess baseline and subsequent changes in renal function. We excluded patients who had end-stage renal disease (estimated glomerular filtration rate [eGFR] < 15 mL/min per 1.73 m2, chronic dialysis or prior kidney transplantation) at baseline. All patients were followed from the date of cardiac surgery (index date) until Mar. 31, 2011.
Validation cohort
The validation cohort included all adults in Alberta who underwent cardiac surgery in Edmonton between Jan. 1, 2004, and Mar. 31, 2009. The exclusion and inclusion criteria were the same as those for the derivation cohort.
Assessment of baseline kidney function
To reduce variation in eGFR between provincial laboratories, serum creatinine measurements were standardized to an isotope dilution mass spectrometry reference standard, and a laboratory-specific correction factor was applied where necessary.15 We determined baseline kidney function by means of the Chronic Kidney Disease Epidemiology Collaboration equation16 using the most recent outpatient serum creatinine measurement within 3 months before surgery. In a sensitivity analysis, we determined baseline kidney function using the most recent outpatient measurement taken within 7 days before surgery.17
Candidate predictors
We identified candidate independent variables based on face validity and inclusion in previous risk scores for acute kidney injury.
Clinical variables included age, sex, diabetes, body mass index (BMI), history of cardiovascular disease, cerebrovascular disease, Canadian Cardiovascular Society angina score,18 congestive heart failure (and New York Heart Association class if congestive heart failure was present), peripheral vascular disease, malignant disease and Charlson Comorbidity Index score.
Laboratory variables included preoperative eGFR, hemoglobin and proteinuria. Proteinuria was defined as a urine dipstick test result of 1+ or greater, an albumin:creatinine ratio greater than 3.4 mg/mmol or a protein:creatinine ratio greater than 17.0 mg/mmol. Information from preceding cardiac catheterization included indication for catheterization, left ventricular ejection fraction, Duke Jeopardy Score (reflecting burden of coronary artery disease)19 and time from catheterization to surgery date. Information about the cardiac surgery included type of proposed surgery and booking status (elective, urgent or emergent).
Outcome measures
The primary outcome was acute kidney injury treated with renal replacement therapy. We identified acute kidney injury using postoperative serum creatinine values and the KDIGO (Kidney Disease: Improving Global Outcomes) definition:20 an increase in serum creatinine of 26.5 μmol/L within 48 hours after surgery or an increase at least 1.5 times the baseline measurement within 7 days after surgery. We determined the requirement for acute dialysis within 14 days after cardiac surgery using a validated algorithm, based on diagnosis and procedural administrative codes with high sensitivity and specificity.21
To determine the validity of the model to predict downstream, clinically important outcomes associated with acute kidney injury requiring renal replacement therapy, we evaluated the accuracy of the model in predicting the following secondary outcomes: length of stay in hospital, 90-day mortality, 1-year mortality and 2-year risk of end-stage renal disease. Postoperative outcomes were determined from the date of surgery to Mar. 31, 2011 (study end).
Statistical analysis
All statistical analyses were performed using STATA/MP software.
Development of risk prediction model
We used multivariable logistic regression to determine the independent association of candidate variables with acute kidney injury requiring renal replacement therapy. All variables were entered into models, and a bootstrapping technique was used to select candidate variables.22 Bootstrapping was performed on 200 random samples drawn with replacement from the derivation cohort; each bootstrap sample was the same size as the derivation sample. Candidate variables were then ranked according to the proportion of bootstrap samples in which they were identified as significant predictors. We selected variables that were significant predictors of acute kidney injury requiring renal replacement therapy in more than 50% of the bootstrap samples for the bootstrapped model. We used stepwise backward elimination to select variables for inclusion in the models based on a p value of less than 0.1.
We used integrated discrimination improvement to assess the contribution of individual variables to the overall estimate of risk, and to develop the final risk prediction model. We used the method by Sullivan and colleagues23 to transform coefficients from the final logistic model into an integer-based risk index.
Validation of risk prediction model
The final risk prediction model was internally validated using a bootstrap resampling method, in which we evaluated the model’s performance using the original data to develop the model. Two hundred samples, of the same size as the derivation cohort, were drawn randomly with replacement from the derivation cohort, and measures of the prediction model’s performance were determined from the bootstrapped samples.
The performance of the risk prediction model was assessed in the derivation cohort and external validation cohorts. We assessed discrimination using the concordance (C) statistic, which is identical to the area under a receiver operating characteristic curve (sensitivity v. 1 – specificity). We compared the performance of the model with the 3 best-performing existing risk scoring tools (Cleveland Clinic score,4 Simplified Renal Index5 and Society of Thoracic Surgeons scoring system6) using variables that were recalibrated in the validation cohort.
We assessed calibration by comparing the observed versus predicted rates of acute kidney injury for each decile of risk and by using the Hosmer–Lemeshow test24,25 and the calibration slope (ideal calibration slope is 1).26,27
We used the Net Reclassification Improvement index to measure the net gain in risk reclassification compared with the best existing model.28 The risk category thresholds used for comparison were determined on the basis of clinically relevant differences in risk and from risk categories (low, medium and high) in previously published risk models.29
Missing data
We imputed missing data for Canadian Cardiovascular Society angina scores, hemoglobin concentration and surgery booking status using single imputation techniques (missing value estimated using multivariable regression). Missing data for proteinuria, left ventricular ejection fraction and Duke Jeopardy Score were modelled as unknown categories. When the New York Heart Association functional classification was unknown, we considered it to be class IV if congestive heart failure was present. We repeated the analyses after excluding records with missing data to test the robustness of the results.
Ethics approval
The study design was reviewed and approved by the Human Research Ethics Board at the University of Alberta in Edmonton and the Conjoint Health Research Ethics Board at the University of Calgary in Calgary.
Results
We included 6061 patients in the derivation cohort and 4467 in the validation cohort (Table 1, Figure 1). Compared with patients in the derivation cohort, those in the validation cohort had fewer comorbid conditions such as obesity, smoking, vascular disease, pulmonary and chronic kidney disease; were less likely to have severe coronary artery disease or previous cardiac surgery; and were more likely to have an inpatient urgent or emergent booking status. The cohorts differed by type of surgery, with more patients in the validation cohort undergoing coronary artery bypass grafting and valve surgery. The overall incidence of acute kidney injury requiring renal replacement therapy after cardiac surgery was similar in the derivation and validation cohorts (2.5% and 3.1%, respectively; p = 0.1).
Development of risk prediction model
Table 2 shows the candidate variables and their associated odds ratios in the bootstrap model and the final prediction model. In addition to risk factors previously identified in other risk models for predicting acute kidney injury, the bootstrapping model also identified obesity, baseline eGFR, preoperative hemoglobin concentration and proteinuria as independent predictors of acute kidney injury after cardiac surgery.
The bootstrapping technique identified a total of 12 variables for the risk prediction model. Minimally informative variables were then sequentially eliminated one at a time starting from the least significant variable; the performance of reduced models was sequentially compared until all variables were significant (p < 0.05) in the final model (Appendix 2, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151447/-/DC1). A reduction from 12 to 8 variables decreased the model’s C statistic from 0.88 to 0.87. The integrated discrimination improvement associated with each deleted variable is shown in Appendix 2. All of the models had good calibration. The final model consisted of 8 variables: congestive heart failure, Canadian Cardiovascular Society class III or IV, diabetes, baseline eGFR, preoperative hemoglobin concentration, proteinuria, type of cardiac surgery and surgery booking status.
The integer-based risk scores and the corresponding risk table are presented in Figure 2 and Appendix 3 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151447/-/DC1). The corresponding risk estimates associated with each score grouping, and the frequency of all measured outcomes in each risk category, are presented in Appendix 4 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151447/-/DC1). Patients in the highest risk category not only had the highest mortality but also the longest duration of hospital stay and the highest risks of short- and long-term mortality and end-stage renal disease.
Performance of risk prediction model
Figure 3 shows the predicted versus observed probability of acute kidney injury requiring renal replacement therapy based on application of the risk prediction model in the derivation and validation cohorts. The C statistics and Hosmer–Lemeshow goodness-of-fit test results for the prediction model and the 3 existing models4–6 are shown in Table 3. Despite recalibration of the 3 existing models within the derivation cohort, our prediction model had the best performance (C statistic 0.87 in the derivation cohort and 0.83 in the validation cohort). Calibration within the derivation cohort was good (p = 0.7 in Hosmer–Lemeshow goodness-of-fit test for our model); calibration was poor for all of the models in the validation cohort.
The net gain in risk reclassification of patients to high- and low-risk strata when we compared the prediction model with the best existing model (Cleveland Clinic Score) is shown in Appendix 5 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.151447/-/DC1). The net reclassification improvement was 13.9% (p < 0.001), with a net proportion of 16.1% events assigned to a higher risk category and a net proportion of 2.2% of non-events assigned a lower risk category. The reclassification improvement was primarily seen among patients who required dialysis, who were more often appropriately reallocated to higher risk categories using the derived model.
Sensitivity analysis
The performance of the model was unchanged when we used the serum creatinine level measured only within 7 days of hospital admission to estimate baseline eGFR, or when we excluded patients with missing data from the cohort before model generation.
Interpretation
We have developed and validated a simple 8-variable preoperative risk prediction model to determine the risk of acute kidney injury requiring renal replacement therapy after cardiac surgery. The model incorporates routinely available clinical and laboratory information, including several recently identified risk factors for acute kidney injury. The prediction model identified the need for acute dialysis after cardiac surgery according to a validated consensus definition and performed as well as or better than existing models.
Among existing prediction models, the most robust for predicting acute kidney injury after cardiac surgery have been those that predict the need for dialysis using combinations of preoperative, intraoperative and postoperative information.4–7,30 However, they consistently underestimate the risk when applied in external cohorts, and risk stratification between low-, medium- and high-risk groups has not been clinically relevant.29 We found that our prediction model performed as well as or better than the 3 best-performing risk scoring tools available for identifying acute kidney injury requiring dialysis in both the derivation and the validation cohorts. Our model uses clinical and laboratory information routinely obtained in preoperative settings, and it provides an easily understandable estimate of risk. Furthermore, our prediction model showed a net improvement of reclassification of patients to clinically relevant high- and low-risk groups, largely gained through the reclassification of patients in with acute kidney injury requiring renal replacement therapy to higher risk categories, thereby directly addressing a challenge with existing scores.
As expected, we observed a slight decrease in the performance of our risk prediction model in the validation cohort. Analysis of the baseline patient characteristics between study centres suggests some important differences between the cohorts in both patient selection and processes of care, including types of surgery and surgery booking status. Variability in these subjective decisions may have accounted for the slight differences observed in model performance between the 2 cohorts.
Limitations
We were limited to using available data; however serum creatinine and hemoglobin concentrations are routinely assessed preoperatively, and more than 99% of patients in our cohorts had these laboratory measurements available. Urinalysis for quantification of proteinuria is not routinely done before cardiac surgery, and about 25% of our cohort did not have urinalysis results available. However, eliminating patients who had missing data did not substantially alter the coefficients obtained for proteinuria in these models. Finally, both the derivation and the validation cohorts were from Alberta. It is possible that population characteristics and decisions regarding candidacy for surgical revascularization, and the decision to start renal replacement therapy, may differ between health care systems, ethnic groups and countries. Prospective multicentre validation studies are needed in non-Canadian settings to determine the broader generalizability of our risk scoring system.
Conclusion
We have developed and validated a practical and accurate risk prediction model for acute kidney injury requiring renal replacement therapy after cardiac surgery based on routinely available preoperative clinical and laboratory data. Our prediction model differs from other risk scoring tools in a number of ways: it uses a small number of variables that may be easily measured preoperatively, and it incorporates routinely collected laboratory measurements as risk factors. Most importantly, it provides a simple and interpretable estimation of risk. The score can be easily calculated at the bedside and may be an effective tool for communicating risk to patients. It could also help patients and care providers in decision-making and be used to identify high-risk patients for future interventional studies.
Footnotes
Competing interests: None declared.
This article has been peer reviewed.
Contributors: Neesh Pannu was responsible for the study concept and design, and data acquisition and analysis. Matthew James was responsible for the study design and data analysis. Feng Ye was responsible for the cleaning and linkage of data, the study methodology and data analysis. Steven Meyer and Teresa Kieser contributed to the study concept and design and to data analysis. Michelle Graham, Brenda Hemmelgarn and Scott Klarenbach were involved wih data analysis and interpretation. Neesh Pannu and Matthew James drafted the manuscript; all of the authors revised it critically for important intellectual content, approved the final version to be published and agreed to act as guarantors of the work.
Disclaimer: This study is based in part on data provided by Alberta Health and Alberta Health Services. The interpretation and conclusions contained herein are those of the researchers and do not necessarily represent the views of the Government of Alberta or Alberta Health Services. Neither the Government of Alberta nor Alberta Health or Alberta Health Services express any opinion in relation to this study.
Funding: This study was funded through an operating grant from the Canadian Institutes of Health Research awarded to Neesh Pannu. The Alberta Kidney Disease Network is funded through a Collaborative Research and Innovation Opportunities team grant from Alberta Innovates — Health Solutions (Interdisciplinary Chronic Disease Collaboration).
- Accepted February 22, 2016.