Application of Machine Learning Algorithms to Predict Acute Kidney Injury in Elderly Orthopedic Postoperative Patients

Background

Acute kidney injury (AKI) is an ordinary post-surgical complication characterized by a sharp decline in glomerular filtration rate within a short period, rapid accumulation of metabolites in the body, and a clinical syndrome caused by a severe disorder of the body’s internal environment. With increasing age, the physiological reserves of elderly patients are further reduced, which increases the proportion of postoperative AKI.¹

Based on previous data, the incidence of AKI after orthopedic surgery fluctuates between 6.7% ~10.8%. The times requires for orthopedic surgery and anesthesia are extended. The body’s stress response, intraoperative and postoperative body position changes, and greater bleeding risk, combined with older patients’ renal conditions themselves, are poor and often accompanied by major heart blood vessel and high blood pressure problems. Thus, such surgeries carry a high surgical risk and complication rate because of the wide range of tissue dissections and large wound surfaces; increasing the possibility of postoperative AKI.² Orthopedic surgery increases the incidence of renal-related complications and mortality, which may increase hospitalization time and costs and even increase long-term mortality.^3,4 Ideal individual-based treatment decisions should replace the “one-size-fits-all” approach suggested in the guidelines. Such generalized decision-making highlights the importance of accurately predicting AKI events with a more personalized therapeutic schedule. Studies have found that the occurrence of postoperative AKI is closely related to patient age, primary diseases, and surgical causes.⁵ This study explored the incidence and risk of AKI in elderly patients after major orthopedic surgery.

Machine learning (ML) as a new type of artificial intelligence (AI) is beginning to be widely used in medical data analysis.⁶ Using the robust predictive ability of ML algorithms, it is possible to develop predictive tools that are superior to traditional statistical modeling in some cases to better predict AKI. ML has been widely applied in other clinical setting such as ICU.^7–10 Unfortunately, there is no research to train ML algorithms to predict AKI in elderly orthopedic patients. Therefore, the purpose of this study is to use the preoperative and intraoperative clinical characteristics to establish a ML-based prediction model, predict the possibility of individualized treatment of AKI, and establish a nomogram to visualize the prediction model.

Materials and Methods

Study Population and Design

This retrospective study was based on the Electronic Medical Record system of patients admitted to the inpatient Department of Anesthesiology of the Affiliated Hospital of Xuzhou Medical University. 1000 elderly patients who had undergone orthopedic surgery from September 2016, to June, 2021 were included finally. The Ethics Committee approved this study of local institutions (Ethics Committee of Affiliated Hospital of Xuzhou Medical University). The requirement for informed consent waived, because this retrospective trial was limited to preexisting data without intervention. We confirm that the data was anonymized or maintained with confidentiality and our study complies with the Declaration of Helsinki. Eligible participants aged over 60 years. Qualified surgical methods include the following: total hip replacement, lumbar laminectomy. Exclusion criteria for clinical data missing. Emergency surgery patients. Patients with severe liver and kidney dysfunction. Combined with severe heart failure, arrhythmia and other cardiac function diseases. Patients with secondary surgery and hip revision.

Data Collection

Participants were assessed at baseline within one week before surgery. Clinicians reviewed the medical records to collect information about the operation process, demographic data, basic disease history, general perioperative conditions, related laboratory tests, surgery type and duration, baseline diagnosis of AKI. The baseline characteristic data, including patient demographics (age, gender, and body mass index (BMI)) and preexisting comorbidities (diabetes mellitus, hypertension, hypoproteinemia, anemia, myocardial infarction (MI)), were utilized to train the AKI predictive model. From this information set, we identify features for our prediction model. Potential predictors need to be feasible in a busy clinical setting through existing sources (such as medical records or standard laboratory data) or through rapid screening tests. We refer to these variables as the full feature set. Some study variables have been previously assessed, including age,^11,12 preoperative renal function,^12–15 preoperative medications,^12,14,16 preoperative hemoglobin,¹⁷ American Society of Anesthesiologists (ASA) Physical Status classification,^11,14,15 and surgical procedure type.^12,14,18 Previous studies have reported that intraoperative blood loss and transfusion, intraoperative hemodynamic fluctuations, especially the duration of hypotension, intraoperative fluid replacement, etc., may be associated with AKI after orthopedic surgery in elderly patients. These risk factors have been proved to promote the occurrence and development of AKI to a certain extent. The baseline serum creatinine level was defined according to the preoperative serum creatinine level closest to the beginning of surgery.

Diagnosis of AKI

This study used AKI diagnosis standard based on the current international guidelines for Kidney Disease: Improving Global Outcomes (KDIGO). The KDIGO guidelines include the respective advantages of the RIFLE standard developed by the previous acute dialysis quality initiative and the AKIN standard developed by the acute kidney injury network working group, so as to achieve the purpose of early diagnosis of AKI. Since postoperative urine volume is affected by many factors, it is difficult to record in retrospective study. In this study, serum creatinine was only used as an indicator for the diagnosis of postoperative AKI.¹⁹ One of the following conditions can be diagnosed AKI: serum creatinine increased ≥ 26.5μmol/L within 48h. Serum creatinine levels increased by 1.5 times or more than the baseline and were clearly or inferred to occur within 7 days.

Statistical Analyses

Fisher’s exact test and the Mann–Whitney U-test were used for the discrete and continuous variables, respectively. To discover the predictors determining AKI, all the predictors were foremost examined for multicollinearity applying the variance inflation factor (VIF) and then enrolled into the model applying multivariable logistic regression analysis. The odds ratio (OR) of independent risk factors for AKI were estimated at 95% confidence intervals (CI) using the stepwise selection method.

ML algorithms are characterized in accordance with their extraordinary performances and perform considerably than conventional regression gets access to in predicting results from large datasets.^20,21 We produced nine types of ML algorithms to model our information: extreme gradient boost (XGBoost), logistic, light gradient boosting machine (LightGBM), randomforest, adaptive boost (AdaBoost), gaussian naive bayes (GausianNB), multi-layer perceptron (MLP), support vector machine (SVC) and k-nearest neighbor (KNN). Of the whole samples, 80% were at random chose to train the models, when 20% were applied for test. For ML-based models, adjustments are considered throughout the training process to prevent overfitting. The best hyperparameter of ML models is 5-fold cross validation. Then according to R application further training ML algorithms to predict the risk of AKI. We evaluated the prediction power of each ML classifier using equivalent hyperparameters on the test sets, and calculated the area under receiver operating characteristic (AUC) and the corresponding sensitivity and value, specificity and overall accuracy of ML algorithms. While comparing the performance of the ML algorithms, an AUC that was nearer to 1 showed a more excellent classification model performance. The risk factors screened by the optimal algorithm were selected, and the nomogram was further drawn. The concordance statistic and calibration curve were used to discriminate and calibrate the nomogram respectively. All tests were two-tailed, and a P value <0.05 was considered statistically significant. All the statistical analyses were executed employing the statistical software package R, version 4.0.4 and SPSS version 16.0 (SPSS Inc., Chicago, IL).

Results

Baseline Patient Characteristics

Based on a review of the results of elderly patients undergoing orthopedic surgery, 99 patients were placed in the AKI group and 901 in the non-AKI group. During the modeling process, the AKI patients who underwent orthopedic surgery being divided into two cohorts, training (799 patients total, 79 AKI, and 720 non-AKI) and test (201 patients total, 20 AKI, and 181 non-AKI). The baseline characteristics of patients with and without AKI in the training and validation cohorts are shown in Table 1. The proportion of AKI after orthopedic surgery in elderly patients was 9.89% in the training cohort and 9.95% in the test cohort. The patients who required AKI after orthopedic surgery in elderly patients had a higher frequency of diabetes, hypertension, anemia, hypoproteinemia and transfusion, and higher ASA classification, age and BMI, and lower MAP, and lower mean arterial pressure time for longer duration.

Table 1 Perioperative Statistical Data of Participants

Machine Learning Algorithms and Comparison Statistical Prediction Model

ML algorithms for prediction of AKI included extreme gradient boost (XGBoost), logistic, light gradient boosting machine (LightGBM), randomforest, adaptive boost (AdaBoost), gaussian naive bayes (GausianNB), multi-layer perceptron (MLP), support vector machine (SVC) and k-nearest neighbor (KNN). We compare ML algorithm with standard AKI prediction method. Firstly, we use standard regression stepwise logistic regression to evaluate the prediction results. For the ML model, we conduct repeated 5-folding cross validation based on the optimization of AUC to identify the optimal model parameters. We compared the models performance on the following criteria: AUC, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy. The following Figure 1 and Table 2 show the receiver operating characteristic curve (ROC) results of each model for AKI prediction. The AUC mean and standard deviation (SD) of the model were calculated by multiple repeated sampling, and the number of repeated sampling was 5. The top 20 important features derived from the the Random Forest Regressor are shown in Figure 2. Ultimately, ten variables were selected and entered into the logistic regression model (Table 3). The variables in the model included age, BMI, diabetes, hypertension, hypoproteinemia, duration of low mean arterial pressure, anemia, transfusion, ASA, mean arterial pressure. In the training cohort, randomforest (sorted by AUC) is the best, and the corresponding scores are: AUC:1.000, Accuracy: 0.996, Sensitivity:1.000, Specificity: 0.995, PPV: 0.971, NPV:0.999. In the test cohort, logistic regression (sorted by AUC) is the best, and the corresponding scores are: AUC: 0.821, Accuracy: 0.742, Sensitivity: 0.793, Specificity: 0. 741, PPV: 0.284, NPV:0.963.

Table 2 Predictive Performance Comparison of the Eight Types of Machine Learning Algorithms

Table 3 Multivariate Logistic Regression Analysis of Participants Based on Preoperative and Intraoperative Data in the Training Cohort

Figure 1 ROC curve analysis and forest plots of machine learning algorithms for prediction of AKI elderly patients under orthopedic surgery. Abbreviations: XGBoost, extreme gradient boost; LightGBM, light gradient boosting machine; AdaBoost, adaptive boost; GaussianNB, gaussian naïve bayes; MLP, multi-layer perceptron; SVC, support vector machine; KNN, k-nearest neighbor; AUC, area under curve.

Figure 2 The top 20 important features derived from the Random Forest Regressor. Abbreviations: BMI, body mass index; ASA, American Society of Anesthesiologists; Ddbp, duration of low diastolic blood pressure; DBP, diastolic blood pressure; Dsbp, duration of low systolic pressure; SBP, systolic pressure; Dmap, duration of low mean arterial pressure; MAP, mean arterial pressure; Toperation, duration of operation.

Univariate and Multivariate Logistic Regression Analyses of AKI

All the predictors were tested for multicollinearity and a variance inflation factor (VIF) <10 and then enrolled into the model. The results showed that age (OR: 1.090; 95% CI: 1.043–1.140), BMI (OR: 1.089; 95% CI: 1.023–1.159), ASA (OR: 2.460; 95% CI: 1.215–4.984), hypoproteinemia (OR: 1.614; 95% CI: 1.009–2.609), hypertension (OR: 2.037; 95% CI: 1.283–3.234), diabetes (OR: 2.148; 95% CI: 1.297–3.559), anemia (OR: 1.859; 95% CI: 1.108–3.119), duration of mean arterial pressure (OR: 1.050; 95% CI:1.003–1.069), transfusion (OR: 1.905; 95% CI: 1.186–3.601) were independent positive predictors of AKI. Mean arterial pressure (OR: 0.958; 95% CI: 0.936–0.980) were independent negative predictor of AKI. It can be seen that in our study age, BMI, ASA, hypoproteinemia, hypertension, diabetes and anemia are intraoperative risk factors for AKI after orthopedic surgery in elderly patients. Dmap, MAP and transfusion are postoperative risk factors.

Development and Evaluation of Nomogram

Age, BMI, ASA, hypertension, diabetes mellitus, hypoproteinemia, anemia, mean arterial pressure, duration of low mean arterial pressure, and transfusion were all independent risk factors for AKI in elderly patients after orthopedic surgery. These 10 variables were incorporated into the predictive model based on the results of stepwise regression. All independent AKI predictors were considered when constructing the nomogram (Figure 3). Compared with the complex logistic regression formula, the nomogram is simple and clear, and the clinical practicability is stronger. According to the score of each independent variable, a vertical projection is made to the top points score axis, which can correspond to a score. Finally, all ten independent risk factor variables are scored. The sum of each index score was projected to the risk axis of postoperative AKI through the corresponding total points below to predict the incidence of AKI in elderly patients after the orthopedic surgery. The higher the total score, the higher the risk of AKI. Therefore, the nomogram can predict AKI for individual patients based on their medical condition. A calibration curve was used for calibration of the nomogram. Calibration curve results show that the ideal curve and the actual curve only small offset. The actual predictive curve is consistent with the ideal curve (B=1000 repetitions, Mean absolute error (A)=0.013, Mean absolute error (B)=0.031), suggesting that the prediction model has good accuracy. After internal verification, this prediction model has certain predictive value (Figure 4A and B). The C-statistic of the nonogram is 0.82 and the nomogram has high predictive value.

Figure 3 Nomogram used for predicting AKI after orthopedic surgery in elderly patients. Logistic regression algorithm was used to establish nomogram. The final score (ie, total points) is calculated as the sum of the individual scores of each of the ten variables included in the nomogram. Abbreviations: BMI, body mass index; Dmap, duration of low mean arterial pressure; ASA, American Society of Anesthesiologists; MAP, mean arterial pressure; AKI, acute kidney injury.

Figure 4 Calibration curve of the nomogram for the training set (A) and the validation set (B). Logistic regression algorithm was used to establish nomogram. The X-axis represents the overall predicted probability of AKI after orthopedic surgery in elderly patients. and the Y-axis represents the actual probability. Model calibration is indicated by the degree of fitting of the curve and the diagonal.

Discussion

This study performs and verified a variety of current machine learning algorithms to predict AKI in elderly patients that underwent major orthopedic surgery. According to the comparison of ML algorithms, the randomforest standing out in predicting AKI of the training set and the logistic regression model provides the excellent performance of the test set. This ML-based model may be used to direct intraoperative decision-making possibly.

Previous studies have reported that the occurrence of AKI after surgery leads to increases in postoperative complications and patient mortality, which constitutes a heavy economic burden to their families and to society. Patients with postoperative AKI tend to have more extended hospital stays and a higher 30-day rehospitalization rate.¹¹ Even after complete renal function recovery, postoperative AKI is associated with a significantly increased long-term risk of death.²² According to a study by Liu et al²³ the incidence of AKI in hospitalized patients in China is between 0.32% and 44.30%; however little literature exists on the incidence of AKI after orthopedic surgery. In this study, 99 of 1000 elderly patients who underwent the orthopedic surgery developed postoperative AKI—an incidence rate of 9.9%, similar to that reported by Ali et al.²⁴ The method used to detect AKI is crucial because the various methods are more or less sensitive. The development and performance of an accurate prediction model will depend to a large extent on the population studied and the factors involved in developing and testing the AKI prediction model. In our study, the incidence of AKI was higher in elderly patients undergoing orthopedic surgery because we used KDIGO to define AKI, which includes elevated serum creatinine (above 0.3 mg/dl (26.4 µmol/L)) and therefore includes milder forms of AKI. However, any level of serum creatinine elevation reflects a clinically relevant reduction in glomerular filtration. Even serum creatinine levels that reach normal before discharge are associated with progressive loss of renal function and increased mortality after release.²⁵

The results showed that patients with a low risk of AKI tend to have similar characteristics (good preoperative renal function, few risk factors), but those who do develop AKI are more difficult to identify for various reasons. Accurate prediction requires a strong predictor. The prediction model identified several preoperative risk factors, including age, BMI, ASA, hypertension, diabetes mellitus, hypoproteinemia, and anemia, as the most critical contributing predictors of AKI. The discovery that older age is extremely predictive of AKI in our study approximate to preceding researches.^26,27 A recent study showed that for patients undergoing total hip arthroplasty, the incidence of AKI increased by 40% when BMI increased by 5 kg/m², and the mechanism is unclear, which may be related to increased inflammatory response and oxidative stress.⁴ However, a study on the relationship between obesity and AKI points out that obesity may have a certain protective effect on AKI in critically ill patients, and its mechanism may be related to the long-term chronic inflammation caused by obesity itself. Long-term occult chronic inflammation has a certain protective effect on the inflammation caused by kidney injury in the late stage. At present, the theory is still controversial. ASA classification is an assessment of the overall health status of patients before surgery, reflecting the severity of preoperative combined diseases; the higher the ASA classification, the worse the patient ‘s health and systemic disease and organ function. Only a few studies have shown that ASA classification is related to AKI in the prediction factors of AKI after orthopedic surgery. For example, Bell et al²⁸ included 10,615 patients with orthopedic surgery and established a prediction model for postoperative AKI. Finally, the prediction factors included in the model included ASA classification. Additionally, patients with hypertension are prone to AKI in our consequences, which are consistent with earlier reports that suggest that hypertension is an incontrovertible predictor of AKI.²⁹ This aspect was previously demonstrated by Panitchote et al.³⁰ In addition, Liu et al³¹ found that diabetes mellitus is significantly associated with an increased risk of AKI. Our study shows that patients with hypoproteinemia are more likely to develop AKI, which is also supported by previous studies.³² Previous studies have also shown that preoperative hemoglobin levels (hemoglobin<8 mg/dl) are associated with a fourfold increase in the risk of AKI.³³ Similarly, transfusions of concentrated red blood cells is specifically correlated with AKI.^34–36 Although the mechanism of influence of such transfusions on AKI has not been fully clarified, transfusion still poses a significant risk factor of AKI.³⁷ Anemia in patients before surgery leads to reduced oxygenation of red blood cells, and intraoperative transfusions may increase the incidence of postoperative AKI. Studies have pointed out that this effect may be attributable to anemia, to insufficient patient blood volume, or to the transfusion product itself.³⁸

However, as mentioned in previous studies, the above factors are mainly based on preoperative information, which is not enough to achieve a credible prediction. In addition, few studies assessed the predictive value of intraoperative factors. Our study evaluated intraoperative medication, urine and hemodynamics. Our study revealed that the durations of low mean arterial pressure, mean arterial pressure and transfusion were independent predictors of AKI and were associated with patients with AKI.³⁹ Therefore, we suggest dynamic intraoperative blood pressure management, which may not only protect the vital organs of patients undergoing surgery but also plays an essential role in predicting postoperative AKI. Intraoperative blood pressure monitoring appears to be more promising as a factor for accurately predicting postoperative AKI risk than do preoperative evaluations alone. The AKI prediction grades in patients undergoing meaningful surgeries have been reported from other settings. All the predictors, nevertheless, originate from merely preoperative information; no aggregated perioperative information was incorporated. The other two AKI prediction scores were reported from ordinary major noncardiothoracic surgical operation by Park et al⁴⁰ and Lei et al.⁴¹

Compared with studies attempting to predict AKI risk,^42–46 our work exhibits several strengths. Foremost, few prior pieces of research have concentrated on patients suffer from clinically low-risk AKI. Furthermore, little research literature is available on applying ML algorithms to predict operative AKI. To the best of our knowledge, this is the first study to evolve predictive models using ML algorithms to guide AKI’s risk assessment from ordinary clinical information. In addition, capitalizing on the progress in computer science and clinical informatics in combination with dynamic (time-varying) clinical parameters may prove more conducive to clinical risk prediction model than relying on a single static clinical parameter. The prediction model of this study establishes a visual nomogram, which can intuitively predict the risk of AKI. Clinical medical staff can calculate the risk probability of AKI according to nomogram, so as to intervene the risk factors of AKI and improve clinical decision-making ability. The calibration curve indicates that the prediction model has good accuracy. Thus, developing clinical decision support systems to strengthen health-related decision making and that can promote timely intervention to prevent or mitigate AKI or promote its rehabilitation is useful.

This study, however, also has limitations. First, retrospective studies’ nature may lead to selection bias. Second, our ML algorithm model is limited to a particular mechanism to a certain extent, which may limit its generalizability. This aspect needs to be further verified in real scenarios. In addition, because our current clinical database has some limitations, its predictive value is insufficiently high. Finally, this study mainly judged the occurrence of AKI according to the serum creatinine concentration of patients during the perioperative period, which can be affected by many factors, such as age and muscle content. For next work, we suggest developing a large, heterogeneous, multicenter, multiethnic cohort to improve, homogenize and validate the prevailing clinical risk prediction models. To date, no risk stratification tool for predicting AKI recovery has been developed. New AKI biomarkers can be incorporated into the clinical risk prediction model.

Conclusions

We developed and evaluate ML algorithms to predict postoperative AKI using available preoperative and intraoperative variables and logistic regression model performing the best. The prediction models and nomogram can accurately identify patients with high risk of AKI. Our future goal is to develop an online risk calculator that provides clinicians with an easy-to-use tool to make accurate surgical decisions and to further improve the performance of our models in the field of personalized medicine. A wider population is also required for further validation.