Establishment and Validation of an Early Predictive Model for Severe Acute Pancreatitis

Introduction

Acute pancreatitis (AP) is one of the main causes of hospitalization for gastrointestinal diseases worldwide, with its main clinical feature being self-digestion of the pancreas. Some patients may have multiple organ dysfunction,¹ with a total mortality rate of about 5%. About 20% of patients will develop severe acute pancreatitis (SAP), which has a high mortality rate, poor prognosis, and a mortality rate of up to 30% to 50%.^2,3 A retrospective study in southern Israel showed that between 2000 and 2012, 602 out of approximately 500,000 hospitalized patients were diagnosed with AP.⁴ In France, there are approximately 15,000 cases of AP per year.⁵ In the United States, over 220,000 people are admitted to intensive care units due to SAP every year, with a mortality rate of approximately 15% −20%.⁶ Therefore, it is of great significance for clinical doctors to pay more attention to and actively and effectively treat and manage high-risk patients with severe illness.⁷

At present, the commonly used AP scores in clinical practice mainly include sequential organ failure assessment (SOFA), Modified CT Severity Index (MCTSI), RANSON, and bedside Index of severity in acute pancreatitis (BISAP). These methods are complex in practical use as they contain multiple parameters, some of which can only be collected after 48 hours.⁸ The RANSON score was the first scoring system to predict AP. Due to its 48-hour requirement to calculate the final score, some believe that it may delay management.⁹ The SOFA score is a score for organ dysfunction, which is a universal score for acute and severe cases, but lacks specificity in assessing the severity of AP. Its predictive value for SAP is relatively low.¹⁰ MCTSI performs outstandingly in predicting local complications, but is poor in predicting severity and mortality,¹¹ and abdominal CT features in AP patients typically appear 48 hours after onset, with little predictive value within 24 hours before onset.¹² Therefore, there is an urgent need for a clinical evaluation method that combines sensitivity and specificity, and is easy and fast to operate.

Methods

Patients

A total of 3421 AP patients admitted to Changsha Central Hospital from January 2015 to August 2022 were included in the study. Inclusion criteria: Patients aged ≥ 18 years and diagnosed with AP. Exclusion criteria: 435 patients were excluded from symptoms exceeding 72 hours (n=377), age less than 18 years (n = 12), pregnancy (n = 1), tumor (n = 9), kidney diseases (n = 5), liver diseases (n = 18), immune diseases (n = 1), chronic pancreatitis or acute exacerbation of chronic pancreatitis (n = 12). (Figure 1)

Figure 1 Flowchart. Abbreviation: AP, acute pancreatitis.

Definitions

The diagnosis of AP needs to meet two of the following three characteristics: (1) gastrointestinal symptoms related to AP, such as abdominal pain, bloating, vomiting, etc.; (2) The activity of serum amylase or lipase is at least three times higher than the normal upper limit; (3) CT enhanced scanning, MRI, or transabdominal ultrasound display the characteristic features of AP.¹³ SAP is defined that AP is accompanied by organ dysfunction, lasting for more than 48 hours.¹⁴ A Modified Marshal Score is used for assessing organ failure in AP and the score of 2 or more can define the presence of organ failure.¹⁵ If serum triglycerides >1000 mg/dl, the etiology of AP is confirmed by hypertriglyceridemia.¹⁶ Biliary pancreatitis is a common disease characterized by pancreatitis caused by biliary obstruction.¹⁷

Data Collection

The collected data include clinical features and laboratory test results of patients with admission time ≤ 24 hours. The collected clinical data of patients include: general information of patients (gender, age, body temperature, systolic blood pressure (SBP), diastolic blood pressure (DBP), heart rate (HR), respiratory rate (RR)), laboratory indicators including albumin (ALB), alanine aminotransferase (ALT), amylase (AMY), activated partial thromboplastin time (APTT), blood urea nitrogen (BUN), calcium (Ca), creatine kinase (CK), hematocrit (HCT), high-density lipoprotein (HDL) lymphocyte (L), low density lipoprotein (LDL), lipase (LPS), neutrophil (N), platelet volume distribution width (PDW), platelet (PLT), prothrombin time (PT), serum creatinine (SCR), total bilirubin (TBIL), triglyceride (TG), white blood cell (WBC), comorbidities (diabetes, hypertension), etiology (biliary pancreatitis or hypertriglyceridemia pancreatitis) and calculation-related severity scores (SOFA, BISAP, RANSON).

Statistical Analysis

In this study, we used SPSS 25 software to analyze the data. The comparison between the two groups is represented by the median (P25-P75), and the Mann Whitney U-test is used to compare the differences between the two groups; The counting data is represented by the number of cases (%), and the chi square test is used to compare the groups. After statistical analysis, P < 0.05 was defined as statistically significant. We established a nomogram model for predicting AP deterioration using R42.2 software.

Divide the data into a modeling group (n = 2112) and a validation group (n = 874) in a 7:3 ratio. In the modeling group, univariate logistic regression was used to screen statistically significant independent variables (P < 0.05), which were included in the multivariate binary logistic regression model. After regression analysis, we identified multiple independent risk factors that affect the severity of acute pancreatitis and constructed a predictive model based on this. Bring the above validation group data into the prediction model, and evaluate the accuracy, consistency and clinical applicability of the prediction model by calculating the area under the receiver operating characteristic curve (AUC), calibration curve and decision curve analysis (DCA) curve. Internal validation is evaluated using Bootstrap.

Results

General Characteristics of All Patients

According to the inclusion and exclusion criteria, a total of 2986 AP patients were selected, including 241 cases (8.1%) in the SAP group and 2745 cases (91.9%) in the non-SAP group. Table 1 showed the general characteristics of the cohort. The proportions of etiologies including biliary and hypertriglyceridemia were 21.27% (n = 635) and 41.39% (n = 1236), respectively. In the SAP group, biliary accounted for 14.9% and hypertriglyceridemia accounted for 53.1%. In non-SAP group, biliary accounted for 21.8% and hypertriglyceridemia accounted for 40.4%. There was no significant difference in gender, age, past medical history (diabetes, hypertension), SBP and DBP between SAP patients and non-SAP patients (all P > 0.05).

Table 1 Comparison of Baseline Data Between SAP Group and Non-SAP Group

In the non-SAP group, variables including temperature, HR, WBC, N, HCT, PDW, PT, APTT, TBIL, BUN, SCR, AMY, LPS, CK, and TG were all lower than those in the SAP group (all P < 0 05), while variables including L, ALB, Ca, and HDL were all higher than those in the SAP group (all P < 0 05). The proportion of hypertriglyceridemia in the SAP group was higher than that in the non-SAP group (P < 0.001), and the proportion of biliary was lower than that in the non-SAP group (P = 0.012).

Comparison of Baseline Data Between the Modeling and Validation Groups

The baseline characteristics of patients in the modeling group (n = 2112) and the validation group (n = 874) are shown in Supplementary Table 1. Except for AMY (P = 0.040), there was no significant difference between the modeling group and the validation group in other laboratory test indicators (all P > 0.05).

Comparison of Baseline Data Between SAP and Non-SAP in the Modeling Group

A total of 2112 patients were included, including 166 in the SAP group (7.9%) and 1946 in the non-SAP group (92.1%). Variables including temperature, HR, WBC, N, PDW, PT, TBIL, BUN, SCR, AMY, LPS, CK, TG, L, APTT, ALB, Ca, HDL, hypertension and hypertriglyceridemia were significantly different between two groups (all P < 0.05) (Table 2).

Table 2 Comparison of Baseline Data Between SAP and Non-SAP in the Modeling Group

Multivariate Logistic Regression

In Table, we conducted statistical analysis on the indicators in the modeling group through single factor binary logistic regression analysis. Seventeen variables including hypertension, hypertriglyceridemia, temperature, DBP, HR, RR, WBC, N, PDW, PT, APTT, BUN, SCR, AMY, LPS, CK and TG were screened out firstly (all P < 0.05). Then, nine variables including RR, HR, WBC, PDW, PT, SCR, AMY, CK and TG were ultimately identified as independent risk factors for SAP by multivariate binary logistic analysis (Table 3).

Table 3 Logistic Regression Analysis of Modeling Group

Nomogram

Based on the multivariate logistic binary analysis results of the modeling group, the nomogram was performed based on the variables including RR, WBC, PDW, PT, SCR, AMY, CK and TG (Figure 2). By using nomogram, we can clearly display the relationships between each variable and risk of SAP. The higher the score calculated in nomogram, the greater the likelihood of a patient developing into SAP.

Figure 2 Nomogram. Abbreviations: SAP, severe acute pancreatitis; RR, respiratory rate; HR, heart rate; WBC, white blood cell; PDW, platelet volume distribution width; PT, prothrombin time; SCR, serum creatinine; AMY, amylase; CK, creatine kinase; TG, triglyceride.

For example, we used simple random samplings to analyze the clinical data of two AP patients. The clinical characteristics of the first patient (ID: 17074957) were as follow: RR: 29 beats/minute (44 points), HR: 142 beats/minute (23 points), WBC: 13 ^10⁹/L (8 points), PDW: 16 fL (10 points), PT: 17s (15 points), SCR: 84 umol/L (14 points), AMY: 688 u/L (2 points), CK: 212 u/L (2 points), TG: 23 mmol/L (6 points). A total of 124 points were calculated and a 76% probability of the patient developing SAP was confirmed by the nomogram. In clinical practice, the patient’s condition was severe, and CT showed acute necrotizing pancreatitis with pelvic and abdominal fluid accumulation. The patient was admitted in the intensive care unit and diagnosed with SAP finally. The second patient (ID: 19071443): RR: 18 beats/minute (25 points), HR: 80 beats/minute (11 points), WBC: 14 ^10⁹/L (8 points), PDW: 17 fL (10 points), PT: 10s (8 points), SCR: 68 umol/L (8 points), AMY: 2425 u/L (21 points), CK: 73 u/L (1 point), TG: 29 mmol/L (6 points). A total of 98 points were calculated and a 15% probability of the patient developing SAP was confirmed by the nomogram. Actually, the patient had no organ dysfunction and discharged within one week.

In addition, dynamic nomogram (https://aaapredictedaaa.shinyapps.io/DynNomapp/) was constructed.

Validation of the Predictive Accuracy of Nomogram in Modeling and Validation Cohort

In the modeling group, the prediction model showed good accuracy in assessing the SAP risk of AP patients, and the AUC was 0.788 (95% CI: 0.747–0.829) (Supplementary Figure 1A). In the modeling group, the model had an internal validation (n = 2112, sampling frequency = 1000) with an absolute error of 0.011. In addition, we also evaluated the predictive performance of the ideal model by drawing calibration curves. Among them, the diagonal dashed line represented the perfect prediction of the ideal model. The solid line represented the performance of nomogram, and the higher the fit with the diagonal (dashed line), the better the prediction effect. The good consistency between the actual model and the predicted model indicated that the model had good calibration (Supplementary Figure 1B).

The accuracy of the prediction model was also verified in the validation group. The AUC was 0.789 (95% CI: 0.730–0.848) (Supplementary Figure 2A). In the validation group, the model had an internal validation (n = 874, sampling frequency = 1000) with an absolute error of 0.009. A calibration curve with diagonal dashed lines representing the perfect prediction of the ideal model was also performed. The relatively close consistency between the predicted probability and the observed probability indicated that the calibration of the model was good (Supplementary Figure 2B).

Evaluation of DCA Curve Model

We constructed a DCA curve to evaluate the net benefits of the prediction model (Supplementary Figure 3), and the results showed that the model had a wide threshold range (0.01–0.88) and good clinical net benefits, which meant the model had a good clinical applicability.

Predictive Performances of Ranson, SOFA, BISAP and Nomogram

The predictive performances of RANSON, SOFA, BISAP scores and nomogram model were shown in Table 4 and Supplementary Figure 4. The positive predictive value (PPV), negative predictive value (NPV), sensitivity, specificity, and AUC were all evaluated. Compared with RANSON, SOFA, and BISAP scores, our predictive model performed better in PPV, NPV, sensitivity, and specificity. The AUCs of RANSON, SOFA, BISAP, and nomogram model were 0.715, 0.621, 0.745, and 0.788, respectively.

Table 4 Clinical Validity Evaluation

Discussion

This study used 9 independent risk factors, including RR, HR, WBC, PDW, PT, SCR, AMY, CK, and TG, to establish a nomogram model that can predict the incidence of SAP in AP patients. Furthermore, our nomogram model had a good clinical predictive effect.

WBC, as a biomarker associated with systemic inflammatory response, may be a potential predictive factor for various diseases.^18–20 WBC is an important indicator for predicting the severity and prognosis of AP.²¹ Due to the increase of WBC in the circulation, it damages lung epithelial and endothelial cells, ultimately leading to respiratory failure,²² exacerbating the patient’s condition. WBC includes neutrophils and lymphocytes, and many studies have found that NLR is a good indicator for predicting SAP.^23,24 Its prognostic value is not inferior to BISAP, and it is equally simple and fast.²⁵ The abnormality in coagulation function, such as prolonged PT, is also related to the severity of AP.⁶ Disorders of the coagulation system can lead to microcirculatory failure and multiple organ failure in both intra and extra pancreatic organs, increasing mortality rates.²⁶ PT can serve as an independent risk factor for predicting SAP, and dynamic monitoring of PT changes during the patient’s course will further enhance its predictive value.²⁷ The increase in SCR reflects the disease status of initial low blood volume and renal dysfunction in SAP, and is an important factor in assessing disease severity.²⁸ Acute kidney injury (AKI) is a common complication of SAP.²⁹ The changes in early SCR levels, especially within 24 hours after admission, are effective predictive indicators of the severity of AP.³⁰ Hypertriglyceridemia (HTG) is a risk factor for AP. In China, HTG accounts for 10–20% of the causes of AP, and even becomes the second largest cause after gallstones.³¹ And the proportion of HTG-AP has been increasing year by year.³² HTG-AP patients are more likely to have a more severe course of disease and a higher likelihood of sustained organ failure.³³ Related studies have shown that TG is an independent prognostic risk factor for the severity of acute pancreatitis.³⁴ PDW displays the changes in PLT size and is considered a marker of PLT morphology.³⁵ When patients develop SAP, PLT adhesion and aggregation lead to a decrease in the number of PLTs in circulation, which results in proliferation of megakaryocytes, and an increase in PDW. Therefore, PDW can serve as a marker for activated platelet release in certain inflammatory diseases.³⁶ Clinical studies have shown that heart rate (HR) is the best predictor of SAP.³⁷ HR can indirectly reflect changes in indicators such as infection, fever, and body metabolism. A study on the effect of silymarin on the severity of Cerulein-induced acute pancreatitis in animal models showed that during the onset of AP, pancreatic digestive enzymes such as AMY are released into the bloodstream and their activity rapidly increases in the early stages. Therefore, serum amylase activity could be used as typical markers to evaluate the severity of AP.³⁸ Many animal model studies related to AP also use AMY as an indicator to determine the severity and prognosis of AP.^39,40 CK-MB is a part of CK, and elevated CK-MB is significantly associated with adverse outcomes in AP patients, making it a potentially useful laboratory parameter for predicting adverse clinical outcomes in AP.⁴¹ In clinical practice both domestically and internationally, the CK levels of AP patients often increase. An increase in CK can be seen in SAP, which may be a predictive factor for SAP.⁴²

In our research, several scores including Ranson, BISAP and SOFA were compared with our nomogram, which showed that the predictive performance of the nomogram model in our study was the best. Previous research showed that Ranson, BISAP and SOFA scores have been applied widely for evaluating the disease severity in AP.³⁰ RANSON is the earliest AP specialist scoring system with high predictive sensitivity and is a complex tool that requires the collection of 11 indicators. In addition, the RANSON score must be evaluated 48 hours after admission, and cannot assess the severity of the disease in the early stages of AP. It has a relative lag and can only be scored once, lacking the effectiveness of dynamic evaluation, which makes some believe it may delay management.⁹ SOFA is a score related to organ dysfunction, which is a universal score for acute and severe cases. It lacks specificity in assessing the severity of AP and has low predictive sensitivity for SAP.¹⁰ In our study, several common clinical scores such as RANSON, BISAP, and SOFA were compared with our predictive model in multiple aspects, and the results showed that our model had the best predictive performance and the evaluation process was also the most concise and efficient.

The sample size used in this study (n=2986) is comparatively large and all variables involved are common indicators in clinical laboratory testing, which has good clinical applicability. However, there are still some limitations in our research. Firstly, this study is a single center and needs to consider differences with other countries, regions, and populations. Secondly, our study is retrospective and may have patient selection bias, which is an inevitable limitation of such studies. Third, this was a single-center retrospective study that was only validated internally and lacked external validation. Therefore, our findings will have to be validated through studies on large, multi-center cohorts, or related meta-analyses. Fourth some other etiologies including alcoholic and idiopathic were not showed due to some data missing based on the retrospective study. Hence, some patients with AP were not accurately identified with the etiologies of AP.

Conclusion

In summary, the nomogram prediction model was established based on independent risk factors and had good discrimination and clinical applicability. Our nomogram could provide a quick and easy assessment of SAP incidence rate for AP patients at the early stage.