Comparative Analysis of Three Atherosclerotic Cardiovascular Disease Risk Prediction Models in Individuals Aged 75 and Older

Introduction

Atherosclerotic cardiovascular disease (ASCVD) is the leading cause of mortality, similar to ischemic heart disease (IHD) and ischemic stroke, and it has been a major public health issue worldwide.¹ Blood lipid, smoking, and blood pressure are adjustable causal risk factors that may be rectified through lifestyle changes or drug use.² Risk prediction and stratification of ASCVD are efficient approaches toward preventive interventions for ASCVD.

China has become an aging society, with an estimated peak approaching in 2053 at approximately 482 million population.³ In China, the proportion of older persons (>65 years) has increased from 6.96% in 2000 to 13.50% in 2020, which is basically from the Fifth to the Seventh Population Census.⁴ This demographic shift and the societal transformation in terms of lifestyle, environment, economy, and the health care system have largely contributed to the major CVD risk factors.⁵ ASCVD is one of the main causes of mortality in older adults in China.⁶ According to The American College of Cardiology (ACC) and the American Heart Association (AHA) guideline, individuals aged 40–75 years should be examined for CVD prevention as a part of a 10-year ASCVD risk assessment program.⁷

So far, several models have been proposed for ASCVD risk evaluation and scientific guidance has been established in the domains of public health and clinical practice for different populations; for instance, Framingham general CVD equations in the United States⁸ and the recent Pooled Cohort Equations (PCEs) for ASCVD.⁹ Few evidence-based studies among older adults aged 75 years and above are available.¹⁰ In 2016, the AHA/ACC/AGS also stated that the participants in most of the previous studies were adults aged less than 75 years.¹¹ According to the Outline of Healthy China 2030 Plan, the average life expectancy for Chinese people has significantly improved from 77 years in 2020 to 80 years in 2030.⁴ Moreover, the older persons population has currently reached 19,060,000, accounting for 13.5% of the total population. The predictive performance of current prediction models for cardiovascular disease risk in elderly individuals is still unclear.

However, there is still no ASCVD risk prediction model specifically for older adults (≥75 years). We utilized data based on Chinese hospitals, strategized this longitudinal research to validate the three models (PCEs; Framingham risk score, FRS; and China-PAR) and compare their performance for predicting the 10-year ASCVD risks among older Chinese adults aged 75–85 years.

Materials and Methods

Study Population

The cohort participants were Chinese adults aged 75–85 years who underwent physical examinations at The General Hospital of Western Theater Command, Chengdu City of Sichuan Province, China, which has 2200 beds and at least 140,000 inpatients annually from Chengdu. The participants’ physical examination data included an assessment of their medical history, anthropometric measurements, blood and urine analyses, and ultrasound examination. They were followed annually from 2011 to 2021. Those patients were seen by the doctors for regular check.

The inclusion and exclusion criteria for the study participants were as follows: As the life expectancy of patients aged over 85 years is not long enough for observing the impact of ASCVD, we included participants aged 75–85 years. Patients who were previously diagnosed with cerebrovascular disease, coronary heart disease, cerebral infarction, heart failure, peripheral artery disease, acute infection, tumor, thyroid dysfunction, and liver and kidney insufficiency and those receiving glucocorticoid therapy were excluded. The study was approved by the institutional research ethics committee of the General Hospital of Western Theater Command (No: 2018EC1-7), and written informed consent was obtained from each subject.

ASCVD Risk Assessment

Trained physicians evaluated the 10-year ASCVD risk using FRS, PCE, and China-PAR. FRS released in 1998 is the world’s first cardiovascular disease risk assessment model.¹² The Framingham risk score predicts the percent risk of developing heart failure, coronary, cerebral, or peripheral artery disease during the 10 years of follow-up, based on the input variables such as age, sex, smoking, diabetes, systolic blood pressure, and BMI.¹² For more information about FRS, please refer to https://objectivehealth.ca/clinicians/framingham/. Patients with the 10-year ASCVD risk were divided into three groups: low risk (<10%), medium risk (10–19%), and high risk (≥20%). The ACC/AHA proposed a summary cohort equation in 2013 that was based on five large, multiregional, multiethnic community cohort studies (PCEs), which was the first risk prediction model to include large data of white and African-American patients.¹³ The 2013 ACC/AHA Cardiovascular Risk Assessment Guidelines recommended using PCEs as a cardiovascular risk assessment tool. The preoperative data in PCEs included sex, ethnicity, age, smoking status, as well as history of hypertension and diabetes and total cholesterol (TC) and high-density lipoprotein (HDL-C) levels. The 10-year ASCVD risk estimation based on the PCEs is available at https://tools.acc.org/ascvd-risk-estimator-plus/#!/calculate/estimate/.¹⁴ The 10-year ASCVD risk was divided into four groups: low risk (<5%), close to the risk threshold (5–7.4%), medium risk (7.5–19.9%), and high risk (≥20%). The China-PAR project (Prediction for ASCVD Risk in China) released in 2016 aimed to develop and validate the 10-year risk prediction equations for ASCVD from 4 contemporary Chinese cohorts.¹⁵ The factors included in China-PAR were as follows: sex, age, systolic blood pressure, waist circumference (WC), smoking status, serum TC and HDL-C levels, and history of drinking and diabetes as well as regional characteristics and disease spectrum characteristics of China (north and south, urban and rural areas, and family history of ASCVD). The 10-year ASCVD risk estimation through China-PAR is available online at https://www.cvdrisk.com.cn/ASCVD/Eval/; https://objectivehealth.ca/clinicians/framingham/. The 10-year ASCVD risk was divided into three groups: low risk (<5%), medium risk (5–9.9%), and high risk (≥10%). The upper age limit for FRS, PCEs, and China-PAR is 80, 79, and 85 years, respectively. If the age exceeded the limit of predictive models, the maximum age of the 3 models was considered.

Covariates

Several potential confounding variables were collected through questionnaires. The smoking status was categorized as current smoker and nonsmoker (including never smoked and ever smoked). A current smoker was defined as someone who smoked 1 or more cigarettes per day at least for more than 1 year. The participants’ medical, family, and plateau life histories were considered. The physical examination parameters including heart rate, blood pressure, waist circumference (WC), height, and weight were measured by trained physicians. Hypertension was defined as diastolic BP ≥90 and/or systolic BP ≥140 mmHg. Their fasting blood and urine samples were collected in the morning and serum TC, HDL-C, fasting blood glucose (FBG), creatinine, and triglyceride levels were measured using a fully automatic biochemical analyzer (Mindray, BS-800). According to FRS, TC was categorized into the following five groups: <4.14 mmol/L (1 mmol/L = 38.67 mg/dL), <5.16 mmol/L, <6.2 mmol/L, <7.23 mmol/L, and ≥7.23 mmol/L. HDL-C was categorized into the following five groups: <0.91 mmol/L, <1.15 mmol/L, <1.28 mmol/L, <1.53 mmol/L, and ≥1.53 mmol/L. Type 2 diabetes was defined as a fasting plasma glucose level >7.77 mmol/L (1 mmol/L = 18 mg/dL) and was diagnosed with diabetes by physicians.¹⁶

Main Outcome Measures

The primary endpoint of this study was defined as the first diagnosis of fatal or non-fatal acute MI or coronary heart disease (CHD) or ischemic stroke. The incident ASCVD was defined as per the use of the International Classification of Diseases-10th Revision (ICD-10) codes (I21-I22, I46, I63-I64).¹⁷ The patient information was checked from medical records in the hospital information system. In cases where the occurrence of the same type of ASCVD event was more than once, the first event was considered the end event. In case the subject died during the follow-up, the cause of death was taken from the cause of death monitoring system. Loss to follow-up is mainly due to death from non-cardiovascular diseases and loss of contact with research subjects.

Statistical Analysis

Normally distributed continuous data were summarized as sex-specific mean and standard deviation (SD) and analyzed using the t-test. Non-normally distributed continuous covariates were presented as medians and quartiles and analyzed using the Wilcoxon rank sum test. Categorical data were presented as counts or percentages and analyzed by using the χ² test or Fisher’s precision probability test.

The performance of the 3 risk models in predicting ASCVD for the older persons Chinese population was assessed based on the separate examination of the calibration and discrimination for either gender. The discrimination of each of the ASCVD risk predictive models was assessed using the area under the curve (AUC) and R² statistic for survival data. High R² or AUC values indicated better discrimination. Calibration indicated how accurately the 10-year ASCVD risk predicted the observed risk. Calibration was evaluated by plotting the calibration curve and calculating the Hosmer—Lemeshow statistic, with P > 0.05 indicating a good calibration. All analyses were conducted using R 4.1.3 (R-project, Vienna, Austria). All Ps presented were two-tailed, and the results were considered significant at P < 0.05.

Results

Baseline Characteristics

As per the flowchart (Supplementary Figure 1), the study included 717 participants (263 females and 454 males) without ASCVD with a mean age of 80.1 years. During the 10-year follow-up, 206 participants (28.73%) were diagnosed with ASCVD [160 (35.24%) males and 46 (17.49%) females]. Males were usually smokers (males: 41.63% vs females: 1.90%) and drinkers (males: 16.30% vs females: 5.32%) and had a family history of hypertension (males: 64.10% vs females: 34.60%). Females had higher serum TC, glutamic pyruvic transaminase, HDL-C, and low-density lipoprotein (LDL-C) levels. The distributions of most predictors were statistically significant between men and women (P < 0.05), except for diabetes, heart rate, serum creatinine, fasting blood glucose, and triglyceride levels. Therefore, we evaluated the three risk models stratified by sex. The details are shown in Supplementary Table 1.

As per the FRS model evaluation, 71 participants (9.90%) were at low risk (all females); 148 participants (20.64%) were at median risk [125 females (47.52%) and 23 males (5.07%)]; and 498 participants (69.46%) were at high risk [67 females (25.48%) and 431 males (94.93%)] (P < 0.001).

As per the China-PAR model evaluation, 221 participants (30.82%) were at median risk [122 females (46.39%) and 99 males (21.81%)] and 496 participants (69.18%) were at high risk [141 females (53.61%) and 355 males (78.19%)] (P < 0.001).

As per the PCE model evaluation, 51 participants (7.11%) were at median risk [44 females (16.73%) and 7 males (1.54%)] and 666 participants (92.89%) were at high risk [219 females (83.27%) and 447 males (98.46%)] (P < 0.001).

The 10-year incidence of ASCVD in males was significantly higher than that in females (35.24% vs 17.49%) (P < 0.001).

Comparison of the Predictive and Observed of Three Different Models

The observed ASCVD events were 28.73% and the predicted events for FRS, China-PAR, and PCE were 22.1%, 12.4%, and 36.1%, respectively (P < 0.001). The observed ASCVD events were 35.24%, while the predicted ASCVD events for FRS, China-PAR, and PCE were 26.1%, 13.0%, and 37.9%, respectively, for men (P < 0.001). In women, the observed ASCVD events were 17.5%, while the predicted ASCVD events for FRS, China-PAR, and PCE were 15.1%, 11.4%, and 32.8%, respectively (P < 0.001).

Statistical differences were noted between the predicted and observed 10-year mean ASCVD risk among the three models (P < 0.001). Expected odd ratio was further calculated; we found that, in men, FRS and China-PAR underestimated the ASCVD risk by 26% and 63%, respectively, while PCE overestimated the risk by 8%. On the other hand, in women, FRS and China-PAR underestimated the risk by 14% and 35%, respectively, whereas PCE overestimated the risk by 88%. The details are shown in Table 1.

Table 1 Predicted and Observed Risks of ASCVD

Comparison of ASCVD Risk Ratings Among the Three Predictive Models

As per the FRS model evaluation, the average absolute ASCVD risk was 22.1%, of which 9.9% was rated as low risk, 20.6% as median risk, and 69.5% as high risk. As per the China-PAR model evaluation, the mean absolute ASCVD risk was 12.4%, of which 30.8% was rated as median risk and 69.2% was rated as high risk. As per the PCE model evaluation, the mean absolute ASCVD risk was 36.1%, of which 7.1% was rated as median risk and 92.9% as high risk (P < 0.001) (Figure 1).

Figure 1 The 10-year absolute CVD risk levels predicted by 3 models.

Discrimination and Calibration

The AUCs of the FRS, China-PAR, and PCE models were 0.618 [95% confidence interval (CI): 0.571–0.665], 0.778 (95% CI: 0.737–0.820), and 0.682 (95% CI: 0.629–0.735), respectively, in men; the corresponding R² were 0.065, 0.249, and 0.147, respectively (Figure 2A). The AUCs of the FRS, China-PAR, and PCE models in women were 0.815 (95% CI: 0.747–0.882), 0.815 (95% CI: 0.752–0.878), and 0.774 (95% CI: 0.703–0.845), respectively; the corresponding R² values were 0.307, 0.255, and 0.214, respectively (Figure 2B).

Figure 2 (A) Discrimination of the 3 equations in men. Framingham (man): AUC = 0.618 95% CI = 0.571–0.665. China-PAR (man): AUC = 0.778 95% CI = 0.737–0.820. PCE (man): AUC = 0.682 95 CI = 0.629–0.735. (B) Discrimination of the 3 equations in women. Framingham (woman): AUC = 0.815 95% CI = 0.747–0.882. China-PAR (woman): AUC = 0.815 95% CI = 0.752–0.878. PCE (woman): AUC = 0.774 95% CI = 0.703–0.845.

Calibration was made by comparing the predictive and observed risks in each decile of model-based probabilities by using the Hosmer—Lemeshow Chi-square (χ²) test. The χ² values of the FRS, China-PAR, and PCE models in men were 1.3962 (P = 0.994), 25.528 (P = 0.001), and 7.9098 (P = 0.442), respectively. The χ² values of the FRS, China-PAR, and PCE models in women were 13.523 (P = 0.095), 6.1433 (P = 0.631), and 8.3685 (P = 0.398), respectively. The calibration plots of FRS, China-PAR, and PCE models are shown in Figure 3. FRS and PCE were well-calibrated in men, and FRS, China-PAR, and PCE were well-calibrated in women.

Figure 3 Calibration of the 3 equations. (A): X-squared = 1.3964, df = 8, P = 0.994. (B) X-squared = 13.523, df = 8, P = 0.095. (C) X-squared = 25.528, df = 8, P = 0.001. (D) X-squared = 6.1433, df = 8, P = 0.631. (E) X-squared = 7.9098, df = 8, P = 0.442. (F) X-squared = 8.3685, df = 8, P = 0.398.

Discussion

We herein evaluated the performance of the FRS, PCE, and China-PAR models stratified by sex in a 10-year prospective cohort of Chinese older adults aged 75–85 years. FRS and China-PAR underestimated the 10-year ASCVD risk, whereas PCE overestimated the risk among all populations. All three risk models had high discrimination among both sexes. Both FRS and PCE were well-calibrated in men. FRS fairly predicted the 10-year ASCVD risk in men and China-PAR fairly predicted that in women.

Smoking, drinking, physical activity, diet, blood pressure, serum LDL-C, blood glucose, and BMI are the main risk factors for ASCVD.^18–20 The disposition of these risk factors for ASCVD was significantly different among sexes in the present study; males tended to be smokers and drinkers and had a higher WC and family history of hypertension, whereas females had higher serum TC, alanine aminotransferase, HDL-C, and LDL-C levels and a family history of ASCVD. A substantial reduction in the CVD burden (age-standardized DALY rate) was observed during 1990–2016, with a greater reduction noted in women (43.7%) compared to that in men (24.7%).²¹ Therefore, we evaluated the performance of the FRS, PCE, and China-PAR models stratified by sex in assessing the 10-year ASCVD risk.

Among the total population in our study, the predictive 10-year ASCVD risks as per the FRS, PCE, and China-PAR models were 22.1%, 12.4%, and 36.1%, respectively. The difference may be because different models incorporate different predictors. The PCE underwent numerous external validations in several populations and under different settings. The majority of results obtained from external validation studies indicated that the PCE overestimated the ASCVD risk,^22–24 which is in line with our findings. In the present study, PCE overestimated the 10-year ASCVD risk in the entire population by approximately 88%, whereas FRS (26%) and China-PAR (63%) underestimated the 10-year ASCVD risk. In addition to the risk factors included in FRS, PCE also included sex, serum LDL-C level, and smoking status and China-PAR included age, HDL-C, SBP, TC diabetes, current smoking, and four additional covariates (WC, geographic region, family history of ASCVD, and urbanization). The other possible reason for the overestimation/underestimation of the ASCVD risk of the three models was the under-ascertainment of ASCVD events, an increase in statin use, and a decrease in ASCVD incidence rates over time. In a prospective American cohort that included Chinese, black, Caucasians, and Hispanic Americans, ASCVD risk overestimation was observed in all ethnicities, with the highest among Chinese.²⁵ In general, East Asians showed lower rates of CVD risk factors, such as obesity, than Western individuals.²⁶ PCE released by the ACC/AHA in 2013 was developed in accordance with the data obtained from cohorts based on the US population, which may account for the overestimation of ASCVD risk by PCE. The present study revealed that the 10-year risks of ASCVD, as assessed by the three prediction models, were significantly higher in men than in women. Other studies also indicated that the CVD risk was significantly greater in men than in women.²¹ Among individuals aged 35–84 years, men showed approximately twice the total numbers of incidence of morbidity and mortality in women.²⁷ This difference is mainly due to dissimilarities in physical structure and gene expression between males and females.²⁸ Female estrogen can directly regulate the response of atherosclerosis, indirectly act on serum lipoprotein and triglyceride levels, and have other protective effects on the cardiovascular system; the lack of estrogen hormone increases the risk of cardiovascular disease in postmenopausal women.²⁹ The discrepancy between sexes may also be because women maintain a lesser probability of the disease than men at any level of the major CVD risk factors (such as smoking and living pressure).^30,31 Thus, it is important to identify sex-specific risks of cardiovascular disease and develop sex-specific interventions.

In the present study, 69.5%, 69.2%, and 92.9% of Chinese older adults aged 75–85 years were rated as having a high risk for ASCVD by the FRS, China-PAR, and PCE models, respectively. A prospective study conducted on 70–80-year-old Australians showed that 20.8% and 12.3% of them were rated as high risk by PCE and China-PAR.³¹ The average age of the study population was 80 years, which may have led to a higher proportion of people being assessed as high risk for ASCVD. Aging is one of the main risk factors for ASCVD.³²

We found that the discrimination capacities in predicting the 10-year ASCVD risk for all 3 risk models were moderate among Chinese older adults aged 75–85 years. For men, China-PAR had the best discrimination abilities and FRS had a relatively better calibration. For women, FRS had the best discrimination abilities and China-PAR had a relatively better calibration. The China-PAR project was developed and the 10-year risk prediction equations were validated for ASCVD from 4 contemporary Chinese cohorts.¹⁵ However, we found that only China-PAR had shown a good ability to predict the 10-year ASCVD risk among women aged 75–85 years and that FRS and not China-PAR fairly predicted the risk of 10-year-ASCVD in men. The validation of the prediction equations for predicting the 5-year ASCVD risk in the rural areas of northwestern China revealed that both China-PAR and FRS fairly predicted that in men, but not in women,³³ which is partially consistent with the findings of our study. China-PAR in the 5-year ASCVD risk prediction in this rural Northern Chinese population at an average population risk level fairly predicted risk in men, but overestimated the risk in women,³⁴ which is consistent with the findings of our study. The performance of the 3 prediction models for ASCVD risks varied across populations.

Nonetheless, there are several limitations: First, it is a single-center study from a particular Province only, and the current sample size was small; thus, there might be a potential selection bias. Second, older adults aged 75–85 years who had multiple comorbidities that could have an impact on ASCVD prognosis were included; thus, the predictive performance of the predictive models might have been affected too. Third, the average age of people included in our study was 80 years. Most of their physical condition changed rapidly, and some of them were unable to finish their annual physical examination; therefore, more participants were lost to follow-up compared with other cohort studies. The covariates having significant differences are not balanced in line with the methodology of the scientific work; therefore, they should be considered for future work.

Conclusion

Among Chinese older adults aged 75–85 years, FRS and China-PAR underestimated and PCE overestimated the observed 10-year ASCVD risk. All three risk equations displayed excellent discrimination. FRS and PC were well-calibrated in men. All three models were well calibrated in women; the China-PAR showed the most accurate predictions, and FRS could well predict the risk in men.