Curve-Modelling and Machine Learning for a Better COPD Diagnosis

Introduction

Chronic Obstructive Pulmonary Disease (COPD) is a respiratory condition that causes persistent airflow obstruction, due to alterations in the airways and/or in the alveoli, which result in chronic respiratory symptoms such as cough and dyspnoea.¹ Even though COPD is a preventable and treatable disease, it is the third leading cause of death worldwide, with a higher burden on low- and medium-income countries.² For example, in Latin America, the PLATINO Study found a 14.3% COPD prevalence, ranging from 7.8% in Mexico City to 19.7% in Montevideo, Uruguay.³ Globally, more than 3 million deaths were attributed to COPD in 2019 alone,⁴ and the disease prevalence is only expected to increase in the following years, because of the world’s population aging and continuous exposure to noxious particles or gases, like tobacco smoking and air pollution.⁵ Due to its progressive nature, timely diagnosis of COPD is key to manage disease development by taking protective measures and treating symptoms, improving the prognosis for the patient.⁶

Current international guidelines indicate that any subject exposed to at least one risk factor should be tested for COPD with spirometry, and the ratio between the Forced Expiratory Volume in the first second (FEV₁) and the Forced Vital Capacity (FVC), after applying a bronchodilator, is evaluated. If this ratio results to be under 0.7 (70%), the patient is diagnosed with COPD.¹ However, authorities like the American Thoracic Society (ATS) and the European Respiratory Society (ERS) advise considering the statistically derived lower limit of normal (LLN) as well.⁷ Even though there has been a global effort in improving its diagnosis, COPD still has high underdiagnosis and misdiagnosis rates. For instance, the PUMA Study, which was conducted in Argentina, Colombia, Uruguay, and Venezuela in 2016, found an underdiagnosis prevalence of 77% and a misdiagnosis rate of 30.4%.⁸

The FEV₁/FVC ratio only considers two instantaneous measures of the spirometry, but more information can be gathered from the flow-volume curve shape. Previous studies have been conducted to quantify and analyse the curvature of this trace with promising results in COPD diagnosis.^9–14 Meanwhile, other researchers have leveraged the increase in computational power and the development of new tools in artificial intelligence. These tools have an outstanding performance in recognizing multidimensional patterns, so they have been proved to be useful in diagnosing respiratory diseases.^15–20 We developed a new COPD diagnostic system that combines these two approaches by modelling the curvature of the expiratory flow-volume trace and feeding the resulting coefficients to different machine-learning techniques to classify patients between those with COPD and those without it. This paper presents the results of the diagnostic accuracy of the proposed system, when tested in patients who are >40 years old and have respiratory symptoms and/or risk factors, in comparison to the classification made by an expert pneumologist.

Methods

Study Design

This was an observational analytical single-centre study that followed the Declaration of Helsinki and was approved by the Ethics Committee of Clínica Universidad de la Sabana. These patients had signed an Informed Consent, in which they authorized the use of their personal data for research purposes. Their records were collected between August 2017 and August 2019.

Eligibility Criteria

The inclusion criteria were patients included in the spirometer database, with signed informed consent and under study due to suspicion of respiratory disease. The exclusion criteria were patients younger than 40 years, having less than 3 post-bronchodilator spirometric trials, incomplete clinical information, or illegible spirometry according to the expert pneumologist. In particular, the record needed to have all the clinical variables that were deemed relevant for COPD diagnosis in a previously conducted expert consensus Supplementary Table 1.

Variables

All included patients had raw spirometry data stored in the same spirometer (Vmax Encore 22, CareFusion, Yorba Linda, California), and demographic information and answers regarding risk factors and respiratory symptoms were collected through questionnaires validated for COPD diagnosis. Spirometry raw data were stored in .fvl format, and each file included the patient’s name and ID, date of test, as well as the registration of flow and volume in time for up to 8 pre-bronchodilator and 8 post-bronchodilator forced-manoeuvre spirometry trials. Each file also had the selection of the best pre-bronchodilator trial and the best post-bronchodilator trial, according to a professional in respiratory therapy, who chose the best traces according to the ATS/ERS criteria (the curve with the largest sum of FVC + FEV1 that fulfilled the Acceptability, Usability, and Repeatability ATS/ERS Criteria for FEV1 and FVC).²¹ An expert pneumologist evaluated each patient’s record and determined which patients had COPD and which did not. This classification was used as a reference test to evaluate diagnostic accuracy.

Study Sample Size

Since this is a study that compares two diagnostic tests, the sample size calculation was performed as indicated by Machin et al.²² With a significance level of 5%, a power of 20%, an increase in sensitivity from 80% in the imperfect gold-standard to 90% in the new test, and a proportion of diseased patients of 30%, we obtained a minimal sample size of 657.

Test Methods

We extracted the raw signals from the spirometer for all the patients, verified the fulfilment of all inclusion/exclusion criteria, selected the best prebronchodilator and postbronchodilator trial traces as tagged by the respiratory therapist and modelled the expiratory phase from the peak expiratory flow to the forced vital capacity, by means of four techniques:

- quadratic polynomial: F = C₂V²+C₁V+C₀

- cubic polynomial: F = C₃V³+C₂V²+C₁V+C₀

- one-term exponential: F = ae^bV

- two-term exponential: F = ae^bV+ce^dV

The goodness of fit for the obtained models was calculated separately for prebronchodilator and postbronchodilator traces by four metrics: Sum Squared Error (SSE), Coefficient of Determination (R²), adjusted Coefficient of Determination (adjR²) and Root Mean Square Error (RMSE).

Then, the subjects were randomly assigned to training or testing, in a 90/10 ratio.²³ This was done to set aside a group of samples (testing) that will not be used to train the machine learning techniques. This allows the evaluation of the performance of the system in the samples used for training and in samples the system has not seen before, to emulate what would occur in a real-life setting. The coefficients of training samples were fed in three groups (prebronchodilator coefficients, postbronchodilator coefficients and all coefficients) to 5 different machine learning techniques: two unsupervised techniques (k-means clustering and hierarchical clustering) and three supervised techniques (Decision Trees – DT, Support Vector Machines – SVM and artificial neural networks – ANN). The artificial neural networks to be tested were fully connected feedforward networks, and the input layer changed depending on the number of coefficients chosen to feed it. The training of all 5 techniques was performed using 10-fold cross-validation.²⁴

Next, the classification made by these techniques on both training and testing groups was compared to the classification made by the expert. The diagnostic performance of each of the machine learning methods was determined based on the accuracy, sensitivity, specificity, positive predictive value, and negative predictive value calculated from the confusion matrix.²⁵

Statistical Methods

Categorical Variables were characterized by absolute and relative frequencies. Quantitative variables were described by medians and interquartile ranges if they were not normal according to the One-sample Kolmogorov–Smirnov test. To compare the groups of patients with COPD and without COPD, we used Chi-Squared Pooled Estimate of Proportion for categorical variables and Wilcoxon Rank Sum Test in quantitative variables.

Data Analysis and Modelling Software

Data were processed using MATLAB Release 2022a (The MathWorks, Inc., Natick, Massachusetts, United States).

Results

The available database had spirometric records and questionnaire answers of 765 patients. After reviewing inclusion/exclusion criteria, 695 patient records were included in the analysis Figure 1.

Figure 1 Patient selection process.

The 695 subjects were divided into two groups (COPD and No COPD), according to the diagnosis made by the expert, and 34% (237/695) had COPD Table 1. The COPD group was significantly older than the No COPD group, and women were the majority in the No COPD group. Most of the subjects (595/695) reported having respiratory symptoms. Out of the whole sample, 27% of the subjects informed having a previous COPD diagnosis, 14% stated they had been diagnosed with asthma, but less than half (46%) claimed they had gone through a spirometry study before. The COPD group had a higher proportion of smokers or ex-smokers (52% vs 42% in the No COPD group), and dyspnoea was the most frequently reported symptom (62%). Also, in the COPD group, the most common risk factor was exposure to wood smoke (68%), followed by tobacco smoking (52%), with a median pack-year of 29, without considering non-smokers.

Table 1 Patient Demographic Data

Figure 2 shows an example of each of the techniques used for curve fitting. The spirometric curve was modelled from the Peak Expiratory Flow (PEF), or maximum flow, to the Forced Vital Capacity (FVC), or maximum volume.

Figure 2 Example of curve fitting by quadratic polynomial, cubic polynomial, one-term and two term exponentials when applied to the spirometric curve.

Table 2 displays the summary of the goodness-of-fit evaluation for each of the modelling techniques, discriminated on prebronchodilator (PreBD) and postbronchodilator (PostBD) traces. The cubic polynomial had the highest goodness-of-fit metrics, while the 1-term exponential had the worse fit performance. Since both SSE and RMSE are measures of error, the smaller they are, the better the fit. On the other hand, R² and adjR² are unity bound and a measure of one means perfect fit.

Table 2 Curve Modelling Goodness of Fit

Table 3 compares, in terms of median and interquartile range, the behaviour of the coefficients between the COPD and No COPD groups, and it shows that most of the proposed coefficients have statistically significant differences between both groups.

Table 3 Coefficient Statistical Description

The methods that displayed the highest accuracy (<85%) when evaluated in the test samples are summarized in Table 4. The highest accuracy was obtained by the Artificial Neural Network, fed by the coefficients of the quadratic polynomial model applied to the postbronchodilator traces, with an accuracy of 92.9%.

Table 4 Best Diagnostic Performance Results in Test Samples

Out of this selection of well-performing methods, this neural network configuration also displayed the highest sensitivity (88.2%) and highest specificity (94.3%). The architecture of this ANN is a 3-neuron input layer and three hidden layers (18, 272 and 1 neurons, respectively), both with hyperbolic tangent as activation function. The output layer has two neurons, one for each class, and the activation function is softmax. Full Results regarding diagnostic accuracy are in the Supplementary Tables 2.1–2.10

For comparison, we analysed the classification made by the current gold standard (FEV1/FVC <0.7) on the 70 patients that were chosen for testing the artificial neural network. Table 5 displays the confusion matrix for the gold standard classification for the testing samples, using the best PostBD curve. Table 6 displays the classification made by the artificial neural network trained with the coefficients from the quadratic polynomial in the same curve and in the same samples.

Table 5 Confusion Matrix for the Gold Standard FEV1/FVC <0.7 on the Same 70 Patients That Were Used to Test the Machine-Learning Algorithm

Table 6 Confusion Matrix for the Testing Samples of the Artificial Neural Network Trained with the Coefficients from the Quadratic Polynomial After Applying Bronchodilator

These two classifications yield the diagnostic accuracy metrics that are summarized in Table 7.

Table 7 Diagnostic Accuracy Comparison Between FEV1/FVC <0.7 and the Artificial Neural Network Trained with the Coefficients from the Quadratic Polynomial After Applying Bronchodilator

Discussion

The goal of this study was to develop an automated tool that leverages the information contained in the curvature of the spirometry traces to assist the process of diagnosing COPD, particularly in primary care settings, where medical professionals might not be as familiarized with spirometric patterns as specialists. We proposed a new COPD diagnostic system that models the curvature of the expiratory flow-volume trace through a mathematical model and feeds the coefficients of the model to a machine-learning technique to classify patients between those with COPD and those without it. We evaluated the combination of four mathematical models and five machine learning techniques. Out of the different combinations, the best diagnostic performance was obtained with a quadratic polynomial and an artificial neural network.

From the goodness-of-fit scores, we found that cubic polynomial model had the best fit to the spirometric traces, while the 1-term exponential model had the worst. Nonetheless, we chose to use the coefficients from all four models because they all had an acceptable fit to the curves. Then, to get an idea of the proposed variables’ capabilities to distinguish between the two groups, we compared the behaviour of the coefficients between the COPD and No COPD groups, and we found that most of the coefficients had a low p-value, when evaluated individually. This result indicated that these features had the potential to correctly classify our subjects. Regardless of this result, we used all of them to feed the machine-learning techniques.

We chose five machine learning techniques to cover a wide range of possibilities. We used clustering techniques to explore the behaviour of our variables, but their diagnostic performance was not as good as that for supervised methods. Regarding the supervised techniques, we tried the simple decision tree because this algorithm could be easily understood and used in the clinical practice. Support Vector Machines were in the mid-range complexity, and we used it because it usually performs well in separating data in multi-dimensional spaces. Finally, the artificial neural network, the most complex technique, is difficult to interpret but it is the most used in the available literature, and it is the most powerful.²³ As expected, the artificial neural network had the best performance. Surprisingly, the best artificial network was the one fed by the coefficients of the quadratic polynomial, when the modelling is applied to the postbronchodilator curves.

As shown in Table 7, our proposed system displays better diagnostic accuracy metrics than the current gold standard FEV1/FVC <0.7. Particularly, the specificity and the NPV show outstanding performance, which would make our classification system very valuable in a screening stage of the diagnostic process, such as in primary care.²⁶

In a previous study, we proposed seven new measures of spirometry and evaluated their diagnostic accuracy.²⁷ Three of these measurements (slope at 50% of FVC, slope at PEF and slope at 75% of FVC) had an outstanding 95% accuracy when compared to the FEV₁/FVC ratio. Other studies have also proposed new metrics for the curvature of the spirometry trace. For example, Das et al⁹ found in a cohort of patients with COPD that the area under the forced expiratory flow-volume loop had good accuracy to detect severe hyperinflation. Mochizuki et al¹⁰ proposed the obstruction index and found that it is very well correlated with emphysema in Computed Tomography (CT). In 2018, Bhatt et al¹¹ explored mathematical modelling using exponentials in the volume–time curve and found that Parameter D had a higher sensitivity than FEV₁/FVC for light obstruction when CT is used as reference.

On the other hand, machine learning techniques have been used for over a decade to facilitate the detection of respiratory diseases. Tang et al¹⁵ evaluated the diagnostic performance of deep residual networks to automate COPD detection in low-dose CT and found that this technique can obtain very good accuracy and positive predictive values. Bodduluri et al¹⁶ compared a deep neural network and a random forest classifier against traditional spirometric measures in their capabilities to differentiate among COPD structural phenotypes, showing the great potential of artificial intelligence for this diagnostic task. Finally, Ioachimescu et al¹⁷ proposed the squared root of the area under the curve to feed an artificial neural network and it had a misclassification rate under 9% in patients with COPD diagnosis obtained through spirometry, plethysmography, and Helium dilution.

The main strength of this study is that we had access to raw spirometry data, which allowed us to work with actual spirometric data. Another of our strengths is that all studies were conducted in the same pulmonary lab, so all the records were acquired by professionals in respiratory therapy who had the same training and applied the same protocols.

One of the limitations of this study is that it was conducted with the data from a single centre, which may limit the extrapolation of the results. Finally, we did not have other more conclusive tests to diagnose COPD in the patients (such as Computed Tomography or Carbon monoxide diffusing capacity), which would guarantee a more certain diagnosis by the pneumologist.

Conclusion

This study explored the diagnostic performance of a combination of several curve modelling techniques and machine-learning techniques to diagnose COPD, based on spirometry alone. We found that an Artificial Neural Network fed with coefficients obtained from quadratic polynomial coefficients had 92.9% accuracy, 88.2% sensitivity and 94.3% specificity to detect COPD. The proposed system had better diagnostic accuracy performance than the current gold standard FEV1/FVC <0.7 in a sample of 70 patients. Hence, Artificial Neural Networks fed with quadratic and cubic polynomial coefficients have interesting potential for assisting the clinical diagnostic process, becoming an important aid in primary care to diagnose COPD in an early stage.