Reliability, Validity, Modification and Expansion of the Chinese Version of the Disease-Specific Anxiety Questionnaire for Chronic Obstructive Pulmonary Disease

Introduction

According to the World Health Organization (WHO), chronic obstructive pulmonary disease (COPD) is the third leading cause of mortality worldwide.¹ A survey published in the Lancet reported that there were 99.9 million people living with COPD in China.² High mortality, disability, and a heavy disease burden contribute to psychological disorders in patients.^3,4 Anxiety is one of the most common psychological comorbidities.⁵ COPD patients with anxiety often experience more frequent exacerbations that lead to longer hospital stays, higher medical costs, higher risks of readmission for acute exacerbation of COPD (AECOPD), and even higher mortality.^6–10 It has been reported that the prevalence of anxiety among inpatients with COPD is 10%–55%, and this is 13%–46% among outpatients,¹¹ although this might be underestimated, including in China.⁸

An increasing number of researchers and health-care providers have begun to focus on anxiety comorbidities in patients with COPD. The main screening tools used in China include the Hospital Anxiety and Depression Scale (HADS),¹² Self-Rating Anxiety Scale (SAS),¹³ Generalized Anxiety Disorder Scale-7 (GAD-7),^14,15 and Beck Anxiety Inventory (BAI).¹⁶ Yet these tools are often criticized for lacking accuracy because they do not address somatic symptoms.¹⁷ A recent study reported that anxiety has structural hierarchies, including general and specific anxiety.¹⁸ The above routine scales are usually used to measure general anxiety, but specific tools are still lacking. The 11-item Anxiety Inventory for Respiratory Disease (AIR), developed by Professor Yohannes,¹⁹ has been translated into Chinese and validated in patients with COPD.²⁰ However, as a specific tool to measure respiratory disease, the AIR might be responsible for true positives without somatic symptoms, which would lead to a decrease in sensitivity. A COPD patient-specific anxiety scale was developed by Zhang, containing 31 items and six dimensions;²¹ however this scale did not include confirmatory factor analysis (CFA) to validate the potential measurement structure, which may hinder the general application of the scale because its structure is unknown. To the best of our knowledge, only a few studies have used this tool in China.

The COPD Anxiety Questionnaire (CAF) is a self-report scale developed in 2011 by Kühl et al to measure specific anxieties in patients with COPD.²² The scale consisted of 27 items and five dimensions: fear of dyspnea (FD), fear of physical activity (FPA), fear of progression (FP), fear of social exclusion (FSE), and sleep-related worries (SRW). In addition, the scale includes eight conditional items related to the partner or oxygen treatment. Keil et al revised the scale to a shorter 20-item version and validated it in a large sample (n = 1024) in 2014.²³ The scale was developed in Germany but has been unavailable in English until now, so the scale has only been used in German-speaking countries.^24,25 The CAF was translated into Danish and validated in 2022.²⁶ As it has been proven to be a valid measurement tool for disease specificity, the revised CAF (CAF-R) has been recommended for use in COPD patients. Therefore, in this study, we aimed to translate the German version of the CAF-R into Chinese and test its reliability and validity in Chinese patients with COPD.

Study Design and Methods

Participants

Participants were recruited from medical institutions (ie, The First Affiliated Hospital of Soochow University, The Second Affiliated Hospital of Soochow University, Suzhou Municipal Hospital, and Suzhou Xiangcheng People’s Hospital) in Jiangsu Province, Suzhou, China, between April 2022 and June 2023. The inclusion criteria were as follows: (1) patients diagnosed with COPD according to the 2023 Global Initiative for Chronic Obstructive Lung Disease (GOLD)²⁷ and (2) females and males aged ≥18 years who were aware of their condition. The exclusion criteria were as follows: (1) cognitive disorders such as dementia, (2) mental disorders,²⁸ or (3) experience of major negative life events within 6 months. All survey respondents volunteered to participate in this study and provided their informed consent. Our study conformed with the Declaration of Helsinki. Ethical approval was obtained from the ethics committee of the First Affiliated Hospital of Soochow University [2022 ethical approval (application) and 251].

Procedures

After receiving authorization from Professors Stenzel and Kühl, the German version of the CAF-R²³ was forward-translated and back-translated according to Brislin’s translation model.²⁹ The main processes were as follows. (1) Forward translation: two translators whose native language was Chinese (an experienced respiratory PhD who studied in Germany for 1 year and a native Chinese nutritionist who lived in Germany for nearly 20 years) independently translated the CAF-R from German into Chinese (two Chinese versions T1, T2). The two translators and our research group members (a respiratory PHD, a respiratory nursing specialist, and a nursing research expert with experience in instrument development) discussed and modified the questionnaire until reaching a consensus on the Chinese version (T3). (2) Back-translation: Two bilingual experts (a medical professor who immigrated to Germany and worked there for at least 15 years, and a master of engineering who had worked in Germany for 8 years), blinded to the original scale, independently translated T3 into German (BT1 and BT2). The two experts then met, discussed, and modified the back-translated version until reaching a consensus on the German version (BT3). BT3 was submitted to Professors Stenzel and Kühl, who were invited to evaluate the translation equivalence between BT3 and the original scale. Next, we group-adjusted and modified T3 according to the suggestions of the original authors and created the Chinese version (T4).

Cross-Cultural Adaptation

Cross-cultural adaptation was applied to achieve equivalence (including semantic, idiomatic, experiential, and conceptual equivalence) between the original questionnaire and the Chinese version.³⁰ An expert committee comprising seven researchers in different disciplines (ie, respiratory nursing, psychology, respiratory medicine, and pulmonary rehabilitation) conducted the cross-cultural adaptation. The expert authority coefficient was 0.87. Furthermore, forward and backward translators were invited to participate in expert consultations because the original scale was unavailable in the English version. Based on clinical experience and professional knowledge, the experts were invited to evaluate the equivalence of the T4 version and whether it conformed to the cultural background of China. They were asked to rate each item on a 4-point scale ranging from 0 (irrelevant to all) to 4 (very relevant). We modified T4 to obtain the Chinese version T5.

Pre-Testing and Cognitive Interviewing

Twenty COPD patients who were hospitalized in the Department of Respiratory and Critical Care Medicine of a tertiary hospital in Suzhou were selected for pre-testing from February to March 2022. After all respondents independently completed the questionnaire (T5), cognitive interviews were conducted to understand their comments and suggestions on item content, semantic clarity, and difficulties in item comprehension. According to the results of the pre-testing and the cognitive interviews, adjustments were made to the Chinese version.

Measurements

This Was a Cross-Sectional Study.

Demographic Characteristics and Clinical Data

A self-developed questionnaire was used that included questions on sex, age, marital status, medical payment form, financial burden, smoking status, pulmonary function (GOLD stage), use of long-term oxygen, diagnosis of COPD for the first time, and acute exacerbation (or not).

The CAF-R-CN is a self-reported measure with 25 items used to assess COPD-specific fear and worry. The scale covers six domains: FSE, FD, FPA, FP, SRW, and economic-related worries (ERW). Each item is rated by the patients themselves using a 5-point scale from “never” (scored 0) to “always” (scored 4), with a total score ranging from 0 to 100. The original CAF-R has good reliability, with Cronbach’s α ranging from 0.78 (SRW) to 0.87 (FP).²³

The Beck Anxiety Inventory (BAI), which consists of 21 items rated on a 4-point scale was applied to assess patients’ anxiety.¹⁶ In the total score, multiplying by 1.19 was considered the standard score, with a higher score indicating a higher anxiety level. The BAI showed good reliability (Cronbach’s α = 0.93) and validity. Selecting a cut-off score of 45 (consistency value of Kappa = 0.82) to assess anxiety disorders in adults generated a sensitivity of 91.66% and a specificity of 91.25%.³¹

The 13-item Beck Depression Inventory (BDI-13) has been validated in Chinese to measure patients’ depression and consists of 13 items rated on a 4-point scale.^32,33 The total score ranges from 0 to 39, with higher scores indicating more serious depression. The BDI-13 demonstrated good reliability and validity.

The modified Medical Research Council Dyspnea Scale (mMRC) was used to assess the degree of breathlessness when participants engaged in physical activity, rated on a 5-point scale,³⁴ with a higher grade suggesting more severe dyspnea.

Analysis

Microsoft Excel was used for data input and processing. IBM SPSS version 25.0 was used to perform descriptive, correlation, difference analysis, and exploratory factor analysis (EFA). AMOS version 27.0 was used to perform CFA, and both were used for convergent and discriminant validity. P-values (two-tailed) for all analyses (≤0.05) were considered statistically significant. For missing data, multiple imputations were performed using multiple linear regression method.³⁵

Sample Size Estimation

The sample size for EFA was five to ten times the number of scale items, as the 25-item CAF-R-CN requires a sample size of 125–250 cases. A structural equation model (SEM) for CFA required more than 200 cases at least.³⁶ However, considering the 10% invalid samples, the sample size was calculated to be approximately 358–495 cases.

Reliability

The internal consistency of the CAF-R-CN was assessed by computing item-total correlations and Cronbach’s α. Individual items with a correlation coefficient total score of less than 0.3 (r <0.3),^37,38 or a p-value that was not significant (p >0.05) were removed. Cronbach’s α >0.9 was considered excellent, 0.8–0.9 was considered good, and 0.7–0.8 was acceptable.^38,39 Test–re-test reliability was used to reflect the external consistency.

Validity

Content validity This is reflected by the content validity index (CVI), which includes the item-level content validity index (I-CVI) and scale-level content validity index (S-CVI), which contains both scale-level CVI/universal agreement (S-CVI/UA) and scale-level CVI/average (S-CVI/Ave). The Delphi expert consultation method was used to assess the content validity of the CAF-R-CN. Seven experts were invited to evaluate and score the relevance of all items and their corresponding dimensions. A 4-point score recorded “not relevant”, “somewhat relevant”, “relevant”, and “highly relevant”. Content validity was considered good with an I-CVI >0.78, S-CVI/UA ≥0.80 and S-CVI/Ave ≥0.90.⁴⁰

Structure validity Two strategies were used to analyze the CAF-R-CN. After randomly dividing the data into two subsamples, one containing 224 participants was used to perform EFA. Only the Kaiser–Meyer–Olkin (KMO) value >0.80, and Bartlett’s test of sphericity was statistically significant (p <0.05), and EFA could be carried.^41,42 The other subsample (n = 224) was used to conduct CFA. The chi-squared degree of freedom ratio (CMIN, x²/df), root mean square error of approximation (RMSEA), comparative fit index (CFI), incremental fit index (IFI) and Tucker–Lewis index (TLI) were used to indicate the fit of the model. RMSEA ≤0.08, CFI, IFI and TLI ≥0.9, indicated a good fit of the model.⁴³

Convergent validity This was evaluated using the average variance extracted (AVE, >0.5) and composite reliability (CR, >0.7).^39,43 Convergent validity was evaluated by calculating the correlation coefficient between the dimensions and the total score of the CAF-R-CN with the BAI, BDI-13, and mMRC. Coefficients of 0.2, 0.40, 0.60, and 0.80 were considered weak, moderate, strong, and very strong, respectively.

Discriminant validity Critical ratio method: Ranked from high to low according to the total score on the CAF-R-CN, the top 27% was defined as the high group and the last 27% as the low group, using an independent samples t-test to explore differences between the two groups. Discriminant validity was considered good when the correlation coefficients between the dimensions were less than 0.5, and the dimensions corresponding to the AVE square root value were greater than the correlation coefficients between this factor and other factors.⁴⁴

Cutoff Score

The BAI was selected as the criterion tool, and a score of 45 was used to evaluate the optimal cut-off score of the CAF-R-CN. Receiver operating characteristic (ROC) curves were generated to evaluate the ability of the CAF-R-CN to screen for anxiety disorders.⁴⁵ An area under the ROC curve (AUC) >0.5, indicated an acceptable screening ability.⁴⁶

Results

Demographic and Clinical Characteristics

In total, 448 participants with COPD were included in the analysis. The mean age was 71.42 years (SD = 9.33). The proportion of women was 17.2%. Completion of the 25-item questionnaire took an average of 8.4 min (SD = 3.33). Table 1 presents the characteristics of the study population.

Table 1 Demographic and Clinical Characteristics (n=448)

Cross-Cultural Adaptation

After emailing BT3 to Professors Stenzel and Kühl, they provided suggestions for the BT3 version. “If I am short of breath, I want to be examined by a doctor” was modified to “If I become more short of breath, I want to be examined by a doctor”. “When I am short of breath, I get fear” was modified to “When I become more short of breath, I get afraid”. “I’m afraid that the long-term illness will be a burden to others” was modified to “I am afraid that my illness will one day make me a burden to others”.

Comments from the expert committee were as follows. A passive style of grammar common in German was used in the original scale, but this style is not common in Chinese, such as “I’m always afraid that one day I will be taken care of for the disease”, and “I believe that my feelings could not be understood by others”. Both items were modified to an active voice. As the item “I believe that my feelings could not be understood by others” was similar to “I believe that my conditions could not be understood by others” in the same dimension of FSE, these were integrated into one item, and one item was added: “I would not like to communicate with others because of the disease”. Furthermore, the experts put forward that people always value the support of family members in China, so the item “I do not feel valued by my family” was added to the dimension of FSE. “I avoid physical exertion” was judged to have a similar meaning to “I avoid physical activity as much as possible”, so only one item (“I avoid physical exertion as much as possible”) was retained. In addition, “I fear that one day I must stay in bed and cannot take care of myself” was added in the dimension of FP.

In the first round of consultation, the experts suggested that there were essential differences in medical security systems between China and Germany. Many patients with COPD in China experience a heavy financial burden. Based on the opinions of the expert committee, we proposed the development of a dimension of economic-related worries (ERW) after a literature review, patient interviews, and group discussions. Five items were included:

I fear that my income will not be enough to cover my treatment; I fear running out of savings for prolonged treatments; I would not like my children to pay for it; I fear an increasing the financial burden on my children; I fear that the final result might be losing both life and money.

The final version of the CAF-R-CN includes six of the 25-item dimensions. The dimension of ERW was added to the original scale, and the order of the items was also adjusted.

Item Analysis

In the first round of expert consultation, the I-CVI of item 9 on the original scale was 0.68, which was removed. The item-total correlation of CAF-R-CN (25-item) ranged from 0.38 to 0.84 (p <0.001).

Validity

Content validity

The dimension of ERW was added and the items on the original scale were revised. Seven experts were consulted in this study. The final results showed that the I-CVI was 0.86–1.00, the S-CVI/UA was 0.92, and the S-CVI/Ave was 0.99, suggesting excellent content validity.

Exploratory Factor Analysis (EFA)

After the content analyses, 25 items on the CAF-R-CN were used to perform EFA. First, we tested the decomposability of the subsample (n = 224). The KMO value was 0.915, and Bartlett’s test of sphericity was significant (X² = 4299.605, p <0.001), both of which were suitable for factor analysis. Using principal component analysis with an orthogonal rotation of the maximum variance, six factors based on the criteria of eigenvalues >1.00 were extracted, and these explained 74.79% of the total variance. The EFA results are presented in Table 2.

Table 2 Results of Exploratory Factor Analysis (n=224)

Confirmatory Factor Analysis (CFA)

CFA was performed using an SEM. Figure 1 shows the standardized model fit parameters. The SEM fit indices were CMIN = 2.326, RMSEA = 0.077 (p <0.05, 95% CI = 0.073–0.090), TLI = 0.912, IFI = 0.924, CFI = 0.924, suggesting a good fit to the data.

Figure 1 Structural equation model of the Chinese version of the CAF-R.

Convergent Validity

Table 3 shows the values of AVE and CR in all dimensions of the CAF-R-CN. All AVE was >0.5 and the CR was >0.7. Spearman’s rank correlation coefficient was between the dimensions and the total score of the CAF-R-CN with the BAI, BDI-13, and mMRC. Moderate to strong positive correlations (p <0.01) indicating good convergent validity are detailed in Table 4.

Table 3 Convergent and Discriminant Construct of CAF-R-CN (n=448)

Table 4 Validity and Reliability

Discriminant Validity

The differences between the two groups with high and low scores were significant (p <0.001). Cut-off scores were 31 (scores of 0–31 were assigned to the low group) and 62 (scores of 62 to the highest score were assigned to the high group), indicating good discrimination. The correlation coefficients between the dimensions and their coefficients between the dimensions corresponded to the AVE square root values, shown in Table 3.

Cutoff Score

Figure 2 shows the ROC curve for the CAF-R-CN. The AUC was 0.899 (p <0.05, 95% CI = 0.871–0.927). The optimal cut-off score was 50.5, with a sensitivity of 0.786 and a specificity of 0.870.

Figure 2 Receiver operating characteristics curve for the Chinese version of the CAF-R.

Missing Values

Seven questionnaires (1.6%) had missing values for items 5, 23, and 24, which involved children and other family members. However, some participants were single or had no children or family members. Data interpolation was performed using the multiple regression method to bring the data closer to the true level rather than deleting the data.

Discussion

It is important to focus on the comorbidity of anxiety in patients with COPD and promptly screen patients to provide treatment and avoid adverse outcomes.²⁷ The CAF-R has been found to be valid and reliable as measured in a large sample (n = 1025) in Germany.²³ However, the lifestyles and social-cultural backgrounds of patients with COPD in China differ from those of their counterparts in Germany. In this study, we translated and revised the CAF-R and examined its psychometric properties in a sample of 448 patients with COPD. Our findings suggested that the CAF-R-CN is an adequate and valid instrument for Chinese patients.

CAF-R-CN Has Good Reliability

Reliability reflects the stability and internal consistency of a scale. The CAF-R-CN showed acceptable to excellent reliability, with Cronbach’s α for the six dimensions ranging from 0.696 (SRW) to 0.910 (ERW). Cronbach’s α of the SRW dimension was 0.696, which was lower than that of the original study (0.79); nevertheless, the result was still acceptable The Cronbach’s α for the test–retest reliability of the CAF-R-CN was 0.949, indicating that the detection performance of the scale is steady.

CAF-R-CN Has Good Validity

Item 9 on the original scale “If I become more short of breath, I’m afraid that I might catch a cold” was removed as its I-CVI was 0.68 in the first round of expert consultation. The results of the final expert consultation indicated good content validity. The results of the item analysis showed good item validity.

Through EFA, six factors were extracted: FSE (items 1–5), FD (items 6–8, 17), FPA (items 10–13), FP (items 14–16, 18), SRW (items 9, 19–20), and ERW (items 21–25). Five factors were in line with Keil et al, except ERW.²³ Nevertheless, compared with the original scale,²³ item 17 (“I’m afraid my breathing problems would become worse”) in the dimension FP was loaded into the factor of FD; item 9 (“When I become more short of breath, I’m afraid I would suffocate”) in FD was loaded into the factor of SRW. It was difficult to distinguish the item “I’m afraid my breathing problems would become worse” from the dimension of “fear of dyspnea” in the current study, but hard to know whether the structure of the original scale or the linguistic or cultural differences between China and Germany led to this result. In addition, this might be owing to the mild condition of included participants who experienced less dyspnea and suffocation, so little obvious impact on sleep, which would result in “When I become more short of breath, I’m afraid I would suffocate” in FD loaded into the factor of sleep-related worries. The results of CFA showed acceptable fitting indicators, although there was a high correlation between the FD, FPA, and FP dimensions. The results were not as good as those of Keil et al, which might be related to the sample size.²³ Marsh et al argued that it was difficult to develop a good-fitting model if the factors were >5.⁴⁷ According to the final discussions in the research group, items 9 and 17 were still classified as dimensions consistent with the original scale.²³ Furthermore, we adjusted the order of the items on the scale.

The convergent validity of the CAF-R-CN performed well in AVE for all dimensions greater than 0.5, and the CR was greater than 0.7. Further explorations were conducted in the next step. The correlation coefficients between the total scores of CAF-R-CN and BAI, BDI-13, and mMRC were all significant (p <0.01) and the same for each dimension of the CAF-R-CN. These moderate-to-strong positive correlations indicated psychological anxiety and depression and symptom measurement properties in breathlessness on the CAF-R-CN. However, the total CAF-R-CN score showed a higher correlation with the BAI and BDI-13 than with the mMRC. This finding supported the fact that the CAF-R-CN mainly measured psychological features rather than the physical construct of disease-specific anxiety. In summary, the CAF-R-CN demonstrated satisfactory convergent validity.

The results for the critical ratio method were statistically significant (p <0.05), indicating that the CAF-R-CN is valid for assessing the level of response in different participants. Nevertheless, discriminant validity was not considered ideal, as the square root of the AVE for the FD dimension was less than its correlation with the FP dimension, which was consistent with the results of EFA. Health providers should recognize the possible overlaps among some items and dimensions and interpret the results with caution in future clinical practice or research.

Strengths and Limitations

The present study has several strengths. First, the face-to-face field method was adopted to gather data, and the efficiency of the collected questionnaire was 97.18% (a total of 13 invalid samples contained six cases because patients were too ill to complete the survey, and another seven were withdrawn), even with a mean age of the sample of 71.42 years. Second, adding the dimension of economic-related concerns can better reflect disease-specific anxiety in patients with COPD in the Chinese culture context. Finally, the BAI was selected as the criterion scale to evaluate the optimal cutoff score, which provided support for the clinical applicability of the CAF-R-CN. Health-care providers should provide early intervention to reduce the prevalence of anxiety among patients with a score >50.5 measured using the CAF-R-CN.

This study also has some limitations. First, the proportion of female participants (17.2%) was very low, which may have led to selection bias. Further investigation is required using a larger sample in the future. Second, participants were recruited from several medical institutions in Suzhou using convenience sampling, and most were hospitalized rather than community dwellers with serious illnesses, which may not be representative of patients with COPD in mainland China. More community-dwelling participants from more geographic areas should be included for validation of our findings in the future. Third, the discriminant validity of the CAF-R-CN in this study was insufficient, which may be due to the methods used by the original authors in developing the CAF or for other reasons.²² Future researchers should consider this when using the scale. Finally, an objective “gold standard” measurement was not used as a criterion; only the BAI was applied to evaluate the cut-off score of the CAF-R-CN, which may have reduced the accuracy of screening anxiety disorders. However, it’s difficult for a psychiatrist to perform “gold standard” assessment by interviewing each patient with COPD to diagnose anxiety disorders. In addition, the fact that some participants in this study were seriously ill may have hindered interview implementation. Researchers in future studies can attempt validation using the “gold standard”.

Conclusion

The CAF-R-CN consists of six dimensions with 25 items and was developed following the strict process of Brislin’s classic translation model, with satisfactory reliability and construct validity. Although the discriminant validity of some dimensions was not considered ideal, revisions and cross-cultural adaptations were applied, and a cut-off score of 50.5 was used. We recommend the use of the CAF-R-CN to assess the level of disease-specific anxiety among COPD patients in the Chinese sociocultural context.