Psychometric validation of a Hindi version of a chronic obstructive pulmonary disease (COPD) assessment test in patients in northern India

Introduction

Chronic obstructive pulmonary disease (COPD) is a common, preventable lung disorder characterized by progressive, poorly reversible airflow limitation often with systemic manifestations, in response to tobacco smoke and/or other harmful inhalational exposures.^[1] It is a widely prevalent disease in India and worldwide, and contributes considerably to health expenditure, morbidity and mortality among adults.^[1],[2] Evaluation and severity assessment of patients with COPD traditionally revolves around clinical and physiological parameters such as symptom enquiry, assessment of respiratory failure and other complications, and pulmonary function testing. Although these tools are useful to describe the general clinical status of a patient with COPD, they fail to capture the overall patient-centred health status. Patients with COPD have to deal with steadily worsening functional limitation resulting from breathlessness, airflow limitation and disease complications. In addition, they also face personal, social, psychological and economic hardships that impair their overall quality of life. These facets are often much more important to the patient than a simple assessment of disease severity by a physician. Unfortunately, clinicians seldom assess the impact of disease on health status from a patient’s perspective.

The COPD assessment test (CAT) is a commonly used disease-specific health status tool for evaluating functional impairment in patients with COPD. It was developed as a simple eight-item instrument through modern questionnaire methodology and formal psychometric evaluation.^[3],[4] The original questionnaire has been translated into several languages, is being used worldwide, and even gets recommended by international guidelines as a useful measure for assessment and severity categorization of patients with COPD.^[5] Only a few investigators in India have reported preliminary findings on the use of the CAT.^{[6],[7],[8],[9],[10],[11],[12]} However, linguistic translations of this instrument have not been formally validated in Indian patients with COPD. The CAT has been earlier validated in several other Asian countries.^{[13],[14],[15]} There is a need to verify the performance characteristics of the CAT translated into local languages, before it can be recommended for routine clinical applications in India. We previously validated the use of the Hindi translation of St George’s Respiratory Questionnaire (SGRQ) in describing health-related quality of life (HRQL) in patients with COPD.^[16] We therefore undertook a detailed psychometric evaluation of the Hindi version of the CAT in patients with COPD in northern India.

Methods

We studied 178 patients with COPD attending the Chest Clinic at our institute. Diagnosis of COPD was based on characteristic symptoms and suggestive clinical signs, documented airflow limitation on spirometry and presence of established risk factors (such as tobacco smoking, domestic exposure to solid fuel combustion or environmental tobacco smoke, or occupational exposure to mineral dust).1 Subjects were included in the study only if they understood Hindi well. Patients with comorbid conditions that could potentially diminish the level of activity or worsen quality of life (such as cardiovascular, renal, arthritic disorders, etc.) were excluded. The study protocol was approved by our hospital ethics committee, and informed written consent was obtained from all participants before enrolment.

All patients underwent detailed symptom enquiry, physical examination and lung function testing at the initial evaluation. Severity of dyspnoea was graded using the Modified Medical Research Council (mMRC) scale.^[17] Spirometry was performed on a rolling seal spirometer (Spiro RS232, P.K. Morgan Ltd., Kent, UK), and patients’ post-bronchodilator forced expiratory volume in first second (FEV1) and forced vital capacity (FVC) values were compared to predicted norms previously derived at our institute.^[18],[19] Severity of COPD was categorized as per recommendations of the national guidelines on diagnosis and management of COPD in India.^[1] Each patient was scheduled for evaluation twice over a 4-week interval.

At each assessment, patients were requested to complete the Hindi version of the CAT questionnaire themselves without any assistance. The Hindi tool used in this study is available at www.catestonline.org/english/index_Hindi.htm and can be freely used for non-commercial purposes. We used the CAT tool exactly as available from the developers without any modification. The CAT questionnaire has eight items related to cough, phlegm, chest tightness, dyspnoea, limitation of activities, confidence, sleep and energy, with each item scored on an anchor-based semantic differential scale from 0 to 5. Response scores for each item were summated to obtain a final score that ranged from 0 to 40, with a lower score suggestive of better health status. CAT scores below 10, 10–20 and above 20 were considered suggestive of low, medium and high impact, respectively.^[20] Missing data were recorded as such.

Patients also completed the Hindi version of the SGRQ, a specific HRQL measure for chronic respiratory diseases, which consists of 50 items (with 76 weighted responses) covering three independent domains. The three domain scores: Symptoms (8 items), Activity (16 items), and Impact (26 items), and a total score, were computed using item-specific weights, as per guidelines from the developers.^[16],[21] Scores for each component ranged from 0 to 100, with lower scores reflecting better HRQL. The Hindi version of the abbreviated World Health Organization Quality of Life (WHOQOL-Bref) questionnaire, which is a generic HRQL instrument, was also administered to all patients.^[22],[23] The WHOQOL-Bref consists of 26 items on a 5-point Likert scale with a 2-week recall period. Four domain scores: Physical (7 items), Psychological (6 items), Social relationships (3 items) and Environment (8 items) were calculated as per standard guidelines, and transformed to a scale of 0–100 to enable direct comparisons between domains having unequal numbers of items.^[23] Higher domain scores indicated better HRQL.

Psychometric properties of the CAT were evaluated through several methods.^[24] Acceptability was assessed by noting the proportion of CAT questionnaires completely answered by the respondents. Convergent validity was assessed by determining correlations of the CAT score with other disease severity and HRQL assessments at the initial evaluation, and by assessing the degree of correlation between each item and the total CAT score. Divergent validity was evaluated by noting lack of significant correlation between the CAT score and WHOQOL-Bref scores of unrelated domains. Spearman’s rho correlation coefficients were used for all of the above, and a coefficient exceeding ±0.4 was considered as good correlation.^[25] Internal consistency (correlation of individual items with each other) was determined through Cronbach’s alpha on data from initial administration, with the coefficient greater than 0.70 being considered acceptable.^[25] The random-effects intraclass correlation coefficient was used to compute test–retest reproducibility of the two CAT administrations among patients who did not have any exacerbation at either visit or between the two evaluations. Responsiveness was estimated by calculating the effect size in the subgroup of patients who had an exacerbation during the initial evaluation, but had recovered by the next assessment. ‘Effect size’ was defined as mean of absolute magnitude of score change between the two assessments, divided by standard deviation of baseline score, and values of 0.2, 0.5, 0.8 considered to signify small, medium, and large effect sizes, respectively.^[26] The Wilcoxon signed ranks test or Kruskal–Wallis test was used for all group comparisons in view of a non-normal distribution of several variables of interest, and p<0.05 considered statistically significant.

Results

A total of 178 patients with COPD were enrolled in the study. Of them, 47 (26.4%) patients lived in Chandigarh, and others were residents of Punjab (42, 23.6%), Himachal Pradesh (33, 18.5%), Haryana (30, 16.9%), Uttar Pradesh (14, 7.9%), or other states (14, 9%). One patient each did not respond to items 4, 6 or 7 of the CAT questionnaire, and two patients each did not respond to items 5 or All these 7 patients were males, and all except one were tobacco smokers. This translated to an overall acceptability rate of 96.1%. Since a baseline CAT score could not be computed for these 7 patients, further analysis was restricted to the remaining 171 respondents [Figure - 1]. There were 160 males and 11 females, with duration of symptoms ranging from 2 to 15 years. In all, 152 males (95%) and only 1 female (9.1%) had smoked tobacco [Table - 1]. All non-smokers reported exposure to smoke from solid fuel combustion at home and/or work. Forty-four (25.7%) patients reported one or more exacerbation in the past year, and the vast majority had moderate or severe COPD [Table - 1]. The median (interquartile range [IQR]) FEV1 (% predicted), FVC (% predicted) and FEV1/ FVC (%) were 53.7 (37.1–73.5), 78.6 (63.5–90.0) and 55.2 (44.6–65.5), respectively. At the initial assessment, 19 (11.1%) patients had an acute exacerbation, all of whom had recovered fully by the subsequent follow-up visit. Only 120 (70.2%) patients returned for the scheduled second assessment during the specified timeframe, and none of them had new-onset disease exacerbation at that time or between the two evaluations [Figure - 1].

Table 1: Baseline characteristics and COPD assessment test (CAT) scores of the study population

Figure 1: Patient flow in the study

Individual responses to each item of the CAT questionnaire were variable at the baseline evaluation, with no clear floor or ceiling effects [Figure - 2]. A broad spectrum of CAT scores (from 1 to 38) was recorded, and individual scores spanned almost the entire range of possible scores (0 to 40). The median (IQR) score for the study population was 16 (10–24). The total number of patients with CAT scores <10, 10–20 and >20 were 33 (19.3%, low impact), 74 (43.3%, medium impact) and 64 (37.4%, high impact), respectively. There were no significant differences in CAT scores with regard to gender, age, tobacco smoking or disease exacerbations in the previous year [Table - 1].

Figure 2: Distribution of responses to individual items of COPD assessment test (CAT) questionnaire

Baseline CAT scores correlated well with all domain and total scores of the SGRQ, as well as the physical domain score of WHOQOL-Bref [Table - 2]. However, there was poor correlation between CAT scores and WHOQOL-Bref scores in the psychological, social relationships and environment domains that appeared unrelated to the primary CAT construct [Table - 2]. There was a trend towards poorer CAT scores with worsening disease severity. Median (IQR) CAT scores got progressively worse for patients with mild, moderate and severe COPD [Table - 1], and these differences were statistically significant. The CAT scores also correlated significantly with the mMRC dyspnoea grade [Table - 2], and the median CAT scores rose significantly with worsening dyspnoea class [Table - 1]. However, 64 of 90 patients (71.1 %) having an mMRC grade 0–1 dyspnoea had the CAT score of 10 or more, and 7 of 81 patients (8.6%) having an mMRC grade 2–4 dyspnoea had the CAT score below 10. Patients having disease exacerbation at the initial evaluation had a marginally worse CAT score than those not having exacerbation (median [IQR] scores 18 [16–28] and 16 [10–23], respectively). Although the CAT scores also correlated statistically significantly with per cent predicted FEV1, the absolute value of Spearman’s rho coefficient was low [Table - 2]. All individual items correlated well with the total CAT score, with Spearman’s rho coefficients ranging from 0.59 to 0.73. The CAT scores correlated Cronbach’s alpha coefficient also had a high value (0.834), implying good internal consistency.

Table 2: Correlation of baseline COPD assessment test (CAT) scores with pulmonary function and other measures of health- related quality of life

Among 152 patients who did not have exacerbation at the initial assessment, 101 (66.4%) returned for the scheduled follow-up assessment. Baseline and follow-up CAT scores were similar in these patients (median [IQR] scores 16 [10–23] and 17 [10–24], respectively, p=0.12). The random-effects intraclass correlation coefficient had a high value (0.83), suggesting good test–retest reproducibility. All 19 patients with baseline COPD exacerbation returned for the scheduled follow-up assessment, and clinical improvement was seen among them. The median (IQR) improvement in the CAT score between the two visits was 4 (3–9). The difference between the two assessments was statistically significant (median [IQR] CAT scores 18 [16–28] v. 16 [12–22], p=0.03), and the overall effect size was 0.45, indicative of moderate responsiveness.

Discussion

Our study provides the first detailed psychometric analysis of the performance of CAT in Indian patients with COPD. Such validation of a linguistic translation of the original questionnaire is essential before the instrument can be recommended for routine use in our patients. The global initiative for obstructive lung disease (GOLD) guidelines recommend the use of CAT in stratifying COPD disease severity. The Indian guidelines, however, do not include this parameter, primarily because of lack of formal data regarding the utility of the instrument in Indians.^[1] Our study attempts to bridge the gap in this important area.

COPD impacts several facets of a patient’s life, and CAT captures some of this information through a patient-reported rather than the physician-enquired method. This is one reason why CAT scores may not correlate well with other clinical and physiological assessments, as different patients perceive the impact of their symptoms and limitations differently. For instance, the vast majority of patients in this study had the mMRC dyspnoea grade of 0–1 (52.6%), and mild-to-moderate disease severity (62.5%). Yet more than 80% patients had a CAT score of 10 or more, which is the cut-off used by the GOLD guidelines to define symptomatically worse disease. The CAT scores correlated well with the symptom of breathlessness as assessed by the mMRC scale. However, the equivalence between a CAT score of 10 or more and the mMRC dyspnoea grade of two or more (as suggested by the GOLD guidelines) was rather poor, with more than 70% less symptomatic patients (mMRC dyspnoea grade 0–1) reporting CAT scores of 10 or more. Recent studies have indicated the lack of such empirical equivalence between the CAT score and the mMRC dyspnoea grade.^{[27],[28],[29]} The correlation of the CAT score with FEV1 was also poor, as has also been noted in earlier studies.^[14],[30] All this suggests that the vast majority of our patients had poor health status, and that there was substantial heterogeneity in health status impairment across patients with different disease severity.

Overall acceptability of the Hindi version of the CAT was high, with only 7 of 178 respondents returning questionnaires with only 1 item unanswered. Some previous studies have considered such questionnaires usable, by extrapolating the missing value from responses to other questionnaire items. However, we chose a stricter criterion by excluding such patients in this initial validation study. There was no questionnaire in which more than one response was missing. Demographic factors of patients, such as age, gender and tobacco smoking did not influence the CAT scores significantly; similar observations were made by other investigators.^[15] However, the number of women and non-smoking individuals in the study was rather small (consistent with the general epidemiology of COPD) and this might pose a limitation in extrapolating our results in this subset of COPD patients.

We studied questionnaire validity using several approaches. As expected, CAT scores progressively increased with worsening severity of COPD. The scores also correlated significantly with the SGRQ scores, with Spearman’s correlation coefficient of 0.634 with the total SGRQ score. Five previous studies have reported that Spearman’s correlation coefficient between the CAT and SGRQ scores in the range of 0.65 to 0.84.15 However, CAT scores correlated only with the physical domain of the generic WHOQOL-Bref, suggesting that the CAT mainly assesses the impact of disease in this domain.

We obtained good reliability statistics for CAT administration. Cronbach’s alpha, a measure of internal consistency, was 0.83, indicating good correlation between questionnaire items. Previous studies have reported this coefficient ranging from 0.85 to 0.98.15 Test–retest reproducibility for CAT administered on two occasions to patients with stable disease was also good, with intraclass correlation coefficient of 0.83. Previous studies have reported similar values ranging from 0.80 to 0.96.15 This suggests that the CAT scores remain largely consistent when the questionnaire is repeatedly administered to stable patients.

The 19 patients with exacerbation of COPD at baseline showed reduction in their CAT scores that paralleled the clinical recovery noted among them, with a median reduction of 4 points. There is no consensus on the minimal clinically important difference (MCID) for CAT under such a scenario. While some studies advocate an MCID of 2 units, others report a decrease ranging from 3.3 to 3.8 units.^[15] Along with an effect size of 0.45, these data suggest moderate responsiveness of the CAT.

Overall, our findings suggest that the scores obtained from administering the Hindi version of the CAT to COPD patients from northern India are a good indicator to quantify patient-perceived impact of disease. The questionnaire has good acceptability, validity and reliability, and can be routinely used for disease evaluation in the outpatient setting.