Dyspnea assessment and pulmonary hypertension in patients with systemic sclerosis: Utility of the University of California, San Diego, Shortness of Breath Questionnaire

Authors

  • Lorinda Chung,

    Corresponding author
    1. Stanford University and VA Palo Alto Health Care System, Palo Alto, California
    • 3801 Miranda Avenue, VA Palo Alto Health Care System, Palo Alto, CA 94305
    Search for more papers by this author
    • Dr. Chung has received consultant fees, speaking fees, and/or honoraria (less than $10,000 each) from Gilead and Actelion, and research funding from Gilead, United Therapeutics, and Pfizer.

  • Hubert Chen,

    1. University of California, San Francisco, and Genentech, San Francisco, California
    Search for more papers by this author
  • Dinesh Khanna,

    1. University of Michigan, Ann Arbor
    Search for more papers by this author
    • Dr. Khanna has received consultant fees, speaking fees, and/or honoraria (less than $10,000 each) from Gilead, Actelion, Bayer, Roche, DIGNA, Pfizer, and Sanofi-Aventis, research funding from Gilead and Actelion, and serves on the Speakers' Bureau of Actelion and United Therapeutics.

  • Virginia D. Steen

    1. Georgetown University, Washington, DC
    Search for more papers by this author
    • Dr. Steen has received consultant fees, speaking fees, and/or honoraria (less than $10,000 each) from Gilead, Actelion, United Therapeutics, Roche, and Sanofi-Aventis, has served on the Speakers' Bureau for Actelion and Gilead, has served as a consultant for Gilead and United Therapeutics, and has received research funding from Actelion, Gilead, Pfizer, and United Therapeutics.


Abstract

Objective

The University of California in San Diego Shortness of Breath Questionnaire (UCSD SOBQ) has been used to assess dyspnea-related activity limitation in patients with airway and parenchymal lung disease. We sought to assess the construct validity and responsiveness of the UCSD SOBQ in systemic sclerosis (SSc; scleroderma) patients with incident pulmonary hypertension (PH) and those at high risk of developing PH.

Methods

We used data from 179 patients enrolled in the Pulmonary Hypertension Assessment and Recognition of Outcomes in Scleroderma Registry with pre-PH (defined by criteria on pulmonary function tests and/or echocardiogram) or definite PH with mean pulmonary artery pressure ≥25 mm Hg by right-sided heart catheterization within 6 months of enrollment. For this analysis, we included those subjects with complete data for self-reported measures at baseline and at 12 months.

Results

At baseline, the UCSD SOBQ had strong correlations in the expected direction with the disability index (DI) of the Health Assessment Questionnaire (HAQ) (r = 0.71, P < 0.0001), dyspnea assessment by visual analog scale (r = 0.71, P < 0.0001), and the Short Form 36 (SF-36) health survey physical component summary (PCS) score (r = −0.77, P < 0.0001), as well as a moderate correlation with the 6-minute walk test distance (r = −0.33, P < 0.0001), Borg dyspnea score (r = 0.47, P < 0.0001), and diffusing capacity of carbon monoxide (r = −0.33, P < 0.0001). Change in the UCSD SOBQ at 12 months correlated in the expected direction with change in the HAQ DI (r = 0.54, P < 0.0001) and change in the SF-36 PCS (r = −0.44, P < 0.0001). Multivariate analysis adjusting for age, sex, and race identified male sex as a significant predictor of death (odds ratio [OR] 7.00, 95% confidence interval [95% CI] 1.55–31.76), while the UCSD SOBQ showed a strong trend toward significance (OR 1.82, 95% CI 0.97–3.41).

Conclusions

The UCSD SOBQ demonstrates good construct validity and responsiveness to change in SSc patients with pulmonary vascular disease.

INTRODUCTION

Systemic sclerosis (SSc; scleroderma) is a chronic autoimmune disease characterized by fibrosis of the skin and internal organs and widespread vasculopathy. Pulmonary involvement, including interstitial lung disease (ILD) and pulmonary hypertension (PH), is common in patients with SSc and is the most common cause of death in recent years (1). PH, defined as a right ventricular systolic pressure (RVSP) on transthoracic echocardiogram (TTE) of ≥40 mm Hg, is observed in 28% of patients (2) and can be related to precapillary pulmonary arterial hypertension (PAH), pulmonary veno-occlusive disease, or secondary to left-sided heart disease or severe ILD. The prevalence of PAH, confirmed by right-sided heart catheterization (RHC), has been estimated between 8–12% (3, 4). Despite the availability of several PAH-specific therapies, patients with SSc-PAH experience poorer survival than patients with idiopathic PAH or other connective tissue disease (CTD)–associated PAH (5). The Pulmonary Hypertension Assessment and Recognition of Outcomes in Scleroderma (PHAROS) registry was established in 2006 to assess outcomes of patients at high risk of developing PAH and of those with incident disease (6). The premise of PHAROS is that early detection and close monitoring of PAH in patients with SSc could improve health-related outcomes.

Validated measures to monitor PAH disease progression and severity are necessary for clinical trials and the routine care of patients with SSc-PAH. Although the 6-minute walk test has been partially validated in SSc-PAH, and is routinely used as a primary outcome in PAH clinical trials (7), musculoskeletal and other systemic comorbidities complicate the interpretation of this test in patients with SSc. Brain natriuretic peptide (BNP) and N-terminal (NT) proBNP levels correlate with hemodynamics and exercise capacity in patients with SSc-PAH (8, 9), but require a blood draw and are affected by renal insufficiency. The University of California in San Diego (UCSD) Shortness of Breath Questionnaire (SOBQ) is a 24-item measure that assesses self-reported shortness of breath while performing activities of daily living (10). It has been validated and used extensively in studies of patients with obstructive lung disease and restrictive lung disease (11, 12). We chose this particular questionnaire because it focuses on dyspnea-limited activities rather than functional limitations. A recent study found that the UCSD SOBQ was strongly correlated with physical function and psychological well-being, and strongly inversely correlated with forced vital capacity (FVC) and diffusing capacity of carbon monoxide (DLCO) in patients with CTD-associated ILD (13). The utility of the UCSD SOBQ in the assessment of dyspnea in patients with pulmonary vascular disease has not yet been evaluated. We sought to assess the construct validity and responsiveness to change of the UCSD SOBQ in SSc patients enrolled in the PHAROS registry with incident PH or who are at high risk of developing PH. We hypothesized that the UCSD SOBQ would correlate in the anticipated direction with other validated patient-reported measures of disability and dyspnea, as well as with clinical measures of disease severity, in such a way to provide preliminary validity of the instrument for use in clinical trials of pulmonary vascular disease in SSc patients.

Significance & Innovations

  • This is the first study to evaluate the construct validity and responsiveness to change of the University of California in San Diego Shortness of Breath Questionnaire (SOBQ) in systemic sclerosis (SSc; scleroderma) patients with pulmonary vascular disease.

  • The SOBQ showed strong correlations with validated measures of disability, dyspnea, and health-related quality of life in patients with SSc from the Pulmonary Hypertension Assessment and Recognition of Outcomes in Scleroderma Registry.

  • The SOBQ was responsive to change in comparison to the Health Assessment Questionnaire disability index and the Short Form 36 physical component summary score.

  • The SOBQ may have utility in predicting mortality in patients with SSc-associated pulmonary hypertension.

PATIENTS AND METHODS

PHAROS is a longitudinal prospective registry involving 22 US scleroderma centers. Each participating center obtained institutional review board approval prior to patient enrollment, and all patients provided written informed consent prior to enrollment. The baseline characteristics and study design for PHAROS are described elsewhere (6).

The PHAROS registry enrolls SSc patients at high risk of developing PH (pre-PH) or those with definite PH diagnosed by RHC within 6 months of enrollment. Pre-PH patients must fulfill one of the following 3 criteria: 1) RVSP on TTE of ≥40 mm Hg, 2) FVC >70% predicted and DLCO <55% predicted, or 3) FVC/DLCO ratio of >1.6. RHC was obtained in pre-PH patients as clinically indicated and determined by the treating physician. Patients with definite PH must have a mean pulmonary artery pressure (mPAP) of ≥25 mm Hg at rest on RHC performed within the 6 months prior to enrollment into the registry.

The investigator at each site confirmed the World Health Organization (WHO) group classification according to the 2009 Dana Point Classification Criteria for PH: 1) WHO group I: PAH: mPAP ≥25 mm Hg on RHC with pulmonary capillary wedge pressure (PCWP) ≤15 mm Hg without significant ILD, defined as an FVC ≥65% predicted and no or only mild fibrosis on high-resolution computed tomography (CT) scan of the chest, 2) WHO group II: PH secondary to left-sided heart disease: mPAP ≥25 mm Hg on RHC with PCWP >15 mm Hg, and 3) WHO group III: PH secondary to chronic lung disease: mPAP ≥25 mm Hg on RHC with PCWP ≤15 mm Hg with significant ILD, defined as an FVC <65% predicted and/or moderate to severe ILD on high-resolution CT.

Pre-PH patients can be recategorized as definite PH patients based on RHC over the course of the study; however, only 2 pre-PH patients had RHC-confirmed PH at 1 year of followup. The following information is collected at baseline and at the annual followup: demographics, medications, physical examination findings, New York Heart Association (NYHA) functional classification assessment, autoantibodies, BNP/NT-proBNP levels if available, pulmonary function tests, TTE results, findings on high-resolution CT, 6-minute walk distance (6MWD), and hemodynamics on RHC if performed. Outcomes including hospitalizations for PH, deaths, and causes of deaths were recorded at the time of the occurrence.

Every 6 months, patients completed the following questionnaires: the UCSD SOBQ, the Health Assessment Questionnaire (HAQ) disability index (DI) (14), and the Short Form 36 (SF-36) health survey. The UCSD SOBQ is comprised of 24 items that assess dyspnea over the preceding week. There are 21 activity-based items that are scored on a 6-point Likert scale, with 0 indicating no shortness of breath and 5 indicating worst or unable to do secondary to shortness of breath. In addition, patients rate 3 questions related to limitations secondary to dyspnea, fear of harm from overexertion, and fear of shortness of breath. The total score is reported as the mean of all items, with 0 indicating no dyspnea and 5 indicating severe dyspnea (11). The minimum clinically important difference (MCID) for the UCSD SOBQ has been estimated as a 5-unit change on a summative scale of 0–120 (equivalent to a change of 0.21 on our 0–5 scale, averaged across items) (15). The HAQ DI includes 20 items in 8 functional domains (dressing/grooming, arising, eating, walking, hygiene, reach, grip, and activities), which assess the patient's usual abilities in the past 7 days. The overall score is calculated by summing the highest item score in each of the domains and dividing the sum by 8; the resulting score range is 0–3, where 0 indicates no disability and 3 indicates severe disability. The scleroderma-specific version of the HAQ (SHAQ) has 6 additional visual analog scales (VAS), which assess the severity of pain, gastrointestinal symptoms, shortness of breath, Raynaud's phenomenon, digital ulcers, and overall scleroderma disease activity in the past week. The HAQ DI and the SHAQ have demonstrated reliability and validity in assessing functional disability in patients with SSc (16, 17).

The SF-36 is a 36-item general health status instrument assessing 8 domains: 1) limitations in physical activities because of health problems; 2) limitations in social activities because of physical or emotional problems; 3) limitations in usual role activities because of physical health problems; 4) bodily pain; 5) general mental health (psychological distress and well-being); 6) limitations in usual role activities because of emotional problems; 7) vitality (energy and fatigue); and 8) general health perceptions (18). Two summary scores, the physical component summary (PCS) and the mental component summary, can be determined, ranging from 0–100, with higher scores indicating better health. The SF-36 has been used as a valid measure of health status in observational studies and clinical trials of patients with SSc (19–22).

For this analysis, we included only those subjects with complete data for key self-reported measures (UCSD SOBQ, HAQ DI, and SF-36) at baseline and at 12 months. Pearson's correlation coefficients were calculated to assess convergent validity of the UCSD SOBQ in relation to the HAQ DI, the breathing VAS of the SHAQ, the SF-36 PCS, the 6MWD, the Borg dyspnea score, and the DLCO corrected for hemoglobin measured at baseline. Correlations of r ≤ 0.29 were considered to be weak, between r = 0.30 and r = 0.49 were moderate, and r ≥ 0.50 were strong (23). Responsiveness to change of the UCSD SOBQ was assessed in a similar manner relative to changes in these same variables between baseline and 12 months. In addition, we calculated effect size, standardized response means (SRMs), and responsiveness statistics for categorical change in measures with reported MCID estimates (6MWD, HAQ DI, and SF-36 PCS) (24–26). These indices are ratios of observed change to a measure of variance, also known as signal to noise. For all 3 indices, the numerator is the mean change from the baseline to month 12 in the changed group and the denominators are the SD at baseline (effect size), the SD of change for the changed group (SRM), and the SD of change for people who are deemed not to change (responsiveness statistics). Univariate and multivariate (adjusting for age, sex, and race) logistic regression models were developed to assess whether the baseline UCSD SOBQ score is predictive of death during study followup.

RESULTS

Baseline characteristics.

Of 434 patients enrolled in PHAROS at the time of this analysis, 179 had complete data for self-reported outcomes of interest at baseline and at 12 months. Baseline characteristics of our analysis group are shown in Table 1. Mean ± SD age was 56 ± 11 years, 87% of patients were female, and 74% were white. Sixty-one percent of the patients had limited cutaneous disease, and the mean ± SD disease duration from the time of the first non–Raynaud's phenomenon symptom was 10 ± 11 years. Thirty percent of patients had definite PH verified by RHC at baseline. The majority of subjects were NYHA class I (43%) or class II (35%) at baseline, while only 18% and 3% were class III and class IV, respectively. The mean ± SD UCSD SOBQ score at baseline was 1.3 ± 1.1. The mean ± SD HAQ DI was 0.9 ± 0.8, the mean ± SD breathing VAS score was 0.9 ± 0.9, the mean ± SD SF-36 PCS score was 34 ± 12, the mean ± SD 6MWD score was 379 ± 139 meters, and the mean ± SD Borg dyspnea score was 2.3 ± 2.0.

Table 1. Clinical characteristics of 179 subjects with systemic sclerosis*
 Mean ± SD or no. (%)
  • *

    FVC = forced vital capacity; DLCO = diffusing capacity of carbon monoxide; RVSP = right ventricular systolic pressure; PAH = pulmonary arterial hypertension; PVH = pulmonary venous hypertension; ILD-PH = interstitial lung disease–associated pulmonary hypertension; NYHA = New York Heart Association; UCSD = University of California, San Diego; SOBQ = Shortness of Breath Questionnaire; SHAQ = Scleroderma Health Assessment Questionnaire; HAQ = Health Assessment Questionnaire; DI = disability index; VAS = visual analog scale; SF-36 = Short Form 36 health survey; PCS = physical component summary; MCS = mental component summary.

Demographic characteristics 
 Age, years56 ± 11
 Female155 (87)
 Race 
  White, non-Hispanic132 (74)
  African American30 (17)
  Hispanic8 (4)
  Other9 (5)
Clinical characteristics 
 Limited109 (61)
 Disease duration, years (n = 174)10 ± 11
 FVC, % predicted (n = 172)79 ± 18
 DLCO, % predicted (n = 171)48 ± 18
 RVSP by echocardiogram, mm Hg (n = 172)45 ± 17
 Definite pulmonary hypertension53 (30)
 Group I PAH38 (72)
 Group II PVH8 (15)
 Group III ILD-PH7 (13)
Health status measures 
 NYHA functional class, baseline 
  I75 (43)
  II61 (35)
  III32 (18)
  IV6 (3)
 NYHA functional class, 12 months 
  I66 (41)
  II57 (36)
  III35 (22)
  IV2 (4)
6-Minute Walk Test 
 Baseline 
  Distance, meters (n = 133)379 ± 139
  Borg dyspnea score (n =130)2.3 ± 2.0
 12 months 
  Distance, meters (n = 133)365 ± 141
  Borg dyspnea score (n =130)2.6 ± 2.1
UCSD SOBQ (range 0–5) 
 Baseline1.3 ± 1.1
 12 months1.3 ± 1.1
SHAQ 
 Baseline 
  HAQ DI total score (range 0–3)0.9 ± 0.8
  Breathing VAS score (n = 165)0.9 ± 0.9
 12 months 
  HAQ DI total score (range 0–3)0.9 ± 0.8
  Breathing VAS score (n = 165)0.8 ± 0.8
SF-36 PCS score (range 0–100) 
 Baseline34 ± 12
 12 months35 ± 12
SF-36 MCS score (range 0–100) 
 Baseline49 ± 11
 12 months49 ± 11

Convergent validity

At baseline, the UCSD SOBQ correlated in the expected direction with all measures (Table 2). Higher UCSD SOBQ scores (greater dyspnea) correlated strongly with worse disability as assessed by the HAQ DI (r = 0.71, P < 0.0001) and more severe dyspnea as measured by the SHAQ breathing VAS (r = 0.71, P < 0.0001). Higher UCSD SOBQ scores also correlated strongly with lower SF-36 PCS scores (worse physical health status) (r = −0.77, P < 0.0001). There were moderate correlations with poorer exercise capacity as assessed by lower 6MWD scores (r = −0.33, P = 0.0001), greater shortness of breath during exercise, as measured by the Borg dyspnea score (r = 0.47, P < 0.0001), and lower DLCO (r = −0.33, P < 0.0001).

Table 2. Correlation matrix demonstrating construct validity of UCSD SOBQ relative to conceptually related health status measures*
 UCSD SOBQHAQ DISHAQ breathing VASSF-36 PCS6MWDBorg dyspnea score
  • *

    UCSD = University of California, San Diego; SOBQ = Shortness of Breath Questionnaire; HAQ = Health Assessment Questionnaire; DI = disability index; SHAQ = Scleroderma Health Assessment Questionnaire; VAS = visual analog scale; SF-36 = Short Form 36 health survey; PCS = physical component summary; 6MWD = 6-minute walk distance (meters); DLCO = diffusing capacity of carbon monoxide.

  • Higher value indicates worse health status.

  • Higher value indicates better health status.

UCSD SOBQ     
HAQ DI      
 r0.71     
 P< 0.0001     
 N179     
SHAQ breathing VAS      
 r0.710.48    
 P< 0.0001< 0.0001    
 N165165    
SF-36      
 r−0.77−0.73−0.60   
 P< 0.0001< 0.0001< 0.0001   
 N179179165   
6MWD      
 r−0.33−0.40−0.240.35  
 P< 0.0001< 0.00010.0085< 0.0001  
 N133133123133  
Borg dyspnea score      
 r0.470.350.46−0.37−0.29 
 P< 0.0001< 0.0001< 0.0001< 0.00010.0014 
 N130130120130122 
DLCO      
 r−0.33−0.18−0.310.260.18−0.13
 P< 0.00010.0171< 0.00010.00060.03950.1541
 N171171157171127125

Discriminant validity.

UCSD SOBQ scores were able to discriminate subjects according to NYHA functional classification status (Figure 1A). Mean ± SD UCSD SOBQ scores increased in a stepwise fashion for each increase in NYHA functional class as follows: 0.8 ± 0.9 for class I, 1.4 ± 0.8 for class II, 2.2 ± 0.9 for class III, and 3.4 ± 0.8 for class IV (P < 0.0001 for trend). Baseline UCSD SOBQ scores were also able to discriminate between pre-PH and definite PH patients with a higher mean score (greater dyspnea) for those with definite PH (mean ± SD 1.1 ± 0.9 versus 1.9 ± 1.4; P < 0.0001) (Figure 1B).

Figure 1.

Discriminant validity of UCSD SOBQ by functional class and presence of pulmonary hypertension. A, SOBQ scores by NYHA functional class, P < 0.0001 for trend, B, SOBQ scores in patients “at risk” for pulmonary hypertension versus those with “definite” pulmonary hypertension, P < 0.0001. UCSD = University of California, San Diego; SOBQ = Shortness of Breath Questionnaire; NYHA = New York Heart Association; PH = pulmonary hypertension.

Responsiveness to change.

The mean ± SD change in UCSD SOBQ over 12 months was −0.06 ± 0.73 in our study. Using an MCID threshold of 0.21, 66 (37%) patients demonstrated a meaningful improvement in the UCSD SOBQ score and 50 (28%) patients demonstrated a meaningful worsening in the UCSD SOBQ score. Change in the UCSD SOBQ correlated in the expected direction with change in the HAQ DI (r = 0.54, P < 0.0001) and change in the SF-36 PCS (r = −0.44, P < 0.0001) (Table 3). There was a weak correlation with change in the SHAQ breathing VAS (r = 0.18, P = 0.025). No significant correlations were found between change in the UCSD SOBQ and changes in the 6MWD, Borg dyspnea score, or DLCO when assessed as continuous variables. Estimates of effect size, SRMs, and responsiveness statistics for categorical change in the 6MWD, the HAQ DI, and SF-36 PCS are shown in Table 4. The UCSD SOBQ demonstrated evidence of good responsiveness with respect to other related patient-reported outcomes, as indicated by effect size changes of >0.20 in either direction. The UCSD SOBQ was only minimally responsive to improvement in the 6MWD and was not responsive to worsening in the 6MWD, as indicated by effect size changes of <0.20. Similar patterns of responsiveness were observed for SRMs and responsiveness statistics.

Table 3. Correlation matrix demonstrating longitudinal construct validity of UCSD SOBQ in conceptually related health status measures over 12 months*
 Δ UCSD SOBQΔ SHAQ totalΔ Breathing VASΔ SF-36 PCSΔ 6MWDΔ Borg dyspnea score
  • *

    UCSD = University of California, San Diego; SOBQ = Shortness of Breath Questionnaire; SHAQ = Scleroderma Health Assessment Questionnaire; VAS = visual analog scale; SF-36 = Short Form 36 health survey; PCS = physical component summary; 6MWD = 6-minute walk distance (meters); HAQ = Health Assessment Questionnaire; DI = disability index; DLCO = diffusing capacity of carbon monoxide.

  • Higher value indicates worse health status.

  • Higher value indicates better health status.

Δ UCSD SOBQ     
Δ HAQ DI     
 r0.54     
 P< 0.0001     
 N179     
Δ Breathing VAS     
 r0.180.28    
 P0.0250.0003    
 N162162    
Δ SF-36     
 r−0.44−0.36−0.23   
 P< 0.0001< 0.00010.004   
 N179179162   
Δ 6MWD     
 r−0.12−0.11−0.19−0.09  
 P0.250.310.090.43  
 N86867786  
Δ Borg     
 r0.200.090.001−0.001−0.19 
 P0.060.420.990.990.11 
 N8484758475 
Δ DLCO      
 r−0.03−0.03−0.300.04−0.040.02
 P0.76540.76020.00250.69900.73290.8875
 N112112971127064
Table 4. Responsiveness of UCSD SOBQ for categorical change in 6MWD, HAQ DI, and SF-36*
Referent measureNChange in UCSD SOBQ, mean ± SDPImprovement of referent measureWorsening of referent measure
ESSRMRSESSRMRS
  • *

    UCSD = University of California, San Diego; SOBQ = Shortness of Breath Questionnaire; 6MWD = 6-minute walk distance (meters); HAQ = Health Assessment Questionnaire; DI = disability index; SF-36 = Short Form 36 health survey; ES = effect size; SRM = standardized response mean; RS = responsiveness statistic; PCS = physical component summary.

Change in 6MWD   0.120.210.150.030.050.03
 >40 meter improvement22−0.13 ± 0.62       
 No meaningful change380.09 ± 0.860.53      
 >40 meter worsening260.03 ± 0.61       
Change in HAQ DI         
 >0.14 point improvement47−0.54 ± 0.76 0.500.711.080.220.300.48
 No meaningful change850.04 ± 0.50< 0.0001      
 >0.14 point worsening470.24 ± 0.81       
Change in SF-36 PCS         
 >2.0 point improvement70−0.25 ± 0.64 0.230.390.320.240.380.33
 No meaningful change51−0.16 ± 0.780.0001      
 >2.0 point worsening580.26 ± 0.68       

Predictive validity.

A higher UCSD SOBQ score at baseline was associated with a greater risk of death during study followup, but this was not statistically significant in univariate analysis (odds ratio [OR] 1.57, 95% confidence interval [95% CI] 0.90–2.72) (Table 5). Multivariate analysis adjusting for age, sex, and race identified male sex as a significant predictor of death (OR 7.00, 95% CI 1.55–31.76), while the UCSD SOBQ showed a strong trend toward significance (OR 1.82, 95% CI 0.97–3.41).

Table 5. Predictive validity of the UCSD SOBQ based on relative risk of death during study followup*
PredictorOR (95% CI)P
  • *

    UCSD = University of California, San Diego; SOBQ = Shortness of Breath Questionnaire; OR = odds ratio; 95% CI = 95% confidence interval.

Univariate model  
 UCSD SOBQ1.57 (0.90–2.72)0.11
 Age1.07 (0.99–1.14)0.07
 Male4.97 (1.29–19.13)0.02
 White, non-Hispanic3.37 (0.42–27.31)0.26
Multivariate model  
 UCSD SOBQ1.82 (0.97–3.41)0.06
 Age1.07 (0.99–1.16)0.07
 Male7.00 (1.55–31.76)0.01
 White, non-Hispanic1.83 (0.19–17.28)0.60

DISCUSSION

When considering the validation of a new measure or descriptor to be used in clinical trials, one needs to consider its feasibility, its face, content, construct, and criterion validity, and discrimination, including its reliability and sensitivity to change, as described through the Outcome Measures in Rheumatology organization (27, 28). Feasibility measures how easily an index can be applied with regards to cost, time, and interpretability (27). Face validity refers to whether a measure appears to measure at least one aspect of the reference condition, while content validity is a measure of the comprehensiveness of the measure (28). Construct validity involves comparing the index score of interest against those derived from other validated indices, while criterion validity involves comparison to a gold standard (28). Reliability encompasses internal consistency, test–retest consistency or stability over time (intrarater reliability), and agreement between observers (interrater reliability) (28).

Sensitivity to change measures the responsiveness of an index to change over time. In this study we provide preliminary validation of the UCSD SOBQ as a measure of dyspnea in SSc patients at high risk of or with incident PH. The UCSD SOBQ demonstrated good construct validity with strong correlations in the expected directions with disability, dyspnea assessment by VAS, and physical health status, and a moderate correlation with exercise capacity and DLCO. The UCSD SOBQ also demonstrated good discriminative properties by differentiating among subgroups of patients stratified according to NYHA functional classifications and pre- versus definite PH status. Responsiveness to change was demonstrated with moderate to strong correlations in the expected directions with measures of physical health status (i.e., SF-36 PCS) and disability (i.e., HAQ DI), which was also further supported by calculated responsiveness indices. The lack of responsiveness to the 6MWD is not surprising as the 6MWD is confounded by musculoskeletal involvement and deconditioning in SSc. Finally, predictive validity of the UCSD SOBQ was suggested by a nonsignificant (P = 0.06) trend indicating a 1.8-fold increased risk of death per 1-point increase in dyspnea score at baseline.

Patient-reported outcome instruments are increasingly recognized as important tools for assessing disease severity and response to therapy. Measures of dyspnea may be particularly important in SSc, but the performance characteristics of these different instruments need further evaluation. In a cohort of unselected SSc patients from Canada, dyspnea as assessed on an 11-point numerical rating scale had a moderate correlation with the HAQ DI and SF-36 PCS (Kendall's τ = 0.30 and τ = −0.46, respectively, P < 0.001 for both) (29). A recent study of SSc patients followed at Northwestern University found strong correlations between the Functional Assessment of Chronic Illness Therapy (dyspnea) and dyspnea-related functional limitation scores and the HAQ DI (r = 0.64 and r = 0.76, respectively) (30). Another study showed moderate correlations in the expected directions between functional disability and 2 measures of breathlessness, the Mahler's baseline dyspnea index (BDI) (r = −0.33, P < 0.01) and the breathing VAS on the SHAQ (r = 0.46, P < 0.01), in patients with SSc-ILD (20). This study also found strong correlations in the expected directions between these 2 dyspnea scales and the SF-36 PCS (r = 0.51 and r = −0.59 for the BDI and the VAS, respectively, P < 0.01 for both) (20).

We chose the UCSD SOBQ for our study since this instrument focuses on dyspnea alone rather than health-related quality of life or functional capacity, which are already measured by the SF-36 and the HAQ DI. Our study found strong correlations between the UCSD SOBQ and disability and physical health status in SSc patients at high risk of or with incident PH (r = 0.71 and r = −0.77, respectively). A small study in 28 patients with SSc and restrictive lung disease found a strong inverse correlation between the Saint George's Respiratory Questionnaire and the 6MWD (r = −0.77, P < 0.0001) (31). Similarly, our study also found an inverse correlation between the UCSD SOBQ and 6MWD; however, the strength of correlation was not as robust (r = −0.33).

The UCSD SOBQ was originally developed for use in the pulmonary rehabilitation setting and has been used extensively in studies of chronic obstructive pulmonary disease (COPD) (11). Mean scores in patients with moderately severe COPD are typically in the mid-50s on a scale of 0–120 (approximately 2.3 on the scale used in this study) (15, 32). A recent study in patients with CTD-ILD found a mean SOBQ score of 46 out of 120 (equivalent to 1.9 on the scale used in this study) (13). Our study is the first to evaluate the utility of the SOBQ in SSc patients with pulmonary vascular disease. The mean ± SD UCSD SOBQ score in the PHAROS cohort was lower (1.3 ± 1.1) than the CTD-ILD population in the Swigris et al study, which is consistent with less severe dyspnea despite similar pulmonary function (mean FVC 72% in CTD-ILD versus 79% predicted in PHAROS patients and mean DLCO 52% in CTD-ILD versus 48% predicted in PHAROS patients) (13). The low UCSD SOBQ scores may reflect the inclusion of a large proportion of patients in classes I and II in the PHAROS registry related to the routine screening and early detection of PH in this cohort. Similar to our findings, Swigris et al showed a strong correlation between the UCSD SOBQ and functional disability (r = 0.58, P < 0.0001), as well as lower DLCO (r = −0.53, P = 0.004) in patients with CTD-ILD (13). Although dyspnea may be qualitatively different among these different pulmonary disease states due to differences in the underlying pathophysiology, the UCSD SOBQ quantifies dyspnea-related activity limitations, which are likely similar among different pulmonary conditions.

To our knowledge, our study was also the first to assess the responsiveness to change of a dyspnea scale in a prospective longitudinal cohort of SSc patients with pulmonary disease. Although the mean SOBQ score across the entire study cohort remained relatively unchanged, 37% of patients improved and 28% of patients worsened by the MCID at 12 months. We found moderate to strong correlations in the change of the UCSD SOBQ with change in SF-36 PCS and HAQ DI, but no significant correlation with change in 6MWD or Borg dyspnea scores. When evaluating the 6MWD as a categorical variable using the MCID of 40 meters as a cut point (24), the mean improvement in UCSD SOBQ was 0.22, slightly greater than the MCID of 0.21, but this was not statistically significant. These findings were not unexpected and are consistent with the fact that dyspnea as assessed by the UCSD SOBQ represents a distinct construct from walk test parameters obtained under artificially imposed conditions. Notably, the UCSD SOBQ may be less confounded by musculoskeletal disease, and therefore may be a more accurate reflection of cardiopulmonary status in SSc patients than the 6MWD. In addition, the UCSD SOBQ provides a more comprehensive assessment of dyspnea than the Borg dyspnea score, which asks for a single rating of breathlessness immediately after exercise. The change in UCSD SOBQ also did not correlate with change in DLCO at 12 months, suggesting that lung function and subjective symptoms may change at different rates in SSc patients at high risk of or with incident PH. This is not unexpected given the overall slow decline in DLCO observed in patients with SSc-ILD (33).

Our study also evaluated the predictive validity of the UCSD SOBQ on mortality in the PHAROS cohort. Male sex was an independent predictor of death, in agreement with other survival studies in patients with SSc (34, 35). There was a strong trend indicating an association between higher baseline UCSD SOBQ scores and death during study followup. Given that there were only 10 deaths recorded, confirmation of the predictive value of the UCSD SOBQ with respect to death in SSc-PH patients will require a longer observation period in a larger cohort of patients with definite PH.

As with all registry studies, our study was limited by missing data, particularly with respect to 6MWD and Borg dyspnea scores. This may have limited our ability to detect smaller correlations between these variables and the UCSD SOBQ. We did not assess for correlations with RVSP or hemodynamic parameters given the poor predictive value of these measures in patients with SSc-PAH compared with those with idiopathic disease (5, 36). We were also unable to assess correlations between the UCSD SOBQ and BNP/NT-proBNP levels because of inadequate laboratory data. All of the centers involved in PHAROS are scleroderma centers at which routine screening for PAH with annual pulmonary function tests and TTEs is performed. Therefore, our results may not be generalizable to SSc-PH populations followed in the general community. Finally, responsiveness to change analyses were limited by the short followup period and the modest proportion of patients meeting criteria for minimal change.

In conclusion, we have shown that the UCSD SOBQ demonstrates acceptable measurement properties in SSc patients with pulmonary vascular disease. Specifically, it converges with related constructs in the anticipated manner and appears to be responsive to changes in related measures of health status. Our study provides evidence that the UCSD SOBQ is a potentially useful measure of disease severity and progression that provides complementary information in addition to the 6MWD in SSc patients at high risk of and with PAH.

AUTHOR CONTRIBUTIONS

All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be submitted for publication. Dr. Chung had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design. Chung, Chen, Khanna, Steen.

Acquisition of data. Chung, Khanna, Steen.

Analysis and interpretation of data. Chung, Chen, Khanna, Steen.

ROLE OF THE STUDY SPONSOR

Actelion and Gilead had no role in the study design or in the collection, analysis, or interpretation of the data, the writing of the manuscript, or the decision to submit the manuscript for publication. Publication of this article was not contingent upon approval by Actelion and Gilead.

ADDITIONAL DISCLOSURE

Dr. Chen is a full-time employee of Genentech. Genentech does not have any products marketed for the treatment of systemic sclerosis or pulmonary arterial hypertension. Genentech had no financial interest in this project and had no input in the design, content, data collection, or analysis, and had no role in the writing or approval of this article, with all opinions and conclusions expressed herein those of the authors.

Acknowledgements

The authors would like to thank Dr. Daniel Furst for critically reviewing the manuscript.

Ancillary