Development of a new instrument for rheumatoid arthritis: The Cedars-Sinai Health-Related Quality of Life instrument (CSHQ-RA)
To update and complement existing instruments, we developed a multidimensional disease-specific instrument, intended to reflect the impact of rheumatoid arthritis (RA) with modern treatment options on patient's Health-Related Quality of Life (HRQOL).
Items were developed from a systematic review of published HRQOL measures and transcripts of RA patient focus groups. Items were refined by an expert panel and administered to 350 patients for psychometric testing.
The systematic review identified 228 potential items, and the focus group transcripts identified 96 additional items. Expert review and pilot testing resulted in an initial 58-item instrument. Twenty-six items were excluded due to floor/ceiling effects, poor response rates, or high item-item correlations. Factor analysis identified a 5-factor structure (eigenvalues ≥1). Multi-trait scaling performed on both completed surveys confirmed the 5 sub-scale structure (Cronbach's > 0.87).
The CSHQ-RA consists of 33 items that address 5 HRQOL domains, each with high internal consistency. Additional testing will assess the instrument's validity and responsiveness.
Rheumatoid arthritis (RA) is a chronic and progressive inflammatory musculoskeletal disease (1–3), affecting approximately 2.1 million Americans (4). The course of the illness can result in joint destruction, impaired psychological and social functioning, as well as increased mortality (2–5). Economically, the disease is responsible for $4.2 billion in medical costs annually (6). Annual indirect costs (primarily due to premature disability) are estimated to be between $13,180 and $65,880 per patient. Indirect costs are estimated to be higher than direct costs due to the disability experienced by the majority of RA patients within 10 years of disease onset (7). Significant resources have been devoted to developing new treatments in an effort to reduce the impact of RA at the individual and societal levels. Accompanying this effort is the need to accurately assess changes in different aspects of patients' health including physical function and health-related quality of life (HRQOL) due to disease activity and therapeutic efficacy.
Health-related quality of life refers to those aspects of human life and activities that are generally affected by health conditions or health services (8). Many instruments have been applied to measure the HRQOL of patients with RA. These instruments are categorized as generic (e.g., the Medical Outcomes Study Short-Form 36 [SF-36]) (9) or disease-specific (e.g., Arthritis Impact Measurement Scale [AIMS]) (10, 11).
Unlike disease-specific instruments, generic instruments contain general health questions that allow for comparisons across diseases (12). However, generic instruments often do not include items that are uniquely relevant to patients with different diseases. Consequently, they tend to be less responsive than those that are disease-specific (3, 13, 14). Yet, few RA disease-specific HRQOL instruments have been developed. Well-validated instruments, such as the Health Assessment Questionnaire (HAQ) Disability Index, which have been used extensively in clinical trials, (15) measure functional status, not the broader and more inclusive construct of HRQOL. Further, HRQOL instruments that are in use were initially developed over 20 years ago (e.g., the AIMS), limiting the extent to which they may completely address current vernacular and patterns of daily life for patients with RA. This is especially pertinent given the new realities of more aggressive and early management of RA in the 1990s with costly interventions that could potentially possess unique adverse effects. Further, such instruments may not be sufficiently sensitive or responsive to detect relatively small changes in HRQOL associated with newly developed clinical interventions (16). Thus valid, reliable, and sensitive RA-specific HRQOL instruments are needed as new therapeutic advances that could potentially reduce the impact of disease on patients' overall health are introduced.
Internal Review Board approval was obtained from the Cedars-Sinai Health System (Los Angeles, CA) in January 2001 for conducting patient focus groups and survey administration.
Item Pool Development.
A systematic search of 3 computerized bibliographic databases (MEDLINE, HEALTHSTAR, and COCHRANE) was performed to identify HRQOL instruments that were validated on (or applied to) patients with RA between January 1975 and December 2000. Based on claims made by the authors, articles were included in the review if they measured outcomes in at least 1 of 4 domains: psychological, social, physical function, and symptoms and assessed at least one of 3 psychometric properties: reliability (i.e., test-retest, internal consistency, and inter-rater), validity (i.e., convergent and discriminant), and responsiveness.
Items of all identified RA- or general arthritis-specific HRQOL instruments were included in the initial item pool. Additional items were developed from the transcripts of 8 patient focus groups (n = 160) conducted across the US in which participants were asked to discuss how RA had impacted their social, physical, and emotional well-being. The resulting item pool was organized based on the 8 domains that comprise the SF-36 (9). The item pool was reviewed for clarity, redundancy, and comprehensiveness by a 6-member expert panel comprised of researchers with knowledge of HRQOL instrument development and clinical experience in treating RA. A pilot instrument was then prepared and piloted on a focus group of 11 RA patients who were asked to score individual items on a 9–point Likert scale based on clarity and relative importance to overall HRQOL. The expert panel further eliminated items felt to be ambiguous, unclear, and/or not directly related to HRQOL. The 6-member expert panel voted anonymously on whether items violating the above criteria were to be retained or deleted. At least 4 panel members were required to be in agreement if an item was to be either deleted or retained. All of the 8 HRQOL domains of the SF-36 were represented by items in the final draft survey.
For purposes of survey validation, 957 adult, English-speaking patients with RA were identified in 13 rheumatology practices in the metropolitan Los Angeles area by their treating physicians. The target sample size of 300 patients was determined in order to generate correlation coefficients with minimal standard errors. The first 350 respondents who returned a mailed study invitation were enrolled. The initial mailing included the draft HRQOL survey in which respondents are instructed to consider how RA has impacted a specified area of their HRQOL during the last 4 weeks. Items are phrased in one of three 5-point Likert response formats: 1) difficulty (“Not Difficult” to “Extremely Difficult”), 2) severity (“Not At All” to “Severe”), and 3) frequency (“Never” to “Always”). Additionally, respondents were mailed a sociodemographic and medical questionnaire and 3 general health status questions: 1) patient's global assessment of functioning (Visual Analog Scale [VAS]) (17), 2) patient's global assessment of joint pain (VAS), and 3) overall rating of health (SF-36). A reminder phone call was made to those who did not return the first mailing within 1 week of mailing.
After 4 weeks, patients responding to the first survey mailing were sent the draft HRQOL survey a second time along with the SF-36 and the Stanford Health Assessment Questionnaire Disability Index (15).
Items were considered for deletion, rephrasing, or collapsing if they violated any of the following 3 criteria: 1) missing responses greater than 5% (18), 2) more than 50% of respondents chose either highest (ceiling effect) or lowest (floor effect) rating offered, or 3) significant item to item correlations ≥0.70 (18).
Principal component factor analysis and multi-trait analysis.
The number of factors was determined based on eigenvalues ≥1, with factor loadings serving as an indication of the degree to which each item was associated with each factor. Items were retained in a given factor if they had a standard regression coefficient ≥0.30. Items with standard regression coefficients ≥0.30 and that loaded nearly equally on 2 or more factors were retained on the factor that was most conceptually similar as determined by the expert panel.
Multi-trait scaling analysis was carried out to evaluate item convergence within scales and item discrimination across scales. A priori instrument reliability criteria included: 1) items needed to correlate ≥0.40 with the total questionnaire (i.e., item-internal consistency) (18), 2) ≥ 80% of item-hypothesized scale correlations ≥ 2 standard errors from all other scales (item-discriminant validity) (19), and 3) Cronbach's alpha coefficients ≥ 0.70 (internal consistency) (20).
Statistical analyses were performed using the Statistical Analysis System version 6.12 for Windows (SAS Institute, Cary, NC), and the Multitrait/Multi-item Analysis Program-Revised (MAP-R) version 1.0 for Windows (Health Assessment Laboratory, Boston, MA).
Development of item pool.
The comprehensive literature review identified 24 HRQOL instruments (22 RA specific, 2 general arthritis) that satisfied the predefined criteria (Table 1). These instruments yielded 228 items, which were combined with 96 items derived from the patient focus groups. Fifty-five items remained following the deletion of redundant items. Based on the feedback from the 11-patient focus group and expert panel, 20 items were reworded, 8 items were deleted, and 11 new items were added. A survey was drafted consisting of 58 items intended to address relevant domains of HRQOL in patients with RA.
Table 1. Health-Related Quality of Life instruments identified from the literature review*
|Rheumatoid arthritis specific|| || || || |
| Arthritis Impact Measurement Scale (10, 11)||Yes||Yes||Yes||Yes|
| Arthritis Impact Measurement Scale 2 (21)||Yes||Yes||Yes||Yes|
| Arthritis Impact Measurement Scale 2–Short Form (22)||Yes||Yes||Yes||Yes|
| Arthritis Helplessness Index (23)||Yes|| || || |
| CSSRD Functional Assessment Survey of RA (24)|| || ||Yes|| |
| Disease Repercussion Profile (25, 26)||Yes||Yes||Yes|| |
| Foot Functional Index (27, 28)|| || ||Yes||Yes|
| Functional Status Index (29, 30)|| ||Yes||Yes||Yes|
| Grip Ability Test (31)|| || ||Yes|| |
| Modified Health Assessment Questionnaire (32)|| || ||Yes|| |
| Overall Status in Rheumatoid Arthritis (33)||Yes||Yes||Yes||Yes|
| Rapid Assessment of Disease Activity in RA (34)|| || || ||Yes|
| Rheumatoid Arthritis Quality of Life (16, 35)||Yes||Yes||Yes|| |
| Rheumatoid Hand Functional Disability Scale (36)|| || ||Yes|| |
| Ritchie Articular Index (37)||Yes|| || || |
| Robinson Bashall Functional Assessment for RA (38, 39)|| || ||Yes|| |
| Shoulder Function Assessment (40)|| || ||Yes|| |
| Sickness Impact Profile-Rheumatoid Arthritis (41)||Yes||Yes||Yes|| |
| Signals of Functional Impairment (42)|| || ||Yes|| |
| Stanford Health Assessment Questionnaire-Disability Index (15)|| || ||Yes|| |
| Thompson's Articular Index (43)|| || ||Yes|| |
|Arthritis specific|| || || || |
| Arthritis Self-Efficacy Scale (44)||Yes|| ||Yes|| |
| MDR index of function (45)|| || ||Yes|| |
Of the 350 people who agreed to participate, 291 (83%) returned the initial survey and were mailed the second survey 4 weeks later. Two hundred seventy-six participants (95%) returned the second survey. Reasons for not participating or for failing to complete the study included poor health, concerns of confidentiality, and incorrect mailing addresses.
The sample was predominately female (85%) with a mean age of 57 years. Most participants were highly educated with 166 (57%) having obtained at least a 2-year degree. Seventy-five percent of the participants were white, non-Hispanic. Nearly 62% of the sample was either married, or lived with a significant other but not married, and 43% were either retired or unemployed. Participants reported, on average, having RA for 12.7 years (± 9.9), and a moderate level of joint pain based on a mean score of 45.7 (± 26.4) on a scale from 0 = no pain to 100 = most severe pain possible.
Item reduction and domain structure.
Three items had missing values equal to or exceeding 5%, 7 items had floor effects ≥ 50%, and none of the items had ceiling effects ≥ 50%. Thirty-four items had correlations ≥ 0.70, suggesting potential item redundancies. Items that violated one or more statistical parameters were presented to the expert panel for possible deletion. Based on expert consensus, 24 items were retained, 17 were deleted, and 6 items addressing side effects of treatment were relocated to a supplemental subscale not included in future analyses. A total of 35 items were retained for further analysis.
Principal component factor analysis and multi-trait analysis.
Principal component factor analysis with Oblique Promax rotation of the 35 items resulted in 4 factors that had eigenvalues exceeding 1.0 and one factor with an eigenvalue of 0.99. A 5-factor principal component factor analysis was then conducted to identify the factor loadings. As shown in Table 2, the results revealed clear loadings for the 5 scales. Assessment of each item's regression coefficient revealed nearly equal variances, thereby allowing for each subscale to be summed. Table 3 displays the mean and standard deviations of each item. Based on a multitrait analysis using the 35 items, 2 items were deleted because they presented greater correlations with competing scales than with their intended scales. The multitrait analysis performed on the remaining 33 items demonstrated item-total instrument correlations ≥ 0.40 for all items, Cronbach's alpha ≥ 0.70 for each subscale, and ≥ 80% of item-scale correlations were ≥ 2 SE greater than the correlations of the item to other scales (Table 4). Results from the multitrait analysis performed on surveys returned in a second mailing confirmed the scale structure.
Table 2. Oblique promax rotated 5-factor pattern matrix of the Cedars-Sinai Health-Related Quality of Life Questionnaire for Rheumatoid Arthritis (CSHQ-RA)*
|M 1||0.68|| || || || |
|M 2||0.79|| || || || |
|M 3||0.75|| || || || |
|M 4||0.54|| || || || |
|D 5||0.39||0.50|| || || |
|D 6|| ||0.56|| || || |
|D 7|| ||0.72|| || || |
|D 8|| ||0.79|| || || |
|D 9|| ||0.83|| || || |
|D 10|| ||0.72|| || || |
|M 11||0.65|| || || || |
|M 12||0.64|| || || || |
|M 13||0.82|| || || || |
|M 14||0.70|| || || || |
|PA 15|| || ||0.46|| ||0.31|
|PA 16|| || ||0.35|| || |
|EWB 17|| || ||0.37||0.40|| |
|EWB 18|| || || ||0.70|| |
|EWB 19|| || || ||0.61|| |
|EWB 20|| || || ||0.78|| |
|EWB 21|| || || ||0.43|| |
|EWB 22|| || || ||0.61||0.31|
|EWB 23|| || || ||0.55||0.34|
|EWB 24|| || || ||0.36|| |
|PA 25|| || ||0.74|| || |
|PA 26|| || ||0.84|| || |
|PA 27|| || ||0.86|| || |
|PA 28|| || ||0.44|| || |
|SF 29|| || || || ||0.77|
|SF 30|| || || || ||0.77|
|PA 31|| || ||0.33|| || |
|PA 32|| || ||0.48|| || |
|D 33|| ||0.46|| || || |
|34†||0.37|| || || || |
|35†|| || ||0.30|| ||0.32|
Table 3. Response distributions for each item of the Cedars-Sinai Health-Related Quality of Life Questionnaire for Rheumatoid Arthritis (CSHQ-RA)*
|M 1||1.95 (0.99)||EWB 18||3.05 (1.20)|
|M 2||2.06 (0.99)||EWB 19||3.45 (1.02)|
|M 3||2.53 (1.26)||EWB 20||3.28 (1.16)|
|M 4||2.22 (1.29)||EWB 21||2.84 (1.37)|
|D 5||1.92 (1.04)||EWB 22||3.31 (1.23)|
|D 6||1.84 (1.02)||EWB 23||3.07 (1.35)|
|D 7||2.04 (1.07)||EWB 24||2.96 (1.19)|
|D 8||1.73 (0.95)||PA 25||3.18 (1.37)|
|D 9||2.12 (1.09)||PA 26||2.90 (1.45)|
|D 10||2.78 (1.18)||PA 27||3.13 (1.39)|
|M 11||3.28 (1.07)||PA 28||2.15 (1.22)|
|M 12||2.96 (1.00)||SF 29||2.82 (1.35)|
|M 13||3.27 (1.33)||SF 30||2.69 (1.32)|
|M 14||2.71 (1.22)||PA 31||2.23 (0.82)|
|PA 15||3.06 (1.17)||PA 32||2.33 (1.19)|
|PA 16||3.31 (1.21)||D 33||1.97 (1.06)|
|EWB 17||3.09 (1.21)|| || |
Table 4. Internal consistency and reproducibility of the Cedars-Sinai Health-Related Quality of Life Questionnaire for Rheumatoid Arthritis (CSHQ-RA)
The new instrument is titled the Cedars-Sinai Health-Related Quality of Life in Rheumatoid Arthritis Instrument (CSHQ-RA). The 5 subscales of the instrument are dexterity (7 items), mobility (8 items), physical activity (8 items), emotional well-being (8 items), and sexual function (2 items). The dexterity subscale is comprised of items that ask respondents to report how difficult it is to do such things as dress themselves and grasp objects. The mobility subscale is comprised of questions regarding the level of difficulty experienced when getting in and out of chairs, bed or the pain associated with walking. The physical activity subscale consists of items related to the frequency in which one was able to do such things as “spend time with others” or “do paid work.” Items addressing emotional well-being centered on worry over being dependent on others, future bone loss, and the side effects of RA treatments. The last subscale, sexual function, is comprised of 2 items that ask if fatigue or pain interfered with either the enjoyment of or ability to engage in sexual activity.
In order to allow for a comprehensive representation of HRQOL issues specific to rheumatoid arthritis, the items of the CSHQ-RA were derived from a rich item pool that incorporated past HRQOL instruments validated in RA samples; an analysis of transcripts from several patient focus groups that enabled the instrument to capture, in patients' current vernacular, their perceptions and concerns; and the input of expert clinicians to verify the clinical relevance of items. Psychometric properties were tested among patients diagnosed with RA.
The CSHQ-RA was developed to assess the broader impact of RA on patients' HRQOL. Recognition of the importance of assessing multidimensional aspects of HRQOL has been demonstrated in both the Arthritis Impact Measurement Scales 2 (AIMS2) (21) and the Multidimensional Health Assessment Questionnaire (46) Yet, unlike many of the RA instruments, the CSHQ-RA employs a consistent response format (i.e., a 5-point Likert scale) throughout all 33 items, along with a single, non-varying (e.g., “at this moment,” “in the past week,” “in the future”) timeframe of reference in an effort to increase reliability and validity. Further, to reduce the chances of attribution error, respondents are instructed to answer each question by considering only how RA has impacted various aspects of their HRQOL. However, it is recognized that such a statement does not rule out the possibility of patients including secondary or unrelated influences that impact HRQOL, such as pain or fatigue.
In the development process, an effort was made to reflect current issues, such as the use of a computer keyboard (versus a typewriter), worry associated with being able to afford RA medications, and concern over the side effects of treatments. Creating an instrument based on input from RA patients at the present time was also intended to address specific attributes of newer RA therapies and their effects, with questions exploring the frequency of fatigue and worry over the long-term consequences of RA, such as disability and joint destruction. Although further testing is necessary, it is hoped that the CSHQ-RA will offer both researchers and clinicians a more comprehensive view of patients' health than traditional clinical outcomes. This is especially important given that physiologic measures, while providing valuable information, often correlate poorly with areas that are of greatest interest to patients such as functional capacity and well-being (47).
There are several limitations to the present study. First, it should be noted that generalizability is limited to patients who are white, college educated, and physically able to complete the questionnaire. Yet, the preponderance of females in the sample is consistent with the typical demographics of this disease (4). Furthermore, in an effort to stay within a short test-retest period, only the first 350 patients who responded to the invitation were included in the study, introducing the possibility of selection bias. Due to the cross-sectional nature of the study sensitivity and responsiveness to changes in patients' perceived health status of the CSHQ-RA were not determined in this study.
Another potential limitation of the CSHQ-RA, in its current form, is that not all domains believed to be potentially relevant to health-related quality of life were included in the final instrument. For example, items intended to assess social well-being were either eliminated or not separated from other subscales as a result of factor analysis. Yet, this domain has been identified as an important area in HRQOL assessment (48–50). Reasons for this occurrence could include a poor choice of initial items representing social well-being, or the incorporation of this construct within items included in other domains. Further development of the instrument focusing on the social domain is currently being undertaken.
Six supplemental questions were designed to assess the frequency of side effects caused by RA treatment. Currently, there is considerable debate concerning the appropriateness of including side-effects as a part of HRQOL assessment. For example, 3 of the 5 responders from the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Quality of Life Guidance Committee (51) agreed that side effects may impact HRQOL but should be considered as separate. However, different drugs have very different toxicity profiles (52), requiring an exhaustive list in order to cover all possible side effects. Therefore, the side effect questions are offered in the present study as supplements that could potentially provide important clinical information.
The CSHQ-RA is a disease-specific health-related quality of life instrument for rheumatoid arthritis that with high internal consistency and reproducibility. The instrument is intended to reflect modern options available to RA patients, including aggressive and costly treatment choices with potent (even unknown or newly emerging) toxicities. Further justification for the development of a new instrument is that the CSHQ-RA is intended to be both multidimensional and disease-specific. These factors allow the CSHQ-RA to assess elements of patients' HRQOL that are not captured by commonly used disease-specific instruments, such as the HAQ, which measures only functional status. The CSHQ-RA may similarly offer advantages over generic HRQOL measures such as the MOS SF-36 that can be relatively insensitive to changes in health states of patients with specific clinical conditions. Additional aspects of reliability (i.e., test-retest) and validity (e.g., convergent and divergent validity) of the CSHQ-RA have been reported elsewhere (53) and are being submitted for full publication.