SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. Supporting Information

Objective

To develop and pilot test a screening tool to identify cases of nephrogenic systemic fibrosis (NSF) among patients exposed to gadolinium-containing contrast agents.

Methods

Sixty English-speaking subjects were enrolled: 10 subjects diagnosed as having NSF, 10 subjects with other fibrosing skin diseases, 20 subjects with nonfibrosing skin diseases, and 20 subjects without a skin disease. Subjects answered a questionnaire with 8 closed-ended (yes/no) questions focusing on cutaneous and musculoskeletal manifestations of NSF. The subjects were evaluated by a dermatologist for the presence of clinical signs of NSF. We compared the number of affirmative responses in the NSF group to those in the other groups, and the optimal cutoff that would differentiate groups was calculated. Discrimination, positive and negative predictive values, and internal consistency were also assessed.

Results

Subjects in the NSF group tended to provide more affirmative answers. Using a cutoff of ≥3 affirmative responses yields a sensitivity of 90% and a specificity of 70%, with an area under the curve of 0.85, indicating good discrimination. Sensitivity analysis using modified control group or weighted scores exhibited only slightly better discriminatory power. The positive predictive value of the questionnaire ranged from 0.3% to 39.7%, and its negative predictive values ranged from 97% to >99% with the different proposed prevalence estimates. The instrument had high internal consistency.

Conclusion

This pilot study demonstrates that this questionnaire has both high internal consistency and good discriminatory ability. Thus, it may be used to screen populations for NSF.


INTRODUCTION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. Supporting Information

First observed in 1997, nephrogenic systemic fibrosis (NSF) is a debilitating systemic fibrosing disease that primarily affects the skin of patients with chronic kidney disease (CKD) (1). Joint contractures often develop when there is tightening of the periarticular skin. There may be extensive extracutaneous fibrosis involving lymph nodes, thyroid, esophagus, heart, lungs, pleura, liver, diaphragm, skeletal muscle, genitourinary tract, sclera, and dura mater (2–7). Among the risk factors for developing NSF, renal dysfunction and prior exposure to gadolinium-based contrast agents (GBCAs) are the most important and have been present in virtually all patients experiencing this disease. A robust relationship has been demonstrated between exposure to GBCAs used in magnetic resonance imaging (MRI) examinations and the subsequent development of NSF; causality has been suggested given this temporal relationship (8–11). Moreover, gadolinium has been detected in skin and other organs of patients with NSF (11–14). NSF typically presents with hyperpigmentation, and hardening and tethering of the skin. In the majority of cases, plaques or nodules first develop on the skin of the distal lower extremities and then progress proximally to involve the skin of the thighs and distal upper extremities and, less often, the trunk (15, 16). The face is characteristically spared. Pain is the most common symptom. Flexion contractures with limitation of joint movement typically develop as the disease progresses, resulting in marked physical disability (15).

The differential diagnosis of NSF includes scleroderma, lipodermatosclerosis, scleromyxedema, eosinophilic fasciitis, and chronic graft-versus-host disease (3, 16). These diagnostic possibilities are refined by the clinical context in which each of these diseases occurs; however, confirmation of an NSF diagnosis requires histologic evaluation of the skin (16).

Following publication of the description of the first 14 cases of NSF in 2000 (1), hundreds of additional cases have been identified. However, many additional cases likely remain undiagnosed (17). The early identification of patients with NSF has become even more important, since promising therapies such as imatinib mesylate have been described (18). The availability of a screening tool to identify cases of NSF among patients exposed to GBCAs would also facilitate the discovery of other risk factors that might predispose patients to develop this devastating condition. We describe the development and validation of a questionnaire to screen patients for NSF.

Significance & Innovations

  • It is essential to develop a screening tool that can identify undiagnosed cases of nephrogenic systemic fibrosis (NSF), especially in early stages of the disease, among patients exposed to gadolinium-based contrast agents. Such an instrument may also detect the other risk factors that might predispose to the development of NSF.

  • A questionnaire consisting of 8 closed-ended questions that focus on cutaneous and musculoskeletal manifestations of NSF has both high internal consistency and good discriminatory ability and may be used to screen populations for NSF.

MATERIALS AND METHODS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. Supporting Information

Instrument development.

We created a brief and objective questionnaire as an instrument to screen patients for the presence of NSF. Questions that focused on cutaneous and musculoskeletal signs and symptoms of NSF were developed in consultation with rheumatology and dermatology experts who were experienced in the diagnosis and care of patients with NSF. Terms and expressions commonly used by patients with NSF in the context of their medical care were considered when constructing the items. Terminology was tailored so as to be understood by subjects in each of the other study groups.

Initially, a questionnaire consisting of 10 questions to detect NSF was developed by a dermatologist and a rheumatologist expert in NSF. These questions were intended to address the most prominent signs and symptoms of the disease: hardening, tethering, and discoloration of the skin, and pain and limitation of joint movement. Each question was constructed so that an affirmative answer (yes) would always count towards increasing the likelihood of having NSF (for a sample questionnaire, see Supplementary Figure 1, available in the online version of this article at http://onlinelibrary.wiley.com/doi/10.1002/acr.21877/abstract).

The questionnaire was designed to be self-administered and completed by subjects in fewer than 10 minutes. After review by 2 other dermatologists and 1 patient with NSF (who was not included as a study subject), phrasing was modified and redundant questions were combined to simplify the questionnaire. The final instrument consisted of 8 closed-ended (yes/no) questions.

Study subjects.

Subjects were recruited from among adult patients (ages ≥18 years) who received medical care in the rheumatology or dermatology units of the Massachusetts General Hospital in Boston. Participants were required to understand and speak English, and were excluded if they had any psychiatric disorder that could, in the opinion of the investigators, interfere with the study evaluations. The study was approved by the Partners Institutional Review Board.

After providing verbal consent, 60 subjects were enrolled in this pilot study and were categorized into 4 groups: 1) patients with a histologically proven diagnosis of NSF (10 subjects); 2) patients with other fibrosing skin diseases (OFSD), such as scleroderma, lipodermatosclerosis, and eosinophilic fasciitis (10 subjects); 3) patients with nonfibrosing skin diseases (NFSD), such as psoriasis and eczema (20 subjects); and 4) patients without a skin disease (20 subjects). Eligible consecutive patients with NSF and patients in each of the other categories were recruited by study staff without attempting to match sex, age, or race. Twice the number of NSF patients in group 1 were enrolled into each of groups 3 and 4 as control subjects without fibrosing skin diseases.

After verbal consent was obtained, each subject was instructed to answer the questionnaire and was queried about demographic information. Subsequently, a dermatologist who was blinded to the subject's responses to the questionnaire performed a skin examination. This examination focused on the presence or absence of 3 fundamentally important signs of NSF: hardening, tethering, and darkening of the skin on upper or lower extremities (9).

Statistical analysis.

The demographic characteristics of each control group were compared to those in the NSF group using Fisher's exact test and Student's t-test. For each group, we calculated the number of affirmative responses to questions on the instrument and performed analyses of sensitivity and specificity to determine the optimal cutoff value that would differentiate patients with NSF from subjects in each of the other groups. Sensitivity and specificity were calculated using different thresholds for the number of affirmative responses. A receiver operating characteristic (ROC) curve was constructed using these thresholds; the area under the curve (AUC) was calculated as a measure of discrimination, i.e., the larger the AUC, the better discrimination the instrument can provide. Positive and negative predictive values were calculated using published estimates of NSF prevalence that have ranged between 0.1% (19) and 18% (20) depending on the type of GBCA, the prevalence of stage 5 CKD, and the definition of NSF used in different studies.

The principal analysis compared the number of affirmative responses to the questionnaire by subjects in the NSF group to those in the other groups combined. In a sensitivity analysis, the OFSD group was excluded from the pooled comparison group, since subjects with OFSD might be expected to provide responses similar to those of patients with NSF. We also analyzed each question individually using the likelihood ratio, which combines information about sensitivity and specificity (21). The likelihood ratio of an affirmative response was used to weight each item; the resulting new threshold values for number of affirmative responses were used to build a new ROC curve. This curve and its AUC were then compared to those of the original ROC curve that was constructed using responses to the questionnaire with unweighted items.

Internal consistency was assessed using Cronbach's alpha, which summarizes the inter-item correlations among all items in a questionnaire. A Cronbach's alpha of ≥0.7 represents acceptable reliability of the instrument (22). Statistical analysis was performed using SAS, version 9.2.

RESULTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. Supporting Information

Patient characteristics.

Demographic characteristics of study subjects are presented in Table 1. Proportions of male and female subjects were similar with the exception of the OFSD group, in which 80% of subjects were women. Subjects in the NFSD group were younger than those in the other groups. In all groups, the vast majority of patients were white. At the time that the study was performed, 9 patients with NSF had stage 5 CKD and 1 had stage 3 CKD. Among the 10 patients with OFSD, 6 had lipodermatosclerosis, 3 had scleroderma, and 1 had eosinophilic fasciitis. Among the 20 patients with NFSD, 13 had psoriasis (including 2 who also had psoriatic arthritis), 6 had eczema, and 1 had a persistent pustular drug reaction.

Table 1. Demographic characteristics of study subjects*
 NSF (n = 10)OFSD (n = 10)NFSD (n = 20)Normal skin (n = 20)
  • *

    NSF = nephrogenic systemic fibrosis; OFSD = other fibrosing skin diseases; NFSD = nonfibrosing skin diseases.

  • Comparisons to NSF group using Fisher's exact test.

  • Comparisons to NSF group using Student's t-test.

Sex, % male50205055
 P 0.351.001.00
Race, % white908080100
 P 1.000.640.33
Age, mean ± SD years64.9 ± 12.853.6 ± 17.748.5 ± 15.261.6 ± 16.7
 P 0.120.0070.58

Study subject and physician assessments.

Subjects in the NSF group tended to provide more affirmative answers to the questionnaire; only 1 member of this group did not respond in the affirmative to any of the 8 questions. Half of the subjects in the OFSD and NFSD groups and all members of the group without skin diseases provided fewer than 3 affirmative responses to the questionnaire (Figure 1). The number of affirmative responses to our questionnaire varied widely among the 10 patients with each of the 3 OFSDs. The 6 individual patients with lipodermatosclerosis provided 0, 1, 3, 3, 5, and 6 affirmative responses, respectively. Of the 3 patients with scleroderma, 1 responded in the affirmative to 6 questions, whereas the other 2 responded in the affirmative to only 1 question. The patient with eosinophilic fasciitis also responded in the affirmative to only 1 question. This wide variation in the number of affirmative responses provided by patients with lipodermatosclerosis and scleroderma highlights the differences among OFSD in patient perception of cutaneous signs and symptoms.

thumbnail image

Figure 1. Study subject responses to the questionnaire. NSF = nephrogenic systemic fibrosis.

Download figure to PowerPoint

On the skin examination performed by a dermatologist, 2 subjects with NSF had 2 of the 3 characteristic cutaneous signs, and 8 subjects had all 3 signs. In contrast, 2 subjects with OFSD had only 1 and 8 subjects had 2 of these cutaneous features. Two of the 20 subjects in the NFSD group had only 1 of these signs (hyperpigmentation), whereas the other 18 subjects in this group, and all 20 subjects without cutaneous disorders, had neither hyperpigmentation, hardening, nor tethering of the skin.

Performance characteristics.

Different thresholds of number of affirmative responses were employed to build an ROC curve, with an AUC of 0.85, indicating good discrimination. Using an affirmative response to 3 or more questions as a cutoff yields a sensitivity of 90% and a specificity of 70%. In a sensitivity analysis that combined only patients with NFSD and subjects without skin diseases as the comparison group, the threshold of an affirmative response to 3 or more questions yielded sensitivity and specificity of 90% and 75%, respectively. This ROC curve exhibited only slightly better discriminatory power, with an AUC of 0.86. Another sensitivity analysis was performed, using the likelihood ratios of an affirmative response (Table 2) to weight each item. Using weighted items, the discriminatory power improved only slightly more, with an AUC of 0.87. Both ROC curves with simple and weighted scores are presented in Figure 2.

Table 2. Sensitivity, specificity, and likelihood ratio of having nephrogenic systemic fibrosis for each item of the questionnaire
QuestionsSensitivity, %Specificity, %Likelihood ratio
150802.50
260885.00
380722.86
490541.96
570803.50
650905.00
770885.83
870722.50
thumbnail image

Figure 2. Receiver operating characteristic curves.

Download figure to PowerPoint

The positive predictive value of the questionnaire ranged from 0.3% to 39.7%, and its negative predictive values ranged from 97% to >99% with the different proposed prevalence estimates. The Cronbach's alpha for this instrument was 0.83, indicating high internal reliability. The last item, which inquired about pain, had the least correlation (0.41) and was the only item that, if removed, would result in a slightly increased Cronbach's alpha coefficient (0.84).

DISCUSSION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. Supporting Information

We have developed and evaluated a questionnaire to screen populations for NSF, such as individuals who have been exposed to GBCA. In this pilot study, the instrument was administered to patients with a biopsy-proven diagnosis of NSF and to subjects without NSF. It discriminated well between these groups and was internally consistent.

This questionnaire has the characteristics of a proper screening instrument, i.e., it is both brief and simple. Although it may identify patients with other conditions that present similarly to NSF, this screening tool is intended to be only the first step in a multistage process to identify individuals who may have NSF; it does not substitute for performing a physical examination and skin biopsy, when indicated. Thus, it is more important that this questionnaire has a lower false-negative than false-positive rate. Using the relatively low threshold of an affirmative response to each of only 3 questions allows reasonable sensitivity and specificity for this screening tool to identify patients who could then be examined for clinical features characteristic of NSF.

The comparison groups included subjects that either had normal appearing skin or skin conditions considered to be either similar or dissimilar to NSF. As expected, responses to questions provided by subjects with OFSD were more similar to the responses of patients with NSF than they were to those of subjects with NFSD or without cutaneous disorders. The main analyses, which included the subjects in all 4 groups, demonstrated that the questionnaire performed well as a screening tool. In a sensitivity analysis that excluded the OFSD group, the instrument had only slightly greater discriminatory power. Given that the prevalence of OFSD in the general population is low, we expect that this questionnaire will perform well as a screening test for NSF among individuals who have undergone imaging studies using GBCAs.

Although awareness and recognition of NSF occurring among patients with stages 4 and 5 CKD has increased, and institutional guidelines restricting GBCA administration to this population have been implemented, there may be cases of NSF that have yet to be identified, especially among individuals with lesser degrees of CKD. Despite an apparent decrease in the number of individuals with NSF that have been identified, and after recent changes in MRI protocols (23, 24), cases of NSF still may remain undiagnosed, especially because these changes were implemented only recently at many institutions (25).

Our screening instrument has several limitations. The polar questions, for which the expected answer is either yes or no, do not allow for detection of different intensities of signs and symptoms that might aid in differentiating certain characteristics of NSF from those of other skin diseases. However, the questions were created with the intent to be simple and objective. The dermatologist who examined the study subjects was blinded to each subject's responses to the questionnaire, but not to their diagnosis. However, the physician's examination served only to confirm the underlying diagnosis, since the objective of this study was to develop and assess patient responses to the screening instrument. The questionnaire included questions only about cutaneous and joint manifestations, which are the features of NSF that can be identified easily by patients. Involvement of other organ systems by NSF does not necessarily provide consistent signs and symptoms that would be recognized by most patients. We purposely omitted questions about renal function because many individuals with mild renal impairment may not be aware of this diagnosis.

We did not include a group of subjects with CKD in the initial validation of this screening instrument. A significant proportion of these patients may have skin and joint alterations, including hyperpigmentation, lipodermatosclerosis, diabetic dermopathy, Dupuytren's disease, and limited joint mobility (26). We anticipate that subjects with CKD likely will answer fewer questions in the affirmative than would patients with OFSD or NFSD. However, the number of affirmative responses provided by subjects in this group will depend on the allocation of individuals with these conditions in the group sampled. Patients with CKD constitute an important group that will be surveyed in the next phase of our research, using the validated questionnaire as a screening tool, to assess the prevalence of NSF in a population of patients previously exposed to GBCAs.

Validation of a screening tool is a continuous process during which information has to be accumulated from different populations. All participants for this initial study were recruited from rheumatology and dermatology clinics and research units at one tertiary care medical center. Further validation of this instrument in a larger number of patients and in other settings, such as radiology departments and dialysis units, will be valuable.

This pilot study demonstrates that this questionnaire has both high internal consistency and good discriminatory ability. However, its validity depends not only upon the questionnaire itself, but also on the purpose for and method by which the instrument is administered and on the population that is surveyed. The initial proposed cutoff of 3 or more questions answered in the affirmative will allow the identification of individuals who should be evaluated further for the possible diagnosis of NSF. Nonetheless, this same set of questions, when provided on a postcard or as an internet-based survey to a different sample of patients, could yield different responses that might warrant revision of this diagnostic threshold. Future studies that administer this instrument to a much larger sample of individuals who have been exposed to GBCAs may identify otherwise undetected cases of NSF and help to clarify the true prevalence of NSF.

AUTHOR CONTRIBUTIONS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. Supporting Information

All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be submitted for publication. Dr. Kay had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design. Lima, Kimball, Kay.

Acquisition of data. Lima, Alora-Palli, Kimball, Kay.

Analysis and interpretation of data. Lima, Kimball, Kay.

Acknowledgements

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. Supporting Information

We thank Eric G. Campbell, PhD, for assistance with the design of the questionnaire.

REFERENCES

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. Supporting Information

Supporting Information

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. MATERIALS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. AUTHOR CONTRIBUTIONS
  8. Acknowledgements
  9. REFERENCES
  10. Supporting Information

Additional Supporting Information may be found in the online version of this article.

FilenameFormatSizeDescription
ACR_21877_sm_SupplFigure1.doc31KSupplementary Figure 1

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.