SEARCH

SEARCH BY CITATION

Keywords:

  • Japanese version of International Restless Legs Syndrome Study Group rating scale for restless legs syndrome;
  • Japanese version of Pittsburgh Sleep Quality Index;
  • reliability;
  • restless legs syndrome;
  • validity

Abstract

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Acknowledgments
  7. References

Aim

This study was conducted to verify the reliability, validity, and responsiveness of the Japanese version of the International Restless Legs Syndrome Study Group Rating Scale for restless legs syndrome (J-IRLS) as a sub-study of a clinical trial of pramipexole against restless legs syndrome.

Methods

After evaluating the test–retest reliability, concurrent validity and construct validity were analyzed. The responsiveness of J-IRLS was confirmed by evaluating the correlations between the changes in J-IRLS total score after treatment, Clinical Global Impression Improvement Scale (CGI-I), and Patient Global Impression.

Results

Test–retest reliability of J-IRLS was good (intra-class correlation coefficient, 0.877; 95% confidence interval, 0.802–0.925). The correlation coefficient of J-IRLS total score and CGI-S score for the first and second visit was 0.804 and 0.796, respectively (both P < 0.0001). Factor analysis of J-IRLS itemsalone identified a two-factor structure. Exploratory analysis on 10 items of J-IRLS together with the Japanese version of the Pittsburgh Sleep Quality Index revealed that several items on the Japanese version of the Pittsburgh Sleep Quality Index appeared as the third factor. The correlations of CGI-I and Patient Global Impression with change in J-IRLS total score after treatment were highly significant.

Conclusions

Reliability, validity, and responsiveness of J-IRLS were considered adequate. The scale is highly applicable both for evaluating the severity of restless legs syndrome and for assessing drug efficacy.

RESTLESS LEGS SYNDROME (RLS) is a disorder characterized by unpleasant leg sensations and an irresistible urge to move the legs that occurs mainly at night.[1] The prevalence of RLS in Western populations is estimated at 5–15%,[2-4] and a relatively smaller prevalence of the disorder in the Japanese population has been reported.[5, 6]

The International Restless Legs Syndrome Study Group Rating Scale for restless legs syndrome (IRLS)[7] was developed as a useful tool for assessing the subjective severity of the disorder, and has become the first-choice method to evaluate the severity of RLS in Western countries. Reports have confirmed the high reliability, validity, and responsiveness of the original English version of IRLS.[7, 8] However, utility of the Japanese version of IRLS (J-IRLS) has not yet been elucidated. Moreover, IRLS contains only one item for evaluating the insomnia symptom (item 4) even though insomnia is the most problematic symptom of RLS.[9] In addition, the relation between the findings of IRLS and widely accepted scales for assessing subjective sleep disturbance, such as the Pittsburgh Sleep Quality Index (PSQI),[10] has not been well elucidated.

To clarify these issues, the present study was designed to ascertain the test–retest reliability, validity, and responsiveness of J-IRLS and to estimate the correlation between J-IRLS and PSQI-J.

Methods

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Acknowledgments
  7. References

Subjects

This study was performed as a sub-study of a randomized, double-blind, placebo-controlled study of the efficacy of pramipexole against RLS in Japan.[11] In that study, male and female patients aged 20–80 years with a diagnosis of primary RLS made according to the essential criteria of the International Restless Legs Syndrome Study Group (IRLSSG)[12] by a sleep disorder expert physician were enrolled. All eight institutions participating in the pramipexole study participated in this sub-study after obtaining approval by the local ethics committee, and all authors of the pramipexole study report agreed to publish the results of this study. Fifty-nine individuals aged 20–78 years who were diagnosed as having idiopathic RLS were enrolled in this sub-study. The presence of any other movement disorders or sleep disorders was ruled out by at least two sleep disorder expert physicians. All patients gave written informed consent before entering the study. Descriptive variables of 59 patients recruited are shown in Table 1.

Table 1. Demographic variables of subjects
  1. CGI, Clinical Global Impression.

Number of subjects59
Age, years
Mean ± SD (range)52.9 ± 16.6 (20–78)
Sex, % (n)
Male54.2 (32)
Female45.8 (27)
Age at onset of symptoms, years
Mean ± SD (range)52.4 ± 16.5 (20–78)
Time since diagnosis, years
Mean ± SD (range)0.38 ± 0.91 (0–4.9)
Treatment history, % (n)
Treated42.4 (25)
De novo57.6 (34)
CGI ‘Severity of Illness’, % (n)
Normal, not at all ill10.2 (6)
Borderline ill11.9 (7)
Mildly ill22.0 (13)
Moderately ill16.9 (10)
Markedly ill27.1 (16)
Severely ill11.9 (7)

Analyses of both reliability and validity of J-IRLS were performed during the screening phase of the above pramipexole study in all patients who consented to participate in the study. After excluding three patients, including two patients who did not answer the Japanese version of the PSQI (PSQI-J)[13] and one patient who dropped out shortly after screening, assessment of the construct validity of J-IRLS by examining its correlation with PSQI-J was performed in 56 patients at baseline of the treatment phase of the pramipexole study. Fifteen patients did not meet inclusion criteria (J-IRLS total score >15) for entering the treatment phase, and four patients did not complete treatment. As a result, we evaluated the responsiveness of change in IRLS over 6 weeks' treatment with either pramipexole or placebo by investigating its correlation with the changes in both the original English version of the Clinical Global Impression-Improvement (CGI-I), an estimate of physician's global impression about the improvement of RLS symptoms, and the original English version of the Patient Global Impressions (PGI), which assesses patients' subjective improvement of symptoms over time, in a total of 37 patients.

J-IRLS

J-IRLS was generated based on the original IRLS (V2.1) after obtaining permission from the IRLSSG through MAPI Research Trust. Linguistic validation of J-IRLS was performed as follows. First, Japanese sleep disorder experts prepared version 1 of the draft J-IRLS by translating the original English IRLS into Japanese. Version 1 was then back-translated into English by native-English-speaking translators; thereafter translation-related issues were addressed and necessary amendments made. After trial use of the amended questionnaire in a cohort of Japanese RLS patients, version 2 was prepared. Version 2 was similarly back-translated into English, and after confirming that no translation-related issues remained, the draft version was finalized and used in the present study.

The J-IRLS scale consists of the same 10 items (questions) as the English version. Each item has a set of five response options graded from no RLS or impact (score = 0) to very severe RLS or impact (score = 4), and is scored by individual patients. The scores of all 10 items are added together to give a total score ranging from 0 to 40.

Study design

Patients who proceeded to the screening phase included 34 de novo patients and 25 patients on existing treatment. The patients already on treatment continuously received the treatment during the screening phase. These latter 25 patients included six symptom-free patients and seven borderline patients (Table 1). However, prior to implementation of this study it had been confirmed by a physician that their symptoms clearly met the IRLSSG diagnostic criteria. Although the 25 patients already on treatment continuously received the treatment during the screening phase, the medications were washed out prior to entering the treatment phase of the present study. These medications included levodopa or dopamine agonists (n = 13), clonazepam (n = 3), and others (n = 9). At the first visit, each patient self-assessed his or her RLS symptoms by J-IRLS under a physician's supervision then underwent assessment of CGI-Severity of Illness (CGI-S) by another physician. Two weeks afterwards at the second visit, J-IRLS and CGI-S were reevaluated in the same manner.

Observation of changes in RLS symptoms was made from baseline to 6 weeks after entering this double-blind, placebo-controlled trial. PGI was assessed by patients themselves and CGI-I by physicians during the treatment period. PSQI-J[13] was self-assessed by patients both at baseline and at 6 weeks after administration.

Statistical analysis

Reliability of J-IRLS was evaluated by a test–retest reliability analysis of both IRLS total score and scores of the 10 items of the scale. The results were assessed by intra-class correlation coefficient (ICC) and linear weighted Kappa statistics, respectively.

The validity analyses consisted of estimation of both internal consistency and concurrent validity in the 59 subjects who entered the screening phase of the main study. In order to confirm one domain structure of J-IRLS, we examined the internal consistency of the 10 items of IRLS. Internal consistency was assessed by Cronbach's alpha calculated for the 10 items whereas concurrent validity was assessed by correlation coefficient of the J-IRLS total score with CGI-S score. The factor structure of J-IRLS 10 items was explored using the principal factor analysis with oblique promax rotation on the 56 subjects who entered the treatment period. The number of factors in this analysis was evaluated by scree plot of eigenvalue. To explore the correlation between J-IRLS and PSQI-J, the factor structure of J-IRLS 10 items and PSQI-J items was extracted by performing principal factor analysis.

According to the previous study reported by Abetz et al.,[8] we compared the effect size used as a measure of the change in J-IRLS scores with CGI-I or PGI to evaluate the responsiveness of the J-IRLS to pramipexole treatment. Responsiveness of the J-IRLS to pramipexole treatment was investigated in 37 patients who completed the 6 weeks' observation by evaluation of the correlations between change of J-IRLS total score over 6 weeks' treatment with pramipexole and CGI-I and PGI. Moreover, change of J-IRLS total score was compared among subgroups on CGI-I and PGI by Kruskal–Wallis test. CGI-I categories were grouped into ‘very much improved’, ‘much improved’, ‘essentially stable’, and ‘worsened’. Meanwhile, PGI categories were grouped into ‘very much better’, ‘much better’, ‘essentially stable’, and ‘worsened’ because the number of patients with CGI-I item scores of ‘minimally worsened’, ‘much worse’, and ‘very much worse’, and PGI item scores of ‘a little worse’, ‘much worse’, and ‘very much worse’ was too small to calculate effect sizes. Therefore, ‘essentially stable’ on CGI included ‘minimally improved’, ‘no change’, and ‘minimally worsened’ while ‘essentially stable’ on PGI included ‘a little better’, ‘no change’, and ‘a little worse’. On both rating scales, ‘worsened’ included ‘much worse’ and ‘very much worse’. Effect sizes were calculated by dividing the change of mean J-IRLS total scores from baseline by the SD of mean scores at baseline.

Significance level in this study was set at 0.05 with two-tailed, and 95% confidence interval (CI) was calculated. All statistical analyses were conducted using sas ver.8.02 (sas Institute, Cary, NC, USA).

Results

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Acknowledgments
  7. References

Test–retest reliability

J-IRLS total score in the 59 patients was 19.5 ± 9.3 (mean ± SD) at the first visit and 19.2 ± 9.5 at the second visit. ICC was 0.877 (95%CI, 0.802–0.925). Percentages of complete agreement for J-IRLS 10 items between test and retest and linear-weighted Kappa statistics are shown in Table 2. The proportion of complete agreement ranged from 49.2 to 71.2%. Linear-weighted Kappa statistics ranged from 0.493 (question 3, relief of symptom while moving around) to 0.745 (question 7, frequency of RLS symptoms).

Table 2. Percentage of complete agreement and weighted Kappa statistics for IRLS items
IRLS itemsnPercentage of complete agreement (n)Weighted Kappa statistics (95%CI)
  1. †Linear weighted Kappa statistics.

  2. CI, confidence interval; IRLS, International Restless Legs Syndrome Study Group Rating Scale.

 1Discomfort in legs/arms5961.0 (36)0.643 (0.504, 0.781)
 2Need to move5954.2 (32)0.574 (0.432, 0.716)
 3Relief from moving around5949.2 (29)0.493 (0.337, 0.650)
 4Sleep disturbance5954.2 (32)0.590 (0.445, 0.734)
 5Tiredness/sleepiness5962.7 (37)0.674 (0.556, 0.792)
 6Severity as a whole5971.2 (42)0.721 (0.594, 0.848)
 7How often5971.2 (42)0.745 (0.627, 0.863)
 8Average severity5959.3 (35)0.613 (0.471, 0.754)
 9Impact on daily affairs5964.4 (38)0.644 (0.500, 0.787)
10Mood disturbance5954.2 (32)0.530 (0.381, 0.678)

Validity analysis

Internal consistency

Cronbach's alpha coefficient among the 10 items of J-IRLS was 0.942 (n = 59) for the first visit and 0.954 (n = 59) for the second visit during screening.

Concurrent validity

Among 59 patients, Pearson's correlation coefficient of J-IRLS total score and CGI-S score for the first and second visits was 0.804 and 0.796, respectively (both P < 0.0001).

Construct validity

Factor analysis on J-IRLS 10 items

Eigenvalues for the first visit during screening were 6.395 for the first factor and 0.691 for the second factor. At the second visit, the values were 6.942 and 0.552, respectively. Scree plots of eigenvalues showed a clear break at the third factor. Therefore, we accepted a two-factor solution, which accounted for 95.3% and 97.0% of the variance for each visit. Eight of the items differentially loaded onto these two factors in both visits (Table 3). Five items (items 1, 2, 3, 7, and 8) could be included in the first factor, each with loadings of >0.57 on the first factor and <0.26 on the second factor. Three items (items 5, 9, and 10) loaded >0.62 on the second factor and <0.24 on the first factor. The two remaining items (items 4 and 6) loaded about equally on both factors. The values for ICC for the first factor (total score of items 1, 2, 3, 7, and 8) and the second factor (total score of items 5, 9, and 10) were calculated to be 0.874 (95%CI, 0.796–0.923) and 0.810 (95%CI, 0.700–0.882), respectively.

Table 3. Factor loadings on the IRLS items in two-factor model
J-IRLSFirst visitSecond visit 
Factor1Factor2CommunalityFactor1Factor2Communality
  1. Factor loading was calculated by principal factor analysis.

  2. Factor loadings for the critical items are indicated in bold and italics.

  3. IRLS, International Restless Legs Syndrome Study Group Rating Scale; J-IRLS, the Japanese version of IRLS.

 1Discomfort in legs/arms0.8750.0940.8820.8240.1570.886
 2Need to move0.996−0.1480.8200.8690.0070.763
 3Relief from moving around0.7020.0430.5340.8470.0010.718
 4Sleep disturbance0.5050.4440.7470.3260.6280.789
 5Tiredness/sleepiness−0.0150.7640.5690.0400.8180.718
 6Severity as a whole0.5730.4310.8400.5920.3760.806
 7How often0.6380.2600.6920.6600.2330.709
 8Average severity0.5770.2600.5980.6480.2210.672
 9Impact on daily affairs−0.0280.9130.8010.0080.8770.780
10Mood disturbance0.2070.6230.6010.2360.6240.653
Factor analysis of PSQI-J sub-items together with J-IRLS

In the 56 examined patients, the correlation coefficient between J-IRLS total score and PSQI-J was 0.495 (P = 0.0001).

A significant correlation was observed between subscales of PSQI-J and J-IRLS. PSQI-J sleep duration showed a weak but significant correlation with the following J-IRLS subscales: discomfort in legs/arms, relief from moving around, sleep disturbance, severity as a whole, and average severity (P = 0.0164, 0.0033, 0.0314, 0.0273, and 0.0033, respectively). In addition, PSQI-J habitual sleep efficacy showed a weak but significant correlation with the following J-IRLS item scores: sleep disturbance, severity as a whole, and average severity (P = 0.0210, 0.0220, and 0.0046, respectively). J-IRLS impact on daily affairs showed a weak but significant correlation with PSQI-J daytime dysfunction (P = 0.0351). However, no significant correlation was observed between sleep latency and sleep disturbance on PSQI and any IRLS item scores.

Further exploration of multiple factor solutions on items of PSQI-J together with J-IRLS was performed on 56 patients at baseline. The first factor and second factor had an eigenvalue >1.0 (5.33, 1.67). The eigenvalue of the third factor was 0.87. The scree plot showed a clear break at the fourth factor. Therefore, we accepted three soluble factors, which accounted for 88.1% of the variance. Factor loading with oblique promax rotation and communality is shown in Table 4. Subscales were constructed from 10 items with both factor loadings of >0.4 and >0.15 of difference in the loading values from other factors. As a result, 10 items of J-IRLS were divided into two factors as follows: items 1, 2, 4, 6, and 8 appeared as the first factor and items 5 and 9 as the second factor. Among PSQI-J variables, item 1 (sleep quality) appeared as the first factor. Item 7 (daytime dysfunction) appeared as the second factor. Item 3 (sleep duration) and item 4 (habitual sleep efficiency) could not be categorized in the above two factors, and appeared as the third factor.

Table 4. Factor loadings of PSQI-J items and J-IRLS items
  Factor1Factor2Factor3Communality
  1. Factor loading was calculated by principal factor analysis with oblique promax rotation.

  2. Factor loadings for the critical items are indicated in bold and italics.

  3. J-IRLS, the Japanese version of International Restless Legs Syndrome Study Group Rating Scale for restless legs syndrome;

  4. PSQI-J, the Japanese version of the Pittsburgh Sleep Quality Index.

PSQI-J1Sleep quality0.4850.0620.3130.494
2Sleep latency0.327−0.2220.1210.139
3Sleep duration0.141−0.0050.6550.525
4Habitual sleep efficiency0.134−0.3020.7770.718
5Sleep disturbance−0.1480.1000.3210.09
6Daytime dysfunction−0.0440.585−0.2240.353
J-IRLS1Discomfort in legs/arms0.970−0.031−0.120.836
2Need to move0.976−0.167−0.250.714
3Relief from moving around0.3040.1400.2870.311
4Sleep disturbance0.4710.1990.1660.438
5Tiredness/sleepiness0.1040.7020.0370.574
6Severity as a whole0.7720.1230.0520.727
7How often0.3120.2600.1550.305
8Average severity0.5130.1670.2700.559
9Impact on daily affairs−0.0480.7710.0820.584
10Mood disturbance0.4190.441−0.0350.507

Responsiveness

The J-IRLS total score after 6 weeks of treatment was 11.7 ± 9.3, the change of J-IRLS total score from baseline to week 6 in 37 patients evaluated was 11.9 ± 9.6 and the correlation coefficients with CGI-I and PGI were 0.686 and 0.715, respectively (both P < 0.0001). Changes in J-IRLS total score by CGI-I and PGI subgroups also showed significant differences (Fig. 1; P = 0.0005 and 0.0004, respectively; Kruskal–Wallis test). The effect sizes for ‘very much improved’ and ‘much improved’ on CGI-I were −3.2 and −1.7, respectively; those for ‘very much better’ and ‘much better’ on PGI were −3.1 and −1.8, respectively.

figure

Figure 1. Responsiveness as determined by change in International Restless Legs Syndrome Study Group Rating Scale (IRLS) total score from baseline to week 6: effect sizes are compared among (image) Clinical Global Impression-Improvement (CGI-I) and (image) Patient Global Impressions (PGI) categories. ‘Essentially stable’ included ‘Minimally improved’, ‘No change’ and ‘Minimally worse’ on CGI-I, and ‘A little better’, ‘No change’ and ‘A little worse’ on PGI. ‘Worsened’ included ‘Much worse’ and ‘Very much worse’ on both CGI-I and PGI. P = 0.0005 for comparison among group for CGI and P = 0.0004 for those among PGI (Kruskal–Wallis test).

Download figure to PowerPoint

Discussion

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Acknowledgments
  7. References

The value of ICC for sum score of J-IRLS in the present study was similar to that of the first validation report on the original version of IRLS made by IRLSSG,[7] suggesting that test–retest reliability of J-IRLS is stable over a certain time period.

Although the above report confirmed the test–retest reliability of the sum score of IRLS,[7] this is the first report to estimate the test–retest reliability of the 10 items of J-IRLS. Herein, all Kappa statistics for 10 items showed >0.4 of the values, suggesting that reliability of the J-IRLS 10 items is moderate or good.[14] Among lower items constructing the J-IRLS, ‘need to move’, ‘relief from moving around’, ‘sleep disturbance’, and ‘mood disturbance’ had moderate inter-rater reliability (weighted Kappa, 0.493–0.590).

When evaluating internal consistency of J-IRLS, Cronbach's alpha was >0.90, consistent with the above-mentioned first validation report on IRLS.[7] Therefore J-IRLS is regarded as having good internal consistency.

Regarding the concurrent validity of the sum score of J-IRLS examined by comparing its value with CGI-S, the correlation coefficient in this study was consistent with that of the previous validation report.[7]

In this study, we found the two-factor structure for the J-IRLS 10 items, as reported previously.[8, 15] The factors relevant to RLS symptoms mainly occurring at night and relevant to the impact of RLS symptoms on daily life were statistically distinguished. The reason for the discrepancies of results for item 3, ‘RLS relief from moving around’, among our findings and those of Allen et al.[15] and Abetz et al.[8] remains unclear. However, considering that tiredness and sleepiness (item 5) as well as mood disturbance (item 10) of RLS have been speculated to emerge as a daytime consequence of chronic sleep deprivation caused by the disorder,[16, 17] it seems reasonable that these items belong to the symptom impact domain. The values of ICC for these factors were also similar to the total score and were stable.

This study revealed that both J-IRLS and PSQI-J included some uncorrelated items, and that the total scores of J-IRLS and PSQI-J showed significant but small correlation. This phenomenon could be ascribed to the difference in the target domain between these two parameters. When evaluating factor structure of the items of PSQI-J together with those of J-IRLS, we found three distinct factor structures. Ten items of J-IRLS were included in the above-indicated RLS symptom domain and symptom impact domain. Cole et al.[18] previously reported that items of PSQI could be divided into three divisions. In this study, item 1 of PSQI-J (sleep quality) was classified as a symptom-related factor, item 7 (daytime dysfunction) as a symptom impact factor, and items 3 (sleep duration) and 4 (habitual sleep efficiency) appeared as a third factor indicative of sleep efficiency – as consistent with Cole's report.[18] This finding implies that additional use of PSQI-J among RLS patients may elicit further information regarding sleep disturbance that cannot be detected by J-IRLS alone.

The change of J-IRLS total score after treatment correlated well with both CGI-I and PGI, which is consistent with a previous report.[8] The effect sizes of the change of J-IRLS for CGI-I subgroups were also consistent with those reported by Abetz et al.[8] These findings strongly indicate that J-IRLS is a useful tool for evaluating symptom response to RLS-specific treatments.

Our study contained a number of limitations. First, this study was performed as a sub-study of a randomized, double-blind, placebo-controlled clinical trial of pramipexole, potentially leading to selection bias. Second, because the study sample comprised a considerable number of patients having low CGI-S, this phenomenon could lead to the increased weighted Kappa value. Nonetheless, J-IRLS demonstrated appropriate reliability and validity to evaluate RLS symptoms compatible with the original version of the scale. Moreover, J-IRLS was shown to have sufficient responsiveness to RLS treatment. Analysis of construct validity revealed that J-IRLS and PSQI-J share common domains for RLS-related symptoms and their impact on daily life; however, only PSQI-J was revealed to be powered to measure sleep efficiency. From these findings, we speculate that J-IRLS is valid for quantifying the severity of RLS in Japanese patients.

Acknowledgments

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Acknowledgments
  7. References

This study was supported by Boehringer Ingelheim. The department chaired by Dr Y. Inoue has been donated by Alfresa Pharma, Astellas Pharma, AstraZeneca, Boehringer Ingelheim, GlaxoSmithKline, MSD, Mitsubishi Tanabe Pharma, Otsuka Pharmaceutical, Takeda Pharmaceutical, Pacific Medico, Philips Respironics, Sanofi-Aventis, and Shionogi. The department chaired by Dr Y. Oka has been donated by Philips Respironics. Dr K. Kuroda has no conflict of interest to declare. Dr K. Hirata has received financial support from GlaxoSmithKline. Mr. T. Kagimura is an employee of Nippon Boehringer Ingelheim. This trial was registered with ClinicalTrial.gov (no. NCT00152997).

References

  1. Top of page
  2. Abstract
  3. Methods
  4. Results
  5. Discussion
  6. Acknowledgments
  7. References