SEARCH

SEARCH BY CITATION

Keywords:

  • Parkinson's disease;
  • dyskinesia;
  • rating scale

Abstract

  1. Top of page
  2. Abstract
  3. Materials and Methods
  4. Results
  5. Discussion
  6. Author Roles
  7. Acknowledgments
  8. Disclosures
  9. References
  10. Supporting Information

We present the methodology and results of the clinimetric testing program for non-English translations of International Parkinson and Movement Disorder Society (MDS)–sponsored scales. The programs focus on the MDS revision of the UPDRS (MDS-UPDRS) and the Unified Dyskinesia Rating Scale (UDysRS). The original development teams of both the MDS-UPDRS and UDysRS envisioned official non-English translations and instituted a rigorous translation methodology. The formal process includes five core steps: (1) registration and start-up; (2) translation and independent back-translation; (3) cognitive pretesting to establish that the translation is clear and that it is comfortably administered to and completed by native-speaker raters and patients; (4) field testing in the native language using a large sample of Parkinson's disease patients; and (5) full clinimetric testing. To date, the MDS-UPDRS has 21 active language programs. Nine official translations are available, having completed all phases successfully, and the others are in different stages of development. For the UDysRS, 19 programs are active, with three official translations now available and the rest in development at different stages. Very few scales in neurology and none in movement disorders have fully validated translations, and this model may be adopted or modified by other scale programs to allow careful validation of translations. Having validated translations allows for maximal homogeneity of tools utilized in multicenter research or clinical trial programs.

The International Parkinson and Movement Disorder Society (MDS) has sponsored a variety of efforts to develop clinical rating scales in movement disorders. The MDS revision of the UPDRS (MDS-UPDRS) was developed as a formal MDS program,[1, 2] and the Unified Dyskinesia Rating Scale (UDyRS) was provided to the MDS after its development and initial clinimetric testing.[3] Both scales were originally developed and tested in English among native English-speaking raters and patients. In both scales, patient perceptions of disease impact are assessed, examination of motor features conducted, and specific instructions are provided for raters. The scale developers placed an active emphasis on clarity, word choice, short sentence structure, and a reading level reflective of completion of the seventh grade. Because clinical research involving Parkinson's disease (PD) increasingly relies on multinational clinical trials, the accurate and uniform translation of scales has been an MDS priority. To this end, and with support from the MDS, we have formalized an ongoing mechanism to develop official and clinimetrically sound non-English translations of the MDS-UPDRS and the UDysRS.

Materials and Methods

  1. Top of page
  2. Abstract
  3. Materials and Methods
  4. Results
  5. Discussion
  6. Author Roles
  7. Acknowledgments
  8. Disclosures
  9. References
  10. Supporting Information

Overall Organization

The scale translation program is publicized and outlined on the MDS website (www.movementdisorders.org), so that all MDS members are aware of procedures and requirements. All languages are invited to be represented regardless of the population served by a given language. A Web-based application identifies the language team composition and is reviewed by the Scale Translation Steering Committee (STSC; C.G., G.T.S., N.R.L., and B.C.T.). Each language program has a leader, responsible for managing the program at each of the five steps (see below) and interacting with the STSC. Electronic communications are utilized, and, at the point of field testing, a Web-based central database is opened for the project, so that the language team can submit completed MDS-UPDRS and UDysRS scores directly. This central database, managed by B.C.T., S.L., and L.W., meets requirements for protection of patient information and allows for immediate checking of missing data and rectification of ambiguities. Data analysis is managed by the author team, and if the data meet the prespecified criteria after field testing, the translation is designated as an official MDS translation of the given scale. Finalized translations are stored on the MDS website, and, as exclusive property of the MDS, the rules for access and utilization of the translations or the original English-language versions follow MDS policies.

Process

The process of developing an officially approved translation of the MDS-UPDRS and UDysRS involves five steps.

Step 1: Registration and Start-up

A formal application is submitted to the STSC by a language team, with a leader identified to organize the effort and be the contact person with the STSC. The language team identifies members to produce the translation and an independent group to back-translate into English. Both translation and back-translation teams must include at least one neurologist fluent in English. After application review and incorporation of any STSC suggestions for other team members or a different geographical representation, the program moves to the next step.

Step 2: Translation and Back-Translation

The translation team translates the full English scale and submits this provisional document to the independent back-translation team to provide an English equivalent that is compared directly to the official English version by the STSC. Professional translating services may be used, but the leader of a given program must submit the documents with confirmation that he or she has approved them for medical accuracy, clarity, and seventh-grade (primary education) reading level for the culture. These documents are reviewed internally by the STSC and, if needed, by a language expert unrelated to the program. Questions or concerns regarding ambiguities or conceptual discord between the original and back-translated English editions, and any identified problems with the translation from the outside consultant, must be resolved by the language team members in a revised document. Attention is placed on clarity as well as simple word choices that are applicable to the different cultures or countries where the language is spoken. The goal of this step is to capture the meaning and clinical utility of each item, not necessarily resulting in a literal word-for-word translation. Once the translation and back-translations are complete, the program moves to the next step.

Step 3: Cognitive Pretesting

The provisional translation undergoes formal cognitive pretesting subsequent to a standard set of guidelines and rating packet prepared by a cognitive testing specialist (N.R.L.) and translated into the given language by the language program leader. Sample items are administered by a rater to a series of PD patients, and both rater and patient respond to questions specifically focused on clarity, ease of understanding, and comfort with the items.[4] Sample items are selected by the STSC from each of the four parts of the MDS-UPDRS or UDysRS. For each item, both rater and patient answer a series of questions regarding the usage and understanding of the item, instructions, and options for answers. This phase is aimed at identifying any parts of the scale that are unclear, difficult to administer or answer, or emotionally awkward and uncomfortable for either rater or patient. Cognitive data are submitted to the STSC, and, in the event that problems are identified, these issues are sent to the program leader with a request for a response and possible modifications to the translation. This process may take several rounds of testing to assure that all issues are resolved before larger field testing is launched. When all issues are finally resolved, the resultant version is termed official working document, and this document is moved to the next step for definitive testing.

Step 4: Field Testing

The language teams organize a set of native-speaking raters willing to perform the scale on the prescribed number of native-speaking PD subjects needed for the validation effort. In the case of the MDS-UPDRS, the number required is 350 PD patients, spread across all H & Y stages and both genders. This calculation is based on the number of items in the MDS-UPDRS (65) and the need for at least 5 subjects per item of the scale (each item has five possible answers: 0–4) in order to perform the statistical tasks needed to validate the instrument. In the case of the UDysRS, with only 26 items, the sample size selected is 250 PD patients. Given that dyskinesia does not occur in all PD patients, up to 10% of the UDysRS cohort is allowed to be enrolled if they do not report dyskinesia and therefore have potential entries of uniform zeroes. All raters or sites must obtain institutional research approval and register this document into the central database. With this information registered, the central database is opened to that rater or site for direct data entry. Raters examine PD patients with the scale and submit scores along with basic demographic data on gender, age, disease duration, and educational level. The database managers (B.C.T., S.L., and L.W.) regularly check the database to identify any problems, such as incomplete entries or entries outside the range of possible values, so that they can direct questions back to the rater or program leader. Once the required number of ratings is entered and checked for completeness, the database is locked and the program moves to the final step.

Step 5: Data Analysis and Report Generation

For MDS-UPDRS, a confirmatory factor analysis is run against the established English-version factor structure, for each part (I–IV).[5-7] M-plus is used to do the confirmatory and exploratory factor analyses. We use an unweighted least squares approach to factor estimation, which minimizes the sum of squared differences between observed and estimated correlation matrices not counting diagonal elements, because the variables are ordered categorically.[8] If the factor convergence fails, we use a weighted least squares with mean and variance adjustment (WLSSMV) approach.[9] To assist in the interpretation of the factors, we use an orthogonal CF-VARIMAX rotation that constrains the factors to be uncorrelated. In order to be designated as an official MDS translation, the prespecified criterion is a comparative fit index (CFI) ≥0.9, relative to the English version, for all four parts of the scale. As secondary analyses, an exploratory factor analysis of the language version is performed to examine the underlying factor structure without constraint of a prespecified factor structure.[10] This analysis uses similar factor extraction and rotation techniques as those used in the CFI, but the number of factors is determined by scree plot examination.[6] This second step allows for cross-cultural differences in the scale to be appreciated as a potential observation for future research efforts.

For the UDysRS, the original English version was not submitted to a large validation study and the first large-scale field testing began with the Spanish translation itself. Therefore, in the Spanish version, we conducted reliability testing (Cronbach's alpha, item-to-total correlations, and alpha-if-item-removed) and construct validity testing using an exploratory factor analysis (EFA), informed by eigenvalues and scree plots to determine the number of factors that best represents the data, followed by a confirmatory factor analysis.[5-7] For subsequent translation programs, a confirmatory factor analysis is run against the established Spanish-version factor structure, and the same criteria of CFI ≥0.9, in this case for the entire UDysRS, is the criterion for designation as an official MDS UDysRS translation.

Upon completion of all analyses, a report is generated to the MDS and the leader of the language team to share locally. The designated MDS official translation is posted on the MDS website and owned by the MDS with access policies established. The translation program publication guidelines encourage individual groups to report their data based on the official report.

Results

  1. Top of page
  2. Abstract
  3. Materials and Methods
  4. Results
  5. Discussion
  6. Author Roles
  7. Acknowledgments
  8. Disclosures
  9. References
  10. Supporting Information

As of 1 March 2014, 21 programs are active for MDS-UPDRS (Supporting Table 1). Nine have completed all five steps and are designated as official MDS translations: Estonian, French, German, Hungarian, Italian, Japanese, Russian, Slovak, and Spanish. The CFIs for these successful translations have ranged between 0.90 and 0.96 for part I, 0.96 and 0.99 for part II, 0.93 and 0.95 for part III, and scored 1.00 for part IV in all languages. Seven languages are in field testing: Chinese, Dutch, Greek, Hebrew, Korean, Polish, and Turkish. Four are in cognitive pretesting: Hindi, Portuguese, Serbian, and Thai. One group (simple Chinese) is developing the translation and back-translation.

Nineteen programs are active for UDysRS: French, German, and Spanish have completed the program and have official UDysRS versions. All completed translations exceeded the CFI criterion of ≤0.90, ranging from 0.97 to 0.99. Chinese, Dutch, Greek, Hebrew, Hungarian, Italian, Japanese, Korean, Polish, Slovak, and Turkish teams are in the process of field testing. Portuguese and Serbian are in the cognitive pretest phase, and Estonian and Russian language groups are in the translation/back-translation phase.

To date, all translations that completed full field testing successfully met the prespecified criteria for designation as an official MDS translation on the first round of large-scale testing. The process in the preceding steps, however, required multiple revisions in some cases. For completed translation/back-translations, the median number of revisions is two (range, 1–3), and for completed cognitive pretesting, the median number of testing rounds before satisfactory completion is two (range, 1–4).

Whereas the primary outcome for determining the adequacy of the translation is the comparative fit index score, a secondary analysis involved an EFA for each translation. The purpose of this analysis is to examine subtle differences in the factor loadings that may be observed across the various languages and cultures. Though similar in most instances, each bit of language data demonstrated a slightly different factor structure than the others. For instance, in a given language, individual items in the MDS-UPDRS sometimes loaded more heavily on a factor that was not the same as that identified in the other languages (Estonian, Italian, Slovak, or Spanish). Sometimes, items did not load heavily on any factor identified in the MDS-UPDRS English version (French or German). Sometimes, an item loaded on two factors, in contrast to the single loading found in other languages (Japanese). Similar differences in factor structure were observed in the UDysRS editions, with each language edition having some minor differences in factor loading from the other languages.

All finalized and validated translations of the MDS-UPDRS and UDysRS have been given to the MDS, where its staff manages requests for access for research or clinical care programs. As of 1 March 2014, 14 official requests have been submitted for MDS-UPDRS translations (seven from pharmaceutical companies, five from universities or foundations, and two from individuals) and three formal requests for UDysRS translations (all pharmaceutical companies). Additional requests have been submitted regarding the anticipated timeline for availability of translations currently in testing.

Discussion

  1. Top of page
  2. Abstract
  3. Materials and Methods
  4. Results
  5. Discussion
  6. Author Roles
  7. Acknowledgments
  8. Disclosures
  9. References
  10. Supporting Information

This translation program is unique in movement disorders and is designed to provide the international community with official translations of MDS-owned scales that are clinimetrically sound and culturally individualized. Because movement disorders clinical trials are usually multinational or include raters and patients who are not native-English speaking, such validated translations provide an assurance of homogeneity otherwise not confirmed. In the past, the original UPDRS was widely used in non-English contexts, but the scale was applied either in English or with the use of nonofficial translations.[11] Likewise, for dyskinesia scales, the widely used Abnormal Involuntary Movement Scales and Rush Dyskinesia Rating Scale were used in international studies involving multiple language teams, but no clinimetric testing of translations validated homogeneity of application.[12] Problems with data differences among sites using an English-based scale in cultures where English is not the native language have previously obscured the final analyses of efficacy and safety and forced subanalyses excluding data from sites in certain countries.[13]

These translation programs have been broadly embraced by movement disorder teams internationally, and the enrollment covers languages that are widely used internationally as well as languages with a very restricted native-speaking population. For languages with a wide geographical representation, the administrative team required that multiple cultures be represented at all phases of the program, and as one example, the Spanish MDS-UPDRS included raters and patients from Spain, Latin America, and the United States. For Portuguese, the leaders for both the MDS-UPDRS and UDysRS represent both Portugal and Brazil. These efforts of cross-cultural representation introduced challenges at both the translation and cognitive pretesting phases, because some phrases in a given language were more comfortably understood in one culture than another, and revisions were necessary to select a text that was clear to all represented groups. On the other hand, with this infrastructural work before field testing, all scales that have been finally submitted to large-scale patient testing have been successful, emphasizing the value of this initial rigor of scale development methodology.

Whereas the protocol for the translation program is strict and uniformly applied, the overall program encourages flexibility and allows for additional programs to be added to the core effort. As an example, in the Spanish MDS-UPDRS program, the language team acquired simultaneous quality-of-life measures and performed additional analyses after the Spanish version was designated as an official MDS translation.[14]

After performing the analyses of the Spanish UDysRS, we suggested that the accuracy of the UDysRS time-based questions on percent of the day with ON dyskinesia and OFF dystonia could be enhanced with the addition of clarifying statements that remind raters and patients to remain focused on the time spent with the two forms of dyskinesia over the past week. As a result, for all the translations and the original English UDysRS, we have added three clarifying statements to ensure harmonization of the time-based questions with the patient/caregiver questionnaire and interview items. In the initial instructions to the full scale and at the end of each part of the questionnaire, we alert the rater to review the patient materials to ensure that the time-based items register at least some dyskinesia over the past week, if the patient has marked any of the questionnaire items reflecting disability from dyskinesia over the same period.

The strengths of this program include a clear, step-wise process for validations anchored in strong clinimetric science and with both movement disorder expertise and direct input from PD patients and caregivers. This model can be utilized for any scale and any condition. Because of the multiple steps, it is not a fast process, but the outcomes to date have been uniformly successful. An inherent limitation is the arduous work and volunteer dedication of the participants, and some language teams have contacted us with initial interest, but felt that the time and work involved was more than their language team could afford. This limitation has been voiced primarily in smaller countries, but the example of Estonia and Slovakia indicate that, with excellent organization and commitment, the patients and colleagues can be identified.

As an MDS-sponsored effort, the development and testing of MDS-UPDRS and UDysRS translation rely on voluntary participation. Clinicians interested in leading a new translation effort for either or both scales can contact Christopher G. Goetz (cgoetz@rush.edu) or the MDS secretariat, Jennifer Wolf (JWolf@movementdisorders.org).

Author Roles

  1. Top of page
  2. Abstract
  3. Materials and Methods
  4. Results
  5. Discussion
  6. Author Roles
  7. Acknowledgments
  8. Disclosures
  9. References
  10. Supporting Information

(1) Research Project: A. Conception, B. Organization, C. Execution; (2) Statistical Analysis: A. Design, B. Execution, C. Review and Critique; (3) Manuscript: A. Writing of the First Draft, B. Review and Critique.

C.G.G: 1A, 1B, 1C, 2A, 2B, 2C, 3A, 3B

G.T.S.: 1A, 1B, 1C, 2A, 2B, 2C, 3A, 3B

B.C.T.: 1A, 1B, 1C, 2A, 2B, 2C, 3B

L.W.: 2B, 2C, 3A, 3B

S.L.: 2C, 3B

N.R.L.: 1A, 1B, 1C, 2B, 2C, 3B

Acknowledgments

  1. Top of page
  2. Abstract
  3. Materials and Methods
  4. Results
  5. Discussion
  6. Author Roles
  7. Acknowledgments
  8. Disclosures
  9. References
  10. Supporting Information

The authors appreciate the MDS support of this program, managed by Ms. Jennifer Wolf. The authors thank the leaders of each language program as well as all the raters and patients who have participated.

Disclosures

  1. Top of page
  2. Abstract
  3. Materials and Methods
  4. Results
  5. Discussion
  6. Author Roles
  7. Acknowledgments
  8. Disclosures
  9. References
  10. Supporting Information

Funding Sources and Conflicts of Interest: The authors receive compensation from the International Parkinson and Movement Disorder Society for the management of this program. The Rush Parkinson's Disease and Movement Disorder Program is supported by the Parkinson's Disease Foundation.

Financial Disclosures for previous 12 months: C.G.G.: has held consulting and advisory board membership with honoraria with Addex Pharma, Advanced Studies of Medicine, Boston Scientific, CHDI, Clarion Healthcare, Partners, Frankel Group, Health ICON Clinical Research, Ingenix (i3 Research), and the National Institutes of Health (NIH); has received grants and research funding from the NIH and Michael J. Fox Foundation (MJFF); directs the Rush Parkinson's Disease Research Center that receives support from the Parkinson's Disease Foundation; directs the translation program for the MDS-UPDRS and UDysRS and receives funds from the MDS for this effort; has received honoraria from the American Academy of Neurology, Movement Disorder Society, New York Academy of Medicine, Phipps University (Marburg, Germany), and the University of Pennsylvania; has received royalties from Oxford University Press, Elsevier Publishers, Wolters Kluwer Health- Lippincott, and Wilkins and Williams; and has been employed by Rush University Medical Center. B.C.T. has received grants and research support from an MDS grant, NIH grants (National Institute of Neurological Disorders and Stroke, National Heart, Lung, and Blood Institute, National Institute on Minority Health and Health Disparities, and The National Institute of General Medical Sciences), the Pfizer Data and Safety Monitoring Committee, and the NIH Data and Safety Monitoring Committee and has been employed by the University of Texas School of Public Health. N.R.L. has work in consulting, cognitive testing, qualitative research, and program/process evaluation consulting with UMass Medical School (UMMS) Lamar Soutter Library, UMass Medical School InterProfessional Development, The Association of Academic Health Sciences Libraries, Medical University of South Carolina (MUSC) College of Nursing and Hollings Cancer Center, and the MDS and is a subcontractor on a variety of research and evaluation grants with principal investigators at UMMS and MUSC. G.T.S. has held consultancies and advisory board membership with honoraria with Adamas Pharmaceuticals, Inc., Ceregene, Inc., CHDI Management, Inc., Ingenix Pharmaceutical Services (i3 Research), and Neurocrine Biosciences, Inc.; has received grants and research frunding from NIH, MJFF, and the Dystonia Coalition; has received honoraria from the MDS, American Academy of Neurology, MJFF; and has been employed by Rush University Medical Center. L.W. has been employed by University of Texas School of Public Health. S.L. has received grants and research support from NIH (National Center for Advancing Translational Sciences) and has been employed by University of Texas School of Public Health.

References

  1. Top of page
  2. Abstract
  3. Materials and Methods
  4. Results
  5. Discussion
  6. Author Roles
  7. Acknowledgments
  8. Disclosures
  9. References
  10. Supporting Information
  • 1
    Goetz CG, Fahn S, Martinez-Martin P, et al. Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS): process, format, and clinimetric testing plan. Mov Disord 2007;22:4147.
  • 2
    Goetz CG, Tilley BC, Shaftman SR, et al. Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS): scale presentation and clinimetric testing results. Mov Disord 2008;23:21292170.
  • 3
    Goetz CG, Nutt JG, Stebbins GT. The Unified Dyskinesia Rating Scale: presentation and clinimetric profile. Mov Disord 2008;23:23982403.
  • 4
    Tilley BC, La Pelle NR, Goetz CG, Stebbins GT. Using cognitive pretesting in scale development for Parkinson's disease: the Movement Disorder Society Unified Parkinson's Disease Rating Scale (MDS-UPDRS) Example. J Parkinsons Dis 2014. [Epub ahead of print] PMID: 24613868.
  • 5
    Gorsuch RL. Factor Analysis. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associations Inc; 1983.
  • 6
    Hatcher L. Step-by-Step Approach to Using the SAS System for Factor Analysis and Structural Equation Modeling. Cary, NC: SAS Institute; 1994.
  • 7
    Muthen LK, Muthen BO. Mplus User's Guide. 6th ed. Los Angeles, CA: Muthen & Muthen; 2010.
  • 8
    Brown TA. Confirmatory Factor Analysis for Applied Research. New York, NY: Guilford Sage Publications Inc; 2006.
  • 9
    Forero CG, Maydeu-Olivares A, Gallardo-Pujol D. Factor analysis with ordinal indicators: a Monte Carlo study comparing DWLS and ULS estimation. Struct Equ Model 2009;16:625641.
  • 10
    Browne MW. An overview of analytic rotation in exploratory factor analysis. Multivar Behav Res 2001;36:111150.
  • 11
    Movement Disorder Society Task Force on Rating Scales for Parkinson's Disease. The Unified Parkinson's Disease Rating Scale (UPDRS): status and recommendations. Mov Disord 2003;18:738750.
  • 12
    Colosimo C, Martinez-Martin P, Fabbrini G, et al. Task force report on scales to assess dyskinesia in Parkinson's disease: critique and recommendations. Mov Disord 2010;25:11311142.
  • 13
    Lewitt PA, Hauser RA, Lu M, et al. Randomized clinical trial of fipamezole for dyskinesia in Parkinson disease (FJORD study). Neurology 2012;79:163169.
  • 14
    Rodriguez-Blazquez C, Rojo-Abuin JM, Alvarez-Sanchez M, et al. The MDS-UPDRS Part II (motor experiences of daily living) resulted useful for assessment of disability in Parkinson's disease. Parkinsonism Relat Disord 2013;19:889893.

Supporting Information

  1. Top of page
  2. Abstract
  3. Materials and Methods
  4. Results
  5. Discussion
  6. Author Roles
  7. Acknowledgments
  8. Disclosures
  9. References
  10. Supporting Information
FilenameFormatSizeDescription
mdc312023-sup-0001-TableS1.pdfapplication/PDF231KSUPPORTING TABLE 1. MDS-UPDRS and UDysRS status (1 March 2014).

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.