SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Acknowledgements
  7. References

Aim

To evaluate the stability over time of the Manual Ability Classification System (MACS) levels.

Method

The study group comprised 1267 children with cerebral palsy (746 males, 521 females) who were followed from 2005 to 2010 with two or more registered MACS classifications rated at least 1 year apart. Thirty-five percent of the children (n=445) had four MACS registrations. The children were between 4 and 17 years old at their first rating, The stability over time was also compared between children who were younger (4y of age) or older (≥10y) at the time of their first classification.

Results

An excellent stability was found between two ratings at 1-year intervals with an intraclass correlation coefficient (ICC) of 0.97 (95% CI 0.97–0.97) and 82% agreement (n=1267). The stability was also excellent for two ratings performed 3 to 5 years apart (ICC 0.96; 95% CI 0.95–0.97) with an agreement of 78% (n=445). Across four ratings, 70% of the children remained at the same level. The results were similar for younger and older children, indicating that stability was not influenced by age.

Interpretation

This study provides evidence that MACS levels are stable over time and that the classification has predictive value.

Abbreviation
MACS

Manual Ability Classification System

Cerebral palsy (CP) is a clinical description and by definition; a permanent disorder of the development of movement and posture, causing activity limitation, attributed to non-progressive disturbances that occurred in the developing fetal or infant brain.[1] The motor disorder in CP is often accompanied by disturbances of sensation, perception, cognition, communication, and behavior, by epilepsy, and by secondary musculoskeletal problems.[1] This description gives no information about the children's functional ability, and it has been suggested that the diagnosis of CP should be used together with functional classifications.[1] One of these classifications is the Manual Ability Classification System (MACS)[2] that has been widely used in recent years, both in clinical practice and in research. However, further investigation is required to determine whether or not children remain at the same MACS level during development and growth and to establish if the MACS ratings have predictive value.

The MACS consists of five levels and was designed to classify how children with CP aged 4 to 18 years use their hands when handling objects in daily activities.[2] Children at MACS level I handle most objects easily and successfully. At MACS level II, they handle most objects but with somewhat reduced ability and/or speed. Children at MACS levels III to V need assistance to different degrees. At MACS level III, children handle objects with difficulty and require help in preparing and/or modifying activities. Children at MACS level IV can handle a limited selection of easily managed objects in adapted situations. At MACS level V, children cannot handle objects and have severely limited ability to perform even the simplest of actions. Further descriptions of the MACS levels can be found online.[3]

If an instrument such as the MACS shows outcomes that are stable over time, it could be used to predict future manual ability levels. Even though children's hand use may change as a result of increasing experience, development, interventions, or as the result of secondary complications, we expect most children to stay at the same MACS level over several years. This is because we do not expect changes to the extent that could generate a transfer across the fairly broad classification levels. The stability of MACS ratings may be influenced by either a child's possible change in manual ability or by possible inconsistencies within or between raters. In several studies, the inter- and intrarater reliability of the MACS has been found to be good to excellent between therapists as well as between parents and therapists, varying from 0.73 to 0.98 and 0.91 to 0.98 respectively.[2, 4-11] There is also evidence supporting the stability of MACS classifications over a short period. In a previous study, the intraclass correlation coefficient (ICC) was high (0.92) and 67% of the children remained at the same level over a period of 1 year when scored by caregivers.[12]

The primary aim of this study was to evaluate the stability of MACS levels over time for children with CP. The secondary aim was to investigate whether MACS levels were less stable for younger than for older children, or if certain MACS levels showed lower stability than others.

Method

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Acknowledgements
  7. References

Participants

Data in this retrospective study were extracted from the Cerebral Palsy follow-Up Programme (CPUP) register (May 2010). The CPUP is a Swedish population-based National Health Care Quality Register and is an annual follow-up programme for the prevention of secondary musculoskeletal problems in individuals with CP, which has been approved by the National Board of Health and Welfare.[13] The data in this study consist of MACS registrations from 2005 to 2010 as determined by the child's occupational therapist as part of the annual follow-up. Inclusion criteria were children with at least two registered MACS classifications at least 1 year apart and who were between 4 and 17 years old at the time of the first rating.

Procedure

The MACS has been included in the CP follow-up programme in Sweden since 2005. All therapists who examine the children are introduced to the included assessment methods. The MACS brochure and the MACS level identification chart are also available on the MACS website.[3] In the CPUP programme, the children are examined twice a year until 6 years of age and once a year thereafter. For children assessed twice within the same year, only the last rating was used and only ratings at least 12 months apart were included in this study. Some children were rated by the same occupational therapist over time, but in many cases there were different therapists involved.

Statistical analysis

Contingency tables were constructed and the percent agreement and the ICC, the two-way random single model was used (2,1) to evaluate stability for two repeated ratings with either 1 year between ratings or with 3 to 5 years between ratings. Two age groups in each model were further investigated to evaluate if stability differed for younger and older children, that is for children who were 4 years of age versus children who were at least 10 years of age at the time of their first classification. A reliability coefficient above 0.80 is desirable for group level comparisons and a value above 0.90 is required if the measures are to be used for individual level comparisons.[14, 15] The ICC has been shown to give equivalent results to the weighted kappa,[16-18] and was used in this study to allow for comparison with other studies. McNemar's test for paired proportions was used to evaluate possible systematic shifts to higher or lower classifications over time. Percent agreement and ICC were also calculated to evaluate the overall stability of four repeated ratings. Friedman's test was used, a two-way analysis of variance by ranks, to determine if there was a tendency for the MACS scoring to differ more at one of the rating times than the others. Fleiss' kappa, an unweighted, chance-adjusted measure of agreement, was used to calculate stability over the four repeated ratings in each MACS level. A Fleiss' kappa of 0.61 to 0.80 is considered good, and between 0.81 and 1.0 is very good.[19]

Data was collected from the national register and follow-up programme (CPUP). Participation in CPUP takes place after informed consent from caregivers. The project was conducted after approval from the ethics research committee at the Karolinska Hospital.

Results

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Acknowledgements
  7. References

In total, 1267 children (746 males, 521 females) fulfilled the inclusion criteria. The mean age at the time of their first MACS classification was 6 years 6 months (SD 2y 11mo; Table 1). The number of ratings for each child varied between two and four, with at least 12 months and at most 5 years from first to last rating. In all, 3694 MACS registrations were included in this study. All MACS levels were represented and the distribution of MACS levels and age at first classification is presented in Table 1. Information on sex, diagnosis, and the Gross Motor Function Classification System (GMFCS) level were extracted from the child's latest registration (Table 1). Some children had not yet received a specific sub-diagnosis of CP and were included in the category of unspecified CP.

Table 1. Demographic characteristics of participants in terms of age and MACS level at first classification
 Participants n (%)SexMACS level
MalesFemalesIIIIIIIVV
  1. 16 children had no registered GMFCS level. MACS, Manual Ability Classification System; GMFCS, Gross Motor Function Classification System.

Age (y)
4434 (34)26317112396786374
5226 (18)1231036764372929
6134 (10)82524736191517
7110 (9)56544023181316
885 (7)5926272512912
966 (5)303621189612
1052 (4)32202313529
1136 (2)24121711242
1252 (4)371519114711
1331 (2)1813155632
1420 (2)11964352
15–1721 (2)11109 6 6
CP subdiagnosis
Spastic unilateral371 (29)  15514158170
Spastic bilateral419 (33)  14676715670
Dyskinetic145 (11)  78203674
Ataxia79 (6)  252613123
Unspecified253 (20)  8155373545
Total (%)1267746 (59)521 (41)414 (32)306 (24)199 (16)156 (12)192 (16)
    GMFCS level
Total (%)   562 (44)202 (16)116 (9)185(15)186 (15)

Stability of two MACS ratings 1 year apart

An excellent agreement was found with an ICC of 0.97 (95% confidence interval [CI] 0.97–0.97) between two ratings 1 year apart for all 1267 children. Eighty-two percent of the children were classified at the same level in both ratings. The percent agreement and the distribution of deviant ratings are seen in Table 2. In cases where the ratings differed, it was generally only a difference of one level. One hundred and twenty-three children were scored to a higher level (indicating lower ability) on the second rating occasion, and 103 children were scored to a lower level (indicating higher ability), but there was no systematic direction for the differing ratings (McNemar's test for paired proportions p=0.13). There was almost no difference in stability between the younger (4y) and older (≥10y) age groups with an agreement of 80% and 82% respectively.

Table 2. MACS levels for two ratings with 1y between ratings (n=1267)
 MACS second rating
IIIIIIIVVTotal
  1. Total agreement 82% (1037/1267). Bold type indicates the number of children rated at the same level at both the first and the second ratings. MACS, Manual Ability Classification System.

MACS first rating
I 364 43340414
II36 234 3330306
III323 148 232199
IV0412 124 16156
V00124 167 192
Total4033041971781851267

Stability of two MACS ratings 3 to 5 years apart

Thirty-five percent of the group (n=445) had been classified with the MACS over a period of 3 to 5 years. The ICC was 0.96 (95% CI 0.95–0.97) and the percent agreement was 78% between the first and last ratings. The distributions of classification for this group are found in Table 3. Disagreement between ratings mostly concerned one level of difference. Fifty children scored at a higher level and 47 children scored at a lower level at the time of the last classification (McNemar's test for paired proportions p=0.84). Age did not influence the scoring and there was similar stability between age groups with a percent agreement of 75% for children 4 years of age and 77% for children 10 years of age or older when first classified. Thus, the MACS levels were equally as stable whether the first classification took place at a younger or older age.

Table 3. MACS levels for children at the first and last ratings out of four ratings performed over a period of 3–5y (n=445)
 MACS last rating
IIIIIIIVVTotal
  1. Total agreement 78% (348/445). Bold type indicates the number of children rated at the same level at both the first and the last ratings. MACS, Manual Ability Classification System.

MACS first rating
I13911310154
II21801410116
III014 43 11068
IV022 35 948
V0017 51 59
Total160107635560445

Stability of four MACS ratings

Agreement between the four ratings was also excellent with an ICC of 0.95 (95% CI 0.94–0.96) for ratings performed about 1 year apart over the course of 3 to 5 years. The percent agreement between the four repeated ratings was 70% and this showed that most children remained in their initial MACS level in all subsequent ratings. When percent agreement was calculated separately for unilateral CP and bilateral CP, the agreements were 72% and 69%, respectively. For children who received different ratings, this commonly involved a change of just one step in the classification. Only 10 children (2.2%) changed two levels in the four-rating model (Table 4). No systematic difference of the scoring at different rating times was shown (Friedman's test p=0.69). When the agreements were analyzed for each MACS level separately, the stability was very good for MACS levels I and V. The stability for levels II to IV was somewhat lower but still considered good as estimated Fleiss' kappa (Table 5).

Table 4. Distribution of the change of MACS levels in children with four repeated ratings over 3–5y (n=445)
No. of levels changedN of children%
  1. MACS, Manual Ability Classification System.

031270.1
112227.4
2102.2
310.2
Table 5. Stability for each MACS level across four ratings
MACSN of childrenKappaSE
  1. Agreements estimated by Fleiss' kappa. MACS, Manual Ability Classification System; SE, standard error.

I1320.850.019
II660.690.019
III360.690.019
IV300.750.019
V480.870.019

Discussion

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Acknowledgements
  7. References

The ICC was, in general, excellent and the agreement was high in the different models. This shows evidence for the high stability of children's MACS levels over time. When comparing results between two ratings, the percent agreements were 82% for ratings performed 1 year apart and 78% for those performed 3 to 5 years apart. Even the four repeated MACS ratings over a period of 3 to 5 years showed an agreement of 70%. Furthermore, there were no differences in stability over time between younger children who were 4 years old and children who were 10 years or older at the time of their first classification.

The stability of the MACS over two classifications with 1 year in between showed a somewhat higher stability in the current study (ICC=0.97) compared with that found by Imms et al.[12] (ICC=0.92). In that study, all children were 11 to 12 years old, but the biggest difference was that the MACS levels were rated by the children's caregivers in a postal survey but in the current study the rating was performed by healthcare professionals in collaboration with the children's parents. The MACS has been used in Sweden since 2005 and professionals might be more familiar with the MACS compared with the Australian caregivers in the study by Imms et al.[12] who were probably scoring the children for the first time. Familiarization with a classification system has previously been suggested to influence the results,[20] and has been mentioned as a possible reason for why the GMFCS showed higher reliability in the second study compared with the first study.[21, 22] The stability of the MACS ratings in the current study is approximately equivalent to that reported for the GMFCS in which 73% of the children remained in the same level when followed over several years.[21]

In cases where the MACS level changed over time, the change was commonly to the next closest level in either direction. Thus, there was no tendency for systematic change over time. When investigating whether certain MACS levels were more prone to change than others, we used the four repeated ratings and their standard deviations as a measure of variance and found that levels I and V were the most stable. The stability was somewhat lower for levels II to IV. This is possibly because change can occur in two directions for these three levels, but for levels I and V changes can occur in only one direction.

It might have been expected that younger children would be more susceptible to change and that their MACS levels would be less stable over time compared with children who were followed from an older age. However, the stability for children 4 years of age and the stability for children of 10 years and older showed similarly high levels. This aligns well with findings of interrater reliability being equally high for different age groups of children.[2] Less stability was seen for the GMFCS in younger ages when comparing children younger and older than 6 years of age,[21] and when comparing children younger than 4 years of age with older children.[23]

The usefulness of a classification system as a predictive instrument depends on its stability over a long period so it was of interest to compare agreement between two MACS ratings with different time periods between ratings. We found that the time between the first and second ratings did not significantly affect the agreement because there was only a very small difference between ratings repeated after 1 year compared with those repeated after a period of 3 to 5 years. This finding also indicates that the MACS may be stable even with different raters, as although it may be likely that the same rater completed the MACS at the end of a 1-year period, over a 3- to 5-year period it is more likely that different raters will have completed the MACS ratings.

This study indicated that children predominantly remain in the same MACS level over time. This is not surprising, for two reasons. Firstly, change between levels is not expected because of the broadness of each of the five categories. These levels are meant to cover the whole range from very mild to very severe functional limitations in the population of CP. Secondly, age-related development is embedded in the MACS, as the handling of objects always refers to objects that are relevant for the child's age. Thus, the expected changes in skill level that are related to age will not make children change their MACS level. The levels instead describe distinct differences between how objects are handled and the amount of assistance needed to handle these objects. In the light of this, it is important to acknowledge that the MACS is a very useful tool for describing functional levels of manual ability for individuals or groups and that this information has a predictive value. However, the MACS is not intended to be an evaluative tool, and MACS levels are not expected to be sensitive to change after interventions and should thus not be used as an outcome measure.

In this study sample there were varying numbers of children at different ages and MACS levels and this reflected the distribution of children included in the CPUP register. The reason for the higher proportion of younger children in this study is the fact that children enter the register at early ages. Since 2005, all regions in Sweden have been included and the register is now based on the entire national population of children born in 2000 or later. Nevertheless, the distribution of MACS levels in this study seems to be comparable with the Victorian Cerebral palsy register study in Australia[12] and a population-based study in Sweden[24] in which more children were found in levels I and II compared with levels III to V. Another limitation of the study is that the child's previous result could have been known to the therapist. However, when completing the CPUP assessments it is recommended that comparison with historical assessments should be done only after completing the annual CPUP evaluation. The stability of MACS has in this study been reported for a time period up to 5 years. Further investigation of the longitudinal stability over periods longer than 5 years is also needed. The CPUP register provides an opportunity for future studies of this issue.

In conclusion, the MACS has previously been shown to demonstrate evidence of validity and reliability. Our study now shows that the MACS is also fairly stable over time and has predictive value.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Acknowledgements
  7. References

This study was supported by Promobilia Foundation, Stiftelsen Frimurare Barnhuset i Stockholm, the Swedish Research Council (K2009-63X14534-07-3, K2012-69X-14534-10-2), and Strategic Research Programme in Care Sciences at Karolinska Institutet. The authors have stated that they had no interests which might be perceived as posing a conflict or bias.

References

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Acknowledgements
  7. References
  • 1
    Rosenbaum P, Paneth N, Leviton A, et al. A report: the definition and classification of cerebral palsy April 2006. Dev Med Child Neurol Suppl 2007; 49(Suppl. 109): 814.
  • 2
    Eliasson AC, Krumlinde-Sundholm L, Rosblad B, et al. The Manual Ability Classification System (MACS) for children with cerebral palsy: scale development and evidence of validity and reliability. Dev Med Child Neurol 2006; 48: 54954.
  • 3
    Manual Ability Classification System for children with cerebral palsy. http://www.macs.nu (accessed 25 March 2013).
  • 4
    Akpinar P, Tezel CG, Eliasson AC, Icagasioglu A. Reliability and cross-cultural validation of the Turkish version of Manual Ability Classification System (MACS) for children with cerebral palsy. Disabil Rehabil 2010; 32: 19106.
  • 5
    Mutlu A, Kara OK, Gunel MK, Karahan S, Livanelioglu A. Agreement between parents and clinicians for the motor functional classification systems of children with cerebral palsy. Disabil Rehabil 2010; 33: 92732.
  • 6
    Ohrvall AM, Eliasson AC. Parents' and therapists' perceptions of the content of the Manual Ability Classification System, MACS. Scand J Occup Ther 2010; 17: 20916.
  • 7
    Kuijper MA, van der Wilden GJ, Ketelaar M, Gorter JW. Manual ability classification system for children with cerebral palsy in a school setting and its relationship to home self-care activities. Am J Occup Ther 2010; 64: 61420.
  • 8
    Morris C, Kurinczuk JJ, Fitzpatrick R, Rosenbaum PL. Reliability of the manual ability classification system for children with cerebral palsy. Dev Med Child Neurol 2006; 48: 9503.
  • 9
    van Meeteren J, Nieuwenhuijsen C, de Grund A, Stam HJ, Roebroeck ME. Using the manual ability classification system in young adults with cerebral palsy and normal intelligence. Disabil Rehabil 2010; 32: 188593.
  • 10
    Jang DH, Sung IY, Kang JY, et al. Reliability and validity of the Korean version of the manual ability classification system for children with cerebral palsy. Child Care Health Dev 2012; 39: 903.
  • 11
    Randall M, Harvey A, Imms C, Reid S, Lee KJ, Reddihough D. Reliable classification of functional profiles and movement disorders of children with cerebral palsy. Phys Occup Ther Pediatr 2013; 33: 34252.
  • 12
    Imms C, Carlin J, Eliasson AC. Stability of caregiver-reported manual ability and gross motor function classifications of cerebral palsy. Dev Med Child Neurol 2010; 52: 1539.
  • 13
    Swedish National Health Care Quality Programme (CPUP) – for individuals with cerebral palsy. www.cpup.se (updated 2012; cited 31 October 2012).
  • 14
    Polit D, Hungler B. Nursing Research. Principles and Methods. 6th edn. Philadelphia: Lippincott, 1999.
  • 15
    Streiner DL, Norman GR. Health Measurement Scales, a practical guide to their development and use. 4thedn. Oxford: Oxford University Press, 2008.
  • 16
    Downing SM. Reliability: on the reproducibility of assessment data. Med Educ 2004; 38: 100612.
  • 17
    Fitzmaurice G. Statistical methods for assessing agreement. Nutrition 2002; 18: 6946.
  • 18
    Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass coefficient as measures of reliability. Educ Psycholog Meas 1973; 33: 6139.
  • 19
    Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 15974.
  • 20
    Harvey A. Stability of parent-reported manual ability and gross motor function classification of cerebral palsy. Dev Med Child Neurol 2009; 52: 1145.
  • 21
    Palisano RJ, Cameron D, Rosenbaum PL, Walter SD, Russell D. Stability of the gross motor function classification system. Dev Med Child Neurol 2006; 48: 4248.
  • 22
    Wood E, Rosenbaum P. The gross motor function classification system for cerebral palsy: a study of reliability and stability over time. Dev Med Child Neurol 2000; 42: 2926.
  • 23
    Gorter JW, Ketelaar M, Rosenbaum P, Helders PJ, Palisano R. Use of the GMFCS in infants with CP: the need for reclassification at age 2 years or older. Dev Med Child Neurol 2009; 51: 4652.
  • 24
    Arner M, Eliasson AC, Nicklasson S, Sommerstein K, Hagglund G. Hand function in cerebral palsy. Report of 367 children in a population-based longitudinal health care program. J Hand Surg Am 2008; 33: 133747.