- Top of page
- What this paper adds
Aim To investigate the validity and reliability of the revised Video-Observation Aarts and Aarts module: Determine Developmental Disregard (VOAA-DDD-R).
Method Upper-limb capacity and performance were assessed in children with unilateral spastic cerebral palsy (CP) by measuring overall duration of affected upper-limb use and the frequency of specific behaviours during a task in which bimanual activity was demanded (‘stringing beads’) and stimulated (‘decorating a muffin’). Developmental disregard was defined as the difference in duration of affected upper-limb use between both tasks. Raters were two occupational and one physical therapist who received 3 hours of training. Construct validity was determined by comparing children with CP with typically developing children. Intrarater, interrater, and test–retest reliability were determined using the intraclass correlation coefficient. Standard errors of measurement and smallest detectable differences were also calculated.
Results Twenty-five children with CP (15 females, 10 males; mean age 4y 9mo [SD 1y 7mo], range 2y 9mo–8y; Manual Ability Classification System levels I–III) scored lower on capacity (p=0.052) and performance (p<0.001), and higher on developmental disregard (p<0.001) than 46 age- and sex-matched typically developing children (23 males; mean age 5y 3mo [SD 1y 5mo], range 2y 6mo–8y). The intraclass correlation coefficients (0.79–1.00) indicated good reliability. Absolute agreement was high, standard errors of measurement ranged from 4.5 to 6.8%, and smallest detectable differences ranged from 12.5 to 19.0%.
Interpretation The VOAA-DDD-R can be reliably and validly used by occupational and physical therapists to assess upper-limb capacity, performance, and developmental disregard in children (2y 6mo–8y) with CP.
Children with unilateral spastic cerebral palsy (CP) have motor impairments such as muscle weakness and spasticity on predominantly one side of the body.1,2 These motor impairments are important causes of activity limitations.3,4 According to the International Classification of Functioning, Disability and Health, the ‘activity’ level can be subdivided into ‘capacity’ (i.e. the execution of an activity in a standardized environment) and ‘performance’ (i.e. the actual performance of an activity in daily life).5 Children with CP not only experience limitations in their capacity, but they also tend to underuse their affected upper limb in daily life (i.e. limited performance) given their individual capacity. This lack of spontaneous use of the affected limb in developing children is also referred to as ‘developmental disregard’.6
To design an individually tailored rehabilitation programme, detailed assessment of upper-limb disability is essential.7 Therefore, it is important to assess bimanual activities because many children who have developmental disregard prefer to use their less-affected upper limb in unimanual tasks. They will only use their affected limb during bimanual tasks. However, tests of upper-limb use during bimanual activities are scarce,8,9 and many functional measures focus on unilateral tasks.10,11 Only the Assisting Hand Assessment12 consists of semi-structured bimanual tasks to assess the effectiveness of use of the assisting upper limb. Although the Assisting Hand Assessment provides a summed frequency score of the effectiveness of upper-limb use, it does not assess the duration of spontaneous use. Because the overall duration of upper-limb use takes into account all motor behaviours, including (unsuccessful) attempts to involve the affected arm and hand, it seems to be a more valid indicator of developmental disregard than merely counting the frequency of successful behaviours.
To assess both the overall duration and frequency of affected upper-limb use, the ‘Video Observations Aarts and Aarts module: Determine Developmental Disregard’ (VOAA-DDD) was developed.13 It consists of two standardized tasks, ‘stringing beads’ and ‘decorating a muffin’, to assess upper-limb use. The beads task was designed to demand the use of both hands to accomplish the task, whereas the muffin task was designed merely to stimulate bimanual activity (the task is most efficiently performed with both hands). By using structured video observations and a custom-designed software program,14 the tasks can be scored offline for the occurrence of specific motor behaviours (i.e. frequency) and the total duration of affected upper-limb use. When used by trained occupational and physical therapists, the VOAA-DDD was shown to be reliable and valid in children between 2 years 6 months and 8 years of age with unilateral spastic CP.13 However, the scoring system of the VOAA-DDD was very elaborate and the numbers of subtasks and repetitions were not consistent in the two tasks.
Recently, the VOAA-DDD was revised (VOAA-DDD-R) to improve feasibility and interpretation. First, the distinction between the beads task (demanding bimanual hand use) and the muffin task (stimulating bimanual hand use) was made more pronounced. Second, the beads and muffin task now have the same number of subtasks, which is also the same for all ages. Third, the motor behaviours that need to be scored were reduced from 10 to the three most important behaviours (i.e. grasp, hold, release). These behaviours were shown to be essential to performing each subtask and did not differ in frequency between the dominant and non-dominant hand in typically developing children.15 Finally, only three scores are used to reflect different aspects of upper-limb use: a capacity score (i.e. the frequency during the beads task), a performance score (i.e. the frequency during the muffin task), and a duration score (i.e. the difference in the duration of upper-limb use between the beads and the muffin task).15 The last score was used as an operationalization of developmental disregard. These revisions required a new investigation of the psychometric properties of the VOAA-DDD-R. The goal of the present study was to investigate the construct validity and the intrarater, interrater, and test–retest reliability of the VOAA-DDD-R in children with unilateral spastic CP.
- Top of page
- What this paper adds
The results of this study indicate that the three scores of the VOAA-DDD-R (i.e. capacity, performance, and developmental disregard) are both valid and reliable. The construct validity was determined by comparing the scores of children with CP with those of typically developing children, because there is no criterion standard available in the literature for the frequency and duration of use of the affected upper limb during bimanual activities. Children with CP had lower scores than typically developing children for capacity (77% vs 98%) and significantly lower scores for performance (55% vs 100%), yielding much higher scores for developmental disregard (23% vs 7%). In addition, the variability in the CP group was much higher compared with the typically developing children. Furthermore, the older children performed better than the younger ones on the capacity score, which may have been related to improvements in bimanual performance that are related to development. This finding needs to be taken into account by therapists when assessing younger children with the VOAA-DDD-R.
Based on the cut-off score for developmental disregard of typically developing children (i.e. 17%), 64% of the children with CP could be identified as having developmental disregard (Fig. 2). This cut-off value is close to the cut-off value reported in our previous study on the VOAA-DDD (14%).13 These results confirm our hypothesis that many children with CP show a discrepancy between what they can do with their affected upper limb when bimanual activity is demanded (i.e. capacity) and what they actually do when bimanual activity is merely stimulated (i.e. performance). These test scores can be used as a basis for designing an individually tailored rehabilitation intervention.15 For instance, Figure 2 shows that the six children with a low capacity score (0–40%) scored 0% on performance. Based on these scores it is advisable that these children are primarily trained to improve their upper-limb capacity. Remarkably, even eight children with a (near) maximum capacity score of 100% showed some degree of developmental disregard, whereas nine others with a maximum capacity did not. This pattern of results suggests that a (nearly) optimal capacity is needed to prevent the occurrence of developmental disregard, but that such a score certainly provides no guarantee for the absence of developmental disregard. Thus, these children should all be carefully monitored for signs of developmental disregard and offered appropriate training (e.g. constraint-induced movement therapy). On the other hand, one or two children with a somewhat lower performance than their optimal capacity scores did not seem to have developmental disregard based on the duration of use of their affected upper limb.
The VOAA-DDD-R showed excellent intra- and interrater reliability, as indicated by high ICCs for capacity, performance, and developmental disregard. Reliability in this context means that repeated measurements result in similar outcomes,17,18 which are not influenced by characteristics of the instrument, differences in performance by the same rater, differences between multiple raters, or by the natural variability within an individual. The ICC values in the present study indicated that the repeated scoring of the assessments was very stable both within (intrarater reliability) and between raters (interrater reliability). This suggests that when a child is assessed twice by the same rater or by two different raters, the results are generally the same and not affected by the measurement instrument. The test–retest reliability was excellent for the capacity and performance scores and good for developmental disregard. Thus, the variability between two assessments caused by variation of the child’s behaviour was larger than the variation caused by the raters. In addition, the results indicate that with repeated testing the frequency scores were more stable than the duration scores. This can be explained by the fact that for the frequency scores a child could obtain maximally one point for each behaviour per subtask, which renders the frequency scores more stable but also less sensitive to repeated behaviours within a subtask. Nevertheless, the absolute agreement between the repeated assessments was good, as indicated by SEMs between 4.5% and 6.8%. These results imply that, when two groups of children with CP are compared, a group difference of 5.1% on capacity, 4.5% on performance, and 6.8% on developmental disregard can be regarded as a real difference and not due to natural variation. For individual children, a change in the VOAA-DDD-R scores needs to be larger to be significantly different, because the smallest detectable differences ranged from 12.5 to 19.0%. These results indicate that although the VOAA-DDD-R is suitable to detect differences between groups, it needs to be further refined to be able to detect smaller changes in individual children.19
Until now, no reliable and valid measure of developmental disregard has been available in the literature. In this perspective, the VOAA-DDD-R is a valuable addition to the existing measures of affected upper-limb use in children with CP. Because the VOAA-DDD-R consists of common daily-life tasks that are attractive and meaningful for all children, it may also have merits for other groups of children with unilateral upper-limb disability, for instance children with peripheral nerve damage, traumatic brain injury, or stroke. A limitation of the present study is that the responsiveness (i.e. sensitivity to change) was not investigated. Thus, future studies need to examine the responsiveness of the VOAA-DDD-R to determine its usefulness and sensitivity in intervention studies. Another limitation is that one could argue that the VOAA-DDD-R is not truly a test of upper-limb performance in daily life, because it requires a standardized test situation. Yet, a drawback of real-life assessments is that they may be too subjective. For instance, self-report questionnaires20,21 are usually completed by the child’s parents or caregivers with a great influence of personal perspectives and proneness to inconsistencies. Recent developments in the use of wearable wrist activity monitors to assess actual daily-life use of the affected upper limb are promising,22 but such monitors have only been tested during standardized activities as well. Finally, the construct validity was determined based on the assessment of typically developing children, who are expected to have no limitations in capacity and performance and show no developmental disregard. To confirm that the cut-off value for developmental disregard used in this study was indeed valid, we need to investigate other groups of children with CP with and without developmental disregard as determined, for example, by experts.
In conclusion, this study showed that the VOAA-DDD-R, using a simplified scoring system, is equally reliable, when performed 2 weeks apart, and as valid as the original VOAA-DDD when applied by trained occupational and physical therapists to children with unilateral spastic CP (2y 6mo–8y). By comparing the use of the affected upper limb during a task demanding the use of both hands compared with a task merely stimulating bimanual activity, upper-limb capacity, performance, and developmental disregard can be reliably and validly assessed offline with a computer-supported video scoring system.