The Melbourne Assessment of Unilateral Upper Limb Function (Melbourne Assessment) is a criterion referenced tool designed to measure the quality of upper limb movement in children aged 5 to 15 years with a neurological impairment.[1-3] Although originally designed for children as young as 5 years, recent studies have established the validity of using a modified version of the tool with children as young as 2 years 6 months of age.[4, 5] Since its publication in 1999, the Melbourne Assessment has been increasingly used as an outcome measure in clinical and research applications, both nationally and internationally.[6-8] Studies investigating the effectiveness of interventions such as intrathecal baclofen and kinesiotaping are using the Melbourne Assessment to evaluate change in children's upper limb movements. For the Melbourne Assessment to be a valid outcome measure, it is critical that it demonstrates strong psychometric properties. Studies undertaken at the time of developing the tool found the Melbourne Assessment to have high internal, test–retest, and intra- and interrater reliability for the overall test scores.
The tool was developed using classical test theory approaches. Test developers undertook an extensive review of the literature and existing assessments, and sought input from expert clinicians to define the aspects or elements of movement that were typically observed when evaluating upper limb movement quality. Four main elements of movement quality were identified: (1) amount of active range of movement at each upper limb joint (ROM); (2) accuracy of reach for, or placement of, an item; (3) dexterity of finger movements when grasping, releasing, and manipulating objects; and (4) fluency or smoothness of the movement. These four elements are observed during completion of 16 tasks, which were selected to mimic everyday activities. Two additional aspects of upper limb movement: ‘Bilateral co-ordination’ and ‘Speed’ were also included in the original scoring (Table 1). Scoring is completed by grading one or more elements of movement quality observed in the performance of each task on a 3-, 4-, or 5-point ordinal scale using specifically developed criteria. Across the 16 tasks there are a total of 37 movement scores, which are summed to provide a single total score of the child's upper limb movement quality.
Table 1. Components of movement evaluated by the original Melbourne Assessment of Unilateral Upper Limb Function
|Upper limb task/action||Item no.||Item title||Elements of movement quality scored|
|Range of movement||Accuracy of reach/placement||Fluency||Dexterity of finger movements||Bilateral coordination||Speed|
|Reach||1||Reach forwards||X||X||X|| || || |
|2||Reach forwards to an elevated position||X||X||X|| || || |
|3||Reach sideways to an elevated position||X||X||X|| || || |
|11||Reach to brush from forehead to back of neck||X|| ||X|| || || |
|12||Palm to bottom||X|| ||X|| || || |
|15||Reach to opposite shoulder||X||X||X|| || || |
|16||Hand to mouth and down||X||X||X|| || ||X|
|Grasp||4||Grasp of crayon|| || || ||X|| || |
|5||Drawing grasp|| || || ||X|| || |
|7||Grasp of pellet|| || || ||X|| || |
|Release||6||Release of crayon||X||X|| ||X|| || |
|8||Release of pellet||X||X|| ||X|| || |
|Manipulation||9||Manipulation|| || ||X||X|| || |
|Pointing (×4)||10||Pointing|| ||X X X X|| || || || |
|Supination||13||Pronation/supination||X|| || || || || |
|Bilateral transfer||14||Hand to hand transfer|| || || || ||X|| |
|Total score items (37)|| || ||10||11||8||6||1||1|
The recent emphasis on the importance of producing psychometrically sound, interval level measures to advance the field of health sciences[9-11] has suggested the need to further investigate several aspects of the Melbourne Assessment scale. The unidimensionality, one aspect of internal construct validity, of the Melbourne Assessment scale requires more targeted evaluation. The unidimensionality of a scale is one of the fundamental properties of any measurement. Unidimensionality indicates that all items within the one scale are measuring the same construct. For a summed score to be valid, all items contributing to that score should measure the same construct. In the Melbourne Assessment it is important that the unidimensionality of the total scale be examined, as currently the 37 movement scores are summed to provide one total score. It is possible that, given the same four elements of movement are scored repeatedly across the 16 tasks, groups of scores that measure the same element of movement may form four separate, unidimensional subscales.
Over the past two decades there has been increasing use of the Rasch measurement model[9, 12-15] in the development and evaluation of clinical tools in the health sciences. Rasch analysis has also been used to verify and refine the psychometric properties of outcome measures initially developed using traditional methods of test construction.[10, 11, 16] The aim of this study, therefore, was to use Rasch analysis to further investigate the internal construct validity of the Melbourne Assessment and to optimize the scaling of the measure to ensure clinicians and researchers could draw meaningful conclusions from scores obtained on the tool. Specifically, the focus of the study was to investigate whether scores on the Melbourne Assessment comprised a unidimensional scale, or instead, contained four psychometrically distinct subscales: (1) ROM; (2) accuracy; (3) dexterity; and (4) fluency.
- Top of page
Rasch analysis, used in this study to further investigate the psychometric properties of the Melbourne Assessment, identified aspects of the tool that required modification. The scale was found to be multidimensional, with support found for four discrete unidimensional subscales, each measuring a distinct attribute of movement quality. This differs from the original assumption made at the time of developing the tool that all scores could be summed to provide one total score to measure upper limb movement quality. The revised version of the scale (Melbourne Assessment 2) resulting from this study, provides a measure of a child's quality of unilateral upper limb movement across four separate subscale scores: (1) range; (2) accuracy; (3) dexterity; and (4) fluency.
Rasch analysis identified a number of additional modifications needed to the Melbourne Assessment. Seven of the original 37 scores exhibited disordered response formats requiring rescoring, and 11 pairs of scores revealed high residual intercorrelations, suggesting item redundancy. Based on these results, and clinical judgement, the Melbourne Assessment was reduced in size, and restructured, resulting in a 9-score active range of movement subscale, an 8-score accuracy subscale, a 7-score fluency subscale, and a 6-score dexterity subscale. The final modified version of each subscale demonstrated good internal consistency, with high PSI values ranging from 0.81 and 0.92. In addition, the absence of evidence of differential item functioning for children's sex or age provides support that Melbourne Assessment 2 scores are not influenced by sex or age, thus clinicians can be confident that any differences in scores observed over time or between children of differing ages is a result of variations in their level of movement quality. Subscales showed overlap, but provide measurement of four clinically discrete elements of movement quality.
The refinements identified from this study have now been implemented in the production of a revised version of the tool, the Melbourne Assessment 2. The Melbourne Assessment 2 is available for purchase from The Royal Children's Hospital, Melbourne via the website (http://www.rch.org.au/melbourneassessment). The Melbourne Assessment 2 comprises 14 tasks and 30 scores. The scaling refinements incorporated in the revised tool improve the measurement potential of the scale and enhance clinician interpretation of test scores. The restructure of the scale into four separate, unidimensional subscales enables measurement of the specific qualitative elements of movement range, accuracy, dexterity, and fluency. Summing of scores into four subscale total scores also allows clinicians to identify the particular element(s) of movement quality a child might be experiencing greatest difficulty with, or where an intervention may have the greatest impact.
The finding that 10 score items across the four subscales showed DIF for different raters was not unexpected. Assessment of inter-rater agreement undertaken on the 37 scores of the original Melbourne Assessment had also identified 11 scores for which inter-rater agreement was moderate or poor (weighted kappa values ranging between 0.39 and 0.60). The presence of variation in raters’ scorings of these items creates a case for providing raters with consistent training in reliable scoring of the tool. Web-based resources which provide demonstration in the administration of assessment tasks and training in the scoring of individual movements are now accessible to purchasers of the tool. Guidelines are also provided for establishing reliable administration and scoring of the tool. If raters achieve the recommended level of scoring reliability, scores from different raters can be used with confidence in clinical and research applications.
The construction of the Melbourne Assessment 2 has arisen from the findings of the Rasch Analysis implemented in this study. It is now important that a study be undertaken with data collected using the Melbourne Assessment 2 to confirm these findings. Linacre also states that the sample size for a definitive analysis should be 20 times the number of items in the scale (i.e. 20×30 items in the refined scale), suggesting that future testing using larger samples is needed to support the findings of this study. In addition, other aspects of the scale also require further investigation. Assessment is needed of the ability of the four subscales to discriminate between differing levels of limitation in children's manual abilities. If subscale scores are found to correlate strongly with the level of upper limb function, it may be that the quantitative scores of the Melbourne Assessment 2 could be used to support applications for educational funding or therapy services for individual children. Subscale scores may also provide clinicians and researchers with an objective means of grouping children of like ability together in studies of treatment effectiveness or long-term follow-up.
Further research is also required to determine the ability of the scale of the Melbourne Assessment 2 to detect small, but clinically important, changes in a child's unilateral upper limb movement quality. A study to establish the standard error of measurement and smallest detectable difference, as well as the amount of change needed for each subscale to show a ‘clinically important change’ is currently being undertaken.