The aim of this review was to evaluate the psychometric properties of outcome measures used to quantify upper limb function in children and adolescents with brachial plexus birth palsy (BPBP).
The aim of this review was to evaluate the psychometric properties of outcome measures used to quantify upper limb function in children and adolescents with brachial plexus birth palsy (BPBP).
Eleven electronic databases were searched to identify studies on the effects of conservative management to improve upper limb function in young people with BPBP. Outcome measures used in these studies were extracted and used in a subsequent search to identify studies that evaluated the psychometric properties of these measures. The methodological quality of these studies was rated using a standardized critical appraisal tool.
Thirty-three outcome measures and 12 psychometric studies were identified. Nine outcome measures had some psychometric evidence, which was variable in quality. The outcome measures which seem to have the most robust psychometric properties include the Active Movement Scale, Assisting Hand Assessment, Pediatric Evaluation of Disability Index, and the Pediatric Outcomes Data Collection Instrument.
Further research is required to determine the psychometric properties of outcome measures used for children and adolescents with BPBP. Caution is required when interpreting the results of commonly used outcome measures in this population owing to their relatively unknown psychometric properties.
Brachial plexus birth palsy
Pediatric Evaluation of Disability Index
Pediatric Outcomes Data Collection Instrument
This paper reports on the results of a systematic search and critical appraisal of literature on the psychometric properties of outcome measures for children and adolescents diagnosed with brachial plexus birth palsy (BPBP). BPBP is the most frequently occurring peripheral nerve injury in childhood, and is often diagnosed soon after birth. The likelihood of sustaining BPBP is multifactorial, with maternal, fetal, and delivery risk factors reported in the literature. Brachial plexus injuries range in severity and location within the brachial plexus. Children with middle (C5–C7) and total (C5–T1) plexus injuries tend to have a high risk of permanent disability, compared with children with upper plexus injuries (C5, C6).[3, 5] Consequently, children and adolescents with BPBP are seen to have a wide range of limitations in the functional use of their affected arm.[6, 7] Varying opinions exist regarding the most appropriate methods to quantify upper limb function.[8, 9] This makes it difficult for clinicians and researchers to prescribe and monitor the effects of various management strategies for these children.
The management of this condition often occurs from infancy, throughout childhood, into adolescence and adulthood. Conservative management, such as physiotherapy and occupational therapy, is usually commenced first. Surgery (either to the brachial plexus or locally at the affected joints) may be undertaken in childhood or adolescence. Therefore, outcome measures used in young people with this diagnosis must be able to accurately quantify upper limb changes over time in the developing body. It is therefore important to identify outcome measures in young people with BPBP that are not only valid and reliable, but also responsive to change over time.
Validity, reliability, and responsiveness collectively are termed psychometric properties of an outcome measure. Reliability is the extent to which a measurement is free from error. In reality, error is present in all types of measurement, and may be increased in young children who are more unpredictable in their movements compared to adults. Validity is the extent to which an outcome measure evaluates a variable or construct of interest, whereas responsiveness refers to the ability of an outcome measure to detect clinically meaningful changes over time. These psychometric properties can be evaluated by using various methodological and statistical approaches (e.g. classical test theory, item response theory, Rasch model of measurement), over long periods and in different samples, to provide clinicians and researchers with evidence of the behaviour of an outcome measure.
It is important for clinicians and researchers to have a thorough understanding of the psychometric properties of outcome measures and to use this evidence to select the most appropriate outcome measure(s) for their measurement needs. This will give users confidence in their measurement process. This is particularly important in children with BPBP, as surgical decisions are often based on variables such as upper limb strength, active upper limb movement, and the ability to perform age-appropriate activities.[2, 6, 8] These are all variables that are targeted during conservative management of children and adolescents with BPBP.[15, 16]
The aims of this study were (1) to identify outcome measures used in the literature to evaluate upper limb function in children and adolescents with BPBP; (2) to identify and critically appraise studies that evaluate the psychometric properties of these outcome measures and synthesize their findings; and (3) to make recommendations regarding the most appropriate outcome measures for children and adolescents with BPBP. This synthesis therefore represents an important first step towards establishing evidence-based recommendations for future intervention programmes and research studies that examine the effect of different management strategies in children and adolescents with BPBP.
This study was undertaken in two phases and was based on the work by Adair et al., who investigated the psychometric properties of functional mobility outcome measures in hereditary spastic paraplegia and other childhood neurological conditions. Phase 1 involved generating a list of outcome measures, which have been used to evaluate children and adolescents with BPBP. Phase 2 involved identifying the psychometric properties of these outcome measures.
The search strategy used to address the aim of phase 1 builds on the work by Bialocerkowski et al.,[15, 16] who collated outcome measures used to evaluate the effectiveness of conservative interventions for infants younger than 2 years of age diagnosed with BPBP. To generate an up-to-date list of outcome measures used in children and adolescents with BPBP, the search strategy used by Bialocerkowski et al.[15, 16] was replicated and broadened to identify primary studies conducted on children and adolescents with BPBP, irrespective of their age. This meant that the original selection criterion ‘participants (in the primary studies) being younger than 2 years of age' was not used in the current study. This methodological approach is considered a robust method of identifying outcome measures in this population as it has previously systematically identified a large body of literature on this topic compared with other systematic reviews on this topic.[18, 19] Moreover, children with BPBP often receive conservative interventions before undergoing primary or secondary surgery if required.
Eleven databases were searched, from January 1992 to August 2012, for quantitative studies, published in the English language, on the effectiveness of conservative interventions for children and adolescents diagnosed with BPBP: CINAHL, MEDLINE, Web of Science, Scopus, ProQuest Central, ScienceDirect, PEDro, Meditext, TRIP Database, MyOAI, and ProQuest Digital Dissertations. The search strategies used by Bialocerkowski et al. in their 2005 and 2009 studies were replicated, as they were custom designed based on the unique features of each database (see Appendix SI, supporting information published online). The Scopus search strategy was developed using the same principles as outlined in Bialocerkowski et al. because this database was not available during previous searches.
Studies identified from the database search were independently evaluated by two researchers (KO and AB) against the inclusion criteria: quantitative studies (randomized controlled trials, pseudo-randomized controlled trials, comparative studies, and case series) which investigated the effects of conservative interventions for the treatment of children and adolescents (<18y) diagnosed with BPBP. Multiple single-subject studies were included, as participants acted as their own comparison and were evaluated using the same outcome measures throughout the course of treatment. Conservative intervention was considered as management that was not surgical, and could include, but was not limited to, physiotherapy, occupational therapy, orthotics, and botulinum toxin A (BoNT-A) injections. Studies which investigated the effects of BoNT-A were included only if they were combined with other conservative management techniques, such as physiotherapy or occupational therapy. Thus, studies which evaluated the effect of BoNT-A following surgery were excluded from this study. Inclusion of studies into this review was reached by consensus of the two independent researchers.
The names of the outcome measures used to evaluate the effectiveness of the conservative management for children and adolescents with BPBP were extracted from the included studies independently by the same two researchers (KO and AB). The list of outcome measures generated was compared between the researchers and consensus gained when any disagreements occurred. The reference lists of the primary studies identified during this phase were also screened to identify additional outcome measures used for children with BPBP. When identified, these outcome measures were added to the outcome measures list.
Four databases were searched (MEDLINE, CINAHL, Web of Science, Scopus), from their date of inception to August 2012, by entering the name of each outcome measure identified in phase 1 and combining it with the phrase ‘brachial plexus’. These databases were chosen because, in combination, they contain the most comprehensive range of journals focusing on health and have provided the majority of evidence in previous systematic reviews on children with BPBP.[15, 16, 18] Studies were included if they were published in the English language and investigated one or more psychometric properties of the outcome measure (i.e. validity, reliability, and/or responsiveness). As psychometric properties of outcome measures are population specific, these psychometric studies needed to be conducted on children diagnosed with BPBP so as to be included in this systematic review. Inclusion of studies into this review was decided by consensus of the two independent researchers (KO and AB). Moreover, studies were included in this systematic review if they reported on the development and preliminary psychometric evaluation of new outcome measures specifically developed for children and adolescents with BPBP.
The outcome measures list was subsequently updated to include all identified outcome measures from phase 1 and phase 2 search strategies. All outcome measures were classified according to the International Classification of Functioning and Health (ICF) as ‘body function’, ‘activity/participation variables’, or ‘other domains’. The latter category, ‘other domains’, was added to capture other outcome measurement types, such as quality of life and patient or family satisfaction with treatment. The proportion of outcome measures which evaluated body function, activity/participation variables, and ‘other domains’ was calculated.
The methodological quality of the included psychometric studies was assessed independently by two researchers (TP and AB) and consensus gained by discussion. The criteria of Brink and Louw were used, which specifically appraise the validity and reliability of outcome measures. This critical appraisal tool consists of 13 items. No composite score was calculated as this critical appraisal tool assesses the impact of each individual item on the quality of methodological procedures implemented in each study. In addition, various types of data were extracted from each of the studies: authors; date of publication; name of the outcome measure; participant, rater, and measurement characteristics; and results of the psychometric evaluation. The psychometric properties of the outcome measures were narratively synthesized and interpreted with respect to the methodological quality of the primary studies. Descriptive statistics were used to summarize the volume of psychometric evidence for each outcome measure per ICF domain. Reliability coefficients gained from the included studies were interpreted using the recommendations by Katz et al.: >0.80, very high agreement; 0.60 to 0.79, moderately high agreement; 0.40 to 0.59, moderate agreement; and <0.4, low agreement.
Figure 1 illustrates the study selection process and search yield. The phase 1 search strategy yielded 29 articles which met the inclusion criteria.[24-52] In these articles, 29 different measures were used to evaluate the effect of an intervention (Table 1). An additional four outcome measures were identified though secondary search during the phase 2 search: the Toronto Test Score, activities to evaluate arm and hand function, the Children's Hand-use Experience Questionnaire, and the Brachial Plexus Outcome Measure. Therefore, in total, 33 outcome measures were identified for children and adolescents with BPBP (Table 1). There was 100% agreement between the two researchers regarding these outcome measures. Two-thirds of the outcome measures (n=22) evaluated body function variables, such as upper limb range of motion, strength, and sensation. Ten outcome measures evaluated activities/participation variables, whereas one assessed quality of life (Table 1). Twelve articles on the psychometric properties of these outcome measures were identified from the phase 2 search strategy[34, 37, 53-62] and were included in data extraction (Table 2 and Table SI, online supporting information) and quality appraisal (Table SII, online supporting information). There was 99% agreement between the two researchers with respect to the methodological quality of these articles. The one disagreement was resolved by discussion.
|Outcome measure||Description||Studies that measured children and adolescents with BPBP (Aim 1)||Reports on psychometric properties (Aim 2)|
|Addresses body function|
|Active Movement Scale||Involves a physical examination where combined active movement and muscle strength is quantified based on 15 upper limb movementsa||Bain et al.; Strombeck et al.||Bae et al.; Bialocerkowski and Galea; Curtis et al.|
|British Medical Council's Muscle Movement Scale||Involves a physical examination in which the strength of various muscle groups is quantified on a six-point grading system||Basciani and Intiso; Heise et al.; Lagerkvist et al.; Laurent et al.; Lindell-Iwan et al.; Murphy et al.; Noetzel et al.; Santamato et al.|
|Coding of sequelae||Sequelae of BPBP are coded according to the range of shoulder abduction, supination and elbow extension, and hand function||Lindell-Iwan et al.|
|Dual X-ray absorptiometry||An evaluation of bone mineral density (bone per unit skeletal area) in grams of hydroxyapatite per square centimetre||Ibrahim et al.|
|Gilbert scale (shoulder)||Involves a physical examination of shoulder abduction or external rotation which is graded on an ordinal scale||Bisinella and Birch; DiTaranto et al.|
|Gilbert and Raimondi Scale (elbow)||Involves a physical examination. Combines the range of active elbow flexion and extension with the magnitude of a fixed flexion deformity at the elbow||Bisinella and Birch; Santamato et al.|
|Global clinical rating scale||Parents' perception of benefit of treatment rated on a 0–100 scale||Desiato and Risina|
|Grip strength||Ability to grip, usually measured in kilograms or expressed as a ratio between two hands, based on a physical examination||Strombeck et al.[48, 49]|
|Impairment rating scale||A five-point scale which evaluates upper extremity movement, strength, and sensation, based on the results of a physical examination||Badr et al.; Eng et al.|
|Mallet Scale (shoulder)||Involves a physical examination where the child performs five functional upper limb movementsa||Basciani and Intiso; Bisinella and Birch; Hoeksma et al.; Leblebicioglu et al.; Philandrianos et al.; Santamato et al.; Waters; Xu et al.||Bae et al.|
|Narakas Motor Scale||Involves a physical examination where the Modified British Medical Council's Muscle Movement Scale is used to quantify the strength of various muscle groups on a four-point grading system||Hoeksma et al.; Leblebicioglu et al.; Santamato et al.|
|Nine-hole Peg Test||A timed test of finger dexterity, measured in seconds to complete the various tasks||Basciani and Intiso; Buesch et al.; Santamato et al.|
|Quantitative electromyography||A physical examination test which quantifies the fullness of the interference pattern and the size of individual motor unit action potential based on a five-point scale||Eng et al.; Strombeck et al.|
|Quantitative sensory testing||A physical examination test to detect cold and warm sensation measured in degrees centigrade and compared with normative values||Strombeck et al.|
|Pick-up test||Relative pick up time in seconds of the unaffected to the affected arm. A comparison is made with normative values and a ratio of <2.2 is considered typical||Strombeck et al.[48-50]|
|Raimondi Scale (hand)||Involves a physical examination. Combines the results of active movement of the fingers, thumb, wrist, and forearm with hand sensation||Bisinella and Birch; DiTaranto et al.|
|Scoliosis (via observation)||Based on clinician observation and quantified as either ‘present’ or ‘not present’||Strombeck et al.|
|Stereognosis||Recognition of objects placed in the hand without the use of visual input||Strombeck et al.|
|Toronto Test Score||Involves a physical examination where the child performs five upper limb movementsa||Bae et al.|
|Two-point discrimination||A test of sensory discrimination, based on a physical examination, and measured in millimetres, with 3mm considered typical||Strombeck et al.[48, 50]|
|Upper limb movement – active||Physical examination of active movement of upper limb joints usually measured in degrees||Basciani and Intiso; DeMatteo et al.; Desiato and Risina; Lagerkvist et al.; Murphy et al.; Okafor et al.; Strombeck et al.[48, 49]; Waters|
|Upper limb movement – passive||Physical examination of passive movement of upper limb joints usually measured in degrees||Ho et al.; Murphy et al.; Waters|
|Addresses activities and participation|
|Activities to evaluate arm and hand function||A set of activities used to evaluate bimanual hand use in children aged 4–6y covering the domains of dressing, eating, washing, and infant activities||Boeschoten et al.|
|Assisting Hand Assessment||A physical test which is recorded to score, which qualifies how effectively a child uses their hand in bi-manual play on 22 items||Buesch et al.||Holmefur et al.; Krumlinde-Sudholm and Eliasson; Krumlinde-Sudholm et al.|
|Bimanual activity||The parents' perception of the child's ability to eat, dress, groom, and bathe, and possible problems associated with play, sports, and outdoor recreation based on open-ended interview questions||Strombeck et al.[48, 49]|
|Brachial Plexus Outcome Measure||Evaluates activities and participation specific to anatomical impairments for children with BPBP||Ho et al.|
|Children's Hand-use Experience Questionnaire||Experience of hand use, measured on three domains (grasp efficiency, time taken, feeling bothered) for bimanual activities, for children and adolescents (aged 6–18y)||Sköld et al.|
|Functional questionnaire||The parents' perception of the child's ability to feed, wash, dress, and ride a bike, with each activity quantified on a four-point scale||Hoeksma et al.|
|Functional skills||Eight functional age-appropriate skills (grasp and release, uni- and bimanual activities) based on the Callier–Azusa Scale and the Denver II||Lagerkvist et al.|
|Level to perform normal activities of daily living||The parents' perception of the child's ability to perform normal activities of daily living according to their age rated on a 0–100 scale||Hoeksma et al.|
|PEDI||An interview which assesses functional skill level, caregiver independence, modifications/adaptive equipment based on 20 items||Ho et al.||Ho et al.|
|The Melbourne Assessment of Unilateral Upper Limb Function||A physical examination test which grades unilateral upper limb function on 16 grasp and release activities||Buesch et al.|
|Addresses other variables|
|PODCI||A questionnaire which quantifies functional health status with a focus on musculoskeletal health in children and adolescents through four domains||Huffman et al.||Huffman et al.; Bae et al.|
|Outcome measure||Author||Participant characteristics|
|n||Age||Sex||Assessor characteristics||Measurement characteristics||Results|
|Outcome measures which evaluate body functions|
|Active Movement Scale||Bialocerkowski and Galea||30 diagnosed with BPBP||7mo–4y 7mo||18 females, 12 males||Two paediatric physiotherapists||One prospective measurement session, comparing the results of the active movement scale with two-dimensional motion analysis for shoulder flexion and abduction, and elbow flexion and extension||Face validity: overestimation of shoulder and elbow active movement compared with two-dimensional motion analysis (per cent agreement=41–70%, ҡ=−0.05 to 0.07)|
|Outcome measures which address activity/participation|
|Assisting Hand Assessment||Krumlinde-Sudholm and Eliasson||37 (seven diagnosed with BPBP; 22 diagnosed with spastic hemiplegia) with a total of 66 assessments||16mo–5y (children with BPBP: 2y 7mo–5y [mean 3y 11mo])||20 females, 17 males (children with BPBP: three females, four males)||One occupational therapist||One prospective session which consisted of scoring of the Assisting Hand Assessment from a video. Rasch analysis used to provide evidence of validity||Unidimensionality: unidimensionality of 21 out of 22 items (<5% misfit)|
|Krumlinde-Sudholm et al.||302 with a total of 409 assessments (47 assessments on children with BPBP; 362 assessments on children with hemiplegia)||18mo–12y (mean 4y 11mo)||181 females, 228 males||Expert raters||One prospective session which consisted of scoring of the Assisting Hand Assessment from a video. Rasch analysis used to provide evidence of validity||Unidimensionality: unidimensionality for 20 out of 22 items (<5% misfit)|
Person-responsive validity: 97% with acceptable goodness of fit to model expectations
Ceiling and floor effects: none present
|Holmefur et al.|| |
Two-rater study: 18 (two with BPBP, 16 with hemiplegic cerebral palsy)
20 rater study: 8 (two with BPBP, six with hemiplegic cerebral palsy)
Two-rater study: 18mo–5y 11mo
Twenty-rater study: 9mo–4y 2mo
Two-rater study: not provided
Twenty-rrater study: not provided
|Occupational therapists who are certified Assisting Hand Assessment raters (3d certification course)||Two prospective sessions (3wks apart) which consisted of scoring the Assisting Hand Assessment from a video and comparing therapists' scores with a criterion standard (Assisting Hand Assessment certifiers' scores)||Construct validity: supported by Spearman's ρ=−0.43 (association between the raters' Assisting Hand Assessment score and the criterion standard)|
|Brachial Plexus Outcome Measure||Ho et al.||306 diagnosed with BPBP||4–19y||180 females, 126 males||One occupational therapist, one occupational therapy student||One retrospective measurement session, with the Brachial Plexus Outcome measure and Active Movement Scale data for each case extracted from case notes||Construct validity: supported by Spearman's ρ=−0.71 (association between the Brachial Plexus Outcome Measure and the Active Movement Scale)|
|Children's Hand-use Experience Questionnaire||Sköld et al.||86 (31 with unilateral cerebral palsy; 26 with BPBP, 29 with upper limb reduction deficiency)||Children with cerebral palsy mean age 11y (SD 3y); children with BPBP mean age 13y (SD 4y); children with upper limb reduction deficiency mean age 11y (SD 3y)||44 females, 42 males||One prospective measurement session. Rasch analysis used to provide evidence of validity|| |
Unidimensionality: underfit by one item
Principal component analysis=57.4–64.1%
|PEDI||Ho et al.||45 diagnosed with BPBP||7mo–6y (mean 5y 1mo)||29 females, 16 males||Not stated||One retrospective measurement session, with the PEDI and Active Movement Scale data for each case extract from case notes||Discriminant validity: supported by z=−2.143, α=0.32 (PEDI score significantly lower in children with BPBP without hand impairment as assessed by the Active Movement Scale, compared with children with BPBP without hand impairment)|
|‘Other’ outcome measures|
|PODCI||Huffman et al.||23 diagnosed with BPBP who are candidates for external rotation tendon transfer surgery||3y 6mo–8y 7.2mo (mean 5.6y 1mo)||14 females, 9 males||n/a||One prospective measurement session. Comparison of children with BPBP to normative data sets and active range of movement of shoulder abduction and external rotation, active and passive movement of elbow motion and forearm rotation, and presence/absence of hand involvement|| |
Discriminant validity: supported by significant difference between children with BPBP and normative PODCI data for four scales (upper extremity, mobility, sports, global function), with p<0.05
Construct validity: decreased active range of movement of shoulder external rotation was significantly associated with lower comfort scores on the PODCI (no statistics provided)
|Bae et al.||150 diagnosed with BPBP||2–10y (mean 5y)||83 females, 67 males||Orthopaedic surgeon trained in the use of the Active Movement Scale, Toronto Scale, and Mallet Scale, with previously established reliability||One prospective measurement session. Comparison of the PODCI with normative data, and Mallet Scale, Toronto Test Scale, and Active Movement Scale scores|| |
supported by significantly lower PODCI values for children with BPBP compared with normative values for the upper extremity, mobility, sports, comfort, and happiness scales (p<0.01). Positive significant associations between the PODCI and the Mallet score in children aged 2–5y, between the PODCI sports score and the Mallet score for children aged 2–5y and between the PODCI global function and to Toronto Scale Score for children aged 6–10y
Of the 20 outcome measures which evaluate body structure and function variables, three had evidence of their psychometric properties for children and adolescents with BPBP: the Active Movement Scale,[53, 58, 59] Toronto Test Score, and the Mallet Scale. This evidence was produced by a total of three studies.[53, 58, 59]
There is low volume of evidence (from only two studies)[53, 59] which suggests that the Active Movement Scale has fair to substantial intra- and interrater reliability in children aged 1 month to 15 years when used by paediatric physiotherapists or orthopaedic surgeons who have undergone training in using this outcome measure (Table SI). In addition, the reliability of the Active Movement Scale tended to dependent on the age of the child, with higher reliability coefficients gained in older children compared with infants. Reliability coefficients also varied with respect to the movement evaluated, with shoulder external rotation and elbow extension tending to be the least reliable, irrespective of the age of the child.[53, 59] Bialocerkowski and Galea questioned the validity of this outcome measure to quantify shoulder and elbow movement in children with BPBP. They found that when the Active Movement Scale was used by an experienced paediatric physiotherapist, range of active shoulder and elbow movement was overestimated (by one Active Movement Scale grade) in children aged 6 months to 6 years when compared with two-dimensional motional analysis (Table 2). These psychometric properties should be interpreted with respect to the methodological limitations of the studies, such as the lack of variation in the order of examination,[53, 58, 59] the lack of detail regarding the application of the outcome measure which would not allow replication, and the lack of details regarding competence of the assessor (Table SII). No evidence was found on the responsiveness of the Active Movement Scale.
There is low volume of evidence which suggests that the Mallet Scale and the Toronto Test Score both have excellent internal consistency (α>0.90 [p<0.001]) (Table SI). The Mallet Scale has evidence of substantial intra- and interrater reliability in children aged 1 month to 15 years when used by paediatric physiotherapists or orthopaedic surgeons who have undergone training in the use of this outcome measure. The Toronto Test Score has moderate intra- and interrater reliability in the same sample of children and raters. Bae et al., however, found that the reliability of these two outcome measures varied and was dependent on the age of the child evaluated, with higher reliability coefficients gained in older children than in infants (Table SI). No studies were found which evaluated validity and responsiveness of the Mallet Scale and the Toronto Test Score.
Of the 10 outcome measures which evaluate activity/participation variables, five had evidence of psychometric properties: the Assisting Hand Assessment,[60-62] Arm and Hand Function, the Brachial Plexus Outcome Measure, the Children's Hand-use Experience Questionnaire, and the Pediatric Evaluation of Disability Inventory. This evidence was produced by a total of seven studies.[34, 54-56, 60-62]
The Assisting Hand Assessment had the greatest volume of psychometric evidence.[60-62] There is evidence which suggests that the Assisting Hand Assessment has almost perfect reliability (intra- and interrater reliability; Table SI) and evidence of appropriate validity when the child's performance is scored from a video by certified Assisting Hand Assessment raters (Table 2). These results should be interpreted with respect to the methodological quality of these studies, where raters were not blinded to their results and the reference standard was not clearly explained[61, 62] (Table SII). Moreover, all of these studies were conducted on samples of children with various upper limb conditions (Table 2 and Table SI), such as hemiplegic cerebral palsy and BPBP. Results were not stratified for each diagnostic group. No evidence was found on the responsiveness of the Assisting Hand Assessment in children with BPBP.
The PEDI had evidence of discriminant validity. This was gained from the results of one study in which retrospectively collected PEDI and Active Movement Scores were extracted from the case notes of children with BPBP aged up to 6 years (Table 2). Explanation of the reference standard was not provided in sufficient detail to allow replication (Table SII) and it is not known whether these data were collected in a consistent manner over time. No studies were found which evaluated reliability and responsiveness of the PEDI in a sample of children or adolescents with BPBP.
The Arm and Hand Function outcome measure rates activities across four domains (hand use, assistance, speed, and deviation of movement and body posture). It is a new outcome measure which has evidence of moderate intrarater reliability across dressing, eating, washing, and infant activities when assessed from a video (Table SI). These results need to be interpreted with respect to the competence of the rater, which was not defined. Moreover, the rater was not blinded to their previous results and the order of examination was not varied (Table SII). No studies were found which evaluate the validity and responsiveness of this new outcome measure in a sample of children or adolescents with BPBP.
The Brachial Plexus Outcome Measure is another new outcome measure. Ho et al. found that there is evidence of good internal consistency of this outcome measure in children and young people with BPBP aged 4 to 19 years (Table SI). There is evidence of construct validity of the Brachial Plexus Outcome Measure when compared with the Active Movement Scale scores (Table 2). However, insufficient information was provided regarding the rater's competence and the period between the two tests (Table SII).
Another recently developed outcome measure is the Children's Hand-use Experience Questionnaire. Sköld et al. provided evidence of its content and internal structure, from Rasch analysis, in children (mean age 11y) with unilateral upper limb disorders, such as cerebral palsy, BPBP, and upper limb reduction deficiency (Table 2 and Table SI). Although the psychometric properties of this outcome measure appear appropriate, these results were not stratified based on the diagnosis of the child.
One ‘other’ outcome measure, the Pediatric Outcome Data Collection Instrument (PODCI), which quantifies quality of life, was identified from the search strategy. This outcome measure has evidence of discriminant validity, with children with BPBP (aged up to 8y) having significantly lower scores on upper extremity, mobility, sports, and global function domains than age-matched typically developing children (Table 2). According to the criteria of Brink and Louw, this study had sound methodological quality (Table SII). Bae et al. provided evidence to support the construct validity of the PODCI in children with BPBP aged 2 to 10 years, when compared with normative data. In children with BPBP aged 2 to 10 years, PODCI and Mallet Scale scores were significantly lower than normative data in the domains of upper extremity, mobility, sports, comfort, and happiness. Positive significant associations were found between various PODCI subscales and Mallet Scale scores (Table 2). However, the order of examination in this study was not varied (Table SII). No studies evaluating the reliability and responsiveness of this outcome measure in children and adolescents with BPBP were found.
This is the first study, known to the authors, which has systematically identified outcome measures used in the literature to assess children and adolescents with BPBP, and which has critically evaluated and synthesized their psychometric properties. We found that there are many outcome measures (n=33) that have been used in the published literature to evaluate children with BPBP; however, few have evidence of reliability and validity for children with BPBP. This means that clinicians and researchers are using outcome measures with unknown psychometric properties to make management decisions. The number of outcome measures used to evaluate the effectiveness of conservative interventions has grown substantially over the last decade. Compared with the outcome measures identified in a previous systematic review by Bialocerkowski et al. in 2005, the number of body function measures has grown by 150% (from nine outcome measures in 2005, to 22 identified in this systematic review). Similarly, the number of activity/participation outcome measures which have been used in the published literature reporting the effects of conservative management of BPBP has increased tenfold.
This systematic review focused on outcome measures that have been used to evaluate the effectiveness of conservative interventions for children and adolescents with BPBP, as conservative management is usually commenced before surgical opinion is gained. Moreover, conservative interventions which focus on facilitation of active upper limb movement and use of the upper limb during age-appropriate activities, as well as the prevention of joint contractures, tend to continue from childhood into adolescence, irrespective of whether surgery has been conducted. Outcome measures used to evaluate the effect of conservative interventions are frequently used to evaluate the effect of surgery and postoperative rehabilitation. Ho et al. found that six body function outcome measures (active movement/passive movement, Active Movement Scale, British Medical Research Council Muscle Grading System, Gilbert Scale, Mallet Scale, and Raimondi Scale) and four activity/participation outcome measures (activities of daily living, bimanual activities, PEDI, PODCI) are frequently used to evaluate the effects of surgical outcomes. All the outcome measures reported by Ho et al. were identified by our search strategy. The search strategy used in this systematic review was likely to be more comprehensive as it employed a greater number of databases and search terms despite identifying only evidence published in the English language. Twice as many outcome measures were identified using this approach as by Ho et al. Therefore, the results of this systematic review are likely to be generalizable to children and adolescents with BPBP irrespective of the type of intervention received.
The high volume of outcome measures identified in this systematic review may have occurred as a result of the varying opinions of surgeons and therapists regarding the most appropriate methods to quantify upper limb function in children and adolescents with BPBP.[6, 8, 9] The outcome measures identified focused on the quantification of body function variables (n=22). However, outcome measures which focus on the quantification of activity/participation have more recently been developed for children with BPBP (e.g. the Assisting Hand Assessment, the Brachial Plexus Outcome Measure, the Children's Hand-use Experience Questionnaire, and the PEDI). This may reflect the growing opinion among clinicians and researchers that assessment and management should focus on all aspects of health and disability and the subsequent move away from using outcome measures which focus solely on body function.
A major finding of this systematic review is that there is a paucity of research evidence on the psychometric properties of outcome measures for children and adolescents with BPBP, with only 12 psychometric studies published in this area. Psychometric evidence was found for only nine of the 33 outcome measures identified by our search strategy. Clinicians and researchers, therefore, are using outcome measures with unknown psychometric properties to determine the effects of interventions and to make management decisions. This is highlighted in Table 1, in which 32 of the 33 studies identified in phase 1 of this systematic review used at least one outcome measure with unknown psychometric properties. Although Leblebicioglu et al. used the Mallet Scale to evaluate the outcome of 105 children with BPBP, this study was undertaken at a time when the reliability of the Mallet Scale was not known.
No outcome instrument identified in this review had psychometric evidence which covered the breadth and depth required for clinical and research purposes (Table SIII, online supporting information). Terwee et al. consider that evidence of the psychometric properties of an outcome measure should cover the domains of content validity, internal consistency, construct validity, criterion validity (if a criterion standard exists), agreement, reliability, responsiveness, ceiling and floor effects, and interpretability. None of the outcome measures was evaluated for responsiveness: the ability of an outcome measure to detect clinically meaningful changes over time. This means that clinicians and researchers have a paucity of evidence on which to make sound measurement decisions.
This systematic review found that outcome measures which evaluate body function were numerous and frequently used in the published literature. Many of these outcome measures are commonly used in clinical practice, despite limited evidence to support this. Moreover, the present psychometric evidence has methodological short comings, which makes it difficult for clinicians and researchers to use these outcome measures in a standardized manner in clinical practice and research. According to the ICF, the measurement of body function is an important aspect in evaluating health and disability. Therefore, future research should focus on establishing the validity, reliability, and responsiveness of outcome measures which quantify body function variables for children and adolescents with BPBP, taking into consideration the methodological short comings (as identified in this systematic review) and the depth of psychometric evidence required for clinical and research purposes.
We identified relatively few outcome measures which quantified activity/participation variables, but a greater proportion of these outcome measures had some psychometric evidence. Many of these outcome measures had limited breadth of evidence that was specific to children and adolescents with BPBP. It must be noted that many of these outcome measures have been developed during the last 5 years, for example the Brachial Plexus Outcome Measure, Arm and Hand Function, and the Children's Hand-use Experience Questionnaire. Consequently, they have undergone only preliminary psychometric evaluation, but with promising results. Future evidence, therefore, may be produced that supports the use of these outcome measures, which quantify activities in children and adolescents with BPBP.
It is acknowledged that published evidence of the psychometric properties of some outcome measures which quantify activity/participation has been produced using samples of children with upper limb diagnoses other than BPBP (e.g. the Melbourne Assessment of Unilateral Upper limb Function, the PEDI, and the PODCI). Validity, reliability, and responsiveness are not inherent characteristics of an outcome measure but exist in the context in which the outcome measure was evaluated. In other words, psychometric properties of an outcome measure should be population specific. They are applicable only when used in the same specified manner as the administration method used and by the same assessor on a specific group of participants, and this, along with the skills and experience of the assessor and the characteristics of the participants, potentially impacts on the measurement process. Hence, psychometric evaluation of outcome measures should be targeted to children and adolescents with specific diagnoses (such as BPBP), to provide clinicians and researchers with evidence that the outcome measure will behave as intended. Moreover, psychometric properties do not encompass the quality of the performance of the outcome measure, that is, the outcome measure's clinical utility. Clinical utility is often described as the administration, scoring, interpretation, and feasibility of using an outcome measure. These properties should also be considered when selecting an outcome measure for use in clinical and research settings.
Based on the evidence identified in this systematic review, the outcome measures which have the most robust psychometric properties include (1) the Active Movement Scale,[53, 59] which evaluates body function variables in infants and young children; (2) the Assisting Hand Assessment,[60-62] which evaluates activity/participation variables in children from the age of 18 months; (3) the PEDI, which evaluates activity/participation in children aged 6 months to 7 years; and (4) the PODCI,[37, 57] which evaluates quality of life in children aged up to 10 years.
These outcome measures should be considered for use in clinical practice. However, intended users must ensure that they possess adequate training, as outlined in manuals, and administer the outcome measure in a standardized manner between children. Further research should be undertaken on the above mentioned four outcome measures, specifically to determine the construct validity of the Active Movement Scale and the reliability and content validity of the PEDI and the PODCI. Moreover, research evidence is required on the ability of these four outcome measures to detect change over time (Table SIII). This evidence will provide clinicians and researchers with confidence in their measurements. Our recommendations are likely to change in the future as a result of publication of further psychometric evidence associated with the outcome measures identified in this review, or the development and subsequent evaluation of new outcome measures for children and adolescents with BPBP. It is essential that further psychometric evaluation should be undertaken on these ‘recommended’ outcome measures as the volume, breadth, and quality of these outcome measures is limited (Table SIII).