Classification systems of communication for use in epidemiological surveillance of children with cerebral palsy

Cerebral palsy (CP) is a permanent but not unchanging disorder of movement, posture, and motor function,[1] ‘often accompanied by disturbances of sensation, cognition, communication, perception, and/or behaviour, and/or by a seizure disorder’.[2] Surveillance programmes monitor trends in CP prevalence and its determinants, and record the functional severity of associated impairments.[3]

Surveillance registers of CP share similar inclusion and exclusion criteria but vary in their methods of data collection.[4] Classification systems of the severity of impairment of gross motor function (Gross Motor Function Classification System [GMFCS])[5] and fine manual function (Bimanual Fine Motor Function Classification System [BFMF]; Manual Ability Classification System [MACS])[6, 7] are now frequently used in surveillance registers,[8, 9] enabling the severity of impairments to be reliably compared across time and regions.[10-12] Most classification systems have been validated to be applied from the age of 4 years, because this age allows both a reliable confirmation of the condition and assessment of the child's functional ability.[1, 13] However, it is recognized that the development of the functions classified is rarely complete at this age.

The 2007 International Classification of Functioning, Disability and Health for Children and Youth (2007) (ICF-CY), characterizes Communication in the ‘Activities and Participation’ domain with subcodes d310 to d329 for ‘Receiving Communication’ and d330 to d349 for ‘Producing Communication’.[14] Classifying both components of communication is important and complex, as the motor disorders of CP can impair the production of speech and gesture and the disorders of cognition and sensation may affect the development of both expressive and receptive communication. Systems that describe speech[15, 16] and communication by multiple modes[17] have been developed but their use for surveillance registers has not been evaluated.

This study was part of the Surveillance of Cerebral Palsy in Europe (SCPE) network project, which aims to promote best practice in describing children with CP and to document variations in access to healthcare and in health outcomes (http://www.scpenetwork.eu). The study aimed to identify classification systems suitable to describe the communication of children with CP for epidemiological surveillance, either as a full activity or some of its components. To be considered suitable, the classification system should have acceptable content validity, reliability, and applicability using different sources of information, such as direct observation of the child and retrieving information from case records.

Method

A systematic review of the literature was undertaken to identify classification systems of communication in children with CP. Medline, CINAHL, EMBASE, PsychInfo, Web of Knowledge, Scopus, First Search, ERIC, Linguistics and Language Behaviour Abstracts, and DARE, were searched up until June 2010 using the terms ‘communication’, ‘speech production measurement’, ‘speech articulation tests’, ‘speech disorders’, ‘articulation disorders’, ‘communication disorders’, or ‘dysarthria’. Key journals were also hand-searched from their inception or from 1980 until the end of March 2010.

Classification systems identified in the search were assessed by 12 experts from eight countries (speech and language therapists [SLTs], occupational therapists, neurodevelopmental paediatricians, paediatric neurologists, and epidemiologists) using a pre-established set of construct, validation, and effectiveness parameters (Table SI, online supporting information).

The selected classification systems were translated into Danish, Latvian, Lithuanian, Norwegian, European Portuguese, European Spanish, and Swedish, following international guidelines that included two independent translators, discussions on phrasing and terminology by two focus groups (parents and health professionals), and back translation, to ensure retention of original concepts and meaning.[18] Translations were made of the classification systems, the instructions, and any examples provided. Back translations were checked and approved by the first author of the classification system.

To assess the comparative effectiveness of the classification systems, parents (or caregivers), SLTs, and other healthcare professionals applied them to classify the communication of children with CP and rated their experience of applying the systems.

An international, multicentre sample of children with CP aged 4 to 13 years was purposively sampled by local clinicians from clinical caseloads of seven registers of members of SCPE: North of England, Portugal (Lisbon and Oporto), Latvia, Lithuania, Norway, Spain (Madrid), and Western Sweden. Target sample size was 30 children per centre, covering all CP types (bilateral and unilateral spastic, dyskinetic, and ataxic), gross motor function (GMFCS levels I–V), and cognition (IQ≥70; 50–69; <50).

The communication of each child was rated by four types of raters: (1) parents/carers applying the systems from their knowledge of their child; (2) SLTs; (3) other healthcare professionals by direct observation of the child; and (4) another healthcare professional retrieving information from case notes. These notes were usually those held by the paediatrician but such notes usually include letters from SLTs describing communication. The communication of Norwegian children was rated by special educators with expertise in language and communication rather than SLTs but, for ease of reporting, we refer to them as SLTs. Video-recording facilities were not available in every centre or clinic and are not used in routine data collection for the registers involved in SCPE, so it was not used in this assessment.

Raters applied the classification systems in their first language. No training on the classification systems was provided. At least 4 weeks later, children's communication was rated again by rater types 2 to 4 specified in the paragraph above. Parent reassessment was not required. Each rater classified children's communication blind to other raters' assessments and to their original rating.

To investigate the construct validity of the systems, respondents were asked to rate how well each system described for each child ‘full activity of communication (both receiver and producer)’, ‘ability to perform as an emitter’, and ‘ability to produce speech’, using Likert scales (1=very well; 5=very badly).

For each classification system, raters were also asked to rate ‘How easy did you find the application of this classification system in this particular child?’ using a Likert scale (1=very easy; 5=very difficult).

Agreement was estimated with the complete agreement rate (CAR) between rater types, with 95% confidence intervals (95% CI). Inter- and intrarater reliability were estimated with Cohen's adjusted Kappa (standard error). Kappa was interpreted as suggested by Landis and Koch.[19]

Ratings of ease of use and understanding of construct were compared between the three classification systems and between the four groups of raters using the Friedman Test. As every child was not assessed by every group of raters, missing values were managed variablewise for the analysis. Analysis used spss 15.0 (SPSS Inc., Chicago, IL, USA) and OpenEpi 2.3.1 (www.OpenEpi.com; Emory University, Rollins School of Public Health, Atlanta, GA, USA).

Parents gave written consent to participate and for their child's communication to be rated for the purposes of the study. Ethics permission was obtained by each of the participating centres.

Results

From the systematic review of the literature, eight classification systems were identified (Table SII, online supporting information) and five were selected by the expert group for further analysis based on their scope, validation assessment, and actual use. In order to improve discriminative power and ease of application, we proposed to the developers of the Communication Function Classification System (CFCS)[17] that the three sublevels of level 4 (which differentiate difficulties in sending, receiving, or both), should also be applied to level 2. This proposal was agreed and the revised instrument, which lacks specific validation, was called Communication Function Classification System – SCPE Version (CFCS-SCPE). The developers of the Verbal Expression Scale (VES) (abstract of poster presented at the 2009 European Society for Paediatric Research Conference, MG Andrada, personal communication), the Speech Production Rating Scale (SPRS),[15] and the Norwegian classification system[10] were invited to merge their three scales into a single, consensual classification system, the Viking Speech Scale (VSS).[16] Table 1 summarizes the description of the three classification systems finally chosen by the expert group for comparative assessment: the Functional Communication Classification System (FCCS), the CFCS-SCPE, and the VSS. The full scales are provided in Tables SIII, SIV, and SV (online supporting information).

The sample included 155 children (94 males, 61 females) aged 4 to 13 years (mean age 6y, SD 1.1). Most children had spastic CP (n=118 [76.1%]; bilateral n=78, unilateral n=40); 30 (19.4%) had dyskinetic type; and 7 (4.6%) had ataxic CP. The results of their functional assessment are shown in Table 2. All children used multiple modes of communication. The most frequently recorded modes were: speech (n=107 [69.0%]); vocalization (n=64 [41.3%]); and gesture (n=40 [25.8%]). Many children used augmentative and alternative communication (AAC) to supplement their natural forms of communication. Twenty-four used manual sign (15.5%); 33 (21.3%) used low-technology AAC book; and 18 (11.6%) used high-technology AAC (electronic devices, usually with speech output).

Overall, 134 children were rated by their parents (106 [79.1%] mothers; 20 [14.9%] fathers; 2 [1.5%] other relatives; and 6 [4.5%] other carers). The communication of 143 children was rated by SLTs; 146 by other healthcare professionals from direct observation (26 [17.8%] physiotherapists; 63 [43.2%] paediatricians, and 57 [39.0%] other healthcare professionals); and 151 were rated using case notes (1 [0.7%] physiotherapist; 1 [0.7%] nurse; 127 [84.7%] paediatricians; and 22 [14.0%] other health professionals). Of those rating children using case notes, 32 (20.6%) stated that they had previous knowledge of the child. The tables with the distribution of the rates given by each group using the three classification systems are provided as in Tables SVI–SVIII (online supporting information).

The agreement and reliability analysis are presented in Table 3. For statistical purposes, the performance of the CFCS-SCPE was assessed for both the full version with nine categories (five levels, two of them with three sublevels) and considering only its five main levels, to allow comparability with the classification systems with only four (VSS) or five levels (FCCS).

The interrater agreement was moderate for the FCCS and fair to moderate for the CFCS-SCPE (also considering the five main levels of CFCS-SCPE only). The interrater agreement for the VSS was moderate to good; the agreement between SLTs rating by direct observation of the child, healthcare professionals rating based on case notes, and the ratings given by parents or caregivers was virtually the same.

The intrarater agreement (test–retest reliability) was assessed in a smaller sample of children (69–104 depending on rater group). It was moderate to very good for the FCCS, good to very good for the VSS and the CFCS-SCPE (moderate to very good considering only its five main levels).

The agreement of score between the CFCS-SCPE considering its five main levels and FCCS has a wide variation (Kappa=0.53–0.85), the highest agreement being achieved for scores given by healthcare professionals rating children through access to case notes (data not shown). The correlations between the scores given by the CFCS-SCPE considering its five main levels and the VSS are good (Spearman's r=0.795–0.89) and very good between the FCCS and the VSS (Spearman's r=0.85–0.92), the highest correlation being achieved for scores given by healthcare professionals rating children through access to case notes.

The comparative assessment of the construct validity of the classification systems is presented in Table 4. The CFCS-SCPE achieved the highest rating for classifying communication comprehensively by every group of raters: it did ‘well’ or ‘very well’ in a majority of the reports. The FCCS achieved the highest rating for classifying ‘communication as an emitter’ by caregivers and by health professionals (other than SLTs) using it either by direct assessment or through clinical notes, but not by SLTs. Most raters considered that the VSS described the child's speech well or very well: 66.4% of parents, 74.1% of SLTs, and 77.3% of healthcare professionals rating through direct observation of the child, and 70.9% of healthcare professionals rating children through access to case notes. The VSS was seldom judged to assess well either the child's communication (both receiver and producer) or the child's ability to produce communication (as an emitter).

The VSS was considered the easiest classification system to apply by every group of raters, including those using case notes. Every group of raters except the SLTs considered the FCCS easier to apply than the CFCS-SCPE. The percentage of raters judging the classification system as difficult or very difficult to apply was small in every rater group, but smaller (Friedman Test, p<0.001) for the VSS (0–9.3%) than for the FCCS (4.5–15.2%) and the CFCS-SCPE (0–21.8%). The highest percentage of raters that considered the classification systems difficult or very difficult to apply was found among healthcare professionals rating children through access to case notes (VSS 9.3%, FCCS 15.2%, CFCS-SCPE 21.8%) (Friedman Test, p<0.001) and the lowest among healthcare professionals rating through direct observation of the child (VSS 0%, FCCS 4.5%, CFCS-SCPE 0%) (Friedman Test, p<0.001).

Discussion

The assessment of communication in epidemiological surveillance of children with CP is important, as communication difficulties are a frequent cause of activity limitation and affect children, their families, and society.[20] This project aimed to identify systems to classify communication which were easy to use, valid, and reliable.

Three classification systems were compared in the field. The CFCS was developed to classify the full activity of communication in five levels, with sublevels for some differentiation between producing and receiving communication; it was validated for children with CP aged 2 to 18 years.[17] The FCCS was developed to classify the activity of the production of communication in five levels; it was validated for children with CP aged 4 to 6 years. The VSS was developed to classify the intelligibility of speech in four levels; it does not assess the receptive activity of communication and does not consider other forms of production of communication; it was validated for children with CP aged 4 to 13 years.[16]

Barely half of the respondents in each rater group considered the CFCS-SCPE a good descriptor of the ability to communicate either as both a producer and a receiver (45.3–58.7%) or as a producer only (54.5–58.1%). The FCCS was considered by every group of raters as a good or very good descriptor of the ability to communicate as a producer (58.8–79.0%). Thus, many raters, even SLTs, thought that neither the FCCS nor the CFCS-SCPE described adequately the full communication abilities of the children they rated.

The construct of the VSS was correctly understood by a large majority in every group of raters (66.4–77.3%). Each rater group classified this system as able to correctly describe the ability to speak intelligibly. The VSS scored consistently highly in ease of application across every rater group (76.1–85.3%). Particularly, it was the classification system that healthcare professionals, who rated children's communication from case notes, found the most easy to apply (84.2%). These data are relevant, as many registers of CP rely mostly on data retrieved from case notes. Detail in case notes will of course vary but a register seeks complete ascertainment and reproducibility, rather than great detail. It is reassuring that this study shows there was sufficient detail for case notes review to be as informative as direct observation.

A limitation of the VSS is that children with CP may be unable to produce any intelligible speech due to cognitive as well as motor impairments. The reason for unintelligibility is not clear from the scale alone. Thirty-one per cent of this sample did not use speech for communication and had to be classified as grade IV, in spite of their eventual ability to produce communication by other means.

The agreement on the classifications given by each group of raters seems to be higher for the VSS than for the CFCS-SCPE or the FCCS, but the confidence intervals of the CAR frequently overlap and it is difficult to state the significance of the difference in the values of Cohen's kappa[21] as the levels of agreement could be considered either different or similar depending on the criteria.[19, 22, 23] The criteria most frequently cited in the literature were followed.[19]

The stability of the three classification systems on test–retest was virtually the same.

Training and increased familiarity with classification systems, such as the GMFCS, improves their usefulness.[24] This may suggest that while the VSS is easy to apply, even by those not familiar with it or using case notes, FCCS and/or CFCS-SCPE may become easier to use with increasing practice.

The main strengths of this study are: (1) inclusion of both published and ‘grey literature’[25] in the review; (2) rigour provided by the research protocol; (3) balanced distribution of clinical types of CP, IQ, and communication abilities in the participating children; (4) varied professional backgrounds of raters; and (5) different sources of information used for rating children. Moreover, its multinational design provides culturally validated versions of the classification systems in eight different European languages and lays the groundwork for their use in international surveillance of CP.

The relatively small sample of children and their cultural heterogeneity are weaknesses of the study. Each centre aimed to recruit 30 children to the study, but for some centres this was not possible for reasons relating either to time or financial constraints. Also, in some centres, the professionals rating the children from case notes had previous knowledge of the children and this may have created bias.

In spite of the careful cultural adaptation during translation, the reliability assessment of the classification systems could differ between countries and languages. Our sample size did not allow this to be investigated but further research could address this.

It has to be acknowledged that the CFCS-SCPE version of CFCS has been approved by the developers of CFCS but has yet to be specifically validated. The FCCS was applied to a sample that included children older than those used by its developers for the validation. The VSS was validated using the same sample reported in this study.[16]

It is possible to classify communication in the setting of epidemiological surveillance of CP with the instruments that already exist. Their ease of use is different, reflecting the complexity of their purposes and constructs. They all are valid and reliable. Choosing between the three assessed instruments depends mainly on the sources of information and characteristics of the reporters and on the surveillance priorities of each register (balancing areas of special interest with the length of the data collection forms). Classifications of the disorders of cognition and sensation that may affect both expressive and receptive communication and of the motor disorders that can impair the production of gesture are usually already collected by registers. On the other hand, as speech is not generally described, there is a lack of information on the most frequent and important form of producing communication.

We recommend the VSS be adopted as the single measure of speech in epidemiological surveillance, supplemented with additional information on cognition, sensation, and manual ability. The FCCS or CFCS may be used in addition to the VSS to obtain a broader classification of communication.

Acknowledgements

The following SCPE centres collected data for this study: Denmark (led by Gija Rackauskaite); Latvia (led by Andra Greitane); Lithuania (led by Audrone Prasauskiene); Norway (led by Guro L Andersen); Portugal (Lisbon and Oporto, led by Maria da Graça Andrada); Spain (Madrid, led by Javier de la Cruz); Sweden (Western Sweden, led by Kate Himmelmann); and UK (North of England, led by Karen Horridge). The data analysis was performed with the assistance of the Epidemiology and Statistics Office of the Research Unit of Centro Hospitalar de Lisboa Central (Lisbon, Portugal). This study was funded by the European Union Health Programme – Grant EAHC 2008 1307 – ‘Surveillance of Cerebral Palsy in Europe: best practice in monitoring, understanding of inequality and dissemination of knowledge’. The authors have stated that they had no interests that could be perceived as posing a conflict or bias.