The Drooling Impact Scale: a measure of the impact of drooling in children with developmental disabilities

Authors


  • ACKNOWLEDGEMENTS
    The authors acknowledge the support of the Marian & E H Flack Trust and the Royal Children’s Hospital Waverley Auxiliary for providing funding for this project, Christine Westbury for help with data collection, Associate Professor Susan Donath for statistical advice, and the children and their families for their willing participation.

Sue Reid at Developmental Medicine, Royal Children’s Hospital, Parkville, Vic 3052, Australia. E-mail: sue.reid@mcri.edu.au

Abstract

Aim  To describe the development and clinimetric properties of a new scale to evaluate changes in the impact of drooling in children with developmental disabilities.

Method  After examining the properties of potential items, 10 items were retained for inclusion in the final Drooling Impact Scale. The clinimetric properties of the scale were evaluated using data from two convenience samples of children attending a saliva-control clinic: a stable group (n=31, 22 males, nine females; mean age 10y 7mo, SD 4y 5mo, range 3y 6mo–18y 3mo; cerebral palsy [CP] n=17, intellectual disability n=10; non-ambulatory n=13, nonverbal n=12) and an intervention group (n=49, 29 males, 20 females; mean age 11y, SD 3y 6mo, range 3y 4mo–16y 10mo; CP n=31, intellectual disability n=15; non-ambulatory n=27, nonverbal n=28). To assess validity, changes in scores on the Drooling Impact Scale over time were compared with a carer’s global rating of change using Pearson’s correlations and t-tests. A concordance correlation coefficient was used to compute the level of agreement between assessments 1 month apart in stable children. Effect size, standardized response mean, Guyatt responsiveness statistic, and an unpaired t-test were used to estimate responsiveness.

Results  The correlation between the global rating and change in Drooling Impact Scale scores was 0.69 (p<0.001). The concordance correlation coefficient was 0.85. An effect size of 1.8, standardized response mean of 1.5, Guyatt responsiveness statistic of 1.4, and mean group difference of 23.5 (95% confidence interval 17.4–29.6) were obtained.

Interpretation  The Drooling Impact Scale is a valid and reliable subjective measure that is responsive to change.

The Saliva Control Clinic at the Royal Children’s Hospital in Melbourne, Australia, was set up in 1988 to manage problems with drooling in children with a variety of neurological disorders, including cerebral palsy (CP) and intellectual disability.1 A multidisciplinary team, comprising a speech pathologist, paediatric dentist, paediatrician, plastic surgeon, and nurse coordinator, provides a range of carefully targeted interventions. These include oral sensorimotor therapies, intraoral appliances, behavioural intervention, anticholinergic medication, botulinum toxin A (BoNT-A), and saliva-control surgery. Assessment of the effectiveness of these interventions is an important part of the work of the clinic, as our aim is to inform clinical practice and add to the body of knowledge available to clinicians.

A major problem for research into interventions to reduce drooling is that there is no valid and reliable measurement tool of saliva control.2 Historically, drooling has been measured with collection units such as urine or suction bags,3,4 or by using radioactive isotopes.5 However, these methods are complicated and invasive, and leakage can be a problem.6 A method involving the weighing of bibs has also been used;7 although there is some evidence to support the validity of this approach, the process is prone to measurement error from evaporation, other liquids being spilled, or saliva missing the bib. Another approach is to measure the frequency of drooling using the Drooling Quotient, whereby a count is made of each occasion drooling occurs.8 However, long stringy drools have a greater volume than small drips, and this measure does not reflect these differences in quantity. Another disadvantage of this method is that it requires long periods of observation to obtain an accurate, representative score, as drooling varies from day to day and hour to hour.9 Absorbent cotton dental rolls inserted into the oral cavity have also been used as an objective measure of drooling. However, a variety of factors contribute to excessive drooling, and merely reducing the amount of saliva produced may not always result in clinically significant reduction in drooling, particularly where oropharyngeal dysfunction is an important contributory factor.10

An alternative option is the use of a subjective scale. In previous studies the Drooling Rating Scale has been used.11 This has been useful for discriminating between children in terms of the severity and frequency of drooling, but it has not proved to be responsive to clinically significant change after interventions. Nevertheless, a subjective assessment of the impact of drooling is still thought to be a useful and appropriate way to measure changes in drooling, on the basis that the impact of drooling on families, carers, and the individuals themselves is of prime importance when assessing satisfaction with the outcome of saliva-control interventions.12

In formulating the requirements for a new scale, we decided to design a questionnaire that could either be self-administered by carers or allow an interviewer to record the carer’s answer. Given the subjective nature of the measure, the same carer would be required to rate the drooling on successive administrations. To take into account variation in drooling over time, it was deemed most appropriate for the items to relate to drooling over the entire previous week. This would also ensure that the measure could be repeated at fairly frequent intervals for both clinical practice and research, while still taking day-to-day variability in drooling into account.

Potential items for the new scale were devised using information gained from parents and carers during clinical consultations in the saliva-control clinic and using the expert opinion of the speech pathologists. Much of the previous research had been focused on questions about the frequency and severity of drooling11 and the number of bib or clothing changes needed each day,12 so these questions were also incorporated into the item pool. End-anchored semantic differential scales with 10 steps, numbered 1 to 10, were chosen for each response. From the total pool of possible items, many were excluded because they were not deemed amenable to change within 1 week after interventions to reduce drooling. The remaining 12 items were tested for their comprehensibility and ambiguity by parents attending the saliva-control clinic, and repeated measures were obtained from families with children who participated in a randomized, controlled trial of BoNT-A injections into the parotid and submandibular glands.13 Endorsement frequencies for each item were examined, and each item was tested for its internal consistency, test–retest reliability, and responsiveness to change. Two items that did not score well for either responsiveness or test–retest reliability were deleted from the original pool, and the remaining 10 items were retained to form the Drooling Impact Scale (Fig. 1). Each of the 10 items was given equal weighting in the total score. The aim of the present study was to assess the clinimetric properties of the new scale.

Figure 1.

 The Drooling Impact Scale.

Method

Setting

This study was undertaken at the Royal Children’s Hospital in Melbourne, Australia, and all participants attended the hospital’s saliva-control clinic. Ethical approval for the project was obtained from the Human Research Ethics Committee at the Royal Children’s Hospital. Informed consent was obtained from families who participated in the intervention trial; in other circumstances the collection of data was part of usual clinical practice.

Selection of participants

Data from two convenience samples were used. The first was a stable group comprising children whose drooling was expected to remain relatively stable over 1 month. These children were mostly from the comparison group of the clinical trial (with equal drooling severity to the intervention group) plus other children attending the saliva-control clinic who were either already on medication or considering their treatment options. The second group, an intervention group, comprised children who had an intervention to reduce their drooling. The characteristics of the study groups are shown in Table I.

Table I.   Characteristics of the study groups
 Intervention n=49Stable n=31
  1. NA, not applicable.

Age, y:mo, mean (SD)11:0 (3:6)10:7 (4:4)
Sex, n
 Male2922
 Female209
Main diagnosis, n
 Cerebral palsy3117
 Intellectual disability1510
 Other34
Mobility, n
 Walks independently99
 Walks with aids139
 Non-ambulant2713
Communication, n
 No speech problems45
 Some speech problems1714
 Little or no speech2812
Epilepsy, n208
Intellectual impairment, n
 None87
 Mild114
 Moderate159
 Severe1511
Intervention, n
 Botulinum toxin A injection44NA
 Anticholinergic medication5NA
Baseline Drooling Impact Scale score, mean (SD)60.6 (13.6)57.1 (12.8)

Data collection

The Drooling Impact Scale was administered twice, 1 month apart, to carers of children in both groups, by the nurse coordinator of the saliva-control clinic, either face to face or over the telephone. The carers needed sufficient English-language skills to understand and complete the questionnaire, and the same carer was required to fill in the questionnaire on both occasions.

For the intervention group, the scale was administered immediately before and 1 month after the initiation of a saliva-control intervention (either injection of BoNT-A into the salivary glands or commencement of anticholinergic medication). To assist the evaluation of the clinimetric properties of the scale, carers were also asked to rate, on a 10-point scale, the degree to which the drooling had worsened or decreased 1 month after initiation of the intervention. This became the carer’s global rating of change. A score of 9 or 10 was categorized as very good to excellent reduction, 7 or 8 as good reduction, 5 or 6 as little or no reduction, and below 5 as deterioration (Table II).

Table II.   Association between carer’s global rating of change and mean Drooling Impact Scale scores
Carer’s global rating of changenChange in Drooling Impact Scale score, mean (SD)
9–10Very good to excellent improvement938.0 (13.1)
7–8Good improvement2128.0 (12.3)
5–6Little or no improvement1210.6 (12.8)
4Slight deterioration2−1.0 (18.4)

Statistical analyses were performed using Stata 10 (Stata Corp, College Station, TX, USA).

Validity

To demonstrate content validity, a reduction in the impact of drooling measured by the Drooling Impact Scale needed to correspond to a noticeable and relevant reduction according to the carers. If the scale really measures a change in the impact of drooling, we predicted that there would be good correlation (>0.5) between the carer’s global rating of change in drooling and the change in total Drooling Impact Scale scores.

The construct underpinning the Drooling Impact Scale is that a perceived decrease in drooling, for example after a saliva-control intervention, will lessen the impact on carers, and this change will be reflected in lower scores on our scale. This construct was tested using mean changes in scores on the Drooling Impact Scale in the intervention group and relating them to the carer’s global rating of change in drooling. We hypothesized that the greater the reduction in drooling according to the carer’s assessment, the greater would be the change in Drooling Impact Scale scores.

Test–retest reliability

The Drooling Impact Scale was completed by the same rater on two occasions, 1 month apart, to assess the impact of drooling in children who were expected to be stable. The paired data were excluded if there was a change in rater, medication, or health status between the two assessments. A concordance correlation coefficient was used to compute the level of agreement between the two assessments.14 The coefficient is a measure of precision around whether the repeated Drooling Impact Scale scores significantly deviate from the line of perfect concordance (i.e. 45°) with the baseline scores. It also includes a bias-correction factor that measures the accuracy of the concordance. In addition, the Bland–Altman limits-of-agreement method for assessing test–retest reliability was used as a complementary approach, whereby 95% of the differences between the two measurements should be less than 2 standard deviations (SDs) from the mean difference, with the proviso that the mean of the differences is close to zero.15 The statistical measure is performed by obtaining the SD of the differences, a coefficient of repeatability (twice the SD), and the 95% limits of agreement, that is, 2SDs below and above the mean.

Responsiveness

An estimate of the responsiveness or sensitivity to change of the measure was obtained using four statistical methods.

An effect size was calculated by dividing the mean change in scores by the SD of baseline scores in children who had received an intervention and were expected to change. Effect sizes were interpreted according to criteria set by Cohen.16 An effect size of 0.2 to 0.49 was interpreted as small, 0.50 to 0.79 as moderate, and 0.80 or greater as large.

A standardized response mean was calculated, whereby the mean change in scores was divided by the SD of the change scores. The interpretation of standardized response means was the same as for effect sizes.

The Guyatt responsiveness statistic was determined by dividing the minimal clinically important difference by the variability in Drooling Impact Scale scores in stable children (represented by the square root of twice the mean square error).17 The minimal clinically important difference was estimated by relating the total Drooling Impact Scale score to the carer’s global rating of change 1 month after the intervention.

An unpaired t-test was used to compare the mean change in Drooling Impact Scale scores in children who had a saliva-control intervention with the mean change in the stable group.

Results

Validity

The assessment of the correlation between the carer’s global rating of change in drooling and the change in total Drooling Impact Scale scores yielded a correlation coefficient of 0.69 (p<0.001). When the mean change in scores from the intervention group was tabulated against the carer’s global rating of change in 44 children, the results supported the construct that children’s scores on the Drooling Impact Scale are related to the degree of improvement perceived by those caring for them (Table II).

Test–retest reliability

The concordance correlation coefficient was 0.85 (standard error 0.05), an indication of a high level of precision around whether the observed data significantly deviated from the line of perfect concordance. The bias correction factor was 0.99. Using the Bland–Altman approach, the average difference between the Drooling Impact Scale scores at the two time points was 0 (SD7.1), the coefficient of repeatability was 14.2 (twice the SD), and the 95% limits of agreement were −13.9 and 13.9 (2SD below and above the mean). In all but two children, the difference between the two pairs of measurements was within 2SD of the mean, showing that the Drooling Impact Scale has good test–retest reliability in children who are stable (Fig. 2).

Figure 2.

 Bland–Altman 95% limits of agreement.

Responsiveness

The mean change in scores on the Drooling Impact Scale was 23.5. The SD of the baseline scores was 13.3, resulting in a large effect size of 1.8. The SD of the change scores was 16, resulting in a standardized response mean of 1.47. This also indicated that large effect sizes can be obtained using this measure. Five carers assessed their child’s drooling as having reduced little after treatment, corresponding to a score of 6 on the global rating of change; in this group, the mean change was 13.6 points (95% confidence interval [CI] −28.6 to 1.4). Using this figure as our estimate of the minimal clinically important difference, the Guyatt responsiveness statistic was calculated to be 1.4. The mean change in the stable group was 0; the difference of 23.5 (95% CI 17.4–29.6) between the groups was highly significant (p<0.001).

Discussion

The Drooling Impact Scale has been devised to evaluate longitudinal changes in the impact of drooling in children with neurological disorders.18 It was specifically designed to quantify the short- to medium-term treatment benefits of saliva-control interventions. The items included in the final Drooling Impact Scale were chosen for their perceived ability to change after an intervention and were scored on a 10-point scale to optimize their responsiveness. Items that did not respond to change or show adequate test–retest reliability in stable children were omitted. Because the degree of drooling is known to be inconsistent over time, variability was minimized by relating each item to drooling over an entire week. Repeated administrations of the Drooling Impact Scale to the same carer have shown the scale to be stable over time in the absence of any intervention or illness likely to have an impact on the frequency or severity of drooling.

There is some support in the literature for the use of a subjective questionnaire to evaluate changes in drooling. After analysing the available literature on measurement of drooling, members of a 1990 consortium on drooling concluded that not only were objective methods inadequate, but objective quantification was not really necessary for clinical management or research, because the ultimate test of treatment effectiveness is whether it makes caregivers’ lives easier and whether the child’s quality of life is improved.12 Despite this conclusion, there has been limited research on the impact of drooling in children. One group from the Netherlands was able to demonstrate that enormous demands are placed on carers in terms of increased workload, such as the need for frequent reminders to swallow, wiping of excess saliva from mouth, chin, and other areas, and changing and laundering of bibs and clothing.19 The same group also showed that drooling has a negative impact on the child’s social integration and self-esteem, as other people may keep their distance from the child. The extent to which this has an impact on the child depends on their social awareness, a function of age and cognitive ability.19

Evidence from the literature supports the inclusion of some of the items used in the Drooling Impact Scale. A rating of the frequency and severity of drooling has been used in the past and has been shown to be somewhat responsive to changes after interventions, even though it was not developed specifically as an evaluative measure.20 The demands of caring for children who drool excessively have been reduced after interventions, particularly the frequency of wiping of the children’s mouths and chins, the number of changes of bibs or shawls per day, and the damage to equipment.21 In addition, a reduction in drooling has been shown to result in improved social contact with peers.22 Even though many children did not have sufficient social awareness for drooling to affect self-esteem, the Dutch study showed that carers’ perception of the level of child satisfaction with their physical appearance and with life in general can improve after intervention to reduce drooling.22 Many of these changes, however, take effect gradually and will not necessarily change over the short or medium term. Measuring effects on social interaction and self-esteem may require a different scale specifically designed for longer-term outcomes.

In the absence of a practical, objective way of measuring the impact of drooling, we used a global rating scale to show a strong relationship between the carer’s perception of reduction in their child’s drooling and the longitudinal changes in Drooling Impact Scale scores after a saliva-control intervention. The Drooling Impact Scale has also been shown to have excellent responsiveness. Scores improved after two interventions of known efficacy (BoNT-A injections to the salivary glands and anticholinergic medication), and the data support the power of the scale to reflect large effect sizes in this population of children, predominantly with CP or intellectual disability. However, the carers in this study were not blinded to whether or not their child was receiving an intervention. Thus, their expectations of the magnitude of the effect may have influenced how they completed the Drooling Impact Scale.

Although the clinimetric properties of the Drooling Impact Scale have been promising in the context of the types of children attending this particular saliva-control clinic, (predominantly children with CP or intellectual disability), it still needs to be tested in different groups and with other interventions. Moreover, the ability of the scale to discriminate between individuals or groups has not been investigated as part of these studies. Positive and negative impacts of saliva-control interventions on other areas such as eating and drinking skills and speech are not included in the scale. We specifically wanted to focus on the impact of drooling, and other information may need to be collected separately to evaluate the overall effectiveness of an intervention, rather than just its efficacy in reducing drooling. Data on other factors that might affect drooling, such as illness or concomitant medications, should also be collected.

Conclusion

These analyses support the usefulness of the Drooling Impact Scale as an evaluative tool to assess the effect of saliva-control interventions on drooling in children with developmental disabilities. The scale has been shown to behave as expected in validity studies, to have good test–retest reliability in stable children, and to be responsive to change in children who have undergone saliva-control interventions.

Ancillary