Evaluation of Screening in Children Referred for an ADHD Assessment

Authors


Abstract

Although guidance from the National Institute for Clinical Excellence recommends the improved identification of children with ADHD, clinical resources are limited. Amongst children (n = 119) referred over the question of ADHD, we evaluated the utility of screening (using parent and teacher questionnaires) prior to offering an ADHD assessment. The introduction of screening contributed to an increase in the proportion of assessed children receiving a clinical diagnosis of ADHD. Although screening of referred children prior to assessment can optimise the use of specialist clinical resources in the identification of children with ADHD, false positives remain common.

Key Practitioner Message:

  • There are concerns about the accurate identification of children with ADHD. Screening of referred children prior to assessment can optimise the use of limited clinical resources.
  • Amongst children referred for an assessment for the presence of ADHD, the introduction of pre-assessment screening (with parent and teacher questionnaires) contributed to an increase in the proportion of assessed children receiving a clinical diagnosis of ADHD.
  • False positives remain common because symptoms of ADHD overlap with other disorders and developmental problems.
  • Key predictors of receiving a clinical diagnosis of ADHD were parent and teacher hyperactivity scores, an account of overactivity in the referral letter and female gender.

Introduction

ADHD is a neuro-developmental disorder affecting up to 5% of school-aged children. The cardinal features involve symptoms of over-activity, inattention and impulsiveness that are severe, pervasive across settings, and impairing (American Psychiatric Association, 1994). It is associated with educational and social difficulties for affected children and has considerable impact on their families (Sayal, Taylor, & Beecham, 2003). It is also a risk factor for children's further development in terms of a range of difficulties such as other psychiatric disorders; problems involving education, employment, and relationships; and offending (Willoughby, 2003). Evidence-based treatments are available, and early identification and intervention may ameliorate the prognosis (The MTA Cooperative Group, 1999). Given all these factors, the National Institute for Clinical Excellence (NICE) (2000) expressed concern about the considerable under-diagnosis of children with ADHD. It recommended that diagnosis and initial treatment with stimulant medication should only be carried out by specialists (paediatricians or child and adolescent psychiatrists). In some areas, child and adolescent mental health services (CAMHS) have developed specialist ADHD teams; in others, specific ADHD clinics have been set up within the generic CAMHS or paediatric services.

As parents and primary care professionals become more aware of ADHD, the consequent increase in recognition and referral rates have to be balanced against available specialist resources that would enable the desired increases in identification. Furthermore, ADHD is a chronic disorder requiring long-term follow-up and ongoing liaison with other agencies especially educational services. In terms of assessment, clinical practice guidelines recommend a comprehensive approach to establish the presence of ADHD and associated comorbidity and impairment (American Academy of Pediatrics, 2000; Hill & Taylor, 2001; Taylor et al., 2004). These assessments are usually multi-disciplinary in nature and can be time-consuming. For example, components include a detailed developmental history from the parents or caregivers, observation and examination of the child, assessment of cognitive and learning ability, obtaining reports from other sources such as teachers and sometimes a school visit. The increase in demand for ADHD assessments raises the issue of how to optimise the use of limited clinical resources for this purpose.

Several recent studies have described the characteristics of which children in the community with high hyperactivity ratings get referred to clinics as well as their treatment and outcomes (Kube, Petersen, & Palmer, 2002; The MTA Cooperative Group, 1999; Sayal et al., 2003). However, relatively little is known about the intermediate stages i.e. which children referred to clinics with possible ADHD receive this diagnosis and whether screening approaches can assist in determining which referred children might benefit from an ADHD assessment. Previous work in a generic CAMHS clinic has indicated that screening of all referrals resulted in a considerable increase in the proportion of children receiving a clinical diagnosis of hyperkinetic disorder (Foreman et al., 2001). These findings raise the possibility that behaviour rating scales are not obtained or inspected systematically in routine clinical practice. In contrast to this description of under-diagnosis, a major remit of the ADHD clinic described in this paper was to actively consider the possible presence of ADHD in children referred with this diagnostic question. We report an evaluation of the impact of screening of children referred to an ADHD clinic on the proportion of children receiving a diagnosis of ADHD. We also explored the demographic and clinical predictors of which children referred over the question of ADHD receive a clinical diagnosis of ADHD.

Methods

Setting

As part of a re-organisation of a multi-disciplinary CAMHS serving an inner city Primary Care Trust (PCT) area with a population of 250,000, a service was developed to take on all new referrals of children with suspected ADHD as well as the ongoing care of children already diagnosed with ADHD. The service also assesses and manages other disorders that are comorbid with ADHD such as pervasive developmental disorders and tic disorders. Initially, the permanent staffing of the service was 0.5 whole time equivalent (w.t.e.) Consultant Child and Adolescent Psychiatrist and one w.t.e Clinical Psychologist. In order to make the best use of the service's expertise for children requiring specialist treatment and to minimise the therapeutic disruption resulting from transferring assessed children to other teams, screening of new referrals was introduced. In this context, the term ‘screening’ is used to describe the use of parent- and teacher- completed questionnaires to inform the selection process for further detailed clinical assessment–this is different to public health definitions relating to wider populations.

Design and measures

A prospective, before and after design was employed to evaluate the effect of screening on the rate of diagnosis of ADHD. The proportion of children diagnosed with ADHD following the introduction of screening was compared with the proportion diagnosed in the previous year (baseline). During the baseline 12-month period, the vast majority of referrals were accepted for assessment on the basis of the information in the referral letter. Where appropriate, after obtaining further information, referrals were occasionally not accepted following discussion at the team meeting if all the available information suggested that a diagnosis of ADHD was very unlikely.

Subsequently, over a 28 month period, all children referred with a question of ADHD were screened with two sets of parent and teacher questionnaires–the Strengths and Difficulties Questionnaire (SDQ) and the short version of the Conners’ Rating Scales (Conners, 1997; Goodman, 1997). The SDQ has five items (scored 0–10) relating to hyperactivity and inattention. In addition, it also informs about other symptoms such as conduct and emotional problems. In contrast, the Conners’ Rating Scales have a greater focus on ADHD symptoms. There are 4 scales (hyperactivity, cognitive problems/ inattention, oppositional, and an ADHD index) which provide age- and gender- standardised t scores. This combination of questionnaires provides both specific information about possible ADHD as well as an indication about the presence of other disorders. In order for a child to be accepted for a full diagnostic assessment, ADHD symptoms needed to be pervasive across home and school settings. To ensure pervasiveness, for each informant, this reflected a score of either 6 or above on the hyperactivity scale of the SDQ or a t score of 70 or above on one of the three ADHD-related scales of the Conners’ Rating Scales. This SDQ cut-off is in keeping with the SDQ diagnostic algorithm (Goodman et al., 2000) and those used in previous studies (Sayal et al., 2003) to identify children in the community with possible hyperactivity disorders. It is likely to be over-inclusive as nearly 10% of children in community samples have pervasive hyperactivity (high parent and teacher SDQ hyperactivity scores) (Sayal et al., 2006). To further minimise the likelihood of false negatives, the scores were then related at the team meeting to the information provided in the referral letter. Where appropriate, decisions about offering a full assessment were made after obtaining further information from the referrer. In practice, screening scores were only over-ruled in the opposite direction (minimising false positives) and involved children with pervasive symptoms who required an initial developmental or psychometric assessment to quantify the possible presence of a learning disorder. As there are monthly meetings involving the community paediatric services and all CAMHS teams in this PCT area, we were able to establish that none of the children who were not accepted for assessment subsequently received a clinical diagnosis of ADHD.

Following acceptance, a detailed multi-disciplinary clinical assessment was carried out and DSM-IV criteria (American Psychiatric Association, 1994) were employed to reach a possible diagnosis of ADHD. Team members were highly experienced with ADHD work and either had previous clinical experience or ongoing close clinical links with the national specialist ADHD clinic at the Maudsley Hospital.

Analyses

We compared the proportion of assessed children who received a clinical diagnosis of ADHD in each of the two time periods. These figures reflect the probability (positive predictive value (PPV)) of receiving a clinical diagnosis of ADHD amongst children accepted for a full diagnostic assessment. The available sample size enabled the detection of a large (0.8) effect size if such a difference existed (based on power of 0.8 and α of 0.05, a minimum of 26 children were required in each comparison group). Amongst all children referred across both time periods for ADHD assessments, predictors of receiving a clinical diagnosis of ADHD were also examined. Predictor measures included child age and gender, information in the referral letter (describing features of inattention, overactivity, impulsivity or raising the likelihood of ADHD) and the SDQ scores (as these were available on most children). As appropriate, bivariate analyses using two-tailed Mann-Whitney U (non-parametric), t and χ2 tests were carried out. A multivariate logistic regression analysis examined the main predictors of receiving a clinical diagnosis of ADHD.

Results

Children aged between 2 and 15 years were referred from a range of sources including general practitioners (GPs), other child health services (CAMHS, paediatricians, neurologists, and speech and language therapists), schools, and Social Services. Demographic and referrer details across the two time frames are shown in Table 1.

Table 1.   Demographic and referrer details
 Baseline n = 31After screening n = 88
  1. Note: Mean (SD) or numbers (%).

Gender (male)25 (81%)76 (86%)
Age7.65 (3.09)7.39 (2.89)
Referrer
 GP6 (19%)13 (15%)
 Paediatrics8 (26%)37 (42%)
 Other CAMHS7 (23%)19 (22%)
 School6 (19%)13 (15%)
 Other professional4 (13%)5 (6%)

During the baseline year prior to the introduction of screening, 31 referrals were received for ADHD assessments (Figure 1). After obtaining further details, four of these referrals were not accepted following discussion at the team meeting as the available information suggested that a diagnosis of ADHD was very unlikely (e.g. the problems were not pervasive). Of the 27 children accepted for assessment, two did not attend appointments. Of the 25 children assessed, four (16%; 95% confidence interval (C.I.) 5–36%) received a diagnosis of ADHD. In terms of the other children, seven (28%) received a main diagnosis of a specific learning disorder (such as a specific reading disorder), five (20%) oppositional defiant or conduct disorder, and two (8%) global learning disability. Two (8%) assessments were incomplete as families stopped attending. Five (20%) children either had no or an unclear diagnosis (mainly child behaviour problems occurring in the context of maternal depression or other family difficulties). For all of these children, a possible diagnosis of ADHD was ruled out.

Figure 1.

 Flow chart illustrating number of children at each stage

In the 28 months following the implementation of screening, 88 referrals were received for ADHD assessments. After obtaining further information, eight of these referrals were not accepted as ADHD seemed very unlikely. Screening questionnaires were returned on 76 of 80 children, of whom 56 were accepted for assessment. Exploratory analyses suggested that accepted children had higher hyperactivity scores on both parent and teacher SDQs. Mann-Whitney U tests demonstrated significant differences (p = .001) on both the teacher (median scores of 10 vs. 8) and parent SDQs (median scores of 9 vs. 7). Two of the eight items on the Conners’ Rating Scales also distinguished between the groups (both at p < .05 level): parent hyperactivity scale (median t scores of 79.5 vs. 69) and teacher ADHD index (median t scores of 75 vs. 67).

As one family moved away from the area and another family did not attend offered appointments, 54 of these children were seen. Following the introduction of screening, the proportion of assessed children diagnosed with ADHD was 48% (26/54; 95% C.I. 34–62%). In terms of the other children, nine (17%) received a main diagnosis of global learning disability, five (9%) a specific learning disorder, five (9%) oppositional defiant or conduct disorder, one (2%) autistic spectrum disorder, and one (2%) reactive attachment disorder. The other seven (13%) children did not receive a diagnosis; four of these children were aged 4 years and had pervasive hyperactivity not reaching ADHD criteria and, for one child, the mother's depression was the major factor contributing to the child's presentation. Management of non-ADHD cases (whether or not the child was assessed) involved recommendations to the referrer or onward referral to either another CAMHS team or to community paediatric services for a developmental assessment. This often involved meetings with the referrer and the close links between local services meant that referrers could contact the service about non-accepted children if they had concerns in the future.

At face value, there appears to be an increase in the PPV for a clinical diagnosis of ADHD from 16% to 48% following the introduction of screening. However, this difference is inflated by an increase in the ADHD diagnosis rate that would have occurred in the second period even in the absence of screening. Had screening not been employed, at least 33% (26/80; 95% C.I. 22–44%) of referred children are likely to have received a clinical diagnosis of ADHD in the second period (Figure 1). Although a change in the PPV for a clinical diagnosis of ADHD from 33% to 48% reflects a more cautious estimate, there is still a modest increase in the PPV through the use of screening. The role of screening in optimising the use of limited clinical resources is reflected in the reduction in the proportion of referred children offered a full assessment - from 87% (27/31) in the baseline period to 64% (56/88) in the second period.

Predictors of receiving a clinical diagnosis of ADHD

Across the two time frames, sufficient information to make a diagnosis about the presence of ADHD was available on 111 children (excluding children who did not attend assessments (two in each period) or on whom screening questionnaires were not returned (n = 4)). Certain types of professionals appeared to be more accurate referrers of children receiving a diagnosis of ADHD (GPs 44% (7/16); CAMHS 33% (8/24); paediatricians 28% (12/43); school professionals 17% (3/18); and other professionals 0% (0/10)). Table 2 shows other bivariate predictors of ADHD status. In a multivariable logistic regression analysis that included the significant predictors from bivariate analyses and controlled for age, each one-point increase on the parent (Odds Ratio (O.R.) = 2.14, 95% C.I. 1.21–3.79, p < .01) and teacher (O.R. = 2.71, 95% C.I. 1.37–5.39, p < .01) SDQ hyperactivity scores, an account of overactivity in the referral letter (O.R. = 4.14, 95% C.I. 1.18–14.50, p < .05), and female gender (O.R. = 6.37, 95% C.I. 1.37–34.68, p < .05) were all significantly associated with receiving a clinical diagnosis of ADHD. Based on a predicted classification cut-off point of .4 (the probability at which children are placed in the positive group), the model (sensitivity 71%; specificity 83%) correctly classified 79% of children.

Table 2.   Predictors of ADHD status
 ADHD n = 30Not ADHD n = 81Significance testing
  1. *Median SDQ score (range 0-10).

Gender (male)22 (73%)72 (89%)p = .07 (Fisher's exact test)
Age7.43 (1.85)7.37 (3.34)p = .92
Information in referral letter
 Inattention21 (70%)48 (58%)p = .09
 Overactivity23 (77%)36 (44%)χ2 test with continuity correction = 7.57, df = 1, p < .01
 Impulsivity9 (30%)25 (31%)p = 1.00
 ADHD20 (67%)51 (63%)p = .89
Parent-rated Hyperactivity*10 (n = 28)9 (n = 63)Mann-Whitney U = 437; p < .001
Teacher-rated Hyperactivity*109 (n = 62)Mann-Whitney U = 610; p < .01

Discussion

Following the introduction of screening, there was an increase in the proportion of assessed children who received a clinical diagnosis of ADHD. Although this increase partially reflected a naturalistic difference in the proportion of children with ADHD across the two periods, some of the increase could also be attributed to the screening procedure. This overall increase is unlikely to reflect a bias in the receipt of clinical diagnoses since all the children in this sample had the same referral question and received the same assessment (Foreman et al., 2001). Clinical diagnoses were based on information gathered from a detailed assessment and not on questionnaire scores. Similarly, a referral bias is unlikely as the characteristics of the children and referrers were similar across the two time periods.

Our evaluation highlights the potential utility of simple screening tools to improve the identification of referred children who might benefit most from a detailed ADHD assessment. Although it is important to improve the matching of clinical need with available resources, improvements in the accurate identification of children with ADHD are only a starting point. This enables children's access to appropriate interventions and effective treatments which, in turn, should help improve their outcomes. Although non-clinical factors often determine which affected children are referred to clinics (Sayal et al., 2003), it is encouraging that the severity of symptoms was associated with the receipt of a clinical diagnosis of ADHD. Treatment guidance recommends that symptom severity should play a major part in determining clinical management decisions (NICE, 2000).

In contrast to findings from a referred sample in the United States (Mulhern, Dworkin, & Bernstein, 1994) that younger age and male gender are associated with the receipt of a clinical diagnosis of ADHD, we found that, after controlling for severity, referred girls were more likely than boys to receive a clinical diagnosis of ADHD. Although the male:female ratio amongst the referrals was 5.5:1, this reduced to 2.75:1 amongst those diagnosed with ADHD. It is possible that this reflects the use of a thorough assessment for the presence of inattentive symptoms in line with DSM-IV recommendations. However, the small sample size means that this finding should be interpreted with caution. There was indirect support for the role of parental concerns (Mulhern et al., 1994) in predicting the diagnosis in terms of the key symptoms highlighted in the referral letter. Although an account of overactivity in the referral letter had good sensitivity (77%) for predicting a clinical diagnosis of ADHD, the specificity (56%) and PPV (38%; 23/59) were lower because of the high proportion of other ‘look-a-like’ disorders in referred children. In particular, specific or global learning disabilities were the most common main diagnoses (47%; 23/49) amongst assessed children not receiving a diagnosis of ADHD.

Methodological issues

This evaluation of screening was constrained by the need to make efficient use of limited clinical resources. The measures used were primarily developed for use at a community level rather than to enable a differential diagnosis of ADHD amongst referred samples. There are drawbacks to screening a heterogeneous sample in terms of age range, associated difficulties, developmental level, and type of referrer, contributing to false positives and negatives. A major limitation of this evaluation is that little information is available on possible false negative cases. Further research should assess the extent of this situation by assessing a sample of screen-negative children. This could establish the false negative criteria of particular cut-off thresholds. The moderate base rate of ADHD, even amongst children referred to a specialist clinic for a diagnostic assessment, means that any screening instrument will have a low PPV (Clark & Harrington, 1999). However, in this instance, the purpose of screening was to narrow down the pool of children for detailed clinic assessment. The application of the screening in our clinic also aimed to minimise the number of false negative children at the expense of being over-inclusive. As the false positive children had significant symptoms, it could be argued that the risks associated with receiving a clinical assessment are outweighed by the potential benefits. Regardless of the final diagnosis received, an assessment for the presence of ADHD is able to identify other needs such as other or comorbid disorders and impairment at home or at school. Even when the possible presence of ADHD was actively considered amongst these referred children, there was only a modest increase in the proportion of children receiving a clinical diagnosis of ADHD. These findings suggest that wider CAMHS services should not ignore the consideration of the possible presence of ADHD amongst children referred with other problems. The resulting increase in the number of diagnosed cases of ADHD might be fairly modest.

Although this paper reflects diagnostic practice over a three year period, the findings should be considered preliminary. It is possible that factors unrelated to the screening procedure contributed to the increase in diagnostic rate over time. The sample size was small and, as a description of a single area, the findings may only generalise partially. Nevertheless, this ADHD clinic is similar to many specialist community child health services (Tier 3 CAMHS or community paediatric services) responsible for the provision for children with ADHD. In particular, the theme of demand for services for ADHD exceeding available resources is likely to be universal.

Clinical implications

Efficient use of clinical resources for ADHD is crucial in the management of the referred population of children. Enhancing the suitability of referrals through stringent screening procedures is one method of achieving this. The findings raise the issue of whether the proportion of assessed children receiving a diagnosis of ADHD can be increased further. Even with screening, less than half of referred children in this sample received a diagnosis of ADHD. In terms of accepting referrals for a specialist ADHD assessment, the findings in Table 2 suggest the need for obtaining clear information from referrers about the presence of symptoms and associated impairment that are pervasive across settings. Differential diagnoses for ADHD include conduct or emotional disorders, tic disorders, pervasive developmental disorders, and specific or general learning disorders. Symptoms of these other disorders often overlap with those of ADHD. Referrers may misattribute these symptoms to ADHD or fail to take the child's developmental level into account. These findings also highlight the risks of misdiagnosing ADHD in the absence of a comprehensive assessment (Kube et al., 2002). This possibility may contribute to the situation of over-diagnosis and misdiagnosis of ADHD in countries such as the United States where assessments predominantly take place in primary care (Rey & Sawyer, 2003).

The introduction of screening entails increased administration in sending out the questionnaires, pursuing those that are not returned, and scoring. As completion also requires time from the parent and teacher, it can lead to both barriers and delays to care. For example, teachers already have a considerable administrative workload and may not know the child particularly well. However, the screening process may also set up expectations from the beginning that the parents and teachers will play a significant role in the assessment and intervention. There should be flexibility in approaches to collecting pre-assessment information and choice of specific screening instruments that takes account of local circumstances and referrer demand. Based on these findings, we have introduced other tools that complement the screening information. These include a more detailed teacher-completed descriptive questionnaire about behaviour and attainment. A further adjunct or alternative to screening in this form includes a brief telephone conversation with parents to obtain key items of history including information about academic performance and to overcome the obstacle of parents not completing questionnaires if they have literacy difficulties.

As health professionals appeared to be the most accurate referrers, ongoing communication with and education of other referrers are also vital. Locally, this has consisted of workshops with Special Educational Needs Co-ordinators (teachers with a specialist remit) and the regular meetings between child health professionals. In the longer term, referrers could be encouraged to make enquiries about the type and pervasiveness of symptoms or use rating scales for children whom they are considering referring. As further reconfiguration of specialist tier 3 CAMHS continues to take place across the country, training by and consultation from clinicians based in a CAMHS ADHD service becomes more crucial. This may help to minimise potential risks associated with CAMHS professionals becoming increasingly sub-specialised according to their interests in specific areas or disorders. The study also highlights the utility of close links between local services since these children often have difficulties in several domains and come into contact with a range of services.

Conclusions

Although the accurate diagnosis of children with possible ADHD is an important task for child health services, scarce clinical resources need to be used in the most efficient manner in order to meet the demand for such services. Clinical judgment should continue to play a major role in both determining the suitability of referrals for assessment as well as in the diagnostic process. However, the findings highlight that referral information and the use of parent and teacher questionnaires are useful adjuncts in informing these processes.

Ancillary