Mind, Brain, and Education

Teachers (including kindergarten [KG] teachers) have an opportunity to play a key role in early identification and referral for children's developmental, behavioral, and emotional problems (2013 US National Conference on Mental Health www.whitehouse.gov/blog/2013/06/03/national‐conference‐mental‐health). Schools are ideal settings for screening and identification because they do not have the same service access and stigma barriers as formal mental health services. Teachers can often be more objective observers of children's development than are parents. Teachers also have the advantage of being able to assess an individual child's functioning in comparison to a naturalistic developmental comparison group displaying a range of normative behaviors (Rothì, Leavey, & Best, 2008).

Developmental, behavioral, and emotional problems in young children are strongly predictive of a variety of poor academic, socioemotional, and functional outcomes later in life (Briggs‐Gowan & Carter, 2008; Ozonoff, 2015). Early detection of such problems and referral into appropriate services can alter negative trajectories and prevent more serious problems from developing (DuPaul, McGoey, Eckert, & VanBrakle, 2001; Jensen et al., 2011). Behavioral symptoms of disorders such as autism, attention‐deficit/hyperactivity disorder (ADHD), depression, and anxiety can be detected before children begin elementary school (Briggs‐Gowan & Carter, 2008; Daniel, Prue, Taylor, Thomas, & Scales, 2009; Egger & Angold, 2006; National Scientific Council on the Developing Child, 2008/2012).

Some behaviors exhibited in very young children, such as affect dysregulation and/or irritability, are associated with higher risk for a variety of mental health problems later on (Beauchaine, Gatzke‐Kopp, & Mead, 2007; Ozonoff, 2015). For example, chronic irritability in preschool age children predicts anxiety disorders, symptoms of disruptive behavior disorders, functional impairment and increased mental health service use at age nine (Dougherty et al., 2015). The preschool age period is a particularly critical time for development of behavioral control that is associated with a variety of clinical and functional outcomes (Beauchaine et al., 2007). Epidemiological studies of obsessive‐compulsive disorder, conduct disorder, schizophrenia, and depression support the notion that the longer the period of undetected diagnosis and intervention the greater the likelihood of lifelong persistent impairment. This is consistent with age‐related decreasing plasticity (e.g., ability of the brain to adapt to environmental influences) of the developing brain whereby early interventions have greatest capacity to induce lasting change. Thus, early detection and intervention to alter maladaptive patterns have the potential to substantially improve long‐term individual and societal outcomes (Cuellar, 2015; Shaw, Gilliom, Ingoldsby, & Nagin, 2003; Suveg, Southam‐Gerow, Goodman, & Kendall, 2007; Wang et al., 2005).

Research has demonstrated that KG teachers are able to identify socioemotional problems in their students; however, there are often gaps in teachers' knowledge about behavioral and emotional problems, and variability in comfort and confidence with the role of problem identification and referral, resulting in limited identification and referral skills for these types of problems (Green, Malsch, Kothari, Busse, & Brennan, 2012; Loughran, 2003; Reinke, Stormont, Herman, Puri, & Goel, 2011). KG teachers in developing countries with more limited mental health training resources are likely even less well equipped for this role.

Ethiopia is a developing country with limited formal mental health resources and educational challenges. Although there has been significant recent development in expansion of adult mental health services, mental health services for children are almost nonexistent. Although the majority of the population of 99 million is under age 18, there are only two, poorly equipped, child and adolescent mental health service centers in the capital city with three million residents.

Ethiopia faces educational challenges as well. According to the latest national statistics, 21.4% of children enrolled in Grade 1 drop out before reaching Grade 2 (2013/14 Federal Democratic Republic of Ethiopia Ministry of Education, Education Statistics Annual Abstract). The economic and cultural impact of these high dropout rates is catastrophic in this poor country. School dropout is attributed to a variety of factors including poor school readiness (i.e., limited preacademic skills), lack of attention to health and mental health needs, and limited family engagement in education.

The current study examines one component of a comprehensive school readiness initiative (SRI) in Addis Ababa, Ethiopia (Fish, 2015). In addition to building children's preacademic skills for success in school the SRI aims to improve preschool teachers' ability to screen children for common developmental, emotional, and behavioral problems.

There is growing recognition of the negative impact of emotional and behavioral problems on children's early school readiness and school achievement, but few established models for effective preschool teacher training in screening (Rimm‐Kaufman, Pianta, & Cox, 2000). Head Start programs in the United States often emphasize the importance of building staff capacity to promote children's socioemotional development (Head Start Impact Study—Final Report, 2010). A study conducted in two Head Start programs found that training staff in early childhood mental health resulted in significant positive improvements including reduced staff stress and increased knowledge of best practices in early childhood mental health (Green et al., 2012). Increasing knowledge and comfort with mental health practices is valuable, but there are associated challenges. Cross‐cultural studies have demonstrated that teachers can recognize symptoms of problems like ADHD, but their actions based on this recognition may be influenced by a variety of factors including child characteristics and cultural expectations (Lee, 2014). Thus, training teachers to identify cases and provide appropriate help including referral options requires more than simply raising awareness of problem behaviors.

There have been efforts to train elementary school teachers in developing countries to identify and refer children with mental health problems. For example, a 2‐day teacher training program in Pakistan reportedly resulted in improved teacher knowledge and awareness of children's mental health issues (Hussein & Vostanis, 2013). Likewise, a training program in India designed to educate teachers about normal child development and common psychiatric problems resulted in teachers' improved ability to identify psychological problems for children depicted in case vignettes (Shah & Kumar, 2012). Results of a study in Chile showed improvements in academic and behavioral functions following early school mental health intervention (Guzman et al., 2015). However, not all such training programs have achieved desired outcomes. One program in the United Kingdom designed to improve teachers' ability to identify adolescent students with depression did not result in more accurate identification compared to standard teacher identification without training (Moor et al., 2007).

Increasing teachers' knowledge about potential signs of problems and offering encouragement to identify and refer affected children can also result in elevated rates of “false positive” cases (i.e., children who do not actually have the identified problem; Loughran, 2003). Potential over identification is not a benign consequence, because it can result in (1) significant family stress, (2) inappropriate labeling of children which can impact behavioral and academic expectations, and (3) delivery of unneeded services. For these reasons, methods to screen for mental health problems should be evaluated for both their sensitivity (ability to identify true‐positive cases) and their specificity (ability to identify true‐negative cases). Accuracy in identifying cases relies on balancing high sensitivity and specificity.

In this study, the long‐term impact of a manualized teacher training program to improve teachers' screening skills in Ethiopia was evaluated by assessing teachers' ability to identify children with common developmental and mental health problems before and 2 years after participating in the training, and compared to a no‐training control group. The accuracy (e.g., sensitivity, specificity, and reliability) of teachers' screening was tested against a well established measure of children's emotional and behavioral strengths and weaknesses (Goodman, 2001). The hypothesis was that the teacher training intervention would result in more accurate identification of children with mental health problems.

RESEARCH METHOD

DESCRIPTION OF TEACHER TRAINING INTERVENTION

The intervention involved a special teacher training program designed to improve teachers' knowledge about the basics of children's common mental health and developmental problems and to facilitate skills needed to identify children who may need additional assessment or treatment of these problems. The training included provision of standardized manuals (in Amharic) for teachers, developed by a multidisciplinary team of local professionals led by the first author (MD) and available upon request. In all, 11 simplified manuals were developed. A manual on common mental/behavioral disorders was developed addressing depression, anxiety, psychosis, conduct disorder, ADHD, oppositional defiant disorder, and temper tantrums. Six of the manuals addressed the various aspects of child development and other domains of child functioning (physical development, language development, intellectual development, social and emotional development, play, and child nutrition and health). Three were regarding parents' role in child protection, disciplining children, and ensuring child‐friendly interpersonal relationship in the family. One manual addressed problems that Ethiopian parents can face while rearing children. The training thus addressed holistic child development and functioning with the aim of enabling teachers to know the whole child while they address mental and behavioral issues.

The teacher training manual on mental/behavioral disorders includes culturally and developmentally appropriate descriptions of these problems, including examples of observable symptoms and behaviors. The manual also addressed concerns teachers might have about identifying students for follow‐up. Face validity of the manuals was evaluated by child development and mental health experts, including five psychiatrists in the Department of Psychiatry at Addis Ababa University and two relevant professionals from each of the Bureaus of Health, Education, and Women's and Children's Affairs of the city government of Addis Ababa. These experts repeatedly reviewed all materials and deemed the contents and format to be accurate, clear, and useful.

Only teachers in intervention KGs were provided the training. Intervention teachers were trained by psychiatrists and pediatricians over 18 months in meetings with SRI staff. Methods of training included lectures, role plays, and case discussions. Prior to discussing individual syndromes/diagnoses, training started with defining health as physical, mental, and social well‐being, followed by definitions of mental health and disorders.

Pre‐ and post‐knowledge, attitude, and practice (KAP) assessments were performed at four points during the 18‐month period with all participants (Figure 1). Analysis of KAP test scores showed progressive improvements in teachers' knowledge.

DATA COLLECTION PROCEDURES AND QUALITY ASSURANCE

Teachers (who were required to have known the child for at least three months) completed questionnaires on all study children at baseline and post‐intervention. The name of each of the children in the teacher's class was written at the top of a sheet of paper. For each child, the teacher was asked to use his/her own observation to indicate the presence of any mental/behavioral problem. At baseline, before the teachers of intervention KGs were trained on mental/behavioral health problems, they were asked to assess each child in their class and answer the question “Does this child have any mental/behavioral disorder?” based on their own routine observation during the preceding 3 months.

Similar to the intervention teachers at baseline, the nonintervention teachers were never provided any training on mental health or disorders. Similar to the intervention teachers at baseline, both groups (intervention and nonintervention teachers) that participated in the postintervention assessment were asked to assess each child in their respective class and answer spontaneously the question “Does this child have any mental or behavioral disorder?” The teachers were expected to answer “Yes” or “No.” The respondents were also asked to give a brief description of the problem if the answer was “Yes.” Teachers completed the questionnaire independently.

These teacher report data were used to stratify the children into two groups, children with mental/behavioral problems and children without mental/behavioral problems. Using simple random sampling, a number of children constituting one quarter of the study sample size was selected from the children (from intervention KGs and nonintervention KGs) identified by teachers as having problems, whereas the remaining three quarters of the study sample was selected randomly from children picked by teachers as having no problems.

The primary caregivers of the children thus selected were then interviewed face to face by trained, experienced psychiatric nurses using the parent version of the Strength and Difficulties Questionnaire or SDQ 4‐17p (Goodman, 2001).

Supervisors assessed data for completeness. Programmed data entry (using EPI‐DATA version 3.0 [EpiData Association, Odense, Denmark]) was used to maximize data quality assurance within expected ranges, examination of frequency distribution, and so on. Missing and/or inconsistent data values were double‐checked with the hard copy instruments.

MEASURES

RESULTS

Table 2b shows demographic characteristics of the base populations of children and the samples of study children drawn from the base populations.

Sociodemographic characteristics of teachers who participated in the postintervention study are shown in Table 3. In all, 166 teachers from intervention schools and 62 teachers from nonintervention schools were included in the study. Participating teachers from the intervention and nonintervention KGs had no difference in age, sex, marital status, or educational level. However, teachers who were in nonintervention schools had served significantly more years than teachers working in intervention schools.

Pre‐ and post‐training test of KAP for intervention teachers was done at the baseline (n = 128), and then at 6 months (n = 148), at 12 months (n = 178) and at the 18th month of the intervention (n = 146) using 10‐item questions. The pre‐ and post‐training scores were significantly different at all four timepoints, but the magnitude of the difference between pretest and posttest gradually reduced through time. At the initial training, the mean score difference was 3.33. The mean score differences at the second, third, and fourth trainings were 1.61, 0.75, and 0.62, respectively.

The initial analyses tested the sensitivity and specificity of teachers' identification of students' emotional or behavioral problems following training on mental health issues compared to (1) teacher identification prior to training and (2) teacher identification in schools that did not receive the training intervention. Table 4 presents the findings for each of the broad categories of emotional and behavioral problems according to the criterion measure, the SDQ subscales. At preintervention baseline, the specificity of teacher identification was relatively strong across all categories, but the sensitivity was weak. For example, for conduct problems specificity was 74.4% and sensitivity was only 38.5%. Following the training intervention, the sensitivity rose to 66%. Sensitivity of teachers' capacity to identify potential cases increased substantially across categories (ranging from 57.1% to 91.2%), while the specificity remained strong (ranging from 73.8% to 87.2%). Thus, teachers were better able to identify true “cases,” but specificity remained high, thus more positive identification of cases did not result in increased false positive identification.

Several of the specific categories deserve particular attention. Sensitivity rates for identifying emotional problems, for example, were very low preintervention and in the control nonintervention sites (31.8% and 27.3%, respectively). However, the sensitivity rate in the postintervention group was 91.2%, indicating that teachers' ability to identify children with emotional problems improved significantly post‐training. Similar patterns were seen for different types of problems such as conduct problems, hyperactivity‐inattention, and peer problems. The p‐values indicate a significant difference in sensitivity and specificity across the teacher groups. Similarly, effect size calculations showed that the proportions of increment in sensitivity, specificity, and Cohen's kappa in the intervention group as compared to nonintervention group for any SDQ problem were 64%, 13%, and 93%, respectively and the increments were statistically significant. Table 5 presents the raw data.

Reliability of teacher identification of problems post‐training was also assessed by examining interrater agreement between teacher identification and the SDQ subscales as represented by the kappa coefficient. Table 6 presents the kappa coefficients for the preintervention baseline, postintervention, and no intervention control group samples. As illustrated, agreement with the criterion SDQ measure was substantially greater in the postintervention group. Kappa coefficients ranged from .010 to .051 in the preintervention and nonintervention samples, reflecting no agreement between rating methods, contrasted to a range of .16 to .61 in the postintervention sample. Guidelines for kappa value interpretation suggest that values between .40 and .60 represent moderate to good agreement (Cohen, 1960).

DISCUSSION

This study provided strong support for the effectiveness of a training intervention to improve KG teachers' ability to accurately identify children with emotional or behavioral problems in Ethiopia. The sensitivity of teacher's identification of problems was significantly greater 2 years after the training intervention and this was not associated with an increase in false‐positive identification. Overall, the resulting levels of sensitivity and specificity of problem identification were generally above the acceptable level of 70% (Sheldrick et al., 2015). Agreement with the established criterion measure was greater postintervention compared to preintervention and nonintervention controls. Future research on this program should also examine fidelity to the teacher training intervention. Pre‐ and post‐training KAP assessments were carried out at intervals during the 18‐month period of teacher training. The results of the KAP tests show that teachers' knowledge improved through time, indicating effectiveness of training across the sites.

In a resource‐poor setting like Ethiopia, schools can have a significant role in mental health service provision given limitations of formal mental health care. Schools are the most convenient venue to access children, given the high improvement in enrollment rate in the country. Training teachers to identify and refer children for follow‐up is also significantly more cost‐effective than relying exclusively on professional mental health providers. In addition, screening and referral through the schools may be more culturally acceptable and may carry less stigma.

As is well known, mental health problems that are unmanaged in childhood can lead to more complicated problems later in life with personal, social, and societal complications (Copeland, Wolke, Shanahan, & Costello, 2015; National Scientific Council on the Developing Child, 2008/2012). It is also well known that mental health problems negatively affect school performance (Valdez, Lambert, & Ialongo, 2011). These factors impose negatively on a developing country's effort to alleviate poverty. Therefore, early identification and management of mental/behavioral problems are invaluable especially in a poor country that aspires to improve the educational success of its citizens in its struggle against poverty. Moreover, teachers are the source of information for the majority of illiterate rural populations constituting 85% of the national population. Therefore, policy makers need to consider using schools to assist the poorly developed mental health care system in their roles in early detection and management as well as in educating society on the relevance of mental well‐being.

Necessary next steps are to examine referral pathways and effective interventions once problems are detected. There is a great deal of promising research on the biosocial processes that confer risk for clinical disorders, as well as the potential positive impact of early interventions to alter those processes (Beauchaine et al., 2007). For example, Raine et al. (2001) demonstrated that a comprehensive supportive preschool program was associated with improved behavioral and biological markers of impulsivity 6–8 years after the program.

More research is needed to identify the processes and mechanisms of developmental psychopathology for specific disorders to improve the effectiveness of interventions to alter developmental processes when predictive signs are detected. For example, given that chronic irritability at age 3 has been shown to predict a variety of maladaptive outcomes, more research is needed to identify targeted interventions to alter the pathological pathways (Dougherty et al., 2015).

STUDY LIMITATIONS

The results of the study should be evaluated in the context of some limitations. For example, the criterion measure (SDQ) is not a psychiatric diagnostic measure, but it is a well established screening measure for the types of behavioral and mental health problems associated with a variety of psychiatric disorders. In addition, while the teachers in the intervention group did exhibit much higher agreement with the SDQ criterion scales, the magnitude of the agreement on most subscales was still only fair at postintervention. Modest agreement across informants and measurement methods for children's emotional and behavioral problems is the norm across a wide variety of studies and international contexts (Rescorla et al., 2014). Finally, as noted in Tables 1 and 2, the comparison groups, although generally equivalent across a variety of sociodemographic indicators, did differ on some variables such as child and caregiver age, caregiver gender distribution, and family income/social status.

CONCLUSIONS

This study provided promising evidence of the ability to train preschool teachers in Ethiopia on how to accurately detect children's mental health and developmental problems. The potential role for teachers in a resource‐poor environment such as Ethiopia is significant to support essential early intervention and referral.