Measuring the impact of a training intervention for early childhood centre staff on child development outcomes: Findings from a cluster randomized control field trial in rural Malawi

Abstract Background Evidence from low‐income settings around early education interventions that can improve young children's development is sparse, particularly with regard to the most marginalized children. This study used a two‐arm parallel cluster randomized control design to evaluate the impact of an adapted staff training programme on the developmental outcomes of children attending community‐based early learning centres in Thyolo district, rural Malawi. Methods At baseline we randomly selected 48 centres, from each of which 20 children were randomly selected, although data from one centre was incomplete resulting in 932 children from 47 centres. Centres were randomly allocated to either the intervention or control arm. Twelve months later, follow‐up data were collected from 44 centres. At baseline and endline, community‐based childcare centre (CBCC) managers provided information about the centre, and parents/guardians provided information on the children, including the primary outcomes of age‐standardized development scores in the language and social domains, measured using the Malawi Developmental Assessment Tool. Children in the bottom 2.5 percentile of either domain were considered to have a delay; a third outcome variable, Any Delay, was developed to indicate children with a delay in either or both domains. Centre‐level mean scores were calculated, and linear regression models were constructed to assess differences between baseline and endline and between allocation groups. Results Analysis of the difference between baseline and endline measures in the allocation groups shows a non‐significant reduction in delay associated with the study intervention across all domains. Adjustment for baseline characteristics within the CBCCs showed little impact on the magnitude of the observed effect, and the difference remained non‐significant. Conclusions Despite no observed differences between allocation groups, the data did indicate a positive change in the intervention groups in both domains, particularly language. Community‐based early learning in Malawi holds tremendous potential for promoting inclusive development and learning.

language. Community-based early learning in Malawi holds tremendous potential for promoting inclusive development and learning.
K E Y W O R D S child development, child disability, inclusion, measurement, randomized trials 1 | INTRODUCTION Evidence from low-and middle-income countries (LMICs) suggests that as many as 250 million children under the age of 5 are at risk of not reaching their developmental potential (Black et al., 2017). An estimated 53 million children globally have a developmental disorder with most of these children living in LMICs (Olusanya et al., 2018).
Children at highest risk of developmental delay are girls, those living in extreme poverty, those with disabilities, refugees and children with multiple characteristics of disadvantage (Black et al., 2017). The implications of unmet developmental potential extend beyond the individual child, to their families, communities and broader societies (Richter et al., 2017). Early detection of these disabilities is vital with early interventions reducing social, behavioural and educational problems (Milner et al., 2019). This, in turn, can reduce future costs for society and enable children to flourish, learn and participate better (Lancaster et al., 2018). Although the impact of the Covid-19 pandemic is not yet fully understood, it is likely that the developmental challenges faced by children in poorer settings will be aggravated, exacerbating the inequity observed in child development globally (Yoshikawa et al., 2020). Interventions to improve child development outcomes that are not only culturally appropriate and acceptable, but also effective for the most marginalized children, are vital to improve this situation (Pence & Marfo, 2008).
Early childhood development (ECD) encompasses the inputs necessary to help shape the lives of children aged 0 to 8 years to survive, thrive and secure solid foundation (Richter et al., 2019). The five key components of ECD are good health, adequate nutrition, responsive caregiving, security and safety and opportunities for early learning (World Health Organization, 2018). While the evidence based on health and nutrition has substantially increased in recent years, that around the consequences and impact of early learning, particularly in informal settings, continues to be scarce (Berlinski et al., 2009;World Health Organization, 2012).
Malawi is a low-income country in southern Africa, with a welldeveloped policy base for ECD, overseen by the Ministry of Gender, Community Development and Social Welfare (MoGCDSW), and delivered mainly in informal community-based settings (Neuman et al., 2014;World Bank, 2015). Despite strong policy buy-in, attendance is relatively low, particularly in rural locations where 46% of households fall into two lowest economic quintiles and where stunting in children is particularly severe (Mussa, 2014;National Statistical Office [Malawi] and ICF, 2017). ECD interventions are delivered in community-based childcare centres (CBCCs) by volunteer caregivers who are usually paid in kind. Significant challenges with many CBCCs include high caregiver-children ratios, untrained staff and high staff attrition (Munthali et al., 2014). Staff training is provided through a small number of government-recognized organizations, for example, Association of Early Childhood Development Malawi (AECDM); however, they are dependent on funding from international donors that can be sporadic and insufficient for the needs. This was confirmed in previously published data from an initial component of this study, where CBCC caregivers emphasized the importance of receiving additional competency training and skill development, particularly in teaching children with disabilities (Greenwood et al., 2020;Soni et al., 2020).
The objective of this study was to assess the impact of an adapted training programme, which integrated a disability inclusive resource pack into the early years curriculum, on children's developmental outcomes (Soni et al., 2020). Results of the baseline published earlier showed high prevalence of both developmental delays and disability, with a strong relationship between the two (Murphy et al., 2020

| Participants
The District Social Welfare Office maintains a list of registered CBCCs, which served as our sampling frame. At baseline, 48 CBCCs were randomly selected from a pool of CBCCs meeting minimum criteria: located in Thyolo district, feeding programme in place, more than 20 children registered and regularly attending, minimum of two caregivers, minimum infrastructure (e.g., permanent location, water supply) and had not participated in an earlier CRCT funded by the World Bank (Ozler et al., 2016).
Within each CBCC, 20 children were randomly selected at both baseline and endline. Selection for participation at both time points was independent, and the children's data are not linked from baseline to endline. Children were eligible for selection if they were at least 2 years old, had been registered for at least 6 months and attended the centre regularly (at least four times a month).

| Sample size
The sample size was calculated to detect a 10% change in the proportion of children whose developmental age is equal to their biological age (expected increase from 70% to 80%). Based on the 95% confidence interval (CI), 80% power, 10% non-response and 50% variation between the clusters (7), we aimed to recruit 960 children (480 per arm) or 48 CBCCs with 20 children per CBCC.

| Randomization
The 48 study CBCCs were randomized to two arms. Randomization was carried out by one of the co-authors (S.B.) before the trial started, using Excel. The children were randomly selected on the day of data collection using the CBCC records. If there were 20 or fewer children available on the day, then all were enrolled. If there were more than 20 children, then a random selection approach was applied by writing all names on pieces of paper, placing in a bowl and drawing at random.
Given the nature of the intervention, it was not possible to blind participants or data collectors.

| Interventions
The adapted 14-day training programme was provided to caregivers from 24 intervention CBCCs. The training was implemented over 4 weeks in July 2017, with caregivers divided in two groups, to allow CBCCs to continue functioning during the training period. Training was facilitated by national trainers from AECDM, Magomero Community Development College and the MoGCDSW. An 'Inclusion Resource Pack' was developed by the University of Birmingham in collaboration with Chancellor College of the University of Malawi, international non-governmental organization, Sightsavers and AECDM and focused on promoting the inclusion of children with disabilities in the daily activities of the CBCCs (Soni et al., 2020). Training covered the following topics: understanding of disability, inclusive games, early literacy and storytelling, well-being and involvement, safety and risk management, early maths, inclusive environment, inclusion of CBCCs, identification of common types of disability and working with parents of children with disabilities. The team enhanced the basic 2-week training for volunteer caregivers building on the local practices, knowledge and strengths that exist in ECD and collaborate with local training providers, community and ECD services (Soni et al., 2020).
In addition to the training, all intervention CBCCs received a basket of simple locally sourced materials (laminated alphabet and number cards, string, pegs, a locally made parachute, etc.) to facilitate learning and inclusion of children with disabilities.
Control centres received the training in November 2018, after the collection of the endline data.

| Outcomes
The primary outcome was the proportion of children with ageappropriate scores in (1) language and (2) social-emotional development. We used two of the four domains of the Malawi Developmental Assessment Tool (MDAT) to ascertain the scores; the tool was developed based on the age-standardized development norms appropriate for children living in rural Malawi (Gladstone et al., 2010). Children carried out up to 68 tasks that are incrementally more difficult. The assessment was conducted in the presence of the parent/guardian who could provide some of the answers to routine-based questions. Numerical developmental scores in both domains were transformed to binary indicators, identifying those scores that fell in the bottom 2.5th percentile of the age-referenced data as representing an observed delay in that domain. A third binary outcome variable, any delay, measured the proportion of children who had delay in one or both domains.

| Statistical methods
Basic descriptive statistics were calculated to understand the range and distribution of responses. We examined baseline characteristics by allocation group to understand whether values were similar. We did not perform formal statistical tests to assess homogeneity as we believe the randomization process was robust.
Raw summary scores of the percentage of children with a delay for baseline and endline were calculated according to allocation group.
CBCC-specific differences from baseline to endline were calculated for each of the three outcomes, which resulted in a mean percentage difference. We used linear regression to assess the difference between the mean percentage difference in the allocation groups and the associated 95% CI.
Additional analyses were conducted to account for baseline values of the CBCCs that were considered a priori, as having potential influence on the outcomes. We did this through adjusting the linear regression models by including the appropriate baseline values and reviewing the new difference in mean percentage difference and associated 95% CIs.

| Baseline data
Summary statistics for 47 CBCCs at baseline are shown in Table 1 indicating similar mean number of children and caregivers per CBCC in the allocation groups. The proportion of CBCCs with attendance registers was slightly higher in the intervention group, while the proportion of CBCCs with learning and play materials was slightly higher in the control group.
Child-level summary statistics are shown in Table 2. Child-level data were available for 932 children at baseline and 881 at endline. At baseline, the mean age was 51.4 months, and 55.4% were girls; the age and sex profiles were similar in the allocation groups. At endline, the intervention CBCCs had more girls (63.4% vs. 54.2%), and the children were slightly older than in the control CBCCs (54.8 vs. 51.5 months). Over 10% of children at baseline were regarded as having a disability using the CFM tool with a slightly higher proportion in the intervention group. At endline, 6% of children had a disability with no difference between the groups. Table 3 shows that at baseline, the intervention group had slightly higher mean prevalence of language delay and slightly lower mean prevalence of social delay. Mean prevalence of any delay was similar in the two groups.

| Raw summary scores
By endline, mean prevalence of delay in the control CBCCs was higher than in the intervention group in all domains.

| Primary analysis
Analysis of the difference between baseline and endline measures in the allocation groups shows a non-significant reduction in delay associated with the study intervention across all domains (Table 4).
Adjustment for baseline characteristics within the CBCCs showed little impact on the magnitude of the observed effect, and the difference remained non-significant.

| DISCUSSION
The data indicate no evidence of an effect of the intervention on the proportion of children observed to experience developmental delay in either or both of the language and social domains, at approximately 12 months after the intervention. Adjustment for a range of contextual variables does not impact this conclusion.
However, non-significant differences in the reduction in delay were observed and need to be considered. A pronounced reduction in the language domain is particularly notable. The primary analysis suggests that the intervention may be associated with a reduction of 3.2% (95% CI indicating this may range from a reduction of 7.0% to an increase of 0.6%), over a period of 12 months. In the social domain, the intervention may be associated with a reduction of delay of 1.8% (95% CI À7.0% to 3.3%). These promising findings suggest that the  We were unable to verify whether those caregivers who received training continued to work at their CBCC for the entire duration of the follow-up period or whether they ceased employment at some point. This scenario would lead to a dilution of the effect of the intervention, and thus, the observed difference may be an underestimate of any change that could have occurred given 100% retention.
There is a noticeable difference in the age and sex composition of children in the allocation groups at endline. Crude analyses (not reported) between the outcomes and sex, by allocation group and at both baseline and endline, indicate no significant association. However, we cannot rule out that an increase in older children or girls in the intervention CBCCs at endline is an unintended consequence of the intervention.
A short training programme such as this can only cover the most fundamental areas with limited time available to explore early literacy and numeracy skills. Encouraging caregivers to support children more flexibly requires more extensive training and opportunities to try out new approaches and to reflect on practice. The results of the other trial conducted in Malawi indicate that improvements in CBCC caregiver resourcing should be accompanied by support to parents.
Community-based ECD in Malawi holds tremendous potential for promoting children's development and learning in their earliest years, but further work and resources are required to ensure the quality required for to produce the desired child outcomes.