Use of monitoring data to improve implementation of a home fortification program in Bihar, India

Abstract This paper describes the use of program‐monitoring data to track program performance and inform activities. Monitoring data were collected as part of an effectiveness trial of multiple micronutrient powders (MNPs) for children 6–18 months in Bihar, India. Communities (n = 70; reaching over 10,000 children) were randomized to receive either counselling on infant and young child feeding or both counselling and MNPs. Government frontline health workers (FLWs) implemented and monitored program activities with support from CARE India and university partners. Monitoring data were collected over the duration of the entire program to assess program impact pathways using various checklists, which captured information about (a) attendance and training of FLWs at health subcentre meetings, (b) distribution of MNPs, (c) receipt and use of MNPs at the household level, and (d) midline mixed methods survey. At the beginning of the program, 72% of households reported receiving and 53% reported currently consuming MNPs. These numbers fell to 40% and 43% at midline, respectively. The main barrier to use by household was a lack of MNPs, due in part to infrequent FLW distribution. However, FLWs rarely reported MNP shortages at Anganwadi centres. Side effects also emerged as a barrier and were addressed through revised recommendations for MNP use. Qualitative data indicated high community acceptance of MNPs and a good understanding of the program by FLWs. The use of real‐time program data allowed for recognition of key program issues and decision‐making to enhance program implementation.

Extensive research documents the efficacy of MNPs, but there exists a dearth of context-specific evidence on the effectiveness of MNPs for child anaemia reduction both globally and within India (Matias et al., 2017;World Health Organization [WHO], 2011;Yousafzai, Rasheed, Rizvi, Armstrong, & Bhutta, 2014). Our program aimed to test the effectiveness of MNPs to reduce child anaemia in one district of Bihar, India. Recognizing that counselling on child feeding practices is also an important strategy for improving child growth (Home Fortification Technical Advisory Group [HF-TAG], 2015;Loechl et al., 2009;Robert et al., 2006), we combined home fortification using MNPs with improved counselling on infant and young child feeding (IYCF) practices, as recommended by the HF-TAG. To ensure sustainability, we implemented the program using existing government health and Integrated Child Development Scheme (ICDS) functionaries.
Few studies have identified context-dependent bottlenecks, examples of which include supply chain inadequacies, transportation challenges, underestimation during stock and budget forecasting and ensuing delays with payments to manufacturers, poor adherence to dosing requirements, side effects and safety concerns, use of MNPs for nontarget children within the household, and sale of MNP products by the household (Loechl et al., 2009;M. Nguyen et al., 2016;Suchdev et al., 2010;Suchdev et al., 2012). There continues to remain a need for program to systematically study implementation processes (Avula et al., 2013). Additionally, the identification of implementation gaps and demand side barriers is crucial to ensure accurate translation of findings and interpret results related to fidelity of intervention delivery (Habicht & Pelto, 2011;White, 2013). Program impact pathways (PIPs) enable theory-driven process evaluation, which spans the spectrum from supply to demand and allow for visualization of a program uptake and impact (WHO & UNICEF, 2008). This approach facilitates the examination of contextual factors that influence effectiveness (Avula et al., 2013). Additionally, this framework informs ongoing implementation and guides measures for course correction and integration of needed complementary interventions (Avula et al., 2013). This manuscript describes how we leveraged PIPs to assess the implementation of an MNP effectiveness trial in Bihar, India. The home fortification with MNPs program was conducted in two phases, the first involving several rounds of formative research to contextualize the MNPs, IYCF messaging, and study acceptability of the delivery platform Young et al., 2013;Young et al., 2015). Phase two, described here, lasted 12 months and was conducted in 70 communities, designated as the catchments of health subcentres (HSCs). Each HSC has between 5 and 12 Anganwadi centres (AWCs) within its catchment area, and each AWC serves a population of approximately 1,000 people. We chose four blocks for implementation based on distance from the district headquarters. A list of all HSCs in the four selected blocks was generated, and those with political instability or in flood prone areas were excluded from randomization. The remaining 135 HSCs were randomized to generate a list of 35 intervention and 35 control HSCs using a random number generator-based simple randomization method. Intervention and control HSCs were distributed across all four blocks; hence, risk of contamination or spillover due to geographic proximity could not be completely mitigated. We however monitored for such risks on an ongoing basis throughout the course of the program.
The program was implemented by government frontline workers (FLWs), including (a) Accredited Social Health Activists (ASHAs) and (b) Anganwadi workers (AWWs), who were trained and managed at the level of the HSC, under the supervision of auxiliary nurse midwives (ANMs) and lady supervisors. AWWs are permanent paid employees who receive monthly salaries, whereas ASHAs receive incentives based on completion of tasks, which include counselling mothers on nutrition, pregnancy care, and supporting women to engage in institutional deliveries. Home visits are a routine component of the job descriptions of both, with an average of 50-60 households per month per worker, depending on the number of pregnant, lactating women and children under 2 years of age per community (Khandelwal, Dayal, Bhalla, & Paul, 2014

| The PIPs
Program impact pathway (PIP) development is an iterative process and involves the documentation of hypothesized linkages between program activities and their expected immediate, intermediate, and final outcomes (Kim, Habicht, Menon, & Stoltzfus, 2011). We developed PIPs in consultation with CARE India, the implementing partner, and through a detailed  Loechl et al., 2009), and reflects aspects related to program implementation (supply) and uptake (demand). An important a priori part of our program design included the handover of all monitoring activities to the government after Month 4. Operationally, this meant scale back of program team involvement in monthly HSC meetings and handover of checklist completion to ANMs.
All data collected from checklists were reviewed by the CARE and Emory teams on a monthly basis and shared with the field program team for dissemination to FLWs. Major decisions with respect to course correction were made primarily following midline data analysis, collaboratively with inputs from CARE, Emory, and government staff.

| Data collection
We used a variety of checklists at the HSC, Anganwadi, and household levels, throughout the program at varied frequencies to monitor program rollout, assess the level of program implementation, and evaluate PIPs. At midline, we collected questionnaire-based data from households and FLWs. Qualitative in-depth interviews (IDIs) were conducted with caregivers and focus group discussions (FGDs) with FLWs at midline. Table 1 describes data collection protocols in further detail.

| Monitoring checklists
All checklists were pilot tested prior to implementation and revised continuously during the initial months of rollout in consultation with the program team and FLWs. This ensured that checklists were not excessively burdensome to complete for field workers (< 30 min), and captured key monitoring information.  From each HSC, 10 AWCs were randomly selected for a total of 30.
Checklists were completed by our program staff in consultation with AWWs. Issues with the supply chain identified during data collection were reported to the study coordinator, and supplies were replenished through our network of CARE couriers or frontline program staff.
Counselling pamphlets with IYCF messaging (in control and intervention communities) and instructions for MNP use (in intervention communities only) were also replenished via our program staff based on information gathered from these checklists.

| HSC checklists
HSC checklists (refer to Supporting Information) were completed monthly by our study team and by ANMs across all 70 study HSCs.
We trained ANMs to complete these independently with the aim to handover this activity to them once initial program staff involvement was scaled back. The checklists were provided to ANMs at weekly meetings held at block-level primary health centres and collected at this forum or at HSC meetings by program staff.

| Household checklists
In intervention and control communities, program staff completed household monitoring checklists with caregivers and those responsible for the feeding of children aged 6-18 months. Checklists were collected monthly for Months 1 to 3 and then bimonthly for Months 6 to 12. Staff did not complete checklists in Month 4 or 5 as these coincided with the midline. During each monthly monitoring, four intervention HSCs and two control HSCs were selected randomly and checklists completed for 60 intervention and 30 control households. Program staff targeted two AWCs per HSC for household monitoring and randomly selected households for monitoring based on catchments targeted during that month.

| Program implementation midline surveys
The midline was conducted in Month 4 to assess program rollout and implementation. We used data from midline to determine the duration of rollout and the timing of program endline. To understand supply and distribution, we assessed FLW reported delivery of MNPs and caregivers reported receipt of MNPs at the household within the last month. To measure uptake, we looked at whether caregivers reported current use of MNPs/child having consumed MNPs in the last month and week.
Low program implementation was defined as <40% of households reporting receipt and uptake of MNPs in the last month, medium as <60%, and high as >60%; these cut-offs were defined a priori by the team and were based on researcher judgement. At midline, we interviewed 840 caregivers with children 6-18 months of age, and 420 FLWs. All 70 HSCs were sampled; four AWCs were randomly chosen per HSC; four households per intervention Anganwadi and   two per control Anganwadi were chosen randomly and interviewed by our program staff based on household lists provided by FLWs from their registers. We also conducted qualitative IDIs with caregivers (n = 20) and ANMs (n = 2) in addition to FGDs with FLWs (n = 4) from one low-and one high-performing intervention HSC. The "high" and "low" performances were decided based on HSC and household level-monitoring data and in consultation with our program staff and ANMs who supervised FLW attendance at meetings, using indicators of home visits and overall performance. The quantitative survey took 1 month to complete, whereas qualitative data collection was completed over the course of 14 days.
Quantitative analyses were conducted in SPSS; we report percentages for key indicators.  Overall knowledge of FLWs on MNPs remained high throughout the program and ranged between 87% (Month 1) and 100% (Months 3, 6, 8, 10, and 12, with an overall mean of 97%) of workers surveyed using AWC checklists reporting the correct age for MNP introduction.

| Home visits and distribution of MNPs
HSC checklist reports of home visits in the first 2 months of the program were very high (95% and 90% reporting home visits in intervention HSCs and 99% and 98% reporting home visits in control HSCs in Months 1 and 2, respectively), with uneven decline thereafter, as seen in Figure

| Household uptake-Demand side PIPs
To help us understand the demand side of program implementation, we collected data on home visit patterns from the perspective of households and uptake of MNP and IYCF practices using household checklists ( Figure 3). Our findings show that on average, 54% (35-85%) of households reported receiving MNPs in the last month and 52% (40-63.8%) of households reported their child was currently taking MNPs, over the course of the program.

| IYCF practices and MNP-related knowledge and use
In the intervention group, throughout the course of the study, a mean of 74% (

| Side effects
We noted through household and HSC checklists and during routine contact with FLWs that children 6-9 months who had initiated A child between the ages of six to eight eats much less food, compared to that quantity of powder is more.
Because of this imbalance in proportion of food and powder, child refuses to eat food. When food is more and powder is less, child does not recognize the presence of the powder in the food. FGD with AWW in low-performing HSC We addressed the issue of side effects such as vomiting, through retraining of FLWs. As well, we organized community meetings to support demand generation and to provide additional sources of counselling and support to community members. We modified FLW trainings on feeding recommendations for children experiencing side effects based on strategies, first suggested by FLWs themselves, elucidated through participatory consultations. To this end, all FLWs were advised to counsel mothers to split MNP sachets across 2-3 meals per day. These recommendations were well accepted and helped improve overall uptake.
I ask the mother to take some cooked rice in a clean bowl and also bring pulses curry or vegetable curry or anything which is cooked. Then I mix one third of the powder by tearing the packet and telling her what she has to do, and also say that if you will give like this, the child will not vomit and eat the powder in mixed food. FGD with ASHA in low-performing HSC

| Program implementation midline survey
The program implementation midline was conducted in Month 4 of MNP rollout. Our results show that overall implementation was low. Receipt of MNP in the last month, as reported at the household level, was low (40%), as was consumption (35%), as presented in Table S1. These data were used to help determine the timing of the endline survey and inform other programmatic course corrections. People told me that it is good for the health of the child.

| Acceptability and use of MNPs
This will increase strength and weight of the child. So, for the good of the child I am giving this powder regularly to my child. HH IDI with mother in low-performing HSC There is no problem sir, child is getting fat and the mind also grows. FGD with ASHA in low-performing HSC

| Barriers to MNP use-Identified at midline
A lack of supply of the powders in the household was identified as the most common reason for not using the powders (21%), implicating bottlenecks in our supply chain. At midline, households also described several additional reasons for not using MNPs: forgot to give MNP (3.4%), side effects (5.9%), poor appetite of child (3.4%), and illness including cough, cold, or fever (3.0%).

| Counselling
A lack of adequate counselling on how to use MNPs emerged as a common concern for poor uptake in both the quantitative household survey and qualitative IDIs with households. Some households reported mixing the MNPs in plain rice (without lentils or sauces) and feeding the child, often leading to children spitting out the powders.
Generally, child eats rice when it is mixed with oil, salt and chilli powder. When child is given rice mixed only with powder, she spits it out from her mouth. HH IDI with mother in high-performing HSC Additionally, we added messaging to advise use of MNPs in wet foods (rice mixed with lentils and vegetables to ensure it is moist), where the powders are mixed in completely and therefore undetectable to the child.
The issue of side effects and feeding during illness emerged during qualitative work and in routine monitoring.
When child is sick, she does not eat just milk is given. HH IDI with mother in low-performing HSC Child resists food when he feels sick. When child resist food we do not force much. HH IDI with mother in low-

performing HSC
We retrained FLWs on counselling mothers with ill children, by emphasizing the importance of responsive feeding with attention to quantity and frequency of feeds during illness.

| Delivery platforms
Additional findings from the program implementation midline show that 40% of families reported receiving the powders by visiting the Anganwadis as opposed to home visits. We revised our delivery approach to include pick-up of MNPs at Anganwadis directly, in addition to home delivery by FLWs.
Many people are not aware of the importance of the powder, for that at initial stage ASHA and Aanganwari worker have to go to each child's home for distribution of the powder. When people start using powder they themselves will collect it from the AWC. IDI with ANM in high-performing HSC To alleviate FLW work burden and ensure adequate household stock over the festival season, we amended recommendations to encourage FLWs to deliver three boxes of MNPs to each household at a time. This two-pronged approach to dissemination of the powders catered to the needs and convenience of individual families and FLWs.

| Working relationships and community perceptions of FLWs
The qualitative data showed that in areas where FLWs shared a good working relationship, effort was distributed relatively evenly between both cadres of workers. Approximately 21% of households reported receiving the powders from ASHAs and 35% reported receiving MNPs from AWWs.
The powder is dropped at AWC first, then Aanganwari worker informs the respective ASHA about the powder, and planning for the distribution is done based on the requirement of the households. FGD with ASHAs in high-performing HSC When asked, about their overall relationships with the communities they serve, FLWs reported that they had improved through the program: One positive side of this program when ASHA and AWW are involved in such programs they are earning good will of people. People benefitting from this powder consider them the benefit provider. IDI with ANM in high-

performing HSC
It was not like we have to give this powder to some and not to others. It has to be given to all the eligible children of the area who are of a certain age group.
Since the beneficiaries are in almost each household, they get benefit. They think that the Aangawari sister has come with something for their children. And rest of the things are based on categories which create dissatisfaction, that few of them are getting and many are not. AWW FGD in high-performing HSC As shown in Table S2, 98% of FLWs at midline thought it was important to provide MNPs to children in their communities, 96% agreed that MNP distribution had improved their overall status in the community, and 96% said they would continue to distribute MNPs in the future.

| Remuneration and payment for FLWs
We observed that 92.6% of FLWs reported not receiving full payment of regular salaries during our midline. Additionally, 64.4% of AWWs and 63.8% of ASHAs reported never receiving their routine payment on time.
Kindly tell us Sir, is it possible for a family to survive on remuneration of Rs 3000? All the field work has to be done by us but the government is not thinking of us ….

FGD with AWWs in high-performing HSC
No additional compensation was provided to FLWs for their participation in this program and resultant increase in workload. These issues collectively were an essential bottleneck in our supply chain that were beyond the scope of this program to address.

| Systems, stakeholder, and community engagement in implementation
Results from midline data were leveraged to engage all levels of the government system within the district. We worked with the district program officer to improve supervision provided directly to FLWs by community development program officers, lady supervisors, and ANMs. We also increased community demand generation by hosting two rounds of additional community meetings in Months 8-10 and Routine monitoring also helped us track implementation, by having our field teams conduct surveys regularly, visit more remote study sites during the trial, and address supply chain issues in real time.
Ongoing training was an essential component of program implementation and improves quality of care delivered and communication by FLWs as has been reported in prior research (Avula et al., 2013;Bryce, Victora, Habicht, Black, & Scherpbier, 2005). Reinstating our program team to provide refresher trainings and continuously reiterate program-related content was a key decision taken after looking at monitoring and midline data.
The overall goal of this study was to assess whether program scale-up would work using existing government structures. Measures to encourage future program sustainability were put in place wherein CARE program staff were never directly involved in delivery of the intervention to households. Our team did retain control of the supply chain from manufacturer to decentralized storage sites (AWC), however due to which we were able to ensure that stock-outs were not an issue up to this point. Scale-up of this intervention would require additional inputs from the government to manage these higher rungs of the MNP supply chain from manufacturer to storage facilities.
Our program team was also involved at higher levels to boost performance by FLWs.
We recognize the impact of potential Hawthorne effect created by intense monitoring activities such as those carried out during this study and that this may influence indicators more than would be seen if carried out solely by existing government structures (McCambridge, Witton, & Elbourne, 2014). Our learnings reinforce, however, that new program require increased systems involvement from the get-go, initial investment in inputs for FLW training, and community demand generation to ensure adequate program uptake (MNP utilization and consumption;Mason, Sanders, Musgrove, & Galloway, 2006).
The training of and provision of supportive supervision to FLWs, in addition to their use in the delivery of program such as these, builds capacity and sustainability. A recent review states that FLWs are the most common delivery channel for MNPs identified in the literature to date (Reerink et al., 2017). However, such engagement is complicated by existing work responsibilities (Avula et al., 2013;Nair, Thankappan, Sarma, & Vasan, 2001). FLWs are often overburdened and underpaid, which likely dilutes the quality and consistency of service provision (Glenton et al., 2010;Nair et al., 2001;Standing & Chowdhury, 2008). The role of adequate and timely compensation for frontline implementers is of critical importance to ensure the success of effectiveness program that aim to reach the most underserved populations (Avula et al., 2013). A recent study conducted in Bihar shows promising results for improved teamwork, empowerment, job satisfaction, and equitable service delivery, in addition to higher levels of motivation among FLWs, employing a team-based goals and incentives model (TBGI) (Grant et al., 2018 (Strachan et al., 2012).
Beyond the delivery of the intervention, several demand side factors should be considered. We were able to use PIPs to examine critical linkages between implementation and uptake (Loechl et al., 2009;Suchdev et al., 2010). Adequate counselling and education on the purpose, use, and benefits of MNPs is essential to ensure acceptability and fidelity at the household level. This includes management of side effects associated with MNPs, use, feeding during illness, and hygiene practices necessary to reduce morbidity. Other authors have noted a reduction in the quantity and frequency of complementary feeds given to children during illness episodes. There remains a need for standardized and consistent counselling messages to support mothers in feeding children during illness episodes (Brown, Stallings, de Kanashiro, de Romana, & Black, 1990;Hoyle, Yunus, & Chen, 1980 Previous studies have focused on either implementation or uptake (Bonvecchio et al., 2007;Guldan et al., 2000;Santos et al., 2001), but few have taken a holistic approach to understanding program impact across the supply chain (Bhandari et al., 2004;Santos et al., 2001;Zaman et al., 2008) or uptake (Bhandari et al., 2004;Zaman et al., 2008). We noted differences in patterns of implementation (FLW reports of MNP distribution) versus receipt and uptake, in our monitoring data. This highlights the need for triangulation of information that looks at both sides. The utility of this approach is its ability to reduce the influence of social desirability bias on interpretation of data by providing a well-rounded picture that seeks to validate the needs and opinions of service providers, beneficiaries, and intended users.
Additionally, this approach enabled timely action on modifiable factors that, in the case of this program, include efficient delivery and replenishment of stocks correction of improper counselling about MNPs, and modification of delivery modalities to accommodate pick up at Anganwadis and delivery to the household. Non-modifiable factors such as salaries not received on time by FLWs are also a likely hindrance to the success of new interventions integrated into existing services.
Such data provide essential context to our overall findings.
Limitations of our approach include small sample sizes for monthly and bimonthly monitoring activities, and burden associated with routine data collection. We were not able to comprehensively test the knowledge of FLWs after providing trainings, and we did not provide monetary or nonmonetary incentives for their work. Our data are not representative of the population and are based on self-report.
These findings span a short time period (1 year), as opposed to several years, thus highlighting an important evidence gap and need for monitoring of MNP program that have a greater longevity.
In addition to PIPs, other authors have devised a global framework for reporting of context and implementation for pilot studies to be utilized in intervention trials (Luoto, Shekelle, Maglione, Johnsen, & Perry, 2014). The uptake of these criteria alongside PIPs in the development of pilots will add to systematizing methods focused on monitoring of large-scale community-based program in low-and middle-income countries. There is a need to balance requirements for field reporting and monitoring with burden on FLWs and other field staff. We believe that our HSC checklist can be streamlined and integrated into existing reporting requirements to capture key indicators without being burdensome to implement. Efforts are ongoing to digitize registers and checklists using information communication technologies for programmatic purposes in this context (Balakrishnan et al., 2016).
In conclusion, we have found that home fortification program can be delivered through FLWs, but routine monitoring is essential to track and course correct when utilizing existing government resources.
In addition, our study provides evidence for the utility of PIPs and assessment of both supply and demand side factors to gain a comprehensive picture of program implementation. Kumar Rai) played a key role in the collection of these data and supporting our program and FLWs. We extend our special gratitude to all the FLWs, ICDS, and Health officials at the district and state levels, who were our partners in implementation and to all the families who participated in this pilot. Finally, we would like to thank CARE India teams at the block, district (Kayum Ansari, Virat Patel, West

ACKNOWLEDGMENTS
Champaran district office), and state (Bihar) levels for their support and cooperation during the implementation of this study.

CONFLICTS OF INTEREST
The authors declare that they have no conflicts of interest.