Mixed methods evaluation explains bypassing of vouchers in micronutrient powder trial in Mozambique

Abstract Micronutrient powders (MNP) are recommended by the World Health Organization as an effective intervention to address anaemia in children. A formative process evaluation was conducted to assess the viability of a model using free vouchers in two districts of Mozambique to deliver MNP and motivate adherence to recommendations regarding its use. The evaluation consisted of (a) an examination of programme outcomes using a cross‐sectional survey among caregivers of children 6–23 months (n = 1,028) and (b) an ethnographic study to investigate delivery experiences and MNP use from caregiver perspectives (n = 59), programme managers (n = 17), and programme implementers (n = 168). Using a mixed methods approach allowed exploration of unexpected programme outcomes and triangulation of findings. The survey revealed that receiving a voucher was the main implementation bottleneck. Although few caregivers received vouchers (11.5%, CI [9.7, 13.6]), one‐fourth received MNP by bypassing the voucher system (26.3%, CI [23.6, 29.0]). Caregivers' narratives indicated that caregivers were motivated to redeem vouchers but encountered obstacles, including not knowing where or how to redeem them or finding MNP were not available at the shop. Observing these challenges, many programme implementers redeemed vouchers and distributed MNP to caregivers. Virtually, all caregivers who received MNP reported ever feeding it to their child. This study's findings are consistent with other studies across a range of contexts suggesting that although programmes are generally effective in motivating initial use, more attention is required to improve access to MNP and support continued use.

, 2016). MNP interventions have been implemented in a wide range of countries using different combinations of delivery models, platforms, and channels, illustrating their potential for MNP distribution (Reerink et al., 2017). However, there is insufficient evidence to guide effective, context-specific delivery of the intervention in different systems, cultures, and communities (Reerink et al., 2017). More investigations to examine implementation are needed in order to understand what happened in a programme and how that could affect programme outcomes or impacts (Vossenaar et al., 2017).
In Mozambique, two third (68.7%) of children 6 months to 5 years suffer from anaemia (Ministerio da Saude [MISAU]  In two districts of Sofala Province, Global Alliance for Improved Nutrition (GAIN), Save the Children (SC), and Population Services International (PSI) supported MISAU to implement a trial using a voucher distribution system. Caregivers of children aged 6-23 months were provided with IYC feeding counselling as part of ongoing MISAU and SC programmes and vouchers that could be redeemed without cost for MNP called VitaMais at vendors of PSI's existing sales platform.
The logic behind the design decision was that using a market-based platform would alleviate the burden on the public health supply system while increasing accessibility of MNP through distribution points located in communities where caregivers live. Testing the use of vouchers was also relevant to inform the feasibility of using them to subsidize MNP cost (rather than provide MNP free of cost upon redemption).
A formative process evaluation was conducted by GAIN with technical assistance from the Division of Nutrition, Physical Activity, and Obesity, Centers for Disease Control and Prevention to assess the viability of the specific model implemented in Mozambique to deliver MNP and motivate adherence to recommendations regarding its use.
To illustrate how the delivery of the programme would result in continuing and appropriate use of MNP, we developed a programme impact pathway (PIP; Habicht & Pelto, 2012) through consultation with programme staff and an extensive literature review on factors affecting MNP adherence (Tumilowicz, Schnefke, Neufeld, & Pelto, 2017). The PIP is composed of two systems: (a) a programme delivery system including the processes in which the intervention is delivered to the household and (b) a household delivery system that consists of all steps needed for a child to consume a biologically impactful dose. Figure 1 presents the flow of the PIP in the form of programme component exposures and actions by the caregiver. The PIP describes the transmission of MNP and behaviour change communication (BCC) activities from the programme delivery system to the household delivery system and the sequence of activities to prepare food with MNP and feed it to the child.
We identified two main questions to be addressed in the evaluation: (a) What proportion of caregivers received the intervention and adhered to MNP recommendations, and (b) how did delivery and utilization processes affect programme outcomes, including coverage and utilization of MNP. Examination of these two questions required a complex study design and mixed methods. Therefore, we created two streams of inquiry: (a) an examination of programme outcomes using cross-sectional survey data and (b) a focused ethnographic study (FES) to investigate delivery experiences and MNP use from perspectives of caregivers, programme managers, and implementers (Cove & Pelto, 2010;Pelto, Armar-Klemesu, Siekmann, & Schofield, 2013).
The central purpose of ethnography is to obtain the emic view-the insider's perspective-that allows for the discovery of conditions and behaviours that are not foreseen (Pelto & Pelto, 1978). The difference between ethnography and other social science methods of investigation is that ethnographers aim to discover what people do and why, then use what is learned to build theories rather than assess the validity of preconceived theories (Schensul & LeCompte, 2012). In this paper, we focus on what we unexpectedly learned through the mixed methods formative evaluation design about the delivery and redemption of vouchers and how and why programme delivery unfolded as it did primarily from the ethnographic study results.

| Description of programme delivery system
The trial aimed to deliver one voucher every 6 months redeemable for 60 sachets of MNP to approximately 20,000 children 6-23 months,

Key messages
• The novelty and complexity of the voucher system impeded delivery of micronutrient powders (MNP) to caregivers.
• Dedicated and resourceful programme implementers found strategies to overcome challenges, primarily through eliminating the need for caregivers to use vouchers.
• Most caregivers who received MNP initiated feeding it to their child, a much smaller proportion continued to use MNP.
• Findings point to the importance of supporting frontline workers in communities to carry out nutrition promotion activities.
• The capital of the province is in Beira, and most of the district is considered urban or peri-urban. The estimated population of Dondo was 161,752 (Instituto Nacional de Estatística, 2013). Dondo is adjacent to Beira, and the main highway from the coast to Zimbabwe transects the district. Along the highway, communities are peri-urban, but a majority of the population resides in rural areas. MNP are locally branded as VitaMais (produced by Piramal, India) and contain 15 vitamins and minerals including iron and zinc at~1× the recommended nutrient intake (FAO/WHO, 2004).
MISAU with the support of partners integrated instructions on MNP use into existing national IYC feeding materials, which were originally adapted from UNICEF's Community-IYC Feeding Counselling Package (UNICEF, 2013). Using these materials and additional protocols how to distribute and redeem vouchers, GAIN trained "master trainers" from MISAU, SC, and PSI, who subsequently trained programme implementers during week-long sessions in the districts.
Free vouchers for VitaMais were distributed at health centres and in communities to caregivers of children 6-23 months. In Beira, vouchers were distributed to caregivers by health workers, promoters, and community nutrition volunteers, called activistas in Portuguese.
The promoters and activistas in Beira were recruited by SC to support the VitaMais trial. The SC promoters were paid a minimum monthly salary and worked at the health centre, whereas SC activistas were volunteers and worked in communities. SC promoters worked alongside District Directorate of Health of Beira City (DDS acronym in Portuguese) health workers to distribute vouchers for VitaMais. SC promoters also provided interpersonal counselling on VitaMais and IYC feeding, gave group education sessions (palestras), and cooking demonstrations to caregivers at the health centre. Supervised by SC promoters, SC activistas conducted the same activities at the community level.
In Dondo, implementation was solely conducted by the District Services of Health, Women and Social Action (SDSMAS, acronym in Portuguese) without support from SC. Vouchers and BCC activities (i.e., interpersonal counselling, palestras, and cooking demonstrations) were delivered by health workers in health centres and SDSMAS activistas (unpaid volunteers) at the community level. The SDSMAS activistas were already actively supporting health services, and the tasks related to the MNP trial were added to their existing activities.
There were far fewer SDSMAS activistas active at the community level compared with SC activistas in Beira.
In both districts, caregivers were instructed to feed food mixed with one sachet of VitaMais to the child 6-23 months every day for 2 months and to return to the health centre or activista for a new supply after 6 months. BCC activities consisted of interpersonal counselling, palestras, and cooking demonstrations; no mass media communication or community activations activities took place.
The only MNP distribution in Beira and Dondo was through distribution of free vouchers redeemable for VitaMais. Caregivers could redeem vouchers in local commercial shops that were registered with PSI and identified with the brand Troca Aki (translated "Exchange Here"). PSI registered 44 Troca Aki vendors in Beira and 21 in Dondo, predominantly located near health centres and to a lesser extent in outlying communities. Upon redemption of the voucher, each caregiver was provided with three boxes containing 20 VitaMais sachets.
There were two types of vouchers distributed (by the same system previously described): paper and electronic. The paper vouchers were physically given to the caregivers. The electronic vouchers were sent using SMS when the caregiver sent a code to the PSI electronic platform, called Movercado. The Troca Aki vendor received payment after sending the serial number of the paper or electronic voucher to the Movercado platform, which indicated that boxes of VitaMais had been distributed. VitaMais was the only available MNP in the districts and only available through voucher redemption at the Troca Aki shops.
Troca Aki vendors were trained on the supply chain procedures and voucher redemption but received minimal orientation on recommendations regarding the appropriate use of VitaMais.

| Cross-sectional survey
The survey was conducted during January and February 2017, approximately 12 months after initiation of MNP distribution. A structured interview with precoded questions was used to collect data on MNP coverage and adherence outcomes, caregiver exposure to the elements of the programme delivery design and perception-of-use factors, and variables that could have affected programme delivery exposure or adherence outcomes. Sampling was done in two areas: Beira district and Dondo district. The sample size calculation (N = 600) was driven by assuming a 70% VitaMais coverage rate with a ±8.5% relative precision (or a ±6% absolute precision) while assuming a design effect of 2.0 and a 75% response rate. The Mozambique National Institute of Statistics, which has an electronic file consisting of enumeration areas (EA) created for the 2007 national census, selected 40 EA in each stratum (district) where the intervention was being implemented. For Beira district, all 40 EA were selected using population proportion to size (PPS). In Dondo district, 30 EA were initially selected using PPS, and then later, another 10 were selected by PPS. Prior to the next stage of sampling, a household listing was carried out in each selected EA to identify all potentially eligible children 6-23 months. The resulting child lists in each EA were used in the next stage of sampling, which consisted of a simple random selection of 15 children 6-23 months old in each EA. The overall individual response rate was 98.4%, with 99.3% of invited respondents providing questionnaire data in Beira and 97.4% in Dondo. The data are representative of children 6-23 months in Beira district, but we treated the sample in Dondo as a convenience sample as all 40 EA were not selected by PPS at the same time during that stage of sampling.

| Focused ethnographic study
The FES was conducted during November and December 2016, approximately 11 months. After initiation of MNP distribution to learn from the perspectives of caregivers, programme managers, and programme implementers (i.e., MISAU health centre staff, SC promoters, activistas, Troca Aki vendors). We used different data collection protocols for each respondent type as described below.

| In-depth interviews with caregivers
Among caregivers, we used in-depth interviews with open-ended, guided questions, which were administered with extensive probing to expand and interpret the initial responses. The interview protocol is described in an accompanying paper of this supplement (Schnefke et al., 2019). Interviews were conducted in caregivers' homes, their language preference (Sena, Ndau, or Portuguese), and averaged 2-3 hr each. Three health areas per district were selected based on reported low, medium, and high voucher redemption according to the Movercado platform data. Two communities in each of the six health areas, the closest and furthest in distance from the health centre, were chosen to conduct caregiver interviews. Respondents were randomly selected from lists of caregivers of children 6-23 months who had been registered by MISAU or SC as receiving VitaMais within 1-3 months prior to the interview date. The sample design required filling respondent categories based on subgroups (age of child and experience with VitaMais). We aimed to interview a total of five caregivers in each community: three caregivers (two with a child 6-11 months and one with a child 12-23 months) who reported were currently using VitaMais (referred to in this paper as "Continuing Users"); one caregiver of a child 6-23 months who reported to have discontinued using (referred to as "Non-continuing Users"); and one caregiver of a child 6-23 months who reported not redeeming vouchers (referred to as "Non-redeemers"). Fifty-nine of 60 (98.3%) caregivers invited to participate agreed to complete the interview; a single caregiver classified as a non-redeemer was unable to attend the interview and was not replaced. Table 2 presents the number of respondents per respondent type (as described above) and a summary of selected sociodemographic characteristics of the respondents.

| Semistructured interviews with programme managers
Semistructured interviews were completed between November and December 2016 with 17 programme managers as described in Table 3

| Surveys with programme implementers
A structured interview with precoded questions was conducted between November and December 2016 among 168 programme implementers in Beira (n = 103) and Dondo (n = 65), as described in Table 3. The respondents included MISAU health centre staff, SC promoters, MISAU/SC activistas, Troca Aki vendors, and community leaders. All of the programme implementers working in the six health a Wealth index such as described by Shea and Johnson (2004).
b Household hunger scale as described by Ballard et al., (2011). c WHO infant and young child feeding indicators (WHO et al., 2008). d Proportion of children 6-23 months who receive foods from four or more food groups the previous day. Food groups include: (a) grains, roots and tubers, (b) legumes and nuts, (c) dairy products (milk, yogurt, and cheese), (d) flesh foods (meat, fish, poultry, and liver/organ meats), (d) eggs, (e) vitamin-A rich fruits and vegetables, and (f) other fruits and vegetables. Diversity scores for breastfed and nonbreastfed children should not be directly compared because breastmilk is not "counted" in any of the above food groups.
e Proportion of breastfed and nonbreastfed children 6-23 months who receive solid, semisolid, or soft foods the minimum number of times or more the previous day. Breastfed children who received solid, semisolid, or soft foods at least twice a day for children 6-8 months and at least three times a day for children 9-23 months and nonbreastfed children 6-23 months who received solid, semisolid, or soft foods or milk feeds at least 4 times a day.
f Proportion of children 6-23 months who receive a minimum acceptable diet (apart from breast milk). This composite indicator defined as breastfed children 6-23 months who had at least the minimum dietary diversity and the minimum meal frequency during the previous day and nonbreastfed children 6-23 months who received at least two milk feedings and had at least the minimum dietary diversity not including milk feeds and the minimum meal frequency during the previous day.
areas selected for the caregiver interviews were invited to participate.
One hundred sixty-eight of 173 (97.1%) agreed to complete the interview; two Troca Aki vendors were unable to attend the interview and were not replaced.

Quantitative data analysis
Quantitative data were double entered into a customized data entry programme and examined for missing values and outliers. The final dataset was converted to IBM SPSS Statistic, Version 22.0 for Is biological mother of index child (n, yes) 28 28 Schooling of caregiver (n) Illiterate 0 3 Primary education (grades 1-8) 16 20 Secondary education (grades 9-12) 13 7 Household size 5.8 ± 1.9 (3-11) d 6.1 ± 1.5 (4-9) d a Caregivers who reported currently using VitaMais (received VitaMais via vouchers or an alternative pathway and are currently using VitaMais or finished their supply).
b Caregivers who reported to have discontinued using (received VitaMais via vouchers or an alternative pathway but stopped using it). c Caregivers who reported not redeeming vouchers (received vouchers but did not redeem them and never received VitaMais via an alternative pathway).

| RESULTS
The results of the cross-sectional survey revealed bottlenecks in the transfer of messages, vouchers, and VitaMais between the programme delivery system to caregivers. About one third of caregivers in Beira and about four in 10 caregivers in Dondo had not heard about VitaMais (Figure 2). A small proportion of caregivers reported having received a voucher (16.8% in Beira and 6.8% in Dondo), making that step the largest programme implementation bottleneck.
However, the survey also revealed that, unexpectedly, many caregivers received VitaMais through mechanisms that bypassed the voucher system (22.5% in Beira and 29.6% in Dondo). Virtually, all caregivers who received VitaMais reported ever feeding it to their child.
We had assumed that the accessibility of vouchers and VitaMais would be improved through activistas located in communities where caregivers live, but health centres were the most commonly accessed distribution point ( 3.1 | Theme 1. Caregivers were motivated to redeem vouchers and initiate feeding VitaMais to their children because of knowledge of its benefits and authority of health centre staff, SC promoters, and activistas Although VitaMais was a new product largely unknown to both programme implementers and caregivers before the programme,  Caregivers reported receiving the product without a voucher from nurses and SC promoters at health centres and activistas during home visits. By distributing VitaMais directly to caregivers, programme implementers helped caregivers to overcome challenges to redeem vouchers as the following quotes demonstrate: The activistas came to do this program, but when they arrived I told them "I have the voucher but I haven't yet been to the market to exchange it." They said "… That's all right … we'll give it to you here so then you can go and give it to the child." … They took it out and gave it to me. I took the voucher and handed it over.  reported having had at least one stock-out of VitaMais. The most com-  (Reerink et al., 2017), and their knowledge and skill were frequently cited as critical for adherence . Moreover, MNP distribution through frontline workers in the community compared with health centres may be conditionally more cost-effective for achieving higher coverage and adherence outcomes, as was recently demonstrated in Uganda (Baker & Vosti, 2018).
Over the last 15 years, MISAU signed codes of conduct for unpaid, volunteer activistas with donor agencies and nongovernmental organizations (Ministerio da Saude [MISAU], 2000[MISAU], , 2005 and revitalized its paid community health worker programme (MISAU, 2010). However, both activistas and community health workers in Mozambique still face challenges in carrying out their activities (Ndima et al., 2015). Many activistas interviewed for this study stated that they faced financial difficulties and requested payment for their services. Considering the low overall voucher and VitaMais coverage and ingenuity of activistas to overcome barriers with the distribution system demonstrated in this evaluation, there may be a critical role and potential for frontline workers in Mozambique to undertake health and nutrition promotion activities. However, more support is likely required to sustain their work.
Mozambique's public health supply chain frequently suffers stockouts of essential medicines (Salomão, Sacarlal, & Gudo, 2017), as do other resource-constrained systems (Mukasa, Ali, Farron, & Van de Weerdt, 2017). The delivery of VitaMais during this short project (15 months) aimed to alleviate the burden of adding another product to this system by using vouchers redeemable through a market-based platform. Other countries have shown success with similar arrangements to deliver health products, such as insecticide-treated nets for malaria prevention in Tanzania (Kramer et al., 2017). In addition to overcoming supply chain constraints, vouchers are also a means to subsidize health goods and services (Brody, Bellows, Campbell, & Potts, 2013). Although the VitaMais voucher system faced difficulties, voucher programmes have shown promise in other contexts where the investment and willingness to make necessary adjustments based on initial challenges can be ensured. The Tanzania National Voucher Scheme required years and considerable course corrections to programme design to achieve high coverage (Kramer et al., 2017). For example, if further assessment shows that the benefits of not using the public health supply system outweigh the costs of sustaining it, activistas could continue to redeem vouchers for caregivers or the role of the Troca Aki shop could be changed to supply the health centre rather than the caregiver directly. Programmes need time and resources to apply lessons learned and iterative improvement processes to take place.
In retrospect, we see that the study could have been strengthened by sequencing the cross-sectional survey and FES rather than conducting them simultaneously. Had the cross-sectional survey results been available before designing the FES, it would have been possible to more fully explore the population's response to the voucher system and the motivations of programme implementers in their activities to circumvent direct household voucher redemption.
For example, we do not have sufficient evidence to more fully explore how demand (in addition to supply) for the product facilitated or inhibited voucher redemption. Conversely, carrying out the FES before the cross-sectional survey could have allowed for quantitative assessment of the frequency of behaviours and circumstances uncovered in the caregiver narratives. The sampling approach of caregivers interviewed as part of the FES, selecting only caregivers who had received vouchers or VitaMais, also limited data collection about why one third of caregivers never heard about the programme.
Using an ethnographic, mixed methods approach in the formative evaluation allowed us to explore unexpected programme outcomes and triangulate findings. Ethnographic methods resulted in key learning about caregiver's attempts to redeem vouchers and how the programme design can be improved. It is still rare in implementation research in nutrition to find evaluations of MNP programmes or international nutrition programmes more broadly, using ethnography to study population responses to interventions Tumilowicz, Neufeld, & Pelto, 2015). We hope this example will encourage others to make use of this powerful tool and the value of mixed methods designs to build knowledge for programme improvement.