Economic evaluations of onchocerciasis interventions: a systematic review and research needs

Abstract Objective To provide a systematic review of economic evaluations that has been conducted for onchocerciasis interventions, to summarise current key knowledge and to identify research gaps. Method A systematic review of the literature was conducted on the 8th of August 2018 using the PubMed (MEDLINE) and ISI Web of Science electronic databases. No date or language stipulations were applied to the searches. Results We identified 14 primary studies reporting the results of economic evaluations of onchocerciasis interventions, seven of which were cost‐effectiveness analyses. The studies identified used a variety of different approaches to estimate the costs of the investigated interventions/programmes. Originally, the studies only quantified the benefits associated with preventing blindness. Gradually, methods improved and also captured onchocerciasis‐associated skin disease. Studies found that eliminating onchocerciasis would generate billions in economic benefits. The majority of the cost‐effectiveness analyses evaluated annual mass drug administration (MDA). The estimated cost per disability‐adjusted life year (DALY) averted of annual MDA varies between US$3 and US$30 (cost year variable). Conclusions The cost benefit and cost effectiveness of onchocerciasis interventions have consistently been found to be very favourable. This finding provides strong evidential support for the ongoing efforts to eliminate onchocerciasis from endemic areas. Although these results are very promising, there are several important research gaps that need to be addressed as we move towards the 2020 milestones and beyond.


Introduction
Human onchocerciasis, also known as 'river blindness', is a parasitic infection caused by the filarial nematode Onchocerca volvulus. It is transmitted by the bites of Simulium blackflies. Of the 120 million people at risk, 99% live in sub-Saharan Africa, although the disease has also been endemic in six countries of Latin America and is present in Yemen [1]. Symptoms can include severe itching, disfiguring skin conditions, visual impairment, and permanent blindness [1]. Onchocerciasis is the world's second leading infectious cause of blindness after trachoma [2]. Onchocerciasis can also cause excess mortality [3][4][5] and is associated with epilepsy, nodding 788 syndrome and hyposexual dwarfism (Nakalanga syndrome) [6][7][8].
During the last 40 years, there has been a remarkable expansion of onchocerciasis control programmes worldwide [9][10][11][12][13][14], summarised in Box 1. These programmes have had a large impact on reducing onchocerciasis as a public health problem. Initially, control programmes (i.e. the Onchocerciasis Control Programme in West Africa (OCP)) only used large-scale vector control as the drugs available at that time were too toxic for large-scale use. This changed in 1987, when ivermectin (produced under the brand name Mectizan â ), a drug suitable for mass treatment, was registered for human use against onchocerciasis, and large-scale chemotherapeutic control programmes became feasible [11].
In 1987, Merck & Co. committed to supply ivermectin to onchocerciasis endemic countries 'as much as necessary for as long as necessary' [15,16]. Over 7.8 billion ivermectin tablets have been donated for the treatment of onchocerciasis and lymphatic filariasis [17]. This unprecedented donation allowed onchocerciasis control in endemic areas where vector control was not feasible or too expensive to sustain and ivermectin began to be distributed in the late 1980s. Initially, mobile teams of paid, local health professionals were used to distribute ivermectin; however, this was costly [18], and programmes mostly switched to using volunteer distributors and community-directed approaches (Box 1).
Motivated by the successful elimination of the infection in some foci of Mali and Senegal [19,20], there was a shift in onchocerciasis control policy in Africa, with the aim of programmes changing from elimination as a public health problem to elimination of transmission and ultimately infection. In 2012, the Joint Action Forum of the African Programme for Onchocerciasis Control (APOC), chaired by WHO and the ministers of health of endemic countries, set the target at elimination in 80% of African countries by 2025 [21]. WHO's roadmap on neglected tropical diseases (NTDs) also included goals for the elimination in several African countries by 2020 [22].
The coverage of NTD mass drug administration (MDA) programmes has notably expanded over the last 10 years. In 2017, 1.762 billion treatments were delivered worldwide [23]. NTD control programmes are becoming increasingly integrated, moving towards targeting multiple diseases within a multipronged programme [24,25]. In May 2016, the WHO launched the Expanded Special Project for Elimination of Neglected Tropical Diseases (ESPEN), a five-year project to provide national NTD programmes with technical and fundraising support to help them accelerate the control and elimination of NTDs amenable to preventive chemotherapy, namely onchocerciasis, lymphatic filariasis, schistosomiasis, soiltransmitted helminthiases and trachoma [26]. The integrated and cross-sectorial response to NTD control is relevant to the Sustainable Development Goals, including among others Universal Health Coverage, Water Box 1 Summary of the control programmes The Onchocerciasis Control Programme in West Africa (OCP, 1974(OCP, -2002: The OCP was launched in [1974][1975], and originally covered a core area in seven countries of West Africa [11]. However, by 1990 the OCP had expanded its operations to include larger zones and four additional countries following the southern and western extensions [10,11]. From 1974 to 1988 the OCP focused on a strategy of weekly aerial larviciding of blackfly breeding sites. In 1987 ivermectin was registered for human use against onchocerciasis, and due to the suitability of this drug for mass treatment, large-scale chemotherapeutic control programmes became feasible [10]. Large-scale mass drug administration (MDA) of ivermectin began in the OCP regions in the late 1980s, initially administered by mobile teams of paid, local health professionals [10].
African Programme for Onchocerciasis Control (APOC, 1995(APOC, -2015: The APOC was initiated in 1995 including 19 African countries [11,149]. The programme pioneered a community-directed treatment approach, within which the local communities rather than health services directed the treatment process; the treatments were delivered by volunteer community-directed distributors (CDDs) [149][150][151]. APOC gradually expanded and by 2014 it had a network of over 699 656 volunteer CDDs [151]. When the programme concluded at the end of 2015 it was supporting onchocerciasis control and elimination activities in 31 African countries (including the 19 original signatories of the Memorandum, South Sudan, and the 11 ex-OCP participating countries) [149].
The Onchocerciasis Elimination Program for the Americas (OEPA, 2002-present): The OEPA started in 1992 in six countries of the Americas (across 13 discrete foci) [11,12]. The strategy is based on the biannual (twice a year) distribution of ivermectin, to all endemic communities (covering at least 85% of the eligible population) [12]. As of December 2016, a total of four countries have successfully completed the World Health Organization process for verification of elimination [152].
Sanitation and Hygiene (WASH) initiatives, global partnerships, alleviating poverty and hunger and improving education and economic growth [27].
The aim of this paper was to provide a systematic review of economic evaluations that have been conducted for onchocerciasis interventions, to summarise current key knowledge and to identify research gaps in this area.

Search strategy
A systematic review of the literature was conducted on the 8th of August 2018 using the PubMed (MEDLINE) and ISI Web of Science electronic databases. Variants of the following search terms were used to find relevant papers: river blindness, onchocerciasis, cost(s), cost-benefit, cost-effectiveness, economic(s), economic evaluation. No date or language stipulations were applied to the searches. A more detailed summary of the search terms and the PRISMA checklist are supplied in Appendix S1. The titles and abstracts of all the identified papers were examined initially for relevance and then the bibliographies of papers suitable for inclusion were scanned for studies not originally retrieved from the databases. The full selection process is outlined in Figure 1. Studies relating to cost recovery and willingness to pay were not explicitly included as an outcome of the systematic literature search (but are referenced within the review where relevant).

Results and discussion
We identified 14 primary studies reporting the results of economic evaluations of onchocerciasis interventions, which are described in Tables 1 and 2. It is important to note that both the OCP and APOC expanded over time (Box 1) and it was not always clear which specific countries/areas were being included in the different analyses.

Estimated cost of onchocerciasis interventions
A key component of any cost-benefit or cost-effectiveness analysis (Box 2) of an intervention is its estimated cost. The studies identified used a variety of different approaches to estimate the costs of the investigated interventions/programmes. Many of the studies based their costs on programme budgets/reports and very few were based on comprehensive costing studies/approaches. There was variation in what types of costs were included within the analyses.
The estimated intervention costs tended to be higher for the studies evaluating OCP activities. This is likely because the control measures used by the OCP (vector control and mass treatment delivered via mobile teams) were more expensive than the community-directed treatment approach subsequently used by other programmes [28] (Box 1). The nominal financial cost (i.e. not adjusted for inflation or discounted) of the OCP  was just under US$1 billion [29] whereas the projected cost of the larger APOC  was US$478 million [30].
Often it was difficult to compare the reported costs from the different studies, as it was not always clear how costs were estimated, what activities were costed, and whether reported values were discounted or not. Some studies appeared only to adjust the costs for inflation and not discount them (Box 3).
The studies evaluating programmes using communitybased mass treatment generally assumed delivery costs of around US$0.50 per treatment. This is consistent with the findings of a systematic review by Keating et al. [31], in which the average of identified onchocerciasis-specific cost estimates was US$0.46 per treatment, as well as with recent MDA cost benchmarks estimated by the WHO [32,33]. However, it is important to note that the costs of MDA delivery vary across different settings. An important driver in this variation is the size of the targeted population [34][35][36]. This is because MDA programmes generate economies of scale; as the number of people treated increases, the cost per treatment tends to decrease [37,38]. NTD control programmes have also become increasingly integrated and instead of using separate disease-specific programmes they now often target multiple diseases within one programme. This can result in economies of scope, reducing the overall cost of the NTD interventions [31,[37][38][39][40][41][42][43]. Most of the studies identified did not consider the potential impact of economies of scale or scope on the costs of the interventions within their analyses.
There are very few studies that have investigated the costs of alternative onchocerciasis interventions to annual community-directed MDA (with ivermectin) [32]. Turner et al. [44] found that the yearly cost of a communitydirected ivermectin treatment programme increases by 50-60% when increasing the treatment frequency from once to twice a year.
Several studies also considered the potential for costrecovery/cost-sharing, where the communities contribute financially to the cost of the programme [28,[45][46][47] and the participants' willingness to pay for ivermectin treatment [48,49].
Economic costs vs. financial costs. The majority of the studies considered only the financial costs of the intervention. However, when performing economic evaluations of healthcare interventions, it is typically recommended to use 'economic costs' [50,51] (Box 2). These conceptualise costs in a broader way than do financial costs and represent the full value of all resources used for an intervention, including the value (opportunity cost) of donated resources. Economic costs are important as they reflect the sustainability and replicability of interventions.
Ndyomugyenyi et al. [34] found that when accounting for the salaries of governmental personnel and the opportunity cost incurred by the volunteer community-directed distributors (CDDs -Box 2), community-directed treatment with ivermectin in Uganda cost US$0.78 per treatment (2004 prices). However, if these costs were excluded, the cost fell to just US$0.17 per treatment (2004 prices). The difference between the financial and economic costs was smaller (US$0.39 vs. US$0.45 (2011 prices)) in a study in Ghana [44].
A key economic cost for many MDA programmes is the value of the unpaid contribution of the volunteer CDDs. Turner et al. [52] found that the average economic costs relating to the volunteer CDDs unpaid time can be significant, varying between US$0.05-0.16 per treatment (cost year variable). They estimated that the time donated by volunteer CDDs to APOC (1997APOC ( -2015 would be valued between US$60-90 million [52].  [16]. Based on these different assumed valuations of ivermectin this donation would be valued between US$4-11 billion.
In the context of these economic evaluations, it should be noted that it is difficult to estimate the true economic value of a donated drug [54]. This is because the manufacturing costs of drugs are proprietary information, and the donating company may potentially mitigate some of the cost through charitable tax write-offs as well as intangible benefits, such as enhanced public image among shareholders and employees [55]. It is also argued that if ivermectin were not donated, it could potentially be procured from other sources at less than the proprietary cost. For example, Hernando et al. [55] estimate the price of an annual 9 mg ivermectin treatment with generic drugs on the global market [56] to be approximately US$0.78. Based on this, they estimated the direct cost for the tablets donated during 2005-2011 to be US$600 million, compared with the stated value of US$3.8 billion [55], and with potential tax write-offs their net cost could be around US$180 million [55].
These arguments need to be interpreted with caution. Firstly, there is the issue of drug quality, which is a major challenge in low-income countries, and especially with anthelmintics and similar popular drugs which commonly attract counterfeit manufacturers. Secondly, the donations themselves have a chilling effect on the markets, driving generic prices downwards. There is in fact, no established way to estimate the 'real' market price of these drugs, and it is notable that even the lowest estimates of the value of the donations are very substantial, measured in hundreds of millions of dollars per year. What is clear is that the ivermectin donation was the first substantial donation of its type, and its success led to the concept of the NTDs as solvable diseases of poverty, and to the massive donation of some 1.5 billion treatments for a range of NTDs by some 14 other pharmaceutical companies during the London Declaration [57].
The foundation of onchocerciasis control programmes is based on the commitment of ivermectin donation from Merck & Co. for as long as needed [17,57]. Therefore, it is debatable when the value of ivermectin should be included within an economic evaluation.

Estimated economic benefits and cost-benefit analyses of onchocerciasis interventions
We identified several cost-benefit analyses of onchocerciasis interventions ( Table 1). The estimated internal rate of return (IRR -Box 2) ranged from between 11-20% for the OCP and 17-24% for APOC (for comparison, an IRR above 10% is considered by the World Bank as the standard for a successful public health programme [58]) ( Table 1). The estimated net present values (NPV) were between US$8 million to US$3.7 billion for OCP and US $54-307 million for APOC (cost years variable). However, there was variation between the different studies regarding the time frame considered and the discount rate used (Box 3), making it difficult to compare the results directly. None of the cost-benefit analyses identified included the economic value of the donated ivermectin tablets in their base case analysis (Table 1). Waters et al. [18] highlighted that this value could outweigh the estimated economic benefits from both the OCP and APOC programmes.
More recently, Redekop et al. [59] and Kim et al. [60] quantified the economic benefits associated with onchocerciasis elimination. Both those studies found that eliminating onchocerciasis would generate billions in economic benefits (Tables 1 and 3).
The identified studies quantified three different types of economic benefits of onchocerciasis interventions (namely, productivity gains, land use gains and reductions in outpatient services and out-of-pocket expenditures), but there was variation across the studies regarding how and which economic benefits were quantified ( Table 3). Many of the broader benefits of onchocerciasis in terms of the Millennium Development Goals are highlighted by Dunn et al. [61].
Productivity gains. Onchocerciasis-associated morbidity can affect an individual's economic productivity. Lenk et al. [62] recently conducted a systematic review of the productivity losses related to the NTDs eligible for preventive chemotherapy and a summary of the studies they identified for onchocerciasis is presented in Table 4. Unsurprisingly, the degree of the productivity loss was dependent on the type of onchocerciasis morbidity. Productivity losses were highest for blindness and lowest for skin disease.
A key component of estimating the economic benefits of onchocerciasis interventions is the productivity gains that result from preventing onchocerciasis morbidity. The cost-benefit analyses identified appeared to estimate only the productivity gains associated with preventing onchocerciasis-associated blindness. However, the productivity losses associated with onchocerciasis-associated skin disease and visual impairment are also significant ( Tables 4 and 5). Consequently, only quantifying the productivity gains associated with prevented cases of blindness underestimates the economic benefits of onchocerciasis interventions [63].
In contrast, both Kim et al. [60] and Redekop et al. [59] included the productivity gains associated with preventing onchocerciasis-associated skin disease when  • The relative total cost of using moxidectin vs. ivermectin was considered for different programmatic scenarios ¶.
• Used the healthcare providers perspective (not including the value of the donated drugs).
Based on a costing study in Ghana [44].
Annual moxidectin treatment would achieve similar reductions in programme duration as using biannual ivermectin treatment.
If the moxidectin tablets were donated its use would lead to substantial incountry cost savings.
NA NA APOC, African Programme for Onchocerciasis Control; NA, not applicable; OCP, Onchocerciasis Control Programme in West Africa; pOTTIS, provisional operational thresholds for treatment interruption followed by surveillance. †Information for this study was taken from Waters et al. [18]. ‡The estimated economic benefits are outlined in further detail in Table 3.
§The Control, Elim I and Elim II scenarios are described in Kim et al. [99]. ¶Assumed that MDA would be stopped (determining the programme duration) once the pOTTIS would have been achieved (defined as the modelled microfilarial prevalence being <1.4%, measured just before the next treatment round). • Based on the estimated number of blindness cases prevented.

794
• Assumed that one blindness case results in 23 years healthy life lost in hyperendemic and 20 in mesoendemic areas.
• Assumed that blindness is associated with a disability weight of 1. • Financial costs from the healthcare providers perceptive.
• Details not specified.
• Based on the estimated number of blindness cases prevented.
• Assumed each case of blindness results in 20 • US$13.4 per healthy life-year added. • Incremental to annual treatment: • US$0.07-0.13 million per 100 000.
• Assumed that once the pOTTIS was achieved, MDA would be stopped ‡.
• Economic cost from the healthcare providers perspective (not including the value of the donated ivermectin).
• Based on a costing study in Ghana [44]. • Estimated using a dynamic transmission model (EPIONCHO).
• Included the excess mortality associated with heavy infections [83].
• Incremental cost- †Information for this study was taken from Waters et al. [18]. ‡Assumed that MDA would be stopped (determining the programme duration and its total cost) once the pOTTIS would have been achieved (defined as the modelled microfilarial prevalence being <1.4%, measured just before the next treatment round). §Three different endemicity levels were explored (ranging between 40-80% microfilarial prevalence).

Box 2 Glossary
Cost-benefit analysis: A type of economic evaluation which compares the cost of an intervention to its monetary benefits. The results are typically expressed as an internal rate of return or net present value.
Cost-effectiveness analysis: A type of economic evaluation in which the cost of an intervention is compared to the quantity of a single non-monetary effectiveness measure (such as the number of deaths or cases averted). This avoids the issues associated with monetising the benefits of healthcare interventions. The results are expressed as a cost per unit of outcome (see cost-effectiveness ratio).
Cost-effectiveness ratio: A statistic used to summarise the cost-effectiveness of an intervention. It is calculated by dividing the cost of an intervention by its effectiveness measure, such as a cost per disabilityadjusted life year (DALY) averted or healthy life year gained. An incremental cost-effectiveness ratio is calculated by dividing the difference in costs by the difference in effectiveness outcomes of two alternative options (it summaries the 'extra cost per additional unit of effect gained').
Community-directed distributors (CDDs): Also referred to as community drug distributors, these are volunteers selected by their communities to distribute treatment.
Disability-adjusted life years (DALYs): A measure of disease burden that is calculated as the sum of the years of life lost due to premature mortality and the years of healthy life lost due to disability. The number of years of healthy life lost due to disability is calculated using a disability weight factor (between 0 and 1) that reflects the severity of the disease/disability. One DALY can be thought of as one year of 'healthy' life lost.
Economic costs (opportunity costs): These define the cost of a resource as its value in its next best alternative use (also known as an opportunity cost). This is a broader conceptualisation of a resource's value than its financial cost, as it recognises that using a resource makes it unavailable for productive use elsewhere. The rationale behind economic costs is that they are intended to represent the full value of all the resources used for an intervention, and they account for the fact that resources can have a value that is not (fully) captured by their financial costs (such as the 'free' use of building space provided by Ministries of Health, and the unpaid time devoted to mass drug administration by volunteer CDDs). This is particularly important when considering issues related to the sustainability and replicability of interventions.
Economies of scale: The reduction in the average cost per unit resulting from increased production/output; in this case, the reduction in the cost per treatment as a result of increasing the number of people treated.
Economies of scope: The reduction in the average cost per unit resulting when providing multiple goods/ services jointly; in this case, the reduction in the cost per treatment when delivering more than one intervention at once (e.g. integrated control programmes or using the CDD platform to deliver more than a single intervention). Examples include administering treatment for both schistosomiasis and soil-transmitted helminthiases within the same programme (instead of by separate vertical programmes).
Financial costs: The actual expenditure (i.e. the amount paid) for the goods, resources and services that are purchased.
Friction cost approach: The approach that takes the employer's perspective for valuing lost productivity, and therefore only counts as lost, the hours not worked by a sick employee before another employee takes over the work [65]. It is based on the assumption that an ill individual will eventually be replaced by another healthy workertherefore, the initial productivity levels are restored after this 'friction period'.
Human capital approach: The approach that takes the patient's perspective for valuing lost productivity and therefore counts all the work they miss, as a productivity loss. With this approach, all potential production not performed by an individual because of morbidity or premature mortality is counted as a production loss [65].
Indirect costs (productivity costs): Indirect cost represents the value of productivity losses that result from illness, treatment, or premature death.
Internal rate of return (IRR): The discount rate applied to the monetised benefits and costs of an intervention, that makes its net present value equal to zero. Also known as the economic rate of return (ERR).
Mass drug administration (MDA): The large-scale distribution of drugs to eligible people within populations at risk of infection, irrespective of current individual infection status, i.e. without the need for screening for or diagnosing infection prior to each treatment round.
Net present value (NPV): The difference between an intervention's monetised benefits and its cost. A positive NPV is an indicator of a successful investment.

Box 2 (Continued)
798 quantifying the economic benefits of onchocerciasis interventions. Kim et al. [60] assumed that severe itching was associated with a 19% productivity loss and this made up 65% of their projected income/productivity gains. In Redekop et al.'s [59] study, the benefits associated with preventing skin disease represented 22% of the total projected economic benefit. This assumed that 'moderate' skin disease is associated with a 10% productivity loss and 'mild' skin disease is associated with no productivity loss (Table 5).
For these types of estimates, it is important to consider which method is used to value the productivity gains. Estimating the income of individuals affected by NTDs is challenging, as many of them are in informal employment (such as subsistence farmers). The analyses identified used a variety of different methods and sources to approximate the typical income of someone with onchocerciasisassociated morbidity (such as the per capita gross domestic product (GDP), subsistence wage, the GDP per capita of the lowest income quintile). In some cases, the income source was not clearly stated. It is important to note that different income sources can give very different estimates of an individual's typical income, even when they relate to the same type of profession/socioeconomic status [40].
Kim et al. [60] also considered the potential income losses of the informal caregivers of patients with low vision and blindness (Table 5). Currently, there are very few primary data quantifying this. Ibe et al. [64] found that within their survey of onchocerciasis patients in Nigeria, the average productivity cost among the informal caregivers was US$3.50 per month (cost year unavailable).
In all studies productivity gains were estimated using the human capital approach, which takes the patient's perspective for valuing lost productivity. It should be noted that if the friction cost approach was used (which takes the employer's perspective, i.e. only counts as lost, the hours not worked before another employee takes over the patient's work [65]), the estimated productivity gains would have been lower [66]. There is continued debate within the field regarding which approach is most appropriate [65]. In the context of studies for NTDs, it should be highlighted that the friction cost approach is hard to apply to populations that are predominately in informal employment. Kim et al. [60] and Kim & Benton [58] made adjustments for employment rates/labour force participation within their estimates.
Land use gains. Onchocerciasis caused many people to abandon the fertile river valleys within the countries covered by the OCP [2]. It has been estimated that as a result of OCP interventions, 25 million hectares of abandoned arable land was able to be reclaimed for settlement and cultivation [58,67] (capable of feeding 17 million people annually [67,68]). Several of the costbenefit analyses of the OCP included the projected economic benefits resulting from the increased agricultural output from the reclaimed land. These estimates varied between US$57-205 million (cost years variable) and were sensitive to the assumed time horizon and the discount rate ( Table 3). The estimate from Kim & Benton [58] was even higher but the specific value was not stated (the NPVs (1974-2012) relating only to land use gains ranged between US$3154 million (using a 3% discount rate) and US$380 million (using a 10% discount rate)). It should be highlighted that such calculations are based on various assumptions surrounding rates of agricultural land use and repopulation of the reclaimed areas (Table 3). It is also important to consider that onchocerciasis may not have been the sole cause of depopulation of these areas [9]. Because the same level of abandonment of river valleys was rarely seen in non-OCP countries [63], the analyses of the countries covered by APOC generally did not include the economic benefits related to land-use gains. One exception is the report by Haddix [69], which is reported to have reevaluated APOC interventions including the estimated economic benefit resulting from increased agricultural output (Table 3).
Outpatient services and out-of-pocket expenditure. Individuals with onchocerciasis-associated morbidity may seek care from their local health services. Kim et al. [60] projected the economic benefits of onchocerciasis elimination resulting from the decreased use of outpatient health services. They estimated that between 2013 and 2045, when compared with a control scenario, the elimination of onchocerciasis in Africa would save the health systems US$35.9-38.6 million in outpatient service costs, and save the patients/households US$24.7-26.0 million in Perspective: The viewpoint from which the intervention's costs and consequences are evaluated. When adopting the healthcare providers perspective, the costs falling outside the healthcare sector are ignored. In contrast, when adopting the societal perspective, all relevant cost categories should be included, including those incurred by the patients.
Time horizon: The time horizon for the analysis; the duration over which outcomes and costs are calculated.

Box 2 (Continued)
out-of-pocket payments (2013 prices) [60]. Primary data on these costs or on how often individuals with onchocerciasis seek outpatient care or use local health services are scarce. An exception is the study by Ibe et al. [64], which found that the average direct cost incurred by onchocerciasis patient's per outpatient visit was US $14.00 (cost year unavailable), with the majority of this being for medications.
Interestingly, a study conducted by the World Bank and WHO reported that on average, people suffering from the manifestations of onchocerciasis-associated skin disease were found to spend an additional US$8.10 (cost year unavailable) on health-related expenditures over a six-month period compared with those from the same community without these manifestations [70].

The cost-effectiveness analyses of onchocerciasis interventions
We identified seven studies performing cost-effectiveness analyses of onchocerciasis interventions. The majority of the studies evaluated annual MDA ( Table 2). As with the cost-benefit analyses, variation in the time frames and discount rates used (Box 3) make it difficult to compare directly some of the results.
Many older studies have used 'healthy life years averted' as their effectiveness metric (which was based on reductions in the number of onchocerciasis-related blindness cases). These studies generally assumed that blindness results in complete disability, and that each year lived with blindness is equal to one full healthy life year lost, i.e. they assumed a disability weight of 1, which is equivalent to death (Table 5). One exception to this was Evans et al. [71], who used a lower disability weight for blindness of 0.5 (based on empirical evidence that blindness does not result in complete disability and that blind people are active both socially and economically). These studies did not appear to quantify the averted burden of other types of onchocerciasis morbidity (such as skin disease) ( Table 5).
More recent studies have used disability-adjusted life years (DALYs) averted, a standardised and more comprehensive effectiveness metric. When excluding the economic value of the donated ivermectin, the estimated cost per DALY averted of annual MDA varies between US$3 0% discount rate 3% discount rate 10% discount rate

Box 3 Discounting
Healthcare interventions typically incur costs and generate health outcomes over a number of years. However, society does not place an equal value on costs or health outcomes that occur now compared to those that occur in the future. This is because there is an opportunity cost to spending money (as it could be invested to yield returns) and a desire to have benefits now rather than in the future. Economic evaluations therefore need to weight differently costs and health outcomes that occur in the future. Discounting is the process used to convert costs or health outcomes occurring in the future into a present value [153][154][155]. It makes costs and benefits occurring in the future worth less than those in the present. This allows the comparison of the costs and outcomes occurring over different time periods. The discount rate determines the strength of the time preferencethe higher the discount rate the lower the value placed on future costs/outcomes. Note that adjusting for inflation (which accounts for the fact that the purchasing power of a currency changes over time) is not the same as discounting.
When different analyses have used different discount rates, it makes the results harder to compare them directly. The WHO's guide to cost-effectiveness analysis recommends using a 3% discount rate for both costs and health outcomes (and testing the sensitivity of the results to using a 0% discount rate for health effects and a 6% discount rate for costs within the sensitivity analysis) [50]. These recommendations have engendered greater consistency in the use of discount rates. However, although using the 3% discount rate has become the standard, there is still debate within the field, particularly on discounting of health effects [154,155].
• The productivity gains were valued based on the GDP per capita of the lowest income quintile §.
-APOC, African Programme for Onchocerciasis Control; GDP, gross domestic product; OCP, Onchocerciasis Control Programme in West Africa. †Information for this study was taken from Waters et al. [18]. ‡Further detail regarding how the productivity gains were calculated are provided in Table 5. kThe Control, Elim I and Elim II scenarios are described in Kim et al. [99].    [54]. The costeffectiveness of onchocerciasis related MDA is also very favourable compared to other interventions conducted in low-and middle-income countries (a comprehensive list of cost-effectiveness estimates for a range of health interventions in these settings is provided within Horton et al. [72]). The most favourable cost-effectiveness estimates relate to interventions in savannah settings [73]. It is important to note that these estimates are not directly generalisable to onchocerciasis interventions in forest areas, where ocular pathology and morbidity is considered to be rarer ( Figure 2) (but see [74]). According to this assumption, intervention cost-effectiveness in the latter areas would be lower. It should be noted that this assumption is based on limited data from forest settings ( Figure 2).

802
The estimated cost-effectiveness ratio was also dependent on the assumed pre-control endemicity level, with the cost per DALY averted being lower in higher endemicity settings. In the long term, this range is narrower than might be expected as, although fewer DALYs are averted in lower endemicity settings, the total cost of the intervention is also lower (as fewer treatment rounds will be needed to achieve a similar degree of control or to reach elimination) ( Table 2).
The majority of studies have taken the healthcare providers perspective, which does not consider the costs falling on those outside of the healthcare sector. All the costeffectiveness analyses appeared to consider only the cost of the intervention, i.e. none of them included potential savings to outpatient services, prevented out-of-pocket costs, or productivity gains that result from prevented morbidity. Only a few identified studies considered the economic value of the donated ivermectin. Including this significantly increases interventions estimated cost and, therefore, decreases the estimated cost-effectiveness. For example, Turner et al. [54] found that the estimated cost per DALY averted increased from US$3-15 to US$29- DALYs, disability-adjusted life years; GBD, Global Burden of Disease Study. †Reflect the severity of the disease sequelae with 0 representing perfect health and 1 representing death. ‡Used a weight representing an overall average for skin disease across more finely disaggregated strata/severity levels. §Used a weight representing an overall average for visual morbidity (i.e. the weight was not stratified by 'low vision' and 'blindness').
Where relevant additional studies that were not performing economic evaluations were included for comparison.
133 (2012 prices) when including the economic value of donated ivermectin (which would still be classed as cost effective). It is also debatable when the value of ivermectin should be included within an economic evaluation, particularly under the healthcare providers perspective. Numerous approaches have been used to quantify the effectiveness of onchocerciasis interventions. Many older studies based the effectiveness on reductions in the incidence of blindness on limited empirical data and projected similar putative reductions with continuing intervention. More recent studies have used mathematical transmission models to project the ongoing future effectiveness of interventions more accurately, explicitly modelling disease dynamics through time [75]. The two main models used for this purpose (EPIONCHO and ONCH-OSIM) are described in Bas añez et al. [76]. An important advantage of this approach is the capacity to account for the indirect benefits/herd effects of interventions (the indirect benefit afforded to individuals not directly targeted by an intervention that arises from the population-wide reduction in transmission) [75]. These models can also account for longer time horizons, accounting for the continued benefits of interventions even after the control programme has stopped.
DALY calculations. DALYs are calculated as the sum of two components; the years of healthy life lost due to disability, and the years of life lost due to premature mortality [77]. In a DALY calculation, the years of healthy life lost due to disability are calculated using standardised disability weights, ranging between 0 and 1. This reflects the severity of the different disease sequelae, with 0 representing perfect health and 1 representing death. The disability weights used for onchocerciasis DALY calculations have changed over time (Table 5).
For the 2010 GBD study, the disability weights for vision loss were notably decreased compared to previous studies, whereas the weights for skin disease were increased (Table 5). Consequently, Coffeng et al. [78] found that when using the newer weights, the estimated number of DALYs averted by APOC (2000)(2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009)(2010)(2011)(2012)(2013)(2014)(2015) increased by 9% compared to when using the GBD 2004 weights. Moreover, skin disease, instead of eye disease, became the most important contributor to the burden of onchocerciasis. Since the GBD 2010 study, onchocerciasis-associated skin disease has not been assigned a specific single disability weight, and more general disfigurement health states (stratified by three severity levels and whether or not the disfigurement is associated with itch or pain) are used. When using these updated GBD disability weights, studies have typically estimated an overall average skin disease weight (Table 5). Subsequent GBD studies have made additional changes to the DALY calculation for onchocerciasis, particularly relating to the weights attributed to the different types of skin disease Forest/mixed settings Figure 2 The relationship between the prevalence of onchocerciasis-associated blindness and the prevalence of skin microfilariae in savannah (a) and forest/mixed forest-savannah settings (b). The figures were adapted from Figures S3 and S4 in Coffeng et al. [30]. The data were originally taken from [142][143][144][145][146][147][148]. [Colour figure can be viewed at wileyonlinelibrary.com] and the types of skin disease included [79]. A more detailed overview of the changes to the GBD study methodology used to calculate DALYs is presented in [80][81][82].
Several studies have accounted for the excess mortality associated with onchocerciasis visual morbidity [3] within their DALY calculations. However, it has generally not been considered that irrespective of visual morbidity, there is an increased risk of mortality associated with increasing microfilarial load, particularly in children and young adults aged below 20 years [4,5]. Estimates of onchocerciasis burden (and the cost-effectiveness of its control) would generally increase if this onchocerciasisassociated excess mortality were taken into account. For example, Turner et al. [83] found that, depending on the pre-control endemicity level, excess mortality accounted for 29-43% of the estimated pre-control DALY burden of onchocerciasis. If this had not been included, the estimated cost-effectiveness would have been reduced substantively.
Cost-effectiveness thresholds. In cost-effectiveness analyses, the cost per DALY averted is compared to a willingness to pay threshold to determine whether an intervention is cost effective. However, the most appropriate cost-effectiveness thresholds are debated and remain somewhat arbitrary [84][85][86]. The cost-effectiveness threshold set by the WHO-CHOICE [87] (a cost per DALY averted < 3 times the country's GDP per capita) is now considered to be too high [84-86, 88, 89]. Most analyses within the NTD field have not used it [40,41,90], many opting instead for the more conservative costeffectiveness threshold set by the World Bank [91] (≤ US $251 per DALY averted, when adjusted for inflation to 2016 prices [92]). Interestingly, recent analyses have indicated that a cost-effectiveness threshold closer to < ½ the country's per capita GDP would be more appropriate for low-income countries [88,93]. For comparison, the Disease Control Priorities project (Third Edition) used a threshold of US$200 per DALY averted to identify priority interventions for consideration in low-income countries [94]. Despite this ambiguity on cost-effectiveness thresholds, onchocerciasis interventions would remain classed as cost effective by any of the proposed measures.

Elimination and evaluation of alternative interventions
Traditional cost-effectiveness analyses of new interventions evaluate their incremental effectiveness and incremental cost compared to the current practice (calculating incremental cost-effectiveness ratios, Box 2). This framework has been widely and successfully used to evaluate the comparative cost-effectiveness of new intervention strategies for disease control. However, this incremental cost-effectiveness framework is less informative and somewhat ill-suited for disease elimination or eradication programmes. For example, Turner et al. [73] found that that increasing the treatment frequency of ivermectin distribution from once to twice per year yielded very small incremental health gains (only a 3-4% increase in the number of DALYs averted) but could have a large influence on a programme's overall total cost and duration ( Table 2). In such cases, where an intervention is aimed at accelerating and sustaining elimination, an incremental cost-effectiveness ratio may not reflect its true value. Instead, as applied in Turner et al., the absolute cost of the intervention and the time it takes to achieve the desired elimination goal can be more informative. The same framework was applied to an economic evaluation of moxidectin [95] (Table 1), a newly registered treatment for onchocerciasis (https://www.medicinesdevelop ment.com/news-180613.htm). Kastner et al. [96] have also highlighted that the number of DALYs averted may not fully capture the long-term consequences and broader benefits of disease eradication programmes.
Another important aspect to consider when evaluating elimination programmes is the time horizon of the analysis [40]. This is because the costs of elimination programmes are typically higher than would be needed for disease control [97]. After elimination is certified, the estimated cost-effectiveness of an elimination programme will steadily increase as the discounted benefits continue to accumulate but the costs have stopped (with the potential exception of ongoing surveillance). The benefits and potential future cost savings resulting from achieving elimination/eradication are not infinite [40,73], as the costs/cost savings being considered must be restricted within a suitable time horizon and are typically discounted into the future (Box 3). In contrast, for disease control programmes, the intervention costs will typically be incurred for the full-time horizon. Because of this, elimination programmes can be cost-saving in the long term. However, it can take time for the longerterm benefits and cost savings associated with achieving elimination to outweigh the initial increases in costs associated with achieving elimination. Consequently, with short-term time horizons, elimination programmes are unlikely to be more cost effective than disease control programmes.
Eradication investment cases have been developed for a number of NTDs [98], including onchocerciasis [53,60,99]. These latter studies have compared onchocerciasis control and elimination/eradication scenarios and have quantified the duration of the programme, its financial and economic cost, the number of ivermectin treatments required, the workload of the community healthcare workers/volunteers, costs related to outpatient healthcare services, and the productivity gains resulting from preventing onchocerciasis-associated morbidity. The overall conclusions of these studies are that eradication and elimination of onchocerciasis are both justifiable on both cost-effectiveness and benefit-cost analysis grounds, and that eradication tends to be more favoured over the longterm time horizon.

Sensitivity analysis
The majority of identified studies either did not perform a sensitivity analysis or undertook only a univariate analysis, changing one parameter at a time to evaluate its impact independent of other parameters. The main exceptions to this were Redekop et al. [59] and Kim et al. [60].
Redekop et al. [59] performed probabilistic sensitivity analysis (PSA) in which the values of three input parameters (the estimates of disease prevalence in 2010, the percentage of productivity loss/the amount of out-of-pocket payments, and the patient's income) were allowed to vary simultaneously. Kim et al. [60] first conducted a one-way deterministic sensitivity analysis to examine which parameters are key drivers. They then conducted PSA to assess the robustness of the results to the joint uncertainties around all selected parameters [60].

Limitations of this analysis
A potential source of bias of the search strategy is that it did not capture economic evaluations published outside of the searched electronic databases (i.e. grey literature such as policy documents/reports, and many non-English language publications etc.). Efforts were made to minimise this bias by searching the bibliographies of selected studies. There could also be a degree of publication bias, with economic evaluations with negative or unfavourable results less likely to be published. It should be noted that the selection of studies was not performed independently by two researchers.

Areas of further research
The results of the systematic review highlight that the standard onchocerciasis control strategies are consistently found to be very cost effective. However, there are some important research gaps that need further research.
Data on intervention costs and the economic burden of onchocerciasis. The costs of MDA delivery have been shown to vary across different settings [31,32,37,41,100]. This variation potentially affects the generalisability of any cost-effectiveness/cost-benefit analysis [75], and future studies need to quantify the impact of this in greater detail. It should be noted that the costs of conducting MDA in areas where onchocerciasis and loiasis are co-endemic may be higherdue to the need for enhanced surveillance and community sensitisation.
In addition, further studies are needed to investigate how integrating NTD control programmes [31,39] may influence the costs/cost-effectiveness of implementing different control strategies [37,40]. There is notable variation in the methodological approaches used to quantify the economic costs incurred by CDDs. It would be beneficial if future studies adopted more consistent approaches (outlined in [52]).
The cost of onchocerciasis control programmes will likely increase significantly as they approach the 'last mile' towards elimination [37]. This is partly because of the increase in the costs resulting from expanding the programmes to target harder-to-reach areas/groups (diseconomies of scale) [37]. This is an important issue for NTD programmes in general, and further costing studies are needed to quantify it.
Further studies are needed on quantifying the medical costs incurred by those with onchocerciasis seeking treatment. Such studies could yield more robust estimates of the economic benefits of onchocerciasis control [60]. It would also be beneficial if future studies sought to quantify the productivity losses incurred by informal caregivers, not just those caring for the blind. Note also that with the rise in sophistication of health systems in the endemic countries, the health system costs and the contribution of out-of-pocket expenses are both likely to rise [101].
Our analysis also revealed that studies used cost data collected in different years, but it was not always clear if and how costs were adjusted for inflation. Future studies should report this more explicitly [102] and within a given study the costs would be standardised to a consistent year.
Economic evaluations of alternative interventions. In certain epidemiological and programmatic circumstances, alternative strategies to annual MDA will be required to achieve the current goals for onchocerciasis control/elimination [21,22,103]. Such strategies include increased frequency of MDA (up to four times per year), localised low-cost vector control and treatment strategies using moxidectin, anti-Wolbachia therapies and new macrofilaricidal drugs [104][105][106]. However, there currently are very few costing studies and economic evaluations relating to these alternative interventions [32,107]. Studies 808 are needed to evaluate the cost and cost-effectiveness of such strategies. When such studies are performed, it will be vital that the generalisability of the estimated cost across different programmatic settings is considered [37]. It will also be important to consider the value of these alternative interventions not only in reducing the disease burden where they are implemented, but also in their capacity to help eliminate onchocerciasis more quickly. This will be particularly important in reducing the risk that 'hot spots' of sustained transmission seed and reestablish transmission in areas where onchocerciasis has been eliminated.
This area of research is particularly important for interventions targeting onchocerciasis in Loa loa co-endemic areas. It has been recently predicted that, at the 2025 horizon, two-thirds of onchocerciasis remaining cases will be living in hypoendemic areas for onchocerciasis where the risk of Loa-related post-ivermectin severe adverse events (SAEs) has so far been considered to outweigh the benefits for the communities. The paradigm shift from control to elimination implies that hypoendemic areas start receiving community treatment with ivermectin. A test-and-not-treat strategy for onchocerciasis has recently been successfully piloted in a health area of Central Cameroon where community-directed treatment with ivermectin had to be permanently halted after a series of SAEs occurred in 1999 [108]. Costing studies of this approach are ongoing. Critical questions when addressing the cost-effectiveness of this intervention include the counterfactualwhat it would cost to leave all those populations untreated?and cost-effectiveness relative to other locally important health issues.
Future economic analyses of alternative interventions need to be tailored to the key policy questions from the different decision makers and stakeholders. These different questions may require different methodological approaches and perspectives for the analyses. This further emphasises the need for transparent reporting of methodology in economic evaluations.
Quantifying the health benefits of interventions. Many older studies only quantified the health and economic benefits resulting from the number of blindness cases averted. More advanced disease models have been developed that account for averted visual impairment, skin disease, and excess human mortality. However, further refinements are needed to better capture the relationship between infection and skin disease [109], and to account for related neurological disorders such as epilepsy and nodding syndrome [6][7][8].
When evaluating an intervention aimed at reducing morbidity, the number of DALYs averted is often the best effectiveness metric, as it allows cost-effectiveness estimates to be directly compared to estimates relating to other interventions/diseases as well as to standardised cost-effectiveness thresholds. This makes the results of economic evaluations easier to interpret by policymakers. However, DALYs do have limitations and there are controversies surrounding their calculation [110]. For example: • The universal disability weights do not account for how the local context may influence the burden of a disease. Consequently, the potential that the burden of a disease or its sequelae may be worse for those that are living in poverty is not accounted for. It has been argued that this aspect of DALY calculations may significantly underestimate the burden of poverty-related diseases [110,111].
• The DALY disability weights do not fully account for the psycho-social implications of a disease or its sequelae [112] and its overall impact on quality of life. They also do not explicitly account for the impact of the disease on patients' informal caregivers. In particular, the updated disability weight for blindness (decreasing from 0.60 to 0.19) has been controversial within the field [113,114]. A possible reason for this significant change is that within the updated GBD framework (post-GBD 2010), the disability weights are intended to be solely measures of losses of 'optimal health' and are not intended to represent losses of well-being/welfare [80,115].
Interestingly, the same age group (those aged below 20 years) for which there is a statistically significant higher risk of mortality for a given microfilarial load in comparison to those aged 20 years and older [5], is the group with the onset and higher incidence of nodding syndrome, a type of epilepsy that is increasingly recognised as associated with onchocerciasis [116,117]. Preliminary studies of the disease burden of onchocerciasisassociated epilepsy have been conducted [118]. These studies should be followed by economic evaluations of both onchocerciasis-associated epilepsy and the impact of onchocerciasis interventions.
Joint/auxiliary benefits. Ivermectin is a broad-spectrum antiparasitic drug that also has an impact on other coendemic parasitic infections (such as soil-transmitted helminthiases, lymphatic filariasis, loiasis and scabies). Krotneva et al. [119] assessed the auxiliary benefits of APOC and estimated that between 1995 and 2010, ivermectin mass treatment (in APOC regions) cumulatively averted approximately 500 000 DALYs from co-endemic soil-transmitted helminth infections, lymphatic filariasis, and scabies. This highlights that the overall cost-effectiveness of onchocerciasis interventions may be even higher than previously reported. Further quantification of these auxiliary benefits would be useful to improve estimations of the impact of onchocerciasis interventions.
Modelling and elimination thresholds. Dynamic transmission models have an important role, particularly for the evaluation of novel interventions and how they compare to the standard strategy of MDA with ivermectin. These models (reviewed in Bas añez et al. [76]) have undergone extensive refinement in recent years to better capture parasite population dynamics during interventions [120] and to produce more robust projections on the likelihood of elimination. These modelling efforts will be particularly useful in identifying epidemiological and programmatic circumstances in which alternative strategies will be required to reach elimination (for example in highly endemic settings, settings with suboptimal responses to ivermectin [121,122], or where the initiation of programmes has previously been delayed [103]). Moreover, transmission models (as opposed to so-called 'static' models) capture explicitly our current understanding of the changing parasite dynamics during interventions [75].

Conclusions
The cost benefit and cost effectiveness of onchocerciasis interventions have consistently been found to be very favourable. This finding provides strong evidential support for the ongoing efforts to eliminate onchocerciasis from endemic areas.
MDA against other NTDs has also been found to be cost effective [40,54,90] and, therefore, a logical next step would be to quantify the net cost-effectiveness of more closely integrating these programmes, which are already often running side-by-side in co-endemic areas. Indeed, the primary rationale for the aggregation of common infectious diseases of the poor under the denomination of 'Neglected Tropical Diseases' was the perception that treating many diseases using a single common delivery system would be inherently cost effective. It would be important to future policy making to explore the evidence base for that perception. Further systematic reviews of this type on other NTDs would also be useful.
We identify three main research gaps in this area. First, the need to be more inclusive in quantifying burden. Originally, studies only quantified the benefits of preventing blindness, and then also captured onchocerciasis-associated skin disease. An improved understanding of other factors, such as onchocerciasis-associated epilepsy [117,118] would enhance the precision of the calculated benefits. Second, the evaluation of interventions targeting Loa loa co-endemic areas which will become more important in the end-game for elimination. Finally, the need to increase the comparability of economic analyses. Greater adherence to standardised guidelines for reporting the results of economic evaluations (such as CHEERS for cost-effectiveness analysis [123]) would be beneficial and increase the reliability and reproducibility of reported findings.
Programmes to eliminate onchocerciasis have always been in the vanguard of global efforts to eliminate diseases of poverty. Lessons learned here from the economic analysis of onchocerciasis programmes have direct relevance to the design of programmes addressing all the other NTDs.