Incorporating Demand and Supply Constraints into Economic Evaluations in Low‐Income and Middle‐Income Countries

Abstract Global guidelines for new technologies are based on cost and efficacy data from a limited number of trial locations. Country‐level decision makers need to consider whether cost‐effectiveness analysis used to inform global guidelines are sufficient for their situation or whether to use models that adjust cost‐effectiveness results taking into account setting‐specific epidemiological and cost heterogeneity. However, demand and supply constraints will also impact cost‐effectiveness by influencing the standard of care and the use and implementation of any new technology. These constraints may also vary substantially by setting. We present two case studies of economic evaluations of the introduction of new diagnostics for malaria and tuberculosis control. These case studies are used to analyse how the scope of economic evaluations of each technology expanded to account for and then address demand and supply constraints over time. We use these case studies to inform a conceptual framework that can be used to explore the characteristics of intervention complexity and the influence of demand and supply constraints. Finally, we describe a number of feasible steps that researchers who wish to apply our framework in cost‐effectiveness analyses.


INTRODUCTION
As new health technologies are developed, the World Health Organisation (WHO; 2012) considers whether to revise its guideline recommendations. Evidence on efficacy, cost and, in some cases, cost-effectiveness may inform these revisions. Country-level decision makers need to consider whether to adopt revised guidelines and fund the introduction of new technologies (Nasser et al., 2014). Countries may consider the results of economic evaluations performed to inform global guidelines. However, unlike efficacy, cost-effectiveness analyses are not necessarily generalisable beyond a specific setting (Drummond et al., 2005). In theory, each country could conduct its own economic evaluation. However, in practice, such country-specific analyses may be considered too time-consuming and expensive for the majority of low-income and middle-income countries (LMICs). To support the rapid scale-up of new technologies, global health actors have therefore invested in developing user-friendly models that enable decision makers in LMICs to assess the cost-effectiveness of introducing a given new technology in their own setting (Lubell et al., 2008a;Winfrey et al., 2011;Pretorius et al., 2014a;Pretorius et al., 2014b).
Multi-country cost-effectiveness models typically apply efficacy data from trials and adjust for heterogeneity in demography, epidemiology, and unit costs (Lubell et al., 2008a;Winfrey et al., 2011;Pretorius et al., 2014a;Pretorius et al., 2014b). Unit costs are adjusted either through cost models (using an ingredients approach and local input prices) or by using statistical models to extrapolate costs from empirical studies conducted in other settings. Cost and efficacy data are commonly entered as uncorrelated parameters. It is, however, rare for these models to explicitly consider other setting-specific influences on costeffectiveness in LMICs, despite the fact that literature from high-income countries (HICs) identifies a wide range of factors that are important to consider when transferring cost-effectiveness estimates between settings (Drummond et al., 2009). In particular, the use and therefore cost-effectiveness of new technologies may be adversely influenced by constraints to the demand for and supply of health services in which the technology is placed.
This paper aims to encourage the consideration and incorporation of supply and demand constraints in economic evaluations of new health technologies in LMICs. Our objectives are threefold: first, to highlight the importance of considering constraints; second, to conceptualise constraints; and third, to propose pragmatic options for incorporating these constraints in economic evaluations. We begin by presenting a review of economic evaluations of new diagnostics for malaria and tuberculosis (TB) control and describe the extent to which these economic evaluations have considered demand and supply constraints and how these constraints influenced results. These two case studies are then used to develop a conceptual framework for identifying demand and supply constraints. Finally, we discuss the options and challenges in applying this framework going forward.

ECONOMIC EVALUATIONS OF RAPID DIAGOSTIC TESTS FOR MALARIA AND TUBERCULOSIS
In this section, we describe how the scope of economic evaluations conducted on new diagnostics for malaria and TB control evolved to incorporate demand and supply constraints. For this review, we broadly define 'demand constraints' as factors that restrict the individual's ability to seek and use healthcare services in a way that maximises the individual's utility. 'Supply constraints' are defined as factors that restrict the provider's ability to provide these services efficiently and with sufficient coverage and quality to have an optimal impact. We examine constraints that are additional to a payer's budget constraints.
In the case of malaria, we examine rapid diagnostic tests (mRDTs) at point of care. Prior to the rollout of mRDTs, most individuals with suspected malaria had limited access to facilities with microscopy (the previous standard of care) and were treated presumptively. Deploying mRDTs has the potential to reduce the overprovision of expensive anti-malarials and to improve the clinical management of non-malaria febrile illnesses. In the case of TB, we examine the implementation of Xpert MTB/RIF (hereafter referred to as Xpert), a test with improved sensitivity compared with microscopy (the previous standard of care) particularly for those co-infected with HIV and TB . Xpert is also able to identify potential cases of multi-drugresistant TB (MDR-TB).
To identify economic evaluations on mRDTs, we searched PubMed, Scopus and the NHS Economic Evaluation Database for the terms 'malaria' and 'cost' and 'rapid diagnostic'. Citations and bibliographies of studies retrieved and WHO malaria guidelines were reviewed for additional articles published between January 2004 and September 2015. For Xpert, we used two systematic reviews, one of modelling studies of novel diagnostic strategies for TB (Zwerling et al., 2014) and one of TB control costs (Laurence et al., 2015) and then updated them through a PubMed search for the terms 'tuberculosis' and 'cost' and 'Xpert'. Citations and bibliographies of studies retrieved were reviewed for additional articles published up until September 2015. increase in the incremental costeffectiveness ratios for Xpert adoption.

Thailand
(1) Microscopy RDT (OptiMal) was the most cost-effective and policy makers should consider using RDT technology to supplement microscopy in remote non-microscope areas.

Sub-Saharan Africa
(1) Presumptive diagnosis RDTs were cost-effective compared with presumptive diagnosis in most settings and probably cost-effective compared with microscopy, reflecting better accuracy in improved conditions. CE mainly reflects improved treatment and health outcomes of non-malaria febrile illness plus savings in anti-malarial drug costs. Results depend on assumption that providers use test results in treatment decisions.

Mozambique
(1) Presumptive diagnosis RDTs are cost-saving when malaria prevalence is low to medium. Compared with treating patients on the basis of presumptive diagnosis, the use of RDTs results in cost savings only when 29% and 52% or less of all suspected malaria cases test positive for malaria and are treated with artesunate plus sulfadoxine/pyrimethamine (AS + SP) and artemether-lumefantrine (AL), respectively.
These cut-off points increase to 41.5% (for AS + SP) and to 74% (for AL) when the use of RDTs is restricted to only those older than 6 years of age.

2008)
B Decision-analytic model using data on actual clinical practice (provider perspective)

Kenya
(1) Presumptive diagnosis In the high transmission district, RDTs as actually used would improve malaria treatment and lower costs, but the majority of patients with malaria would not be correctly treated. In the low transmission district, RDTs as actually used would yield a minor reduction in under-treatment errors with 41% higher costs. In both districts, adherence to revised clinical practices with RDTs has the potential to further decrease treatment errors with acceptable costs.

Zambia
(1) Presumptive diagnosis RDTs were the most cost-effective method at correctly diagnosing malaria in primary health facilities when compared with presumptive diagnosis and microscopy. Prescribing practices can impact on the potential of RDTs to achieve overall cost savings.

Nigeria
(1) Presumptive diagnosis RDT is cost-effective compared with presumptive diagnosis at malaria prevalence of 43% and therefore suitable in Nigeria.
Cost-effectiveness was affected by malaria prevalence level, ACT adherence level, cost of ACT, proportion of non-malaria febrile illness cases that were bacterial, and microscopy and RDT sensitivity.
(2) RDT (Continues) A. VASSALL ET AL. Decision regarding use of RDTs in these remote areas depends on the current microscopy accuracy in the field.

2010)
A Decision-analytic model based on cohort study data (provider perspective)

Senegal
(1) Presumptive treatment of all fevers RDT use for all clinically suspected malaria and prescribing ACT only to patients tested positive was cost-effective in areas where microscopy is unavailable. Full compliance of providers with RDT results was required in order to avoid severe incremental costs.

Tanzania
(1) Microscopy RDTs reduced drug costs in this setting but did not offset the cost of the tests. Non-monetary benefits included improved management of patients and increased compliance with test results.

2011)
A Decision-analytic model based on trial data (societal perspective)

Uganda
(1) Presumptive diagnosis RDT was more cost-effective than microscopy in both low and high transmission settings and considered cost-effective with respect to presumptive treatment.
(2) Microscopy, (1) Facility (RDT and treatment) Home management of uncomplicated malaria using community health workers was more cost-effective than facility-based management. Utilisation and adherence to clinical guidelines was higher with community health workers than at a health facility.
(2) Home management using community health workers (RDT and treatment) Malaria (Lemma et al.,

Ethiopia
(1) Presumptive diagnosis RDT (pan/pf) was more cost-effective than both RDT (pf) and presumptive disgnosis (2) RDT (pf/pan),  and should be preferred in health posts in rural Tigray and rolled out nationwide.

Senegal
(1) Presumptive diagnosis RDT was cost-effective compared with presumptive diagnosis, but highlight impact of drug stock outs, and improved adherence to RDT results over time.

Brazil
(1) Microscopy Microscopy is more cost-effective than RDT in the remote areas studied if high accuracy of microscopy is maintained in the field.
( RDTs were cost-effective compared with microscopy. (2) RDT (Continues) A. VASSALL ET AL. Six countries in sub-Saharan Africa (1) Presumptive diagnosis A threshold transmission rate exists under which malaria testing is more cost-effective than presumptive diagnosis and treatment.
( (1) Microscopy Introducing RDTs with enhanced training was more cost-effective than with basic training when each was compared with microscopy.

2015)
C Decision-analytic model (provider and patient perspective)

Myanmar
(1) RDT but no supporting intervention Private provider subsidies with IEC or a combination of IEC and financial incentives may be a good investment for malaria control.

Afghanistan
(1) Presumptive diagnosis RDTs were cost-effective compared with microscopy and presumptive diagnosis across the moderate and low transmission settings. RDT remained cost-effective even when microscopy was used for other clinical purposes.
(2) Microscopy (3) RDT Where the Scope of Economic evaluation is defined as (A) Economic evaluations that include attributes of the new technology, setting-specific unit costs and epidemiological considerations; (B) Economic evaluations that include parameters listed in (A) and also potential demand and supply constraints; (C) Economic evaluations that include parameters listed in type B, and the intervention includes activities that address demand and supply constraints.
We identified 27 economic evaluations on mRDTs and 10 economic evaluations on Xpert MTB/RIF (Table I). We categorise them into three groups, according to the way in which they consider demand and supply constraints: A. No adjustment. Economic evaluations that include attributes of the new technology (such as test accuracy/efficacy), setting-specific unit costs, and/or epidemiological considerations (such as disease prevalence) only. B. Adjust for constraints. Economic evaluations that include parameters listed in group A and also explore the influence of setting-specific demand and supply constraints (such as care-seeking preferences, patient adherence to treatment and/or provider practice). C. Address constraints. Economic evaluations that include parameters listed in groups A and B and also consider interventions to address demand and/or supply constraints (such as provider training, patient education and/or incentives).

A. Economic evaluations that do not adjust for demand and supply constraints
Half (14 of 27) of the economic evaluations conducted on mRDT did not adjust for any demand or supply constraints (Table I). Diagnosis using mRDTs is compared with presumptive diagnosis, microscopy or both, and the studies include parameters on the accuracy of the alternative diagnostic methods, combined with setting-specific data on unit costs of diagnosis and treatment and the prevalence of malaria. The costeffectiveness of mRDTs and microscopy varied across the studies and was influenced by setting-specific unit costs and prevalence. While demand and supply constraints were not investigated directly, some studies noted that their findings were dependent on assumptions made regarding the 'field' accuracy of microscopy, prescribers' use of test results and patients' adherence to treatment.
In the case of Xpert, four of the 10 economic evaluations did not explicitly consider supply or demand constraints. In a similar manner to the studies on mRDTs, these studies considered unit costs and difference in epidemiology by setting. While there was no explicit consideration of any constraints, some studies included costs for training and quality control of Xpert (Vassall et al., 2011). Sensitivity analyses were also conducted around the performance of other tests in the diagnostic pathway and the level of HIV care coverage, but no constraints on either provider or patient adherence to the TB diagnostic or the treatment pathway were considered (Abimbola et al., 2012;Andrews et al., 2012;Menzies et al., 2012).
B. Economic evaluations that adjust for potential demand and supply constraints Ten of the economic evaluations conducted on mRDTs explicitly adjusted for demand and supply constraints. One of the first studies explored how provider behaviour can impact on the cost-effectiveness of mRDTs using trial data from Tanzania (Lubell et al., 2008b). The authors concluded that neither mRDTs nor microscopy would be beneficial given the high rate at which anti-malarials were prescribed to patients who tested negative for malaria. This study found that costs incurred would be 10-250% higher (depending on transmission rate) than if prescribing practice were fully consistent with test results. Similar findings were reported in Kenya, where an economic model populated with observed data on provider practice found that introducing mRDTs in a low transmission area would incur 41% higher costs and achieve only a small reduction in under-treatment errors than the standard of care (Zurovac et al., 2008). Other studies investigated the sensitivity of findings to the locally achieved accuracy of microscopy (Harchut et al., 2013).
In the case of Xpert, this category contains two sub-categories of studies. In the first set are three studies, which considered how the organisation of health services may affect the cost-effectiveness of Xpert. These studies explored how the placement of Xpert in sites with different workloads influenced costs (Meyer-Rath et al., 2011), the availability and use of other diagnostic tests (van't Hoog et al., 2013) and capacity constraints at the facility and district level (Lin et al., 2011). The second category of studies evaluate the cost-effectiveness of Xpert during scale-up in 'early adopter' countries using large-scale pragmatic trials. Preliminary results from these studies suggested that effective presumptive TB treatment, lack of access to HIV treatment, high levels of default from the diagnostic process and poor adherence by both clinicians and patients to guidelines following negative test results may substantially reduce the cost-effectiveness of Xpert (Theron et al., 2014a;Theron et al., 2014b;Churchyard et al., 2015). When the results of these trials were taken into consideration, estimates of incremental cost-effectiveness ratios (ICERs) for a range of settings increased by up to 60% (Menzies et al., 2015).

C. Economic evaluations that address demand and supply constraints
Three of the economic evaluations on mRDTs extended the scope of the 'mRDT' intervention to include activities to address demand and supply constraints. For example, in Cameroon, an economic evaluation assessed the introduction of mRDTs with two types of provider training: basic training on revised malaria guidelines and an enhanced training, which supplied information on the guidelines and targeted provider behaviour . The study showed that introducing mRDTs with enhanced training was more expensive than mRDTs with basic training, but also more cost-effective. In Myanmar, researchers evaluated four strategies for introducing mRDTs and concluded that introducing subsidised RDTs in the private sector accompanied by information, education and communication (IEC) and/or a combination of IEC and financial incentives would be cost-effective (Chen et al., 2015).
Only one Xpert study has analysed the cost-effectiveness of activities to address supply or demand constraints. This study examined the combined cost-effectiveness of Xpert with activities to support adherence to the diagnostic algorithm (follow-up of those defaulting, follow-up of negative test results in those with HIV and linkage to HIV care) in South Africa and found much higher ICERs than originally predicted .
In summary, our case studies suggest that failing to consider demand and supply constraints may overestimate cost-effectiveness. Although these case studies do not represent the vast array of new technologies in all LMICs, they are particularly important cases given the substantial investments being made in rapid diagnostics. The inclusion of constraints in both cases substantially influenced estimates of ICERs, and this suggests that constraints should be routinely considered in economic evaluations, at least to the same extent as heterogeneity in epidemiology is considered across settings. It could be argued that evaluating new technologies in the context of demand and supply constraints may 'unfairly' penalise new technologies for factors that are outside their control. However, including the cost of activities that address constraints (or reducing the estimated effect in their absence) rewards more feasible technologies, which is the primary innovation for many new technologies designed for use in LMIC's. As a decision and policy-oriented field, economic evaluation is concerned with minimising the actual opportunity cost in the decision makers' jurisdiction, rather than a hypothetical one.

CONCEPTUAL FRAMEWORK
Informed by the case studies, we consider two approaches to incorporate demand and supply constraints in economic evaluations. The first is to view the technology and activities to address constraints (sometimes referred to as 'critical enablers' (Schwartlander et al., 2011)) as distinct interventions and to assess their costeffectiveness separately. This approach is analogous to those distinguishing between the efficacy of a technology and its effectiveness during implementation in the clinical literature and underpins the economic evaluation approach taken in our case studies (first evaluate based on efficacy, then examine the incremental costeffectiveness of supporting activities).
Alternatively, we propose that researchers should start from the perspective that new technologies are not 'interventions' in themselves. Many new technologies will require some supporting activity to achieve satisfactory efficacy and coverage objectives, given the extensive demand and supply side constraints in LMICs. We recommend that researchers should therefore routinely assume that the introduction of new technologies is likely to be an inherently complex intervention with many component activities. There is a substantive literature on how to determine whether an intervention is complex (Hawe et al., 2004;Shiell et al., 2008). In general, complex interventions are characterised as having several interacting activities and typically require those delivering or receiving the intervention to modify their behaviour. We accept that not all new technologies will fulfil these criteria. The extent to which introducing a new technology requires a complex intervention may depend on the nature of the technology, the extent of constraints, and the scale of the technology required. However, we propose that, given the extent of constraints, complexity needs to be 'ruled out' rather than 'ruled in' in economic evaluations in LMICs.
To assist in this process, we outline in the succeeding discussions a framework for researchers firstly to define the intervention they are evaluating and thereafter to guide their economic evaluation design. Some argue that while complexity may raise additional questions, with adequate time and resources, existing economic evaluation frameworks remain appropriate (Shiell et al., 2008). However, in the case of economic evaluation in LMICs, the extensive task of characterising, identifying and modelling intervention complexity remains a barrier for widespread global application.
Previous work on transferability frameworks from high-income country (HIC) settings identifies a range of factors that influence economic evaluation results by setting, including patient, clinician, healthcare system and wider socio-economic factors (Sculpher et al., 2004;Drummond et al., 2005). The ISPOR Good Research Practice Task Force report on the 'Transferability of Economic Evaluations across Jurisdictions' also identifies similar considerations (Drummond et al., 2009). Many of these setting-specific determinants of costeffectiveness fall within our definition of demand and supply constraints in the context of LMICs. There is also a body of literature from health systems research assessing the relationships between various new technologies, the health system and patient outcomes (Atun et al., 2010a;Atun et al., 2010b); a number of frameworks such as the WHO's 'Building Blocks' that provide insight into the range of supply constraints; and conceptual frameworks examining demand-side constraints for health services (Levesque et al., 2013).
Building on this work, in Figure 1, we present a visual conceptual framework to help researchers identify demand and supply constraints. This conceptualisation promotes a comprehensive description of the interven- tion, including the technology and activities to support its optimal use. The disease/illness/care pathway lies at the centre of the conceptual framework. We depict a simple disease/illness/care pathway that starts from onset of symptoms and continues to care seeking, diagnosis, treatment offered and treatment taken. This form of pathway needs to be specified in sufficient detail to include both disease progression and key decisions taken to seek/provide care. Additional stages could be added, for example, if the diagnostic process involved multiple sequential tests.
Having specified the pathway, including key decision points and options at each stage, the next step is to consider (i) how key decision points to seek/provide care are influenced by the new technology, (ii) the extent to which decisions/options are likely to be constrained by demand and supply factors, and (iii) what actions are needed to address these demand or supply constraints. Specifying the theory of change (De Silva et al., 2014) outlining how the new technology is expected to impact the pathway and outcomes may help in this task.
Some constraints on decision points in the pathway may only become apparent as new technologies are scaled up and may go beyond a direct influence on the technology. For example, while suboptimal provider behaviour may be noticeable at low coverage, more general, human resource constraints may only become apparent at scale. Likewise, on the demand side, uptake of a new technology may be relatively straightforward for users who are frequent health service attendees, but there may be additional costs of ensuring access to hard-toreach groups at higher levels of scale. We have therefore sought to distinguish proximal and distal constraints in our framework. On the demand side, proximal constraints include those that directly restrict access, uptake or adherence to the optimal patient pathway. For instance, care seeking may depend on the ease and cost of accessing care, individuals' awareness of symptoms and treatment options and perceptions about the quality of care. These proximal constraints are in turn influenced by household economics and underlying values and preferences that are shaped by the economic, social and cultural context. On the supply side, proximal constraints may include the knowledge and behaviour of healthcare providers and the availability of complementary resources and services. However, these may be constrained by distal factors, such as overall human resource availability; mechanisms for healthcare financing, planning and management; and the functioning of private sector providers.
We demonstrate how the conceptual framework can be used by applying it to the introduction of mRDTs (Table II). The first column of Table II depicts the different stages in the pathway, and the subsequent columns then consider (i) the key decisions (and options) faced by the patient and provider; (ii) how the new technology should change the pathway; (iii) which proximal demand and supply factors may constrain the use of the technology; and (iv) which interventions may be required to address the demand and supply constraints. For example, mRDTs should extend access to malaria testing and should reduce the number of febrile patients who unnecessarily receive anti-malarials. However, these outcomes may be constrained by the knowledge and preferences of both patients and providers, by patients' ability to access and pay for care, and by whether mRDTs, anti-malarials and treatments for non-malaria febrile illnesses are in stock. Table II has been used in practice. Many of the demand and supply constraints listed here were identified when designing interventions to support the rollout of mRDTs in Cameroon and were underpinned by a theory of change evaluation and formative research . The final column lists activities to address these demand and supply constraints. In Cameroon, the interventions focused on provider knowledge and behaviour Mangham-Jefferies et al., 2014), although elsewhere, activities have aimed to raise community awareness of mRDTs (Onwujekwe et al., 2015) or subsidise the cost of RDTs (Chen et al., 2015).
It should also be noted that we do not recommend that the process of developing Table II is one conducted by the economic evaluator alone. Defining the intervention to introduce a new technology should be a multistakeholder process. It is feasible for economists to work with programme/technology experts to arrive at an outlined Table II before embarking on the model and cost-effectiveness analysis design (Onwujekwe et al., 2015). It should also be noted that the resulting intervention may not address all constraints. Some activities may still be considered by stakeholders as areas to address during implementation. Even in this case, our framework encourages the researcher to consider the full cost of those activities that are agreed and to understand  where the case pathway should be constrained in their analysis, even if the implementation of activities to address constraints is postponed.

APPLYING THE CONCEPTUAL FRAMEWORK
When considering how to account for or address demand and supply constraints, the potential gains from avoiding sub-optimal investment decisions must be weighed against the risk of delaying introduction of technologies with high benefits by waiting for new evidence to emerge from more complex economic analyses. While it may be feasible to work through the framework a priori, the data required to explore such a framework empirically are potentially immense. We identify in the succeeding discussions several approaches to expedite and support the application of this framework in different analytical situations.

Economic evaluations alongside early trials
Work on transferability from HICs identifies various options for using trial data in economic evaluations either when individual patient-level data are available or when the trial took place in the country of interest (Manca and Willan, 2006). In the majority of cases of new technology adoption in LMICs, however, neither of these two situations exists. Nevertheless, early trials of new technologies may still offer some opportunities to inform economic evaluations incorporating constraints. Many trials of new technologies in LMICs are multi-centre and thus provide scope to identify and collect data on location-specific constraints. However, opportunities may be limited as the research process masks a number of constraints that would be observed elsewhere. Some argue that trial sites could be selected to represent the range of settings into which the new technology may be placed, incorporating any prior knowledge about key demand and supply constraints to improve representativeness (Gheorghe et al., 2014). However, given the shortage of 'trial ready' sites in LMICs, and practical issues such as distance and transportation to and from sites in LMICs, there may be a substantial trade-off between speed and expense of trial start-up and ensuring that sites are representative. Despite these limitations, at the very least, detailed descriptive reporting of the cost to ensure the implementation of both the standard of care and the intervention in trials should become standard practice. This would not only help define the intervention but may also provide some information to estimate cost for other settings. This recommendation is supported by the literature on the methodological implications of evaluating complex interventions, which emphasises the need for detailed and standardised reporting about the nature of the intervention and the implementation process (Shiell et al., 2008).
In HICs, considerable attention has been placed on the analysis of patient-level data from trials to explore heterogeneity across locations and individuals (Grieve et al., 2005;Gomes et al., 2012). Where constraints are likely to operate at a sub-group or individual level, this additional data may add considerable insight. However, to date, there have been few trial-based cost-effectiveness analyses that report patient-level data on costs and cost-effectiveness from LMICs. This is likely to be due in part to the substantial costs of data collection involved, where routine systems for resource use are non-existent. Nonetheless, work is beginning to emerge in this area (Barton et al., 2013;Mangham-Jefferies et al., 2014), and this approach offers the promise of adding critical insight into a number of constraints, most notably, demand constraints that may operate at an individual level.

Adapting model-based economic evaluations to consider supply and demand
Given the substantial limitations of trials, it is likely that for some time to come, analysts will continue to use both trial data and models to predict the cost-effectiveness of new technologies in LMICs. In this case, there are several approaches that may be considered. The first option is perhaps the most intensive, viewing demand and supply constraints with the same importance and rigour as any other determinant of cost-effectiveness incorporated in a model. This approach would involve applying our framework and conducting a systematic review on the influence of supply and demand constraints on key pathway probabilities and conducting a wide range of sensitivity analyses. This would serve the purpose of further defining which constraints are most important and could help focus activities to address these constraints.
However, the data scarcity around many of these constraints in LMICs means that populating models with plausible data may be problematic. For example, in our case studies, there was very little prior data available on the levels of presumptive treatment in TB (Theron et al., 2014a). Even if the presumptive treatment rate is identified as a key determinant of cost-effectiveness, it is hard to adjust for presumptive treatment rates in any particular location. It may, however, be feasible to collect additional data as part of model-based evaluations, rather than solely rely on systematic reviews. For example, it may be possible to estimate the costs of activities to address constraints from other new technologies within the same country and conduct small observational studies in areas like clinician behaviour. Such studies may have the wider benefit of ensuring results resonate with decision makers, who may have priors from previous experiences regarding the 'real' costs of implementation in their setting.
Where the literature indicates a constraint may be important but no local data are available, we recommend at the very least that the uncertainty associated with the constraint be communicated with decision makers. There are several examples of this approach emerging in the literature. Recent work by Gomez et al. models shortened TB drugs regimens in a set of scenarios where full guideline adherence by healthcare providers and patients is compared with 'real world' scenarios (Gomez et al., 2015). Another approach to explore constraints is 'cascade' modelling (Hallett and Eaton, 2013), which models the cost-effectiveness taking into account different levels of adherence to each key decision point along the case pathway. The specific costs and effectiveness of bundles of 'enabling' interventions to improve key decision points in the cascade may then be explored. A threshold analysis can then also be used to identify maximum levels of investment to address constraints to the care pathway.
A further emerging approach is to link epidemiological models with bespoke economic or health system models analysing demand and/or supply constraints in a single analytical frame. For example, in one of the Xpert studies cited previously, Lin et al. (2011) linked the results of a setting-specific operational model parameterised with local health facility data on patient flows with a TB transmission model to investigate the influence of capacity constraints on the cost-effectiveness of Xpert. On the demand side, another paper in this supplement (Terris-Prestholt et al., 2016) links the results of discrete choice experiments and new technology uptake models with economic evaluations.

Economic evaluations as part of impact evaluations
The preceding approaches may assist researchers to predict cost-effectiveness prior to new technology introduction. However, substantial implementation research may still be required to understand the full influence of demand and supply factors on the cost-effectiveness of new technologies in LMICs as technologies are rolled out. Economic evaluations alongside implementation research or after scale-up remain scarce. Implementation research-based economic evaluations should be encouraged in order to provide further empirical evidence and deeper understanding of key constraints when assessing the cost-effectiveness of introducing new technologies globally.

CONCLUSION
In conclusion, we argue that economic evaluations of new technologies in LMICs should take a complex intervention perspective and routinely consider demand and supply constraints. If demand and supply constraints are not considered rigorously, local decision makers may consider that the economic evaluations are not relevant to their local context, and scarce resources may be allocated away from more cost-effective interventions. Proposing this paradigm shift, in terms of defining an 'intervention' but considering the substantial empirical challenge, we present a conceptual framework, examples of the framework's application and discuss feasible approaches. We recommend that considering these constraints explicitly be seen as standard 'best practice' in economic evaluations in LMICs, even if the analysis remains partial given the substantial analytical and data challenges. We recommend that demand and supply constraints be considered from the earliest stages of the decision process, and that they be incorporated, applied and evaluated throughout the technology adoption process, with at least as much rigour as epidemiological and price heterogeneity.