Multiple parameter evidence synthesis—a potential solution for when information on drug use and harm is in conflict

Authors


The quality of the evidence on estimating drug-related harm is not yet as advanced as that for intervention effectiveness. Multiple parameter evidence synthesis offers a potential solution, in which ‘all available evidence’ is combined into a single coherent model. We present a case study of estimating the number of people infected with hepatitis C.

A common approach to estimating the amount of drug-related harm [crime, HIV or hepatitis C virus (HCV)] is simply to multiply an estimate of the prevalence of problem drug use (PDU) or people who inject drugs (PWID) with an estimate of the risk of harm [1, 2]. In principle, there is nothing wrong with this approach, but it is not without problems. The estimates being multiplied are subject to considerable uncertainty. PDU prevalence is difficult to measure with any degree of reliability; population surveys undercount and indirect estimation techniques may provide inconsistent and conflicting estimates [3]. In the United Kingdom, estimates of the prevalence of injecting opiate and cocaine use sometimes vary more than twofold—far more than any statistical uncertainty around the individual estimates [4-7]. Further, the methods used tend to lack any ‘formal’ test that can determine which estimate is closer to the true value.

Rather than simply multiplying two estimates together and hoping for the best, a better approach is to compare the estimate of harm derived ‘indirectly’ with any ‘direct’ data that are available on the prevalence of harm. If the two estimates agree, allowing for statistical uncertainty, it is possible to combine all three sources of information. The resulting estimates will probably be more comprehensive and consistent with the available data, i.e. valid.

Bayesian multi-parameter evidence synthesis (MPES) can integrate diverse sources of information into a single coherent model [8, 9]. The methods and philosophy behind this approach are very similar to systematic reviews that pool multiple study effects, and in particular to mixed treatment comparison methods. Mixed treatment studies increase the evidence base by allowing the combination of studies that, for example, measure intervention A versus B directly with those measuring either A versus C or B versus C which, taken together, measure A versus B indirectly [10-12]. Similarly, MPES can be applied to complex networks of ad-hoc epidemiological survey evidence and routinely collected data, which may offer multiple opportunities to check and ‘triangulate’ estimates.

HCV prevalence: case study

In England, two contrasting and competing estimates of the number of people infected with HCV were available: 250 000 produced for the Department of Health or 500 000+ produced for liver charities. Neither estimate was necessarily better or more valid than the other.

Calculating reliable estimates of HCV prevalence or other drug-related harm is complex [13, 14]. No single representative data source of HCV prevalence is available. Instead, partial information is available from multiple sources on the size of the at-risk populations (PWID, ex-PWID, non-PWID) and risk of HCV in different population subgroups (e.g. among antenatal or genitourinary clinic attenders and PWID surveys). There are also ‘mixture surveys’, i.e. surveys including PWID/ex-PWID and non-PWID, but without specific information on the relative proportions or how representative is the survey of the general population. In addition, some data sources such as population surveys of PWID prevalence may provide biased estimates of prevalence but useful information on other functional parameters (such as PWID geographical or age distribution, and ratio of ex- to current PWID). The two rival estimations were based on a mixture of data sources and assumptions. However, in order to account properly for the uncertainty, a model is required that links all the available data sources explicitly and incorporates information on how data sources and quantities relate to each other.

An example is shown in Fig. 1 [14]. The boxes show the data sources. The circles show the quantities to be estimated and the lines indicate relationships and whether the information is ‘mixed’ or believed to be biased. What is critical for the method is that the number of data sources outnumber the number of parameters (quantities to be estimated) in the model [8].

Figure 1.

Direct acylic graph representing multiple parameter estimation model of hepatitis C virus prevalence. PWID: people who inject drugs; STI: sexually transmitted infection

A model is developed that accounts explicitly for the relationship between data sources and potential biases. It jointly estimates the size of the at-risk populations and the prevalence of HCV within the risk groups [13, 14]. In our first exercise, we suggested that the number of people aged 15–59 years with HCV antibodies was 191 000 (95% confidence interval 124 000–311 000), with more than 85% among PWID and roughly equal numbers between current and ex-PWID [13, 14]. This resolved the first conflict between competing estimates. However, another controversy soon arose—whether HCV among migrants may have been underestimated. The MPES framework provides a flexible framework to update estimates through incorporating new information (on HCV risk among non-injectors by ethnic group) [15]. Neither the overall estimate of the number with HCV at 203 000 nor the number of HCV infections among non-PWID at 15 000 changed markedly, but the new estimation suggested that approximately 40% of non-PWID HCV infections were among migrant populations [15].

Implications and future work

MPES has been employed successfully for other infections, including toxoplasmosis and chlamdyia [16, 17], and it is now the key method for estimating the prevalence of HIV/AIDS for policymakers in the United Kingdom (as methods that relied on back-projecting HIV from trends in AIDS cases are no longer reliable) [18, 19]. We see considerable scope for MPES in other areas where there is a lack of clarity over the true size of the risk of specific health problems in the population. This technique may be useful in situations in which multiple sources of information exist, but each provides only partial information and is subject to biases. These situations will be familiar to people working in the addictions field. MPES is time-consuming and technically complex. However, as more exercises are undertaken, quicker solutions to combining data will be developed (as has happened with systematic reviews incorporating mixed treatment comparisons), and we hope that the expectations for the quality of information on drug-related harm informing ‘evidence-based policy’ will be raised.

Declarations of interest

None.

Acknowledgements

We are grateful for funding from MRC NIQUAD Addiction Cluster.

Ancillary