Insight into treatment of HIV infection from viral dynamics models

Summary The odds of living a long and healthy life with HIV infection have dramatically improved with the advent of combination antiretroviral therapy. Along with the early development and clinical trials of these drugs, and new field of research emerged called viral dynamics, which uses mathematical models to interpret and predict the time‐course of viral levels during infection and how they are altered by treatment. In this review, we summarize the contributions that virus dynamics models have made to understanding the pathophysiology of infection and to designing effective therapies. This includes studies of the multiphasic decay of viral load when antiretroviral therapy is given, the evolution of drug resistance, the long‐term persistence latently infected cells, and the rebound of viremia when drugs are stopped. We additionally discuss new work applying viral dynamics models to new classes of investigational treatment for HIV, including latency‐reversing agents and immunotherapy.


| INTRODUC TI ON
HIV, the causative agent of AIDS, infects nearly 40 million people worldwide 1 and represents one of the highest overall global burdens of disease. 2 After an estimated entry into the human population in the early 20th century, 3

Insight into treatment of HIV infection from viral dynamics models
Alison L. Hill 1 | Daniel I. S. Rosenbloom 2 | Martin A. Nowak 1 | Robert F. Siliciano 3,4 1 period of a few weeks to a "setpoint" typically between 10 3 and 10 6 c/mL, where they can remain relatively stable for many years. 9 During this time, viral populations diversify and diverge from the strains that founded infection, 10 often displaying population genetic signs of strong selection. 11,12 CD4+ T cells slowly decrease over the course of chronic infection and eventually become so low (<200 cells/uL blood) that opportunistic infections occur and the individual classified as having AIDS. Early in the epidemic, these characteristic trends inspired the use of mathematical models to understand these dynamics and help generate ideas about how to treat the infection.
Mathematical models are sets of equations or rules that describe how different entities in a system interact and change over time. 15 Different models may consider dynamics at very different scales -from individual molecules to cells to people to countries. Most commonly, models are formulated as systems of nonlinear differential equations or as sets of stochastic reactions constituting a Markov process. Roughly speaking, the use of models in biology can be divided into two cases.
In one scenario, models may be constructed with the goal of explaining patterns that are observed in existing data, perhaps for generating and comparing hypotheses about the mechanisms that lead to the observed data or to estimate values of particular model parameters. While this approach has the advantage of allowing direct comparison of models with data, it has the downside that it is generally always possible to create a model that reproduces observed data, but this does not mean that model is correct or useful. Alternatively, models may be constructed in the absence of directly related data, by starting from a basic mechanistic understanding of the biological processes involved and choosing only the processes considered most critical to the outcome. Values for reaction rates can ideally be taken from direct measurement of individual steps in the process. Constructing such a model is a formal way of integrating often disparate data into a single framework, and can be used to predict the outcomes of studies that have not yet been conducted based on the optimal use of prior information. Ideally, models can be developed and refined by iterating between these two approaches.
In this paper, we will review some examples of how mathematical models have improved our understanding of HIV treatment, including both successes and failures. The models we will discuss are commonly called "viral dynamics" models and track levels of virus and immune cells over time within individual infected people or animals (and thus are often referred to as "within-host" models). A huge amount of other work that will not be discussed here uses "between-host" models to describe how HIV spreads between individuals in a population (eg, . The first half of the paper will focus on antiretroviral drugs, which are still the only approved drugs for treating HIV. The second half of the paper will discuss investigational therapies being tested with the hope that they may one day replace combination antiretroviral therapy (ART) by permanently curing the infection. Many other excellent reviews of viral dynamic modeling of HIV exist in the literature (eg, . Here, we do not attempt to cover the entire field but rather to detail some topics we personally have studied or feel are illustrative examples of these methods.    where V 0 is the viral load at the time of therapy initiation. Thus, the decay dynamics only depend on the lifespans of free virus and infected cells: viral load will decay with the slower of these two values after a shoulder phase approximately equal to the length of shorter lifespan. Since the lifespan of free virus is estimated to be around 1/c ~ 1 h, 26 but the observed decay rate is around 1/d, we must have d I > c and d I ~ 1/d.

| Basic viral dynamics model
When this decay was first observed and interpreted in the context of this model, [29][30][31] it was very surprising that virus-producing cells had such a short lifespan. This lifespan implies that many new cells must be infected each day to maintain setpoint viremia (estimates of d I I at setpoint from the model in Figure 2 19 ), these numbers allow for a tremendous amount of diversity to be generated, explaining the rapid rates of evolution observed.
Despite these and many other insights into HIV infection that have come from the viral dynamics model, it is important to note that the model does make a number of unrealistic assumptions. For example, this model assumes that cells start producing virus immediately upon being infected, whereas in reality a cell must pass through multiple stages of the viral lifecycle before infectious virions are released. Additions to this model include this time delay, [32][33][34] which has many interesting effects, but most importantly, changes the relationship between the early viral growth rate and estimates of R 0 . 7 CD4+ T cells obey very simplified dynamics in these equations, but are actually governed by more complicated homeostatic mechanisms that increase cell proliferation when numbers get low. 35,36 While CD4 + T cell levels can decline dramatically during chronic infection, generally only activated cells are highly susceptible to infection, and only a very small fraction of them are infected at any given time (around 1/1000). 37,38 Including more of these details can improve the agreement between model predictions and observed CD4 counts but still cannot explain the entire progression to AIDS. 39 Infected cells and free virus are not generally cleared at a constant rate throughout infection because they are targeted and cleared by adaptive immune responses that expand in response to infection. Many models of antiviral immunity have been developed to explain different features of infection. 12,19,40,41 Inclusion of immune system effects is needed to reproduce the large drops from peak viremia to setpoint 42,43 and explain patterns of viral evolution (eg, References 40,44,45). When treatment reduces R 0 < 1 in this model, the simplest forms of the model predict that infection will eventually be completely cleared. However, early studies demonstrated that no matter how long antiretroviral therapy is given and plasma viral levels remain undetectable by standard clinical assays, the infection always returns once therapy is stopped. 46,47 This was found to be due to the presence of a "latent reservoir" of integrated proviral genomes in resting memory CD4+ T. These latent genomes are not transcribed into mRNA and translated in protein to complete the viral lifecycle due to the quiescent state of these cells. 48 However, upon cellular activation, transcription and translation can resume. Latently infected cells can persist despite decades of therapy, 49,50 and reactivate later to restart infection. [51][52][53] Consequently, antiretroviral therapy is not curative and currently must be taken for life. Models that include viral latency are now common in studies of both antiretroviral therapy and new curative strategies (Reference 54 and discussed in later section).
Interestingly, many of these more complicated facets of infection can actually be inferred from looking more closely at viral load Further insight has been gained by comparing viral load decay curves in treatment with and without the integrase inhibitor (II) class of drug. Early on after this class was introduced, it was noticed that viral loads became suppressed faster than with reverse-transcriptase (RTI) or protease inhibitor therapy (PI). This was initially taken as evidence that these drugs were more efficacious, but for the reasons detailed above ( Figure 2, lack of dependence of decay curves on drug efficacy), modelers cautioned against this interpretation and hypothesized that the altered kinetics may be due to the later stage in the lifecycle at which the integrase inhibitor class acts. [63][64][65] Recent work by Cardozo et al. 58 used densely longitudinally sampled viral load data 66 during treatment with either (a) 3 RTIs + 1 PI, (b) 1 II, or (c) 2 RTIs + 1 II to compare various models to fit the decay curves. Based on the various alterations in kinetics seen with the II (first phase viral decay separating into two phases, (1a) and (1b), second phase decay occurring later and slower), they identified the model that fit the data best without unnecessary complexity. They concluded that the virus infects two distinct cell subsets, one with a fast rate of integration and another with a slow rate of integration, but that once integration occurs, production of virions occurs with similar rates in each subset. Additionally, their results suggest that the decay curves can only be explained if integrase inhibitors are not 100% effective even at the high concentrations administered, so that some integration proceeds slowly even in the presence of the drug. This agrees with direct measurements of drug efficacy in ex vivo assays (discussed in next section), 67 and could be due to the ability of HIV genes to be expressed at low levels from unintegrated viral DNA. In Figure 3, we show the infection model that has emerged from these combined studies and the decay curves that are produced under different treatment regimes.

| How efficacious are antiretroviral drugs?
HIV drugs rapidly reduce viral loads, but they do not eliminate all of  These results were surprising for a few reasons. First, HIV drug efficacy (as measured by older assays), was previously only reported in terms of the IC 50 , but inhibition at the higher concentrations which are required clinically is highly dependent on the slope as well (Figure 4). The total viral inhibition at clinical drug levels calculated from these assays is higher in drug combinations recommended for first-line treatment and in those that outperform others in head-to-head randomized clinical trials. 67   In situations where drug levels are suboptimal, viral replication can occur and drug-resistant variants can arise. 74 Resistance is not an all-or-nothing phenomenon, and most mutations only confer partial resistance. To quantify the degree of resistance, viruses can be generated in the laboratory with specific suspected drug resistance mutations, and then subjected to the same dose-response curve measurements described above. 75 Overall, the dose-response curve shifts in three possible ways for each resistant strain. In the absence of drug, mutant strains tended to have lower infection rates than wildtype strains. This "cost of resistance" is well-documented in many systems and occurs because of compromises in the function of viral proteins that occur when they undergo amino acid changes to avoid drug effects. [76][77][78][79][80] Since this fitness cost shifts the entire dose-response curve down ( Figure 4C)

| How does antiretroviral efficacy and adherence influence treatment outcomes?
Dose-response curves tell us how much infection is instantaneously Actively infected cells transition into latent infection at rate γ, a is the rate at which latently infected cells reactivate, and d L is the death rate of latently infected cells. Q is a matrix that includes information both on the mutation rate and the genetic structure of the population, ie, is the probability that a cell initially infected by a virion of genotype i ends up carrying genotype j due to mutation during the reverse transcription process is Q ij . The rates governing latently infected cells tend to be much smaller than those for activated cells or virus (eg, d L , γ, a ≪ d I , d T ).
Even without dynamically simulating such a model, important insight can be gained on potential treatment outcomes just from at the relative dose-dependence of mutant and wildtype viral fitness 69,89 ( Figure 4D)  Figure 4C). These proxies are significantly better than simply measuring time-averaged drug concentration, which misses the highly nonlinear relationship between drug levels and viral fitness. However, they still have limited predictive power, since they ignore the fact that resistant strains do not always exist but instead must be generated stochastically via mutation before being available to be selected, and can go extinct if outcompeted temporarily. 69 Consequently, the specific time-course of drug levels can influence outcomes.
More predictive models of viral dynamics under drug treatment can be created by (a) moving from differential equations, which assume populations can be arbitrarily small and all processes occur Beyond the overall adherence level, more detailed characteristics of the drug time-course can influence treatment outcomes. Wahl and Nowak 89 showed that resistant strains are more likely to flourish when drug doses are taken more evenly as opposed to in a more "clumped" fashion, even when the total fraction of doses taken is the same (assuming that resistant strains always exist). When drugs are given in combination, the overlap between missed doses, which can differ depending on whether the drugs are packaged together in a "combo-pill" or allowed to be taken separately, can determine whether or not a drug combination is "resistance-proof". 69 Long-acting therapy, which is taken much less frequently than current daily dosing due to extended half-life formulations, is currently under development, 101 and there are worries it may be more prone to resistance development in the presence of missed doses. Models can be used to explore this possibility, and for preliminary investigation of a once-weekly formulation of the drugs dolutegravir and raltegravir, and suggest failure rates should be similar to daily pills with similar average drug concentrations. 102 The periodic highs and lows of drug levels during regular therapy can also promote resistance in an unexpected way. For example, viral populations may be able to evolve the ability to "synchronize" their lifecycle with the drug period, so that they only undergo a particular lifecycle stage when drug level blocking it is at their lowest, and therefore avoid the drug effect. 34 Whether this effect is responsible for any clinical resistance patterns for HIV is still unknown. There are two basic ideas for how this could be accomplished. One approach, often called a "sterilizing cure", is to purge the body of enough residual latently infected cells that the chance that infection will be rekindled when treatment is stopped is extremely low.

| MODELING NOVEL THER APIE S TO PERTURB L ATENT INFEC TI ON OR BOOS T IMMUNE RE S P ONS E S
Another approach, often called a "functional cure", is to equip the body with the ability to control the infection, rendering small amounts of virus released from reservoirs inconsequential. 105 As was the case for antiretroviral therapy, mathematical models are being used to predict how and when these therapies would work, interpret their outcomes in trials, and help guide drug development efforts (see related reviews 54,106) ( Figure 6).

F I G U R E 6
Schematic of the barriers to HIV cure and conceptual approaches to cure. Combination ART rapidly suppresses viral loads (solid red) to below clinical detection limits, but low-level viremia released from long-lived latently infected cells continues. Whenever therapy is stopped, viral load rebounds (solid red). "Sterilizing cure" approaches aim to reduce or completely clear the latent reservoir, or render cells in it incapable of reactivating (possible infection scenario shown in bottom red dotted line). "Functional cure" approaches aim to equip the body with the ability to control reactivating infection before full-blown rebound occurs (effectively by reducing R 0 < 1) (three possible control scenarios shown in red dotted lines).

| What maintains the latent reservoir and how can we reduce or clear it?
One branch of HIV cure research is focusing on developing therapeutics that can perturb the latent reservoir, ideally reducing its size or activity such that the risk of latently infected cells reactivating and rekindling infection when ART is stopped is removed. 107 In imagining such therapies, researchers have sought to better understand the processes that maintain a nearly stable population of latent cells despite decades of treatment and extremely low levels of detectable virus. The latent reservoir persists mainly as proviruses integrated into the genomes of infected resting memory CD4+ T cells. The frequency of these latently infected cells is around 1 per million cells 53,108,109 (depending on the particular assay used and the requirement for virus functionality), and its size decays with a half-life of 44 months on average. 49,50 The majority of evidence supports the fact this reservoir is maintained by the underlying dynamics of these cells, and not by ongoing viral replication, which could lead to continual reservoir seeding despite antiretroviral therapy (Reference 54,110,111).
While it was originally believed by many that latently infected cells must be intrinsically long-lived, since cell division was expected to reactivate viral expression and lead to eventual cell death, a series of studies over the past few years have convincingly demonstrated that cells in the reservoir can proliferate while remaining latently infected (Reference 110,112). These studies have identified multiple latently infected cells -even in small samples -with virus integrated into identical sites [113][114][115] in the genome or with sequence-identical virus 116-119 -two findings that would be exceedingly unlikely to occur in two independent infection events and likely reflect division of infected cells.
The first class of drugs to be investigated to target latent infection was the so-called "latency-reversing agents". The rationale for these drugs is to increase the rate at which HIV expression is restarted in latently infected cells. If these drugs are given along with antiretroviral therapy, then these reactivated cells will release virus but the released virus will not be able to spread infection to other cells. Eventually, the productively infected cells should die -either by viral cytopathic effects or cytotoxic immune responses. 120 Now that the role of proliferation in maintaining the reservoir has been established, there is renewed interest in developing "antiproliferative" therapies for HIV, which would reduce the ability of latently infected cells to self-renew. Mathematical models have been developed to predict how effective these treatment strategies are likely to be. 121,122 Two recent papers used a similar approach which we will summarize here. If it is assumed Red and blue lines are for alternate parameter sets. B) Hypothetical therapy that increases the activation rate (a) of latently infected cells during ART. When pretherapy a is varied (to 10a* or a*/10), p is kept constant at p* but d is adjusted to keep δ the same. (C) Hypothetical therapy that decreases the proliferation rate (p) of latently infected cells during ART. When pretherapy p is varied (to 10p* or p*/10), a is kept constant at a* but d is adjusted to keep δ. the same. (D/E) Comparison of the relative magnitude of dynamic rates for the corresponding scenarios in the figure above. The height of the bar is proportional to the log10 of the value of the rate. The bar above the horizontal axis represents the process that contributes to reservoir increase ("gain rate", p) whereas bars below are processes that contribute to reservoir decay ("loss rate", a, d). Latency-reversing agents have had some success in increasing HIV gene expression but have not impacted reservoir size, [124][125][126] perhaps because of their lack of specificity for the HIV promotor, posttranscriptional blocks, and lack of recognition of cells by cytotoxic immune responses. Antiproliferative therapies are still at an early stage, but it will likely be difficult to find compounds that substantially reduce division of infected cells without being overtly immune suppressive or triggering compensatory mechanisms to maintain cell population sizes.
The differential equation-based model above can give estimates for the expected decay rate of the latent reservoir, but to achieve cure, the probability that at least one cell remaining in the reservoir reactivates and restarts high-level infection before dying must be zero. To estimate these odds, a stochastic model is needed. An example of this type of calculation is given in Hill et al., 123 Like the above calculation, the exact relationship between reservoir size and probability of cure predicted from the stochastic model is highly dependent on estimates of the underlying parameter values.

| What can viral dynamics tell us about the mechanism of action of new immunotherapies?
Another approach to treat and ideally cure HIV infection involves immunotherapies, which perturb the immune response to infection, either by boosting antiviral immune responses or reversing infection-induced immune suppression. 127 There are many types of immunotherapeutic agents, ranging from small-molecules that act on immune signaling pathways, to biologics like broadly neutralizing antibodies, checkpoint inhibitors, or vaccines, to cell therapies including chimeric antigen receptor T cells. These drugs are being examined alone or in combination with ART for their ability to promote either sterilizing or functional cures for HIV. Even in the few trials that have already been conducted, mathematical models are helping to understand the mechanism of these therapies.
In recent studies by Caskey et al., 128  A few earlier studies conducted this type of "structured" or "analytic" treatment interruption and have provided proof-of-principle for using rebound as a measure of preinterruption infection status.
In the AUTOVAC study, individuals on long-term suppressive ART underwent a series of consecutive treatment interruptions. 46 During each interruption, viral loads rebounded, and once levels passed a threshold therapy was restarted for three months before another interruption. This study found that in the second and third interruptions, the rate of exponential increase in viral load was decreased compared to the first interruption (doubling time increased from 1.4 to 1.9 days), whereas the inferred initial level of viremia from which rebound started -which is directly related to the "reservoir" size and exit rate -was higher (by ~10-fold). These findings suggest that during later interruptions, the immune response may have been boosted compared to the first, which would be expected F I G U R E 8 Modeling viral rebound following ART and immunotherapy. (A) Design of a study in which two novel immunotherapies, a TLR7-agonist and a therapeutic vaccine (Ad26/MVA), where administered during ART treatment of SIV-infected rhesus macaques, followed by a treatment interruption. 130 The time-course of viral loads for one example animal is shown. (B) A mathematical model of viral dynamics augmented to include an antiviral immune response that is stimulated in a viral load-dependent way. (C) Example time-courses of viral load for one animal from each treatment group, along with fits to the model. Each animal was fit to the model individually in a Bayesian framework (with six estimated parameters), and maximum a posteriori values for each parameter were used to plot the results. (D) Group mean values (for 8-9 animals per group) and standard deviations of two parameters that displayed significant variation between groups. Although rebound after long-term ART is generally assumed to arise from reactivated latently infected cells, it is unlikely that these short interruptions substantially increased the reservoir size compared to everything that was seeded before initial therapy. 131 136 Therapy was started at a range of times between 3 days and 2 weeks after infection, and then after 6 months treatment was withdrawn. All animals experienced viral rebound, but the kinetics differed between groups. We would expect that animals starting treatment earlier would have smaller latent reservoir sizes (less opportunity for seeding) and weaker antiviral immune responses. Both experimental assays and fitting viral dynamics models to rebound trajectories supported these hypotheses: very early initiation of therapy lead to the steepest increase in viremia during rebound but the longest delay until the first detectable viral load, which are the predicted effects of lower rates of reservoir exit and decreased effective viral fitness (eg, Figure 8).
Neither very early therapy initiation or repeated treatment interruptions are effective or scalable interventions, but these studies do provide a proof-of-concept that viral rebound kinetics are reflective of preinterruption interventions and they have informed the analysis of two recent preclinical immunotherapy studies. The main drug of interest in these studies was an agonist of Toll-like receptor 7 (TLR7), which is involved in the innate immune system response to viral infections. In the first study, the TLR7-agonist was given to SIV-infected macaques during suppressive ART, and later all treatments were stopped. 137 Most animals rebounded in both treatment (TLR7 + ART) and control (ART only) groups, and mathematical modeling of rebound kinetics showed that rebound trajectories were altered slightly in groups receiving the TLR7 agonist in a way that suggested a partial reduction in the latent reservoir along with alterations to target cell levels and viral immune responses. 137 Consistent with these suggestions, many animals experienced transient increases in viral load during TLR7-agonist administration, despite ART, suggesting that this therapy had an unexpected latency-reversing effect, and two of the thirteen animals in the intervention group never had detectable viremia after therapy cessation.
In a follow-up study, 130 the TLR7 agonist was tested along with a therapeutic vaccine product (both given during ART). In some animals treated with the vaccine, with or without the TLR7-agonist, viremia rebounded rapidly to high levels but was then controlled to very low or completely undetectable levels. These dynamics are never produced by the basic viral dynamics model, which always leads to chronic infection. Alternative models were explored to explain the observations. A model that includes a population of cells belonging to the adaptive immune response which expand in response to viral antigen and act to reduce infection could explain the kinetics, and allowed for estimates of the relative contribution of reductions in the latent reservoir vs enhanced immunity in the altered kinetics. 130 Overall, the modeling analysis suggested that the role of the vaccine was not in boosting clearance of latently infected cells prior to therapy interruption, but in creating an effective primed population of immune cells that do not exist in animals treated only with ART.
While these models have provided insight into treatment interruption trials was a way to evaluate HIV cure studies, there is significant room for improvement in future studies. A major limitation is the lack of detailed longitudinal data on levels and functionality of a panel of components of the immune response, which would allow modelers to conduct more formal hypothesis testing about potential mechanisms. The models used to explain these data are completely deterministic, whereas reactivation from latency, especially following reservoir-reducing interventions, may be highly stochastic. 123,138 They also only track a single strain of virus, but it is possible that fitness differences between multiple strains that exit the reservoir and contribute to rebound, or that new strains that arise via mutation early in rebound contribute to viral and immunologic dynamics. For example, the number of antigenically distinct strains that reactivate may impact the chance of immune control. Another limitation is the uncertainty about the time it takes antiretroviral therapy to effectively "wash out" of the system after the last dose is taken. Hence, the relative contribute of drug washout, waiting time to latent cell reactivation, and time for infection to grow to the detection limit are hard to separate, which limits the quantitative interpretation of reservoir reactivation rates estimated from models. Closer connections between modelers and experimentalists in the early-stage design of HIV cure trials will help ensure that mathematical model can be as informative as possible.

| CON CLUS IONS
Mathematical models have been used to understand the dynamics of HIV within individual patients ever since the infection was first identified. These "viral dynamics" models have provided many im-

ACK N OWLED G EM ENTS
We thank Alan Perelson and Fabian Cardozo for helpful discussions and feedback on the paper. This work was supported by NIH grants (DP5OD019851, P01AI131385, P01AI131365, and 5P30AI060354-15), and Bill & Melinda Gates Foundation award (OPP1148627).