When does attrition lead to biased estimates of alcohol consumption? Bias analysis for loss to follow‐up in 30 longitudinal cohorts

Abstract Objectives Survey nonresponse has increased across decades, making the amount of attrition a focus in generating inferences from longitudinal data. Use of inverse probability weights [IPWs] and other statistical approaches are common, but residual bias remains a threat. Quantitative bias analysis for nonrandom attrition as an adjunct to IPW may yield more robust inference. Methods Data were drawn from the Monitoring the Future panel studies [twelfth grade, base‐year: 1976–2005; age 29/30 follow‐up: 1987–2017, N = 73,298]. We then applied IPW imputation in increasing percentages, assuming varying risk differences [RDs] among nonresponders. Measurements included past‐two‐week binge drinking at base‐year and every follow‐up. Demographic and other correlates of binge drinking contributed to IPW estimation. Results Attrition increased: 31.14%, base‐year 1976; 61.33%, base‐year 2005. The magnitude of bias depended not on attrition rate but on prevalence of binge drinking and RD among nonrespondents. The probable range of binge drinking among nonresponders was 12–45%. In every scenario, base‐year and follow‐up binge drinking were associated. The likely range of true RDs was 0.14 [95% CI: 0.11–0.17] to 0.28 [95% CI: 0.25–0.31]. Conclusions When attrition is present, the amount of attrition alone is insufficient to understand contribution to effect estimates. We recommend including bias analysis in longitudinal analyses.

have been even greater (Johnson & Wislar, 2012;Tolonen et al., 2006). Attrition rates in large-scale longitudinal studies in the United States depend on the modality of data collection [e.g., mail, cellphone, and web] (Patrick et al., 2017) and range, for example, from 25% in the National Longitudinal Survey of Adolescent to Adult Health (Harris et al., 2019), to 40-55% in the National Longitudinal Study of Youth (Aughinbaugh & Gardecki, 2007), 40-60% in the Population Assessment of Tobacco and Health study (Hyland et al., 2017), 44% in the Panel Study of Income Dynamics (McGonagle, Schoeni, Sastry, & Freedman, 2013), and 30-50% in other large-scale longitudinal studies in the United States and other countries (Bureau of Labor Statistics, 2020a, 2020b).
Attrition and nonresponse do not necessarily result in biased associations between exposures and outcomes with respect to the target population if random (Groves & Peytcheva, 2008). However, even studies with low attrition can generate biased associations if existing attrition is nonrandom; that is, when attrition is related to examined exposures and outcomes and selection out of the cohort (Greenland, 1977;Gustavson, Von Soest, Karevold, & Roysamb, 2012;McCabe & West, 2016), depending on assumptions regarding the unobservable causal structure of attrition and the measure of association (Daniel, Kenward, Cousens, & De Stavola, 2012;Howe, Cole, Lau, Napravnik, & Eron, 2016;Westreich, 2012). The increases in overall attrition and nonresponse across historical time in many longitudinal studies have brought these issues to the forefront of attention in generating appropriate inferences in the presence of high rates of attrition.
Well-developed methods for adjusting for nonresponse and attrition are in widespread use; inverse probability weight [IPW] (Little & Rubin, 2002;Rubin, 1976) is commonly applied, although not the only approach that could be used. For IPWs to eliminate bias, however, the model to estimate weights needs to be correctly specified, including no omitted variables contributing to attrition that are causes of study exposures and outcomes (Hogan & Lancaster, 2004;Robins, Hernán, & Brumback, 2000). One way to additionally bound the potential for bias due to attrition when it is unclear whether assumptions are met is through quantitative bias analysis (Greenland & Lash, 2008;Lash et al., 2014;Lash, Fox, & Fink, 2009). For attrition, quantitative bias analysis can augment weighting algorithms to provide more information regarding the potential for residual bias. Given that the long-term outcomes among those lost to follow-up are unknown and that methods to adjust longitudinal studies for attrition require assumptions, bias analysis can provide a transparent assessment of the potential for incorrect inference after adjustment for known causes of attrition.
Assessing the potential for increasing bias due to attrition in recent historical time is especially important for examining effects with early onset and potentially long duration, such as the consequences of adolescent alcohol use. Heavy and prolonged alcohol use is a leading contributor to global morbidity (Rehm & Shield, 2013;Whiteford et al., 2013;World Health Organisation, 2014) and is most likely to begin during adolescence (Patrick & Schulenberg, 2016).
Assessing the extent to which adolescents increase, decrease, or stably engage in binge drinking as they become adults informs developmental science and public health (Schulenberg & Maggs, 2002).
Longitudinal data are critical to assessment of binge drinking stability during the transition to adulthood. Yet, substance use, including alcohol use, is a key factor that predicts attrition out of longitudinal studies (McCoy et al., 2009;Osler, Kriegbaum, Christensen, Holstein, & Nybo Andersen, 2008;Zhao, Stockwell, & Macdonald, 2009)  . From the annual survey,~2,450 students are randomly selected for longitudinal followup, with oversampling for students who report drug use . These respondents begin follow-up assessments either 1 [modal age 19] or 2 [modal age 20] years later and are followed every 2 years thereafter Schulenberg et al., 2019).
We limited analysis to cohorts who had the opportunity to be followed through follow-

| Covariates included in inverse probability of attrition weights [IPW]
Stabilized weights were estimated based on predicted risk of attrition, conditional on correlates of attrition over the study follow-up. These included: binge drinking status at each wave [yes/no], study year, gender, race/ethnicity, highest education level of mother and father as reported by the adolescent, high school grade point average, and a survey weight adjusting for oversampling of some groups. Supporting Information Table S1 provides an overview of the study covariates, and their association with attrition.

| Statistical analysis
First, we estimated the unadjusted risk difference [RD] for age 29/30 binge drinking between those who engaged in baseline binge drinking versus those who did not, with no adjustment for attrition. RD and 95% confidence intervals were estimated from generalized linear models using a log-binomial link. Units of the RD are per 100 persons observed.
Second, we reestimated the RD of age 29/30 binge drinking, weighting estimates based on the inverse of their probability of nonresponse [attrition] at each wave of follow-up.
Third, we applied bias analysis to the assessment of the relationship between age 18 binge drinking and age 29/30 binge drinking.
We simulated a range of 0-100% of those lost to follow-up engaging in age 29/30 binge drinking, at the same proportion among those who were binge drinking at baseline. This assumption fixes the unobserved RD of adolescent to adult binge drinking to 0 among those lost to follow-up. This analysis allows us to examine the scenario in which no students who were lost to follow-up engaged in age 29/30 binge drinking, or alternatively, that all students who were lost to follow-up engaged in age 29/30 binge drinking. Because losses to follow-up increased over historical time, the number of respondents who were imputed increased across historical time.
However, it is unlikely that the unobserved RD among those lost to follow-up is zero, or that 0% or 100% of those lost to follow-up were binge drinking at age 29/30. The question, then, becomes, "What is a valid estimate for the unobserved RD?" For the fourth step, we simulated three scenarios. (a) That the unobserved RD among those lost to followup is equal to the observed RD among those not lost to follow-up, within each cohort group; (b) that the unobserved RD among those lost to follow-up was lower than the observed RD, by 0.1 points. The rationale for this assumption is that adolescents who have a greater prevalence of risk factors for adult binge drinking, other than adolescent binge drinking, may be more likely to be lost to follow-up, which would serve as competing risks; and (c) that the unobserved RD among those lost to follow-up was higher than the observed RD among those not lost to follow-up, again by 0.1 points. The rationale for this assumption was that binge drinking at baseline may be a particularly strong risk factor for study drop-out, and for adult binge drinking, and thus may be a stronger risk factor among those lost to follow-up compared to those observed.
Finally, in a fifth step, we attempted to estimate the most plausible range of likely prevalences of binge drinking as if there had been no attrition. We assumed those who have a high probability of attrition, but did not drop out, are our best estimate of the prevalence of age 29/30 binge drinking among those who did drop out. We created quartiles of sequentially increasing probability of attrition [e.g., 0-24% and 25-49%] among those who remained in the study, using the same predicted attrition risk model used to calculate IPWs. Within each stratum, we estimated age 29/30 binge drinking for those who were not lost to follow-up, and also estimated the RD of age 29/30 binge drinking based on baseline binge drinking status.
All analyses were conducted using SAS [version 9.4], and figures were created with the ggplot2 package in R (Wickham, 2016). Code for all analyses is available in the Supporting Information.

| RESULTS
Across all cohorts, and without incorporating any attrition weights or adjustment, 37.2% of adolescents reported binge drinking at age 18, with declines across historical time, from 48.6% in 1976-1980 to 34.1% in 2001-2005. 3.1 | Association between baseline and age 29/30 binge drinking with no attrition adjustment 3.3 | Association between baseline and age 29/30 binge drinking with IPW attrition adjustment and bias analysis: varying prevalence of binge drinking among those lost to follow-up, RD fixed at 0 The risk difference association between age 18 and age 29/30 binge drinking status, by 5-year cohort groups, with no adjustment for attrition, IPWs adjustments, and change between the two the bias analysis in which we iteratively increased by 10 percentage points the imputed proportion of age 29/30 binge drinking that we would have observed if those lost to follow-up had been observed, assuming that the RD among those lost to follow-up is 0. All imputed models indicated a lower estimate of RD than what was observed in the IPW-adjusted estimates of RD [listed in first column in Figure 2, as well as in Table 1]. Thus, under these assumptions, the observed RD for the association is overestimated in all scenarios; however there is no scenario in which the relationship between baseline and followup binge drinking is null. 3.4 | Association between baseline and age 29/30 binge drinking with IPW attrition adjustment and bias analysis: varying prevalence of binge drinking and varying RD among those lost to follow-up The magnitude of the differences in RDs remains relatively stable across the amount of attrition across cohorts; variation in the magnitude of the RDs is driven by assumptions about increases in the outcome among those lost to follow-up as well as the association with exposure. in the sample. Indeed, the only scenario in which the "true" RD would be greater than the observed RD is one in which the RD between baseline and follow-up binge drinking was assumed to be higher than observed.

| DISCUSSION
The present study used a long-standing multicohort panel study with variation in the amount of attrition across historical time to address the potential for bias in estimating stability of drinking during the transition to adulthood. The magnitude of attrition alone was insufficient for drawing conclusions about the presence of bias; instead, such conclusions depend on the proportion of the outcome [i.e., age 29/30 binge drinking] among those lost to follow-up, as well as the RD between baseline exposure and the outcome among those lost to follow-up. We find that under most scenarios, observed estimates, even with IPW, likely overestimate the association between adolescent and adult binge drinking. However, our estimate of the likely range of effect sizes indicates that age 18 binge drinking is strongly related to follow-up binge drinking even when accounting for attrition. Importantly, there were no scenarios in which the estimated association between baseline and follow-up binge drinking was close to zero, underscoring that longitudinal panel data remain essential epidemiological designs even when attrition is high. The types of bias that we have described here are often considered selection bias, represented in graphical diagrams as a set of observed or unobserved factors that influence both the outcome of interest as well as selection out of the cohort (Greenland, 1977;Howe et al., 2016;Mansournia, Higgins, Sterne, & Hernán, 2017;Pizzi et al., 2012). The observed cohort, then, becomes a subset of the potential study population, in essence stratifying the analytic cohort on the condition of observed follow-up. This stratification may result in collider bias [bias that arises due to conditioning or controlling for a common consequent of exposure and outcome] as well as residual confounding (Westreich, 2012), and as we demonstrate in the present analysis, result in observed estimates that may over or under estimate the unobserved true association. However, common criticisms of longitudinal studies tend to focus on the amount of attrition, rather than the magnitude of the association between exposure and outcome among those lost to follow-up. Indeed, the principle that the amount of attrition matters less than the association with exposures underlies, in part, the interpretation of cumulative incidence case control studies as estimators of risk (Westreich, 2012).
We demonstrate that an IPW for adjusting estimates due to known variables associated with attrition generally attends to some degree of bias, getting closer to a potential "true" effect. Indeed, IPW in the present study reduced the magnitude of estimates by approximately 10%. However, IPW alone may not be sufficient to completely account for differential attrition, especially when the underlying model to predict attrition exhibits suboptimal accuracy. Including bias analysis in ongoing cohort studies may be a useful adjunct to existing methods to increase confidence in the validity of results. Prior studies that elucidate the impact of selection bias in longitudinal cohort studies have also demonstrated that residual selection bias and confounding can render standard adjustment and weighting approaches to be potentially insufficient (Nohr & Liew, 2018). We extend those approaches here, in cohorts with substantial variation in the level of attrition over historical time, demonstrating that the strength of unobservable associations with losses to follow-up become principle determinants that threaten inference.
The implications of these results for developmental patterns of substance use across the life course underscore that patterns of adult alcohol consumption can be predicted from adolescent use but that there are many sources of variation that also influence the magnitude of this risk. Indeed, the relationship between adolescent binge drinking and age 29/30 binge drinking is moderate, and varies across birth cohorts. Adolescent binge drinking has been declining for more than 30 years , and in 2018 was at its lowest level recorded in surveillance studies of adolescents . Concomitantly, adult binge drinking and other indicators of alcohol consumption are remaining steady or increasing across at least the past decade Schulenberg et al., 2019). Furthermore, there is an inverse correlation by birth cohort in relationship between age 18 alcohol consumption and the growth of consumption through the transition to adulthood (Jager, Keyes, & Schulenberg, 2015;Jager, Schulenberg, O'Malley, & Bachman, 2013), suggesting that although public health success in reducing adolescent drinking has been an achievement, adult drinking continues to increase, and public health efforts aimed at the transition to adulthood should be increased.
There are limitations to generalizability and inference from the present study that should be noted. As in most epidemiological observational cohort studies, assessments are based on self-report of respondents and include measurement error. The validity of IPW as a source of confounder control depends on proper functional form of included variables and the lack of omitted variables, which cannot be empirically verified. Additionally, the MTF study excludes students who left school prior to the twelfth grade, thus the present results are unlikely to generalize to individuals who did not complete high school.
In summary, we recommend for future work that investigators include bias analysis along with IPW strategies, to determine the sensitivity of results to a range of potential attrition patterns. Userfriendly statistical macros and programs for including bias analysis are available (Lash et al., 2009)