Factors related to retention in a longitudinal study of infants at familial risk for autism

Abstract Background Reporting retention data is critical to determining the soundness of a study's conclusions (internal validity) and broader generalizability (external validity). Although selective attrition can lead to overestimates of effects, biased conclusions, or overly expansive generalizations, retention rates are not reported in many longitudinal studies. Methods We examined multiple child‐ and family‐level factors potentially associated with retention in a longitudinal study of younger siblings of children with autism spectrum disorder (ASD; n = 304) or typical development (n = 163). The sample was followed from the first year of life to 36 months of age, for up to 7 visits. Results Of the 467 infant siblings who were consented and participated in at least one research visit, 397 (85.0%) were retained to study completion at 36 months. Retention rates did not differ by familial risk group (ASD‐risk vs. Low‐risk), sex, race, ethnicity, age at enrollment, number of children in the family, maternal employment, marital status, or parent concerns about the child at enrollment. A stepwise regression model identified 4 variables that, together, provided the most parsimonious predictive model of study retention: maternal education, maternal age at child's birth, travel distance to the study site, and diagnostic outcome classification at the final study visit. Conclusions The retained and not‐retained groups did not differ on most demographic and clinical variables, suggesting few threats to internal and external validity. The significantly higher rate of retention of children diagnosed with ASD (95%) than typically developing children (83%) may, however, present biases when studying recurrence risk. We conclude by describing engagement and tracking methods that can be used to maximize retention in longitudinal studies of children at risk of ASD.

from enrollment shortly after birth through the window when a diagnosis can reliably be made, is critical. In studies of infants at risk for ASD due to a positive family history, some data collected earlier in infancy is not interpretable until diagnostic outcomes have been documented. Participants who do not complete the study may be missing this critical information, rendering much of their earlier collected data unusable. Attrition in longitudinal studies also reduces statistical power and inflates research costs.
It has been recognized for many years that attrition may also present threats to causal inference by impacting the internal and external validity of longitudinal studies in ways that are less relevant to cross-sectional studies (Campbell & Stanley, 1963). Internal validity refers to confidence that findings are sound and not influenced or confounded by other factors or variables. Internal validity is largely determined by study design and procedures-blinding, randomization, selection of measures-but can also be influenced by attrition if those who do not complete the study share something in common that could provide an alternative explanation for findings or bias the conclusions. External validity refers to the extent to which results from a study can be applied to other samples, situations, or settings. This is driven initially by selection criteria, but attrition can produce biases if those who remain in the study are not representative of the initial sample or the population to which it is hoped to generalize.
Despite the importance of participant retention, many longitudinal studies do not report retention rates or examine effects of attrition on the data. A systematic review of 60 longitudinal studies of health education (Barry, 2005) found that 48% did not record retention rate, and of those that did, less than half examined whether those dropping out differed in systematic ways from those analyzed.
Similarly low rates of reporting attrition were found in a review of 57 publications of very preterm infants (Teixeira et al., 2021). These authors suggested that journals should require publications of longitudinal cohorts to report attrition data.
Although retention data are often not well described, when they are reported, several determinants of retention are commonly identified in longitudinal samples. These include parental age, education level, minority status, and distance from the study site. Factors associated with retention are largely similar across different types of samples, including typically developing adolescents (Ewing et al., 2022), older adults (Heid et al., 2021), and young children with ASD (Bradley et al., 2018).
Studies of infants at elevated likelihood of developing ASD often report rates of missing data or missed visits and may account for it through imputation and other statistical methods (e.g., Bussu et al., 2018;Landa et al., 2013;Ozonoff et al., 2015); however, such studies have not reported retention rates. The threat to internal validity of systematic attrition in infant sibling studies is particularly relevant when the focus is on diagnostic rates, timing of symptom emergence, diagnostic stability, and longitudinal follow-up into later childhood, since biases may be introduced by selective loss of participants with specific characteristics. As one example, estimates of recurrence risk could be inflated if retention rates are higher in children with developmental concerns or signs of ASD than in children who appear to be developing typically. The one published study focused on the risk of ASD recurrence in families with an affected child and a new infant did not report retention data (Ozonoff et al., 2011).
Thus, studying retention is critical in a number of ways, from helping researchers develop strategies to increase engagement and address barriers to completing a longitudinal study, to increasing the likelihood that conclusions will be sound and broadly generalizable. In the current paper, we report on family-level demographic and child-level clinical variables predictive of retention in a 3-year longitudinal study of infants at higher or lower risk of ASD due to family history.

METHOD Participants
The study was conducted in Sacramento, California at the UC Davis MIND Institute. Younger siblings of children with autism spectrum disorder (ASD-risk group) or typical development (Low-risk group) were recruited, primarily from the greater Sacramento Valley (median distance from study site = 25.5 miles). Some families who lived further away were also enrolled if they committed to travel to the MIND Institute for assessments. Inclusion criteria for the ASD-risk group were status as a younger sibling of a child with ASD, with the older sibling's diagnosis confirmed by meeting ASD criteria on both the Autism Diagnostic Observation Schedule (ADOS-2;Lord et al., 2012) and the Social Communication Questionnaire (Berument et al., 1999). Exclusion criteria for the ASD-risk group included birth before 34 weeks of gestation and a known genetic disorder (e.g., Fragile X syndrome) in the older affected sibling. The primary inclusion criterion for the Low-risk group was status as a younger sibling of a child (or children) with typical development, confirmed by an intake screening questionnaire and scores below the ASD range on the Social Communication Questionnaire. Exclusion criteria for the � Few longitudinal studies of children developing ASD report retention rates. If selective attrition occurs, this could skew the results of studies that examine recurrence risk, diagnostic stability, and symptom severity.
� The current study found only four factors that predicted retention, most of which did not present major threats to internal or external validity. However, the significantly higher retention rate of participants who developed ASD compared to those found to be typically developing could bias results, particularly in studies examining recurrence risk.
� Publications of longitudinal cohorts should report both rates of retention and analyses comparing retained and not-retained samples to examine the effects of attrition on variables important to study validity and generalizability. Low-risk group were participant birth before 37 weeks of gestation, developmental or learning disabilities in any older sibling, and ASD in first-, second-, or third-degree relatives.

Recruitment of infants occurred between 2003 and 2018 in
three cohorts corresponding with three funding periods. Recruitment occurred from the prenatal period through 18 months of age (M = 4.1 months, SD = 5.7 months), with most participants (87%) recruited by 9 months of age, less than 5% recruited after 12 months, and only 1.3% recruited at 18 months. Recruitment methods included community outreach events, the MIND Institute clinic, other MIND Institute study referrals, UC Davis Health billing records, and word of mouth. The first point of contact was initiated by parents who called the study phone number listed in recruitment materials to express interest. Study personnel administered an intake screening interview via telephone to prospective families to determine eligibility. The interview collected date of birth and gestational age, qualifying diagnostic status of an older sibling, exclusionary family history information, referral source, and contact information. Those who met eligibility criteria were either scheduled for their first visit at the end of the intake interview or informed that they would receive a follow-up contact for scheduling closer to the date of the first visit, depending on the current age of the infant. Parental informed consent was obtained at the first visit. All procedures were approved by the University of California Davis Institutional Review Board.
The intake interview was administered to the parents of 816 infants to determine study eligibility. Of these, 165 infants were determined to be ineligible during intake interviews (e.g., had a known, exclusionary genetic disorder, infant was out of age range, gestational age was below 34 weeks, etc.), and 50 parents decided not to participate after learning more about the study requirements.
No data was collected on these 215 infants. This left 601 eligible infants, 396 of whom had one or more older siblings with ASD and 205 who had older sibling(s) who were typically developing. Thirtynine of these infant siblings were recruited from a family who had previously enrolled another infant sibling; therefore, the sample of 601 infant siblings derived from 562 separate families. For purposes of analysis, only the most recently enrolled child from a given family was included as the indicator of study retention at the family level.
Of the 562 families eligible to participate, 95 decided not to enroll; the only data collected on these families was recruitment group (ASD-risk vs. Low-risk) and referral source. A total of 467 families (304 ASD-risk and 163 Low-risk) were enrolled, defined as giving informed consent and completing at least one study visit.
Informed consent was obtained at the first visit, documented by the signature of one parent on an IRB-approved consent form. Visits were conducted longitudinally at up to 7 ages (6, 9, 12, 15, 18, 24, and 36 months). Each visit took approximately 2-3 h to complete, depending on the age of the child.
For the primary analyses, a retained family was defined as an enrolled family (i.e., one who had signed a consent form and completed at least one visit) who also completed the 36-month visit.
A not-retained family was defined as an enrolled family who did not have a 36-month visit.

Retention procedures
Participant tracking database A custom database was created to track participant information.
Tracking variables included the reason (e.g., intake, scheduling, sending report) and mode (e.g., email, phone call) of each contact, visit completion status, developmental concerns identified at each visit, diagnostic status, current age of the child, scheduling window for the next visit, and other pertinent information (e.g., preferred mode of contact, best contact times, etc.).

Scheduling
At the completion of each visit, families were reminded of when the next visit was due and told that they could request an interim appointment if any developmental concerns arose before then.
Weekly scheduling attempts were made using the family's preferred mode of contact (email, text, or phone call) beginning 6 weeks ahead of each visit target date and continuing through the scheduling window (4-6 weeks post-target date). When a family did not respond to scheduling attempts in the first 2 weeks, the contact method was varied (e.g., work phone, other parent email) and the day of the week and time of day that contacts were attempted was broadened. In cases in which phone and email were no longer in service, contact information for an extended family member that was collected at the first visit was utilized. In instances where the family was still unreachable, further attempts to contact the family were made by utilizing public records tracking services to find current information.

Clinical feedback and reports
Families were given verbal feedback about their child's development after every visit and provided with written reports beginning at the 18month visit and at all subsequent ages. Verbal feedback and written reports included behavioral observations, scores on normed measures, interpretation of the child's developmental status, including any concerns noted, a reminder of the age of the next visit, and a statement of gratitude for the family's continued participation. When relevant, a diagnostic formulation section, including DSM criteria met and formal diagnoses, was also included, as well as treatment recommendations, referrals, and community resource information.

Compensation
Families received financial compensation for attending each study visit and additional compensation for completing questionnaires before or at each visit. Families were also offered reimbursement for travel expenses on request.
Study "graduation" certificate Following the completion of the 36-month visit, families were given a certificate of participation including their child's name and photos of RETENTION IN A LONGITUDINAL ASD STUDY -3 of 10 them taken at previous study visits, beginning at their first appointment and ending with the most recent visit. The purpose of the certificate was to provide a token of appreciation and a visual reminder of the longitudinal relationship the family had established with the study and the significance of their contribution over time.

Measures
Demographics questionnaire Basic demographic information, including parental age, education, employment, and marital status, family income, race, ethnicity, and sibling information was collected at the first visit and, for retained participants, updated at the 36-month visit. Mailing address was confirmed prior to each visit since questionnaires were sent in advance via U.S. postal service; therefore, travel distance was up to date at the exit visit for both retained and not-retained participants.

Parent concerns
Using a measure previously developed in our laboratory (Ozonoff et al., 2009), parent concerns were collected during the intake interview in response to the prompt: "Do you have any current concerns about your child's development or behavior?". Responses were coded by the intake interviewer into eight categories of concerns, which were then collapsed into two variables for analysis: ASD concerns (the sum of speech/language, social, stereotyped/repetitive behavior, and unspecified autism concerns), and general concerns (the sum of concerns falling into motor, medical/regulatory, behavior/ temperament, and general development categories). Intake interviewers were trained in coding parent concerns to 80% reliability with a training set.
Mullen Scales of Early Learning (MSEL; Mullen, 1995) This standardized assessment of development for children birth to 68 months was administered at all study visits to assess developmental progress and determine outcome classification. Subscales 82.5%) in the not-retained group who exited the study before 18 months were classified as TD. The Non-TD group included seven participants who did not meet DSM criteria for ASD but had: (1) examiner concerns that ASD might be emerging in the future (e.g., child did not meet criteria at the current visit but the examiner felt that he/she might meet criteria at a later visit) and/or (2) MSEL T-scores below 30 on any subtest or below 35 on two subtests. This procedure resulted collectively, across the retained and notretained groups, in the following outcome classifications: ASD (n = 55; 53 ASD-risk; 11 females); Non-TD (n = 108; 84 ASD-risk, 42 females); TD (n = 300; 165 ASD-risk, 151 females). Four participants, all in the not-retained group, did not have enough data to allow classification (2 ASD-risk, 1 female).

Analytic plan
The analysis began by examining differences between families who did and did not enroll in the study and whether enrollment refusal rates varied as a function of recruitment group or referral source.
The rest of the analyses focused on the 467 enrolled families (i.e., consented and completed at least one study visit), comparing those who were retained (completed the study with a 36-month visit) to those who were not retained (ended participation in the study prior to the 36-month visit). Using simple comparisons (chi-square analyses or t-tests) we examined whether retention rates (retained = 1 vs. not retained = 0) varied as a function of multiple demographic variables, including recruitment group, household income, racial or ethnic minority status, maternal education, and others. In order to 4 of 10 -OZONOFF ET AL.

Enrollment refusal
Of the 562 eligible families, 95 (16.9%) decided not to participate after initial contact. There was no difference in the rate of enrollment refusal by recruitment group: ASD-risk 17.4%, CI = 13.9%-21.6% vs.

Study retention
Of the 467 infant siblings who were consented and participated in at Calculated as a proportion, since number of expected visits differed by cohort (up to 7 visits for cohort 2, up to 5 visits for cohorts 1 and 3) and, for not-retained participants, timing of last visit. Finally, we conducted a stepwise regression model, using all participant and family characteristics that differed significantly between retained and not-retained families, to identify a set of uniquely significant predictors of study retention.

DISCUSSION
Reporting retention data (or its converse, attrition rates) is critical to determining the soundness of a study's conclusions (e.g., internal validity) and broader generalizability (e.g., external validity).
Compromising internal or external validity can lead to overestimates  biased results had they been significantly different. There were also no differences between families who expressed initial interest in the study but did not enroll and those who did enroll. It is important to note that we did have missing data for some of the predictors of retention in our models and this may have attenuated our results if missing data was not completely random.
Travel distance was an expected predictor of attrition, based on both previous studies (Bradley et al., 2018;Ewing et al., 2022) and that over a third of those not retained cited a family relocation or travel issues as the primary reason for leaving the study prematurely.
This suggests that longitudinal studies should be wary of enrolling families who live far from where data are collected and should consider including travel funds in their budgets to maximize retention for families who move out of the area after enrollment. Previous studies of varied samples have found that demographic variables like parental age and education level are also common predictors of both study enrollment (Bradshaw et al., 2020) and completion (Bradley et al., 2018;Ewing et al., 2022;Teixeira et al., 2021) and we replicated these findings in our sample as well. Lower household income, along with maternal age and education, were associated with greater attrition in our sample. This suggests that families with lower financial or educational resources may eventually determine that participation in a longitudinal research study is too burdensome to continue, unless additional support to maintain enrollment is provided.
The number of administrative contacts at both enrollment and later visits was also positively associated with retention in the study.
This finding may seem counterintuitive, in that families who are nonresponsive and in danger of dropping out of a study would seem to require more outreach from study staff. However, families who agree to a visit then have multiple follow-up contacts to agree upon a specific date and time and reminders of an upcoming visit, resulting in a significantly higher number of contacts than not-retained families.
Another factor driving the lower number of contacts in the notretained group was outdated contact information; although staff typically made multiple scheduling attempts, these would be aborted if phone numbers or email addresses were no longer in service. A higher number of administrative contacts may also reflect greater responsivity to scheduling negotiations, greater willingness to reschedule an appointment, and greater likelihood of updating contact information if changed.
One finding that does have implications for internal validity is the significantly higher rate of retention of children who were diagnosed with ASD during the course of the study than of children who were found to be typically developing. While retention of participants with typical development was still high (83%), it was significantly lower than retention of participants with ASD (95%). Greater loss of typically developing participants could, in particular, inflate estimates of recurrence risk. A large international multi-site study conducted by the Baby Siblings Research Consortium reported an 18.7% recurrence rate of ASD in families with an affected older sibling (Ozonoff et al., 2011). That study did not report retention data, since it was not collected systematically by the many sites contributing to the analyses, so it is unknown how attrition may have affected recurrence rate. It will be critical for future studies to explicitly address this issue in order to calculate the most accurate and unbiased recurrence rates for families who wish to have this information. If future studies, like the current study, report higher retention of participants diagnosed with ASD, this may suggest that the sibling recurrence is lower than the 18.7% reported previously (Ozonoff et al., 2011). Many publications describe engagement and tracking methods that can be used to maximize retention (e.g., Abshire et al., 2017;Cotter et al., 2002Cotter et al., , 2005Scott, 2004). Teague et al. (2018), in their meta-analysis of retention in longitudinal cohorts, identified 95 innovative strategies used across research projects to improve study completion. These strategies fell into four categories: scheduling and reminders, community building, tracking methods, and barrier reduction, all of which we utilized to improve retention across the 3 years of participation required by our longitudinal study. We had a detailed protocol and tracking database for visit scheduling and reminders. We contacted families between visits with occasional lab newsletters detailing new findings and invitations to recreational and community events. We provided reports for all participants (including typically developing children) with information about developmental functioning and referrals to services when needed, which were highly valued by parents. We had institutional approval to collect next-ofkin locator information and use public records tracking services when we could not reach a family. We offered travel funds, from reimbursement for gas, to airline tickets and hotel rooms for families who had moved but were willing to travel for study visits if costs could be covered. These strategies contributed to a high retention rate of 85% from enrollment near birth through the third birthday. They may also have minimized the potential impact on retention of demographic factors (race, ethnicity) or family stressors (number of children, maternal employment, marital status), none of which were associated with study completion in this sample.
This study found relatively few threats to internal and external validity due to selective retention, suggesting that findings from this lab, and others using similar longitudinal methods with similar samples and similar research questions, are producing largely accurate results that are generalizable to the broader community of families with a child with ASD. It will be critical, however, for future infant sibling studies to track, report, and account analytically for attrition to fully understand the potential effects of selective retention on study results, particularly of sibling recurrence rates.