Edinburgh Research Explorer Commentary: What conflicts of interest tell us about autism intervention research—a commentary on BottemaBeutel et al. (2020)

Woy-naroski (2020) have performed a Herculean and invaluable task in their investigation of conﬂicts of interest (COIs) in nonpharmacological early autism intervention research. Drawing on a meta-analysis of 150 articles reporting group designs, they found COIs in 105 (70%), only 6 (5.7%) of which had fully accurate COI statements. Most reports had no COI statements, but among the 48 (32%) which did, the majority of those declaring no COIs had detectable COIs (23 of 30; 77%). Thus, COI reporting in the literature examined is routinely missing, misleading, and/or incomplete; accurate reporting is the excep-tion rather than the rule. That 120 of the 150 reports were published in 2010 or later, compared to 6 pre-2000, tells us this is not about practices conﬁned to decades past. Instead, it reﬂects and is a telling indictment of established standards in autism intervention research. Achieving a more accurate picture of COIs in autism intervention research required determined detective work on the part of the authors. Neverthe-less, some COIs may be well-hidden and were thus poorly detectable or not detectable at all. In their thoughtful discussion, Bottema-Beutel et note improbable it is that no one at all, in the of intervention papers, for speaking about the they work in

have performed a Herculean and invaluable task in their investigation of conflicts of interest (COIs) in nonpharmacological early autism intervention research. Drawing on a meta-analysis of 150 articles reporting group designs, they found COIs in 105 (70%), only 6 (5.7%) of which had fully accurate COI statements. Most reports had no COI statements, but among the 48 (32%) which did, the majority of those declaring no COIs had detectable COIs (23 of 30; 77%). Thus, COI reporting in the literature examined is routinely missing, misleading, and/or incomplete; accurate reporting is the exception rather than the rule. That 120 of the 150 reports were published in 2010 or later, compared to 6 pre-2000, tells us this is not about practices confined to decades past. Instead, it reflects and is a telling indictment of established standards in autism intervention research.
Achieving a more accurate picture of COIs in autism intervention research required determined detective work on the part of the authors. Nevertheless, some COIs may be well-hidden and were thus poorly detectable or not detectable at all. In their thoughtful discussion, Bottema-Beutel et al. note how improbable it is that no one at all, in the authorship of 150 intervention papers, had received fees for speaking about the interventions they work on. This and other problems inherent in using web searches to unearth unreported COIs suggest that Bottema-Beutel et al.'s findings underestimate both the prevalence and the poor reporting of COIs in autism intervention research. Indeed, with reason, the authors raise the possibility that few of the included studies were free of COIs. Uncertainty about the true extent of COIs in turn makes it difficult to interpret analyses of their association with reported effect sizes, which here were inconclusive, only suggesting smaller intervention effects for studies with no coded COIs.
Importantly, declaring COIs regardless of their potential impact is a basic standard in research. In a literature where COIs are poorly reported, as is the case for early autism intervention research, their potential as a source of bias is obscured, with consequences for both research and practice. Readers need to be able to take into account types of conflict, and calibrate these against study quality (with respect to registration, protocol adherence, reporting standards, sample size, randomization, allocation, blinding, harms reporting, attrition, etc), and the ways in which an intervention has been promoted and disseminated to potential participants and recipients. These factors all contribute to decisions about whether an intervention may be more beneficial than harmful to the targeted population, and to any individual. A failure to accurately declare COIs leaves decision-makers-including researchers, clinicians, educators, and policy-makers, among many others-partially in the dark.
In this light, what does it tell us that Bottema-Beutel et al.'s investigation is unprecedented? This fact is underlined by their provision of a basic primer on COIs in intervention research, a topic conspicuous by its near absence from the autism literature (Milner & Cho, 2014, for a rare exception). This omission contrasts with the vast amounts of public and private resources invested in early behavioral and developmental autism interventions. These high-profile interventions have high costs, whose benefits to providers have attracted equity funds (See https://www.blackstone.com/thefirm/press-releases/article/blackstone-to-acquirecenter-for-autism-and-related-disorders-(card)), yet their benefits and harms to autistics remain uncertain at best due to the poverty of existing evidence (Green & Garg, 2018;Rodgers et al., 2020). A recent systematic review of early autism interventions applied at least some basic standards in evaluating evidence and found, in the entire literature, only six (12.5% of the total) RCTs at low risk of bias (French & Kennedy, 2018). And this was despite disregarding blinding and other issues of pressing concern in nonautism areas, such as outcome switching, adverse events reporting, and, of course, COIs (Heneghan, Goldacre, & Mahtani, 2017). The confluence of poor quality early autism intervention research with the proliferation of high cost early autism interventions suggests that the interests of autistic people have not necessarily been paramount. That it has taken until now for even one autism research group to investigate the other interests involved is telling about the state of the field.
It is important to recognize that while Bottema-Beutel's study is thorough and meticulous, it tells only a fraction of the story. It does not include the plethora of autism single-case designs, whose relentlessly positive results, poor quality, and amplification via misuse of meta-analysis are cause for concern (Bottema-Beutel & Crowley, 2020). It does not include uncontrolled studies, narrative reviews, systematic reviews, umbrella reviews, commentaries, or practice guidelines; nor the COIs of journal editors and reviewers. It does not include several systems of autism-specific low standards for evaluating evidence (e.g., Reichow, Volkmar & Cicchetti, 2008;Wilczynski et al., 2009;Wong et al., 2015), which continue to be influential in research and practice. These systems have been devised to accommodate the poor quality of the autism intervention literature, group and single-case designs both, and have led to the dissemination of 'evidence-based practices' based on poor standards of evidence. Possible COIs in the development of autism-specific low standards range from unmistakeable (service provision) to more nuanced (researcher allegiance), and to our knowledge have never been reported or even mentioned as such in the literature.
The autism research literature's silence on COIs contrasts with the outsized role poor quality intervention research has played in determining how autistics are treated in research and practice, but also in policy, law, and society in general. While poor COI reporting is not unique to autism intervention research, there is reason to be concerned that COIs have had greater, wider, and more disturbing consequences for autistics. Expectations of improvement in themselves produce reported improvements in autistic outcomes even in the absence of any intervention (Jones et al., 2017). With rare exceptions, autism interventions have been hyped and distributed as effective, complete with certification and training programs, without being fairly tested in large well-designed much less independently-conducted RCTs. This kind of plausibly COI-driven process is known to generate spurious positive results (Cuijpers & Cristea, 2016). It has featured dire predictions for autistics deprived of hyped interventions. It has led individuals, organizations, and governments to invest their money, and their reputations, in interventions, training, and certification. Having invested themselves so heavily, they have no incentive to acknowledge, much less encourage, good quality research which may show they have erred. This process has also produced volumes of litigation (with paid expert witnesses, who have COIs) propagating claims that autistics not receiving hyped interventions (whose benefits vs harms are uncertain at best) are incapable of learning and doomed. This goes beyond what COI reporting can uncover, but tells us what may ensue when this and other basic research standards are denied to a specific population.
A place to start, then, in addressing Bottema-Beutel's findings, is that journals publishing autism intervention research should require accurate COI disclosure statements in all papers of all kinds, from editorials to meta-analyses. Readers should not have to search entire manuscripts to locate these statements, which should not be buried in the fine print or main text or acknowledgments, but made prominent in a clearly marked dedicated COI section. This in turn should be visible alongside or within the open-access abstract for each paper. Such a requirement would signal the importance of COIs to both authors and readers, and we dare hope the high visibility would discourage inaccurate reporting. The feasibility of improved reporting is demonstrated by the preprint server medRxiv, which requires COI disclosure statements in their online abstracts. The information is instantly findable, before you download the paper. Authors must also include with their submission a detailed form for COI disclosure (See http://www.icmje.org/conflicts-of-interest/) which prompts authors to consider specific kinds of COIs and has been available for more than a decade. Journals which publish autism intervention research should not have lower standards. Their readers should not have to be detectives to locate information necessary to understand and interpret the autism intervention literature.
Another avenue for improving COI reporting should be in the purview of universities, which are responsible for training and hiring autism researchers, who also take roles as editors and reviewers. Bottema-Beutel et al.'s findings tell us that universities have not made education about, and enforcement of, adequate standards with respect to COIs and COI reporting a priority. Meanwhile, universities reap financial and reputational gains from the reported success of autism interventions for which they provide training and certification programs. Such sources of COI may be encouraged by universities, which-the evidence suggests-do not encourage their accurate reporting. We might then ask, more generally, to what extent inadequate standards in autism intervention research are enabled by the actions or inactions of universities.
Concerns raised by this unprecedented paper thus extend beyond the level of individual author COIs. Bottema-Beutel et al.'s work tells us to be alert to these issues when conducting, reporting, and evaluating research. Despite perverse incentives, individual researchers and research groups have an obligation to improve the entrenched low standards in autism intervention research. Perpetuating and taking advantage of low standards is anathema to the true purpose of science: to make discoveries that ultimately make people's lives better. Knowing and applying the best available standards is essential to the development of an evidence base for early intervention research in autism that we can trust.

Acknowledgements
S.F-W. has published one early autism intervention evaluation study, based on an iPad app, and reporting null results. She included the following Conflict of Interest (COI) statement in the published paper: The iPad app described in this article has been licensed by a commercial developer and is now available as a free version on the Apple App Store and also as a priced 'Pro' version. Dr Fletcher-Watson, Professor Pain, and Professor McConachie may receive royalty payments in future if downloads of the Pro version exceed a certain threshold. Since that statement was made the app has been withdrawn from sale. No royalty payments were ever made. However, she was also the developer of the app and has received consultancy / speaker fees for delivering talks and workshops on the topic of technology-based autism support. S.F-W. also has the following potential Conflicts of Interest: 1. She is related (as sister-in-law) to the former Chief Executive of autism research charity Autistica and has received funding from that body as co-investigator; 2. She is currently Co-I on a clinical trial of the Managing Repetitive Behaviours Intervention though she does not have any COIs for that intervention. The remaining author has declared that they have no competing or potential conflicts of interest.