fMRI as an outcome measure in clinical trials: A systematic review in clinicaltrials.gov

Abstract Introduction More than one‐thousand trials with functional magnetic resonance imaging (fMRI) as an outcome measure were registered in clinicaltrials.gov at the time of writing this article. However, 93% of these registered trials are still not completed with published results and there is no picture available about methodological dimensions of these ongoing trials with fMRI as an outcome measure. Methods We collected trials that use fMRI as an outcome measure in the ClinicalTrials.gov registry on 13 October 2018 and reviewed each trial's record entry. Eligible trials’ characteristics were extracted and summarized. Results In total, 1,386 clinical trials were identified that reported fMRI in their outcome measures with fMRI as the only primary outcome in 33% of them. 82% of fMRI trials were started after 2011. The most frequent intervention was drug (pharmacological intervention) (29%). 57% of trials had parallel assignment design and 20% were designed for cross‐over assignment. For task‐based fMRI, cognitive systems (46%) based on Research Domain Criteria (RDoC) was the most frequent domain of tasks. Less than one‐third of trials (28%) registered at least one region of interest for their analysis. Food cue reactivity task, pain perception task, n‐back task, and monetary incentive delay task were recruited in more than 25 registered trials. Conclusion The number of fMRI trials (fMRI as an outcome measure) with both task and rest protocols is growing rapidly. Our study suggests a growing need for harmonization and standardized checklists on both methods and analysis for preregistration of fMRI‐based outcomes in clinical trials.


| INTRODUC TI ON
In our expectation, the role of fMRI in clinical trials would hopefully be as a biomarker by definition, that is, "a characteristic that is objectively measured and evaluated as an indicator of normal biological processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention" (Biomarkers Definitions Working Group, et al., 2001). To be more specific, fMRI is supposed to be used in order to facilitate the development of new interventions by objectively monitoring functional changes in the brain, hypothetically targeted by the interventions. For example, changes in specific brain activity after taking pharmacological agents are detected by fMRI to evaluate the effect of treatment in early phases in drug trials. In this regard, fMRI would be promising for accelerating the development of new treatment; In drug development for mental health disorders, the urgent need for new treatment was addressed by an NIMH program, Fast-Fail trials (FAST), utilizing target-engagement biomarkers including fMRI as one of the potential functional biomarker. fMRI also might play an exploratory role in clinical trials to providing a more objective evidence base for submission to regulators, for example, in later phases of drug trials (Carmichael et al., 2018). In this study, we examined fMRI as an outcome measure, but speaking more broadly, by NIH definition of interventionmanipulation of the subject or subject's environment for the purpose of modifying one or more health-related biomedical or behavioral processes and/or endpoints-it seems that fMRI can also be considered as an intervention in clinical trials in certain conditions (e.g., presurgical mapping) (National Institutes of Health, 2018, case #18e).
Nonetheless, for fMRI to become a powerful biomarker in clinical trials, still some technical and logistical measures need to be taken in order to make the results reliable and valid. There is a high-range of false-positive results in fMRI activations (Wager et al., 2009) and the vast choice of analysis is one of the main reasons ; Carp showed that 6,912 unique analysis pipelines are possible for one single dataset which leads to 34,560 significance maps that vary in activation strength, location, and extent (Carp, 2012). Registration of fMRI methodology before the study or preregistration which restricts "research's degree of freedom"-a term coined by Simmons (Simmons et al., 2011), meaning the flexibility of researcher at methodological level-is one of the suggested logistical measures to deal with falsepositive results (Button et al., 2013;Munafò et al., 2017;Poldrack et al., 2017;Carmichael et al., 2018). Clinical trials' protocol registration in publicly available clinical trial registries is mandatory by laws and policies (https://clini caltr ials.gov/ct2/about -site/history). ClinicalTrials.gov is the largest clinical trials registry with over 200,000 registered studies. The registry was first established to make possible the public access to clinical trials for participation purposes, and as the need for transparency and reproducibility in science emerges, it extended its goals to also serve as a registry for better monitoring and tracking of clinical trials (Zarin & Keselman, 2007). As time passed, the site tightened its rules and modified the protocol registration's required information for more accurate and complete registration. However, for fMRI in clinical trials, the best practices for preregistration needs to be well-defined through a detailed checklist. If following that checklist becomes obligatory by policymakers and granting agencies as a module in the protocol registration in formal clinical trial registries, like clinicaltrials.gov, hopefully, it will result in more valid and replicable fMRI outcomes in clinical trials in long term.
In this study, we completed a systematic review to investigate the scope of fMRI clinical trials in clinicaltrials.gov. We extracted the available data for characteristics of fMRI clinical trials. For categorizing a large number of tasks in task-based fMRI trials, we chose NIMH Research Domain Criteria (RDoC) construct definition (https:// www.nimh.nih.gov/resea rch-prior ities/ rdoc/index.shtml). As a potential preregistration role that clinicaltrials.gov could play in the future for an fMRI-specific data element, we also explored the extent of available preregistered data in clinicaltrials.gov and provided a few recommendations.

| Search strategy and study selection
We searched clinicaltirals.gov on 13/10/2018 for potential clinical trials that used task-based or resting-state fMRI as their outcome measure by using the search term fMRI. Clinicaltrials.gov is a registry that includes more than clinical trials (or interventional studies). Hence, the search was then restricted to clinical trials by applying one of the potential search filters "study type: Interventional". For further analysis, we downloaded the results in a tab-separated value format and transferred it to an excel table to fix the dataset for later analysis. The dataset encompassed eligible trials' protocol mandatory required information (data elements) which was submitted to the clinicaltrials.gov registry by the sponsor or principal investigator at the start of the study. In order to include completed or ongoing clinical trials, we excluded Suspended, Terminated, Withdrawn, and Unknown study states. We also excluded the following studies by manually reviewing each trial's record page: 1. Trials that the outcome measures were recording data of realtime neurofeedback fMRI to reduce methodological complexities 2. Trials with empty (not recorded) outcome measures 3. Trials that didn't report fMRI in their outcome measures either as a consequence of incomplete reporting or in other cases, for example, if fMRI was used as a part of an intervention in the trial, like "brain surgery based on presurgical planning with fMRI" 4. Explicitly stated Not BOLD fMRI trials, for example, perfusion fMRI 5. Nonrelevant trials that appeared in our result as a result of the similarity of keywords Trials' selection process is outlined in the flow diagram ( Figure 1).
To follow the best practices in reporting, we applied relevant "Preferred Reporting Items for Systematic Reviews and Metaanalyses" PRISMA guidelines (Moher et al., 2010). See Checklist S1 for PRISMA checklist of items.

| Data collection
The following variables are the general characteristics that were available in the dataset:

| Definition and classification of variables
Details of all data elements' definitions in the XML file are available at the ClinicalTrials.gov (https://prsin fo.clini caltr ials.gov/defin itions.html).

| fMRI as an outcome measure
It defines whether fMRI serves as the "only primary outcome measure" (for clinical trials that have no other primary outcome measures except for fMRI), "one of the primary outcome measures", "one of the secondary outcome measures" (but not in the primary outcome measures), or "other outcome measures" (but not in the primary or secondary outcome measures) of the trial. It is better to mention that primary outcome measures' groups also encompass trials that use fMRI as secondary in addition to primary outcome measures, and we didn't categorize trials that use fMRI in both primary and secondary outcome measures in an individual group due to the importance of primary compared with secondary outcome.

| fMRI method
fMRI method used as an outcome measure is categorized on three main groups of "Resting-state fMRI", "Task-based fMRI", "Resting-state fMRI-Task-based fMRI"; Studies that didn't mention their fMRI method explicitly were labeled as "Not specified" with two exceptions: If the study mentioned the stimuli (e.g., brain response to visual food cues) or an action (e.g., moving hand), we included them in task-based fMRI.

| Task name
In order to be able to categorize the tasks, the task name provided by authors' manipulation (e.g., "brain response to visual food cues" F I G U R E 1 Inclusion flowchart of fMRI trials registered in clinicaltrials.gov on or before 10/13/2018 Trials registered in clinicaltrials.gov on or before 10/13/2018 (Search term: "fMRI") (n= 2714) Interventional fMRI trials (n= 2026) Task-based and resting-state fMRI interventional trials (n=1386) Excluded (n= 688) Non-interventional trials were considered as "food cue reactivity task"). We categorized tasks within the framework of RDoC. For the sake of clarity, the "cognitive systems" domain is broken into its sub-constructs (attention, perception, working memory, declarative memory, language, and cognitive control). Because of the ambiguity in names of the tasks, the ones related to each RDoC domain are classified into three groups of "Well-specified", "Partially specified", and "Not specified (Table S2).

| Reported region of interests
We recorded data on whether trials reported at least one major region in which the activation will be examined or have a hypothesis included the activation of at least one region; for resting-state fMRI, we also included common networks (e.g., default mode network (DMN)) as reported regions of interest. As this section was a part of a bigger plan to examine trials from preregistration aspect (see the introduction and discussion), we excluded studies with results for this part. Furthermore, fMRI trials that included drug (pharmacological intervention) as the only intervention or one of the interventions were reported separately as well.

| Statistical analysis
All characteristic data are summarized and analyzed using Excel 2016.
Categorical variables are reported as frequencies and percentages.

| RE SULT
The ClinicalTrials.gov registry includes 2026 entries for interventional studies with fMRI as a keyword. Among these, we identified 1,386 completed (46%) or ongoing clinical trials (54%) that used BOLD fMRI as an outcome measure. The reasons for exclusion of 640 interventional trials are listed in Figure 1. Among completed trials, 84% didn't report their results yet.
The most frequent intervention was drug (28.7%) ( Table 1). All 396 clinical trials with drug as the intervention (pharmacological interventions) and 96 clinical trials with multiple interventions including drug were reported exclusively as "drug intervention" (494 trials) in Tables 1-5. In 33% of trials, fMRI was the only primary outcome ( Table 2). 50% of trials had less than 50 participants and 6% more than 200. 82% of fMRI trials were started after 2011. 57% of trials had parallel assignment design and 20% were designed for cross over assignment (Table 3). More details on the characteristics of trials can be found in Table 2 and Table 3.
Forty-two percent of trials registered task-based fMRI, 19% resting-state, and 12% of trials registered both as the outcome measures. There were 963 tasks in 753 trials that reported task-based fMRI as a part of their outcome measures. 20% of trials used more than one task (Table S1 provides details of the number of tasks in these trials). Positive valence systems (19%), cognitive control (15%), perception (10%), and systems for social processes (10%) were the most frequent domains/subdomains of tasks based on RDoC (Table 4). Food cue reactivity task, pain perception task, n-back task, and monetary incentive delay task were recruited in more than 25 registered trials (Table 5). You can find more details on the fMRI tasks in each domain in Table S2. The trend of use in the most frequently recruited fMRI tasks is also reported in Table 6. We found that 28% of trials have reported at least one region of interest. We also included some of the best pre-registered data samples of preprocessing and analysis plan that was found in our clinical trials dataset in Table S3. The dataset is also shared in the Supplementary Dataset.  domain of tasks, followed by positive valence systems (19%), systems for social processing (10%), and sensorimotor systems (5%).

| D ISCUSS I ON
Emotional processing tasks, cue reactivity tasks, pain perception tasks, N-back task, emotional faces processing task, motor tasks, and Go/no-go task were the most frequent tasks that used in these trials (please see Supplementary Dataset 2 for more details). fMRI statistical analysis details in registered trials are scarce and less than one-third of trials registered at least one region of interest for their analysis.
We reviewed all registered materials for each trial to extract fMRI-specific information that might be reported in different sections such as outcome measures, study arms, detailed description. In about 25 percent of trials, the resting-state or task-based fMRI types were not specified. In about 10 percent of task-based fMRI trials, there was not any clear indication (even of task-domain). Task's name as well as reporting model was varied according to the following categories: (a) Conventional name mentioned explicitly (e.g., monetary incentive delay task) (b) Task was introduced in a general way (e.g., decision-making task, cognitive task) (c) The procedure was indicated (e.g., listen to baby's cry, exposure to auditory and visual food cues, luminous stimulation) (d) Task's name was not mentioned but the task's details were described, and (e) Evaluation of brain response to stimuli in study implicated the task (e.g., brain activity in response to noxious stimuli, as assessed by fMRI). We organized the data to be consistent and placed them in tabular format by categorizing the task domains according to the RDoC definition.
In an effort to search for preregistered data of fMRI specification, we had a plan to collect details of registered fMRI information including data design, data acquisition, data preprocessing, and analysis plan, but the minute amount of available data and the lack of similarity in reporting style posted restrictions on this procedure so that we were unable to take our preregistration classification any further. Even for basic details like regions of interest (ROI), less than one-third of trials registered at least one ROI for their analysis.
At this point, there is sufficient evidence that higher standards of preregistration information are needed to help transparency and reproducibility of research, which would lead to bias reduction.
Overall, as Munafò et al. (2017)  As we reported in Table S3 however, there are still details needed to increase the replicability of the outcome measure such as task version, task analysis contrast, and many details in the preprocessing, ROI definition and data analysis pipeline. We believe that higher level of reproducibility could be facilitated with a detailed checklist that should be developed in a consensus process (Ekhtiari, et al., 2020). It is true that clinicaltrial.gov website is not technically designed to provide, encourage or fully support this checklist, however, we should mention that there is still an available space in the clinicaltrial.gov website for each trial to provide detailed descriptions up to 32,000 characters, and also there is a possibility for uploading additional documents for preregistration including structured checklists for methodological details.
The current study is the first one that tried to provide an over- Go/no-go task 19 9 Stop signal task 17 8 Smoking cue reactivity task 16 7 Regulation task (emotion) 15 5 Stroop task 13 6 Fear conditioning task 13 9 Drug cue reactivity task 11 7 Alcohol cue reactivity task 11 10 4.

| Recommendation 2
Providing official reference checklist of specifically required items for preregistered clinical trials with fMRI as an outcome measure will be important. Committee on Best Practices in Data Analysis and Sharing (COBIDAS) offered a comprehensive reporting checklist for an fMRI study . From the authors' perspective, for a strong form of preregistration, the content of conventional method section of a reported fMRI study should be specified beforehand (registered-report). Hence, we recommend specifying items in Tables D1-D5 of the COBIDAS reporting checklist for preregistration.

| Recommendation 3
Preregistration of fMRI analysis details in a replicable way is a labordemanding job. Having a citable list of most acceptable fMRI analysis pipelines within major analysis platforms without wiggling room for further analytics explorations could be very helpful to increase transparency and replicability. Authors could still enjoy exploratory analysis in the trial outcomes, but explicit differentiation between (a) confirmatory analysis based on a well-defined preregistered analysis pipeline and (b) exploratory analysis should be clearly reported in the publications.

| Recommendation 4
Definition of minimum requirements for fMRI outcome preregistration in major platforms like clinicaltrials.gov and then suggestions of the optimum list of items to be registered is required. The expectation for a comprehensive preregistration for fMRI analysis in the first step might be significantly complex for investigators with less experience and can act as a serious barrier.

| Recommendation 5
All preregistration efforts should not suppress potentials for innovations in new task design and more efficient fMRI analysis pipeline development. We also support secondary data exploration when it is transparently reported.

TA B L E 6
The trend of well-specified tasks with more than 10 repetitions from 2001-2018 Currently, there is a large gap in harmonized collective efforts with shared fMRI protocols and study designs to help accumulate replicable knowledge in the field over time. Meanwhile, there is a growing effort in the neuroimaging community to develop domain-specific and domain-general checklists for better quality reporting of fMRI studies (Ekhtiari et al., 2020). We hope this systematic review and its recommendations will help to move one step forward for this endeavor.

ACK N OWLED G M ENT
Authors would like to thank Dr. GholamAli Hossein_Zadeh, Iranian National Brain Mapping Laboratory, and Peyman Ghobadi-Azbari, Shahed University for their valuable comments on this manuscript.

CO N FLI C T O F I NTE R E S T
None declared.

AUTH O R CO NTR I B UTI O N
A.S., M.P. and H.E. conceived of the presented idea and H.E. supervised the project. A.S. gathered data, designed the tables and performed the analytic calculations. All authors discussed the results and contributed to the final manuscript.

Pe e r Rev iew
The peer review history for this article is available at https://publo ns.com/publo n/10.1002/brb3.2089.

DATA AVA I L A B I L I T Y S TAT E M E N T
The dataset is included in the Supplementary Material.