Test–retest reproducibility of a multi‐atlas automated segmentation tool on multimodality brain MRI

Abstract Introduction The increasing use of large sample sizes for population and personalized medicine requires high‐throughput tools for imaging processing that can handle large amounts of data with diverse image modalities, perform a biologically meaningful information reduction, and result in comprehensive quantification. Exploring the reproducibility of these tools reveals the specific strengths and weaknesses that heavily influence the interpretation of results, contributing to transparence in science. Methods We tested–retested the reproducibility of MRICloud, a free automated method for whole‐brain, multimodal MRI segmentation and quantification, on two public, independent datasets of healthy adults. Results The reproducibility was extremely high for T1‐volumetric analysis, high for diffusion tensor images (DTI) (however, regionally variable), and low for resting‐state fMRI. Conclusion In general, the reproducibility of the different modalities was slightly superior to that of widely used software. This analysis serves as a normative reference for planning samples and for the interpretation of structure‐based MRI studies.

characterization, it aggravates statistical problems related to multiple comparisons. Methodologies designed to reduce the dimensions of information are imperative. A well-known strategy is to aggregate voxels that represent a given structure in regions of interest (ROIs), resulting in a biologically comprehensive quantification. As manually drawing ROIs creates practical challenges (Tae, Kim, Lee, Nam, & Kim, 2008), automated methods for imaging segmentation of multiple contrasts represent a viable strategy (Faria, Liang, Miller, & Mori, 2017;. MRICloud (www.MRICl oud.org)  is a recently developed web-based tool with which to perform automated segmentation and quantification of multiple MRI modalities. MRICloud provides a platform to characterize anatomy (using T1 high-resolution-weighted images for volumetric analysis), white matter (using diffusion tensor images [DTI]), and resting-state functional connectivity, built on structure-based analysis. MRICloud can analyze all these modalities in the same anatomical framework, thus facilitating the integration of information from multiple domains in a biologically meaningful set of structures. In addition, MRICloud is a widely available tool, which is free online, completely automated and, therefore, meets the requirements for a neuroimaging tool that is widely applicable to large-scale multimodal processing.
The reliability and accuracy of MRICloud for whole-brain segmentation, based on DTI or T1-WIs, have been extensively tested and validated (Ceritoglu et al., 2009;Liang et al., 2015;Oishi et al., 2008Oishi et al., , 2009Tang et al., 2015;Wu et al., 2016). A few other software that perform high-resolution T1-based automated segmentation, including FreeSurfer (Fischl, 2012), FSL (Jenkinson, Beckmann, Behrens, Woolrich, & Smith, 2012), SPM (Penny, Friston, Ashburner, Kiebel, & Nichols, 2007), ANTS (Avants et al., 2011), also underwent detailed reliability analysis, including testing the robustness of the respective pipelines to technical factors and artifacts (Ceritoglu et al., 2009;Han et al., 2006;Jovicich et al., 2009;Tustison et al., 2014;Ye et al., 2018). Most of these segmentation tools perform admirably when compared with the "gold standard" manual segmentation of selected structures, particularly when tested by the developers, in healthy subjects. A different aspect, less often reported, is the reproducibility of such technologies, over the entire brain, particularly when applied to DTI and resting-state fMRI measurements, which has raised concerns about the interpretation of the results of these methods in the past (Huang et al., 2012;Morey et al., 2010;Shou et al., 2013;Vollmar et al., 2010).
Here, we assess the test-retest reproducibility of MRICloud

| Participants and images
We used the two independent and public datasets to measure the reproducibility of MRICloud multimodality results: • Kirby21, the "multimodal MRI reproducibility resource" (Landman et al., 2011). Kirby21 is a public dataset available in the Neuroimaging Informatics Tools and Resources Clearinghouse (www.nitrc.org). This database consists of 21 healthy volunteers with no history of neurologic diseases (11 male, 22-61 years old), scanned twice in a day, on a 3T Phillips Achieva Scanner. One subject (#8) was excluded because the original DTI scan was not available. A brief description of the image protocol follows: (a) in variable intervals (4.7 ± 2 months interval, minimum = 1 month, maximum = 11 months). Two individuals have no rest-retest DTI.
Briefly, the process for segmenting the T1-WI, used for volumetric analysis, involves orientation and homogeneity correction; two-level brain segmentation (skull-stripping, then whole brain); image mapping based on a sequence of linear, nonlinear algorithms, and large deformation diffeomorphic mapping (LDDMM); and a final step of multi-atlas labeling fusion (MALF) (Tang et al., 2013), adjusted by PICSL (Wang & Yushkevich, 2013). Please read Wu et al., 2016) for technical details. As for the multi-atlas library, we chose "Adult_22_55yrs_283Labels_26atlases_M2_252_V9B" under MRICloud atlas choices. This atlaset contains 26 healthy individuals, 22-55 years old, demographically close to our cohort, as recommended for atlas mapping (Ye et al., 2018).
For the DTI, the tensor reconstruction and quality control followed the algorithm used by DtiStudio (www.MRISt udio.org). The automated DTI segmentation was similar to that used for T1-WIs, except for the use of complementary contrasts (mean diffusivity [MD], fractional anisotropy [FA], and eigenvector [fiber orientation]) and a diffeomorphic likelihood fusion algorithm (Tang et al., 2014) for multi-atlas mapping. Please read (Ceritoglu et al., 2009) for technical details. We used the only atlas library available for DTI mapping in MRICloud: "Adults_168labels_12atlases_V1," which contains 12 healthy individuals, 20-50 years old.
For the rsfMRI postprocessing (Faria et al., 2012), the T1-WI and the respective segmentations obtained as described above were coregistered to the motion and slice timing-corrected, resting-state dynamics. Time courses were extracted from all the cortical and subcortical gray matter regions defined in the atlases and detrended, regressed for motion and physiological nuisance (Behzadi, Restom, Liau, Liu, 2007). Intensity and motion "outliers" were extracted with ART (https ://www.nitrc.org/proje cts/artif act_detect). Seed-by-seed correlation matrices were obtained from the "nuisance-corrected" time courses, and z-transformed by the Fisher's method.
After the multimodal brain segmentation and quantification, each individual, in each session, was represented by a vector of image features. The image features considered in this study were 226 structural volumes from T1-WIs processing (listed in Table 1), 97 white matter structural FA, and 97 white matter structural MD measures from DTI processing (listed in Table 2), and 1,431 pairwise, resting-state z-correlations between 54 gray matter seeds from rs-fMRI (listed in Table 3).

| T1-WIs volumetric analysis with FreeSurfer
Volumes of Kirby21 cortical labels and the deep gray matter were obtained from FreeSurfer v.5.3, for further comparison of reliability with MRICloud. Briefly, images are aligned to the Talairach and Tournoux atlas, corrected for magnetic field inhomogeneity, skullstripped, and the tissues are classified as gray matter, white matter, or CSF. Next, the white surface (the interface between gray and white matter) and the pial surfaces are estimated by triangle meshes and smoothed with a Gaussian filter of 10 mm FWHM (Fischl & Dale, 2000). Cortical thickness is calculated as the shortest distance between the pial and white surface at each vertex across the cortical mantle. The cortical volume is the multiplication of cortical thickness and surface area. The volumes for subcortical regions are calculated as well (Fischl et al., 2002). For comparison of reliability, we determined the correspondence between MRICloud and FreeSurfer labels (listed in Table 1), which is feasible since both methods label according to structural anatomy.

| rsfMRI analysis with SPM CONN toolbox
We used the SPM CONN toolbox, version 17e, to preprocess and perform first-level statistics of Kirby21 rsfMRI data, and further compared the reliability of these results with those from MRICloud.
The CONN toolbox is the most widely used tool for processing rs-fMRI and uses a combination of SPM12 and native-implemented functions. The preprocessing was attuned to keep the rsfMRI in the native space and used the default CONN parameters for slice-time correction and realignment. As in the MRICloud pipeline, ART identified the outlier scans (97th percentiles in a normative sample). The effect of the rest model and its first-order derivative were used as first-level covariates (individual regressors). Sequentially, the processed functional images were detrended and band-pass-filtered (0.008-0.09 Hz). After tissue segmentation and skull-stripping, the T1-WIs and the respective parcellation maps obtained from MRICloud were brought to the rsfMRI space. The use of the same anatomic labels enabled a direct comparison between SPM CONN and MRICloud seed-by-seed correlations.
Although there are descriptions of automated white matter parcellations using cortical-parcellation-based strategies and fiber clustering parcellation (Zhang et al., 2019), to the best of our knowledge, MRICloud is the only automated pipeline available for whole-brain, DTI structure-based analysis. Therefore, the reproducibility of DTI regional quantification outputted from MRICloud was not directly compared with that from other software.

| Test-retest reproducibility
To assess the test-retest reliability of the different metrics in each region of interest, we used the intraclass correlation coefficient (ICC) (Shrout & Fleiss, 1979). To access a global measure of the reproducibility for a given modality, we used the image intraclass correlation coefficient (I2C2) (Shou et al., 2013), which takes in account the total variability of the data. Because I2C2 is less sensitive to regions with low variability, it tends to be lower, and more realistic, than the average of regional ICCs.
A problem that cannot be intuitively solved by looking at I2C2 or ICCs is whether the global individual pattern of image features is reproducible. This "fingerprint" problem has recently been explored in neuroimaging (Finn et al., 2015;Liu, Liao, Xia, & He, 2018;Mars et al., 2018). Assuming that the MRI data itself are reproducible (which is a valid assumption for structural data, such as T1-based volumes), if the postprocessing and quantification tool is reliable, a given individual will be closer to him/herself rather than to someone else in the space of the image features.
In order to explore this idea, we used principal component analysis (PCA) to reduce the dimensionality of the data in each modality.
TA B L E 1 Regional intraclass correlation coefficients (ICCs) for the T1-based volumetric analysis, outputted from MRICloud (using Kirby21 and Human Connectome Project [HCP]) and FreeSurfer ( TA B L E 2 ICCs for regional fractional anisotropy (FA) and mean diffusivity (MD), outputted by MRICloud, using Kirby21 and HCP spect to the anatomical variability of the population. We also ranked the Euclidean distance among subjects in the three-dimensional PCA space. The lowest rank of 1 represents a pair of individuals that are closest in the feature space (i.e., a pair with the lowest variability). If the measurement variability is lower than the anatomical variability, a test-retest pair has a low score, ideally, 1. We judged "correct classification" when the two closest neighbors were the first and second scans of the same subject (the "test-retest" pair), and "misclassification" otherwise.
Finally, we checked for significant differences in the diverse met-

| Power analysis: illustrating the effect of the data variability
Power analysis was used to illustrate the effects of data variability (both biological and technical) on the automated imaging quantification. For a proof of concept, we chose two regions (one with a large ICC and the other with a low ICC), from the volumetric T1-based analysis and from the DTI analysis. We calculated the sample size necessary to detect group differences at an alpha of 0.05 and a power of 0.8, using GPOWER (http://www.gpower.hhu.de/).
The sample size that resulted was inversely proportional to the data variability, which was inversely proportional to the ICC.

| Volumetric (T1-based) test-retest reliability
The global I2C2 coefficients for the MRICloud volumetric analy-  Figure 2), including those in the white matter, which we were able to obtain because MRICloud performs whole-brain parcellation. Only a few small parcels, primarily in the brainstem, had ICCs below 0.9, and no region had an ICC below 0.8.
Compared to MRICloud, the I2C2s for Kirby21 were slightly lower for the FreeSurfer results: 0.920 (CI: 0.871-0.951) for the cerebral cortex and 0.967 (CI: 0.933-0.988) for the deep gray matter.
The regional ICCs were also lower, in general (Figure 2,   There were no significant differences in regional volumes, as TA B L E 3 Averaged ICCs of seed-byseed correlations outputted by the rsfMRI processed with MRICloud (using Kirby21 and HCP) and SPM CONN (using Kirby21)

| DTI test-retest reliability
The global I2C2 coefficients for the MRICloud analysis of the fractional anisotropy (FA) and mean diffusivity (MD) were as follows:  (Figure 1). The regional ICCs (Table 2, Figure 5) were higher for FA than for MD. The FA ICCs were virtually higher than 0.8, while, for MD, a few regions scored below this level, particularly in the brainstem. In contrast to the volumetric analysis, there was more variation on the ICCs, with some areas scoring high (ICC > 0.9) and others low (ICC < 0.5).The cerebellar peduncles had the lowest ICCs.
Although higher than in the volumetric analysis, the measurement variability for Kirby21 FA and MD (the distance between the rest-retest individuals, or dots with the same color in the PCA plots of Figure 6) was, on average, lower than the anatomical variability (the distance among different individuals). The measurement variability was higher for MD (average Euclidian distance between the test-retest pair 0.147 ± 0.079 and 0.119 ± 0.072 for Kirby21 and HCP, respectively) than for FA (average Euclidian distance between the test-retest pair 0.060 ± 0.039 and 0.057 ± 0.032 for Kirby21 and HCP, respectively). Again, the ranked distance matrices offered a different view of the same findings. Using the FA metrics, individuals in a test-retest pair were the closest (ranked distance of 1) in the majority of cases ( Figure 6, bottom right), although there were 9 "misclassifications" (i.e., the closest individual in the first scan was not him/herself in the second scan). Using MD, individuals in the test-retest pair were often not the closest ( Figure 6, bottom left, 19 "misclassifications").
F I G U R E 1 I2C2 for the results of T1volumetric analysis, fractional anisotropy (FA), and mean diffusivity (MD) from DTI, and resting-state fMRI, in two independent datasets (Kirby21 and HCP), using different platforms (MRICloud [MC],

FreeSurfer [FS], Connectivity toolbox in SPM [CONN-SPM])
F I G U R E 2 Color-coded regional ICCs for the volumetric outputs of MRICloud (MC) and FreeSurfer (FS), in two independent datasets (Kirby21 and HCP), overlaid on a representative brain There was no significant difference in FA or MD between the test and the retest sets. The difference between the test and retest metrics was 0.64% for FA and 1.79% for MD, in Kirby21, and 0.5% for FA and 1.9% for MD, in HCP. The Dice indices between pairs (test-retest) of parcels were high (0.896 ± 0.05 and 0.838 ± 0.066 for Kirby21 and HCP, respectively).

| rsfMRI test-retest reliability
The rsfMRI showed the lowest global I2C2 and regional ICCs among all the tested modalities. The global I2C2 for the MRICloud outputs  Individuals are color-coded, that is, the same color represents a "test-retest" pair. Bottom: matrix of ranked distance between individuals in the three-dimensional PCA plot. If the variance in the measurement between scan sections was minimal, a test-retest pair was scored 1 (dark blue). Test-retest pairs that scored higher than 1 (i.e., the individual was closer to someone else rather than him/herself in the second scan) are framed in red For the SPM CONN outputs, the I2C2 was 0.227 (CI: 0.164-0.293) in Kirby21 (Figure 1). The ICC for a given label was calculated as the mean of the ICCs for correlations between that given seed to each other seed (Table 3). The maximum ICC for a parcel using the MRICloud outputs did not exceed 0.6, with the majority of ICCs fluctuating around 0.4 (Figure 7). For the SPM CONN processing, the maximum ICC did not exceed 0.5, with the majority of ICCs fluctuating around 0.25.
The measurement variability, or the distance among test-retest pairs (same color dots) in the PCA plots created with the pairwise z-rsfMRI correlations (Figure 8, top), was lower, on average, than the anatomical/functional variability, or the distance among different individuals, although the variability was seemingly higher than that obtained for volumes or DTI metrics.The ranked distance among Kirby21 individuals (Figure 8, bottom) showed predominantly "misclassifications" (the closest individual in the first scan was not him/herself in F I G U R E 4 Top: 3D PCA plot created with the volumes of the Kirby21 deep gray matter areas, outputted by MRICLoud (MC) and FreeSurfer (FS). Individuals are color-coded; that is, the same color represents a "test-retest" pair. Bottom: matrix of ranked distance between individuals in the three-dimensional PCA plot. If the variance in the measurement between scan sections was minimal, a test-retest pair was scored 1 (dark blue). Test-retest pairs that scored higher than 1 (i.e., the individual was closer to someone else rather than to him/ herself in the second scan) are framed in red There were no significant differences in the outputs of MRICloud between the test and the retest sets.

| Power
The power analysis illustrated the effects of data variability on the automated imaging quantification. For proof of concept, we chose regions with the highest and the lowest test-retest reliability, as measured by ICCs. The power analysis (Table 4) showed that the volumetric data were very stable, meaning that there is a small effect size among scan sets and that thousands of subjects would be needed to detect differences between them. The results are even more drastic for regions with a high ICC (e.g., precentral gyrus > globus pallidus). For the DTI analysis, here represented by the fractional anisotropy, the results were the same for highly reproducible areas (e.g., projection fibers at the pons level), namely, a small effect size between examinations, and a large sample needed to detect differences between them. However, as the range of ICCs was wider compared to volumes in areas with low ICCs (e.g., inferior cerebellar peduncle), the effect size between scans was relatively high (0.69) and less than 100 patients would be needed to detect differences between the scan sets. As this uses a test-retest design, these differences are technical, rather than biological.

| D ISCUSS I ON
We test-retested an automated web-based tool (MRICloud) that performs segmentation and quantification of multimodality MRI (volume from T1-WIs and FA, and MD from DTI and rsfMRI seedby-seed synchrony). The reproducibility rivaled, or was slightly superior, to that from other well-established methods (FreeSurfer, SPM CONN). As discussed in detail below, the reproducibility was (a) globally very high for T1-volumetric analysis; (b) high for DTI analysis, but regionally more variable than for T1-volumetric analysis; and (c) globally low for rsfMRI. To shed light on the reproducibility of postprocessing and quantification tools for MRI is essential, particularly when, by their nature (automated, user-friendly), these tools are used for processing data on a large scale.

| Reproducibility of volumetric quantification
The I2C2 and the regional ICCs were high for the volumetric analysis (vast majority > 0.9, while perfect agreement is 1), reflecting the high stability of the volumetric data and suggesting that subtle F I G U R E 5 Color-coded regional ICCs for the DTI outputs of MRICloud (FA, fractional anisotropy, MD, mean diffusivity) in two independent datasets (Kirby21 and HCP), overlaid on a representative brain differences appointed by them are reliable (Wonderlick et al., 2009).
Although we observed a tendency of small areas to have lower ICCs than large areas, the small variation of ICCs prevented the determination of a significant relationship between the ROI volume and the respective reproducibility of its volume measures. We found reproducibility similar to that in previous studies for data processed with FreeSurfer (Morey et al., 2010;Wonderlick et al., 2009)  Individuals were color-coded; that is, the same color represents a "test-retest" pair. Bottom: matrix of ranked distance between individuals in the three-dimensional PCA plot. If the variance in the measurement between scan sections was minimal, a test-retest pair scored 1 (dark blue). Test-retest pairs that scored higher than 1 (i.e., the individual was closer to someone else rather than to him/herself in the second scan) are framed in red reproducibility, suggesting both methods perform adequately in anatomically normal data. Nevertheless, understanding the source of ICCs variability can lead to improvements in data postprocessing by identifying factors (such as the parcellation scheme, mapping algorithm or set of atlases) that may impact reproducibility. For instance, MRICloud uses diffeomorphic mapping (LDDMM) and multiple atlases, with a large range of anatomical variability. This makes the method effective on both normal brains and brains with a large range of nonlocalized deformations ("atrophy-like"), while methods that assume a stable, healthy pattern may perform worse on these scenarios {Oishi, 2009 #1831}. This is illustrated in Figure 9, where MRICloud outputted a qualitatively reasonable segmentation for an individual with marked brain atrophy, due to hereditary spastic paraplegia type 11.

| Reproducibility of DTI results
The I2C2s for DTI-derived data, processed with MRICloud, were high, while lower than those from the volumetric analysis. Although most of the previous studies looked at DTI reliability on different scans and/or from multiple centers (Deprez et al., 2018;Fox et al., 2012;Jovicich et al., 2014), a few previous studies that addressed test-retest reproducibility (Huang et al., 2012;Shou et al., 2013;Zhang et al., 2019) found results comparable to ours. DTI-derived metrics are calculated from multiple images and, thus, are inherently more noisy and affected by coregistration errors and other types of stability-related issues (Morey et al., 2010). In addition, DTI is highly prone to voxel-level motion of the subject, which would lead to various types of intensity-modulating artifacts (Alexander, Lee, Wu, & Field, 2006;Ni, Kavcic, Zhu, Ekholm, & Zhong, 2006). Finally, a long retest interval may introduce technical and biological effects in the test retest analysis, which may partially explain the slightly lower DTI reproducibility we found for HCP, compared to Kirby21.
Regionally, we found more variation in the DTI ICCs than in the volumetric ICCs. Again, small parcels, which are more susceptible to noise and partial volume effects (Deprez et al., 2018;Vollmar et al., 2010), and parcels in the extremes of the sample (e.g., brainstem, extreme frontal and occipital areas), where the mapping is more challenging, tended to have lower ICCs. In addition, labels with a clearly predominant direction of fibers (high FA) tended to have high ICC, which was corroborated by the observed higher reducibility for an anisotropic phantom compared to human subjects (Morey et al., 2010). This has to be taken in account when planning or interpreting the results of clinical studies. Since DTI measures experience large variability, their sensitivity to detect biological effects may be low. For instance, our power analysis revealed that, while the scan session has a very small effect size in the volumetric analysis (and thousands of subjects would be needed to detect volumetric differences), the effect size of different sessions is much higher for FA, and less than hundred subjects would be needed to detect a significant difference between scan sessions for the same individuals, in areas of low ICC. Therefore, group differences in DTI metrics must be carefully evaluated depending on effect size, location, and related technical conditions.
Despite the lower reproducibility of DTI compared to T1 volumetric data analysis, the regional ICCs were, in general, high, and the measurement variance was still lower than that of the population variance (the distance between rest-retest pairs was lower than the distance among difference subjects, as demonstrated in Figure 6), revealing that MRICloud is a reasonably stable tool.
F I G U R E 7 Color-coded regional mean ICCs for the resting-state fMRI outputs of MRICloud (MC) and CONN-SPM, in two independent datasets (Kirby21 and HCP), overlaid on a representative brain, overlaid on a representative brain

| Reproducibility of rsfMRI results
The I2C2s for rsfMRI data, processed with MRICloud, were lower than those for volumetric and DTI data; the averaged ICCs for the regional correlations among seeds fluctuated around 0.4. Although there are a few reports of higher ICCs, the majority of previous studies that addressed the reproducibility of rsfMRI across individuals are in agreement with our findings (Andellini, Cannata, Gazzellini, Bernardi, & Napolitano, 2015;Deprez et al., 2018;Huang et al., 2012;Shou et al., 2013). As for the DTI data, but on larger scale, the (well-known) rsfMRI low reproducibility is attributed not only to the postprocessing (which is extremely variable in methodology), but F I G U R E 8 Top: 3D PCA plot created with z-transformed correlations between the Kirby21 fMRI time courses of a pair of seeds, outputted by MRICloud (MC) and SPM CONN. Individuals were color-coded, that is, the same color represents a "test-retest" pair. Bottom: matrix of ranked distance between individuals in the three-dimensional PCA plot. If the variance in the measurement between scan sections was minimal, a test-retest pair scored 1 (dark blue). Test-retest pairs that scored higher than 1 (i.e., the individual was closer to someone else rather than to him/herself in the second scan) are framed in red also to the actual nature of the sequence Noble et al., 2017;Patriat et al., 2013). For instance, here we found reproducibility slightly superior for HCP data than for Kirby21. HCP has more frames per run and higher resolution that Kirby21, which may have contributed to the observed difference. Multiple technical factors (magnetic fields, sequence artifacts, motion) and biological conditions (physical and mental states, even in healthy individuals) contribute to data variability, some with a biologically relevant effect, and others as just noise (Airan et al., 2016;Kelly, Biswal, Craddock, Castellanos, & Milham, 2012). To isolate and quantify the contribution of each of these factors is one of the biggest challenges in the field and is not within the scope of this study.
We found that the reproducibility of the outputs of MRICloud is comparable, and slightly higher, to that obtained using SPM CONN. As the parcellation scheme applied in both methods is the same, as well as most of the postprocessing steps (slice-time correction, coregistration, motion correction, outlier rejection, nuisance correction, etc.), the differences in the reproducibility are likely attributable to methodological differences in the image mapping. Likewise in the T1-volumetric analysis, while this seems to have low influence in anatomical normal data, the differences in mapping may affect anatomically abnormal data differently and the indices of reliability may present a great variation, both absolutely and comparatively, among methods.

| CON CLUS ION
We tested-retested the reproducibility of MRICloud, a free, automated method for multimodal MRI segmentation and quantification, on two public, independent datasets. The reproducibility was extremely high for T1-volumetric analysis, high for DTI (however, regionally variable), and low for resting-state fMRI. The reproducibility for T1-volumetric analysis and rsfMRI slightly over performed that of widely used software. The knowledge about the global reproducibility of each modality pipeline, as well as the regional reproducibility for each label, is essential for both study planning and data interpretation and is in line with the efforts to increase reproducibility and transparence in science.

ACK N OWLED G M ENTS
The authors would like to thank Dr. Martin Lindquist for his statis-

CO N FLI C T O F I NTE R E S T
None declared.

DATA AVA I L A B I L I T Y S TAT E M E N T
The data analyzed in this study are publicly available as Tables S1-S3.

O RCI D
Andreia V. Faria https://orcid.org/0000-0002-1673-002X F I G U R E 9 Segmentation of cortex and white matter outputted from MRICloud (left) and FreeSurfer (right) of a brain with large degree of atrophy Abbreviations: DTI, diffusion tensor images; ICC, intraclass correlation coefficients.
TA B L E 4 Power analysis illustrated for regions with high and low ICCs in the T1volumetric analysis and DTI quantification performed with MRICloud