There are at least two fundamental unanswered questions in the literature on autism spectrum disorders (ASD): Are abnormalities in white (WM) and gray matter (GM) consistent with one another? Are WM morphometric alterations consistent with alterations in the GM of regions connected by these abnormal WM bundles and vice versa? The aim of this work is to bridge this gap. After selecting voxel-based morphometry and diffusion tensor imaging studies comparing autistic and normally developing groups of subjects, we conducted an activation likelihood estimation (ALE) meta-analysis to estimate consistent brain alterations in ASD. Multidimensional scaling was used to test the similarity of the results. The ALE results were then analyzed to identify the regions of concordance between GM and WM areas. We found statistically significant topological relationships between GM and WM abnormalities in ASD. The most numerous were negative concordances, found bilaterally but with a higher prevalence in the right hemisphere. Positive concordances were found in the left hemisphere. Discordances reflected the spatial distribution of negative concordances. Thus, a different hemispheric contribution emerged, possibly related to pathogenetic factors affecting the right hemisphere during early developmental stages. Besides, WM fiber tracts linking the brain structures involved in social cognition showed abnormalities, and most of them had a negative concordance with the connected GM regions. We interpreted the results in terms of altered brain networks and their role in the pervasive symptoms dramatically impairing communication and social skills in ASD patients. Hum Brain Mapp 35:2073–2098, 2014. © 2013 Wiley Periodicals, Inc.