Grapheme–color synesthetes perceive particular colors when seeing a letter, word or number (grapheme). Functional neuroimaging studies have provided some evidence in favor of a neural basis for this type of synesthesia. Most of these studies have reported extra activations in the fusiform gyrus, which is known to be involved in color, letter and word processing. The present study examined different neuroanatomical features (i.e. cortical thickness, cortical volume and cortical surface area) in a sample of 48 subjects (24 grapheme–color synesthetes and 24 control subjects), and revealed increased cortical thickness, volume and surface area in the right and left fusiform gyrus and in adjacent regions, such as the lingual gyrus and the calcarine cortex, in grapheme–color synesthetes. In addition, we set out to analyze structural connectivity based on fractional anisotropy (FA) measurements in a subsample of 28 subjects (14 synesthetes and 14 control subjects). In contrast to the findings of a recent neuroanatomical study using modern diffusion tensor imaging measurement techniques, we did not detect any statistically significant difference in FA between synesthetes and non-synesthetes in the fusiform gyri. Our study thus supports the hypothesis of local anatomical differences in cortical characteristics in the vicinity of the V4 complex. The observed altered brain anatomy in grapheme–color synesthetes might be the anatomical basis for this particular form of synesthesia but it is also possible that the detected effects are a consequence (rather than the primary cause) of the life-long experience of grapheme–color synesthesia.