Early visual areas within each hemisphere (V1, V2, V3/VP, V4v) contain distinct representations of the upper and lower quadrants of the contralateral hemifield. As receptive field size increases, the retinotopy in higher-tier visual areas becomes progressively less distinct. Using functional magnetic resonance imaging (fMRI) to map the visual fields, we found that an intermediate level visual area, the lateral occipital region (LO), contains retinotopic maps with a contralateral bias, but with a combined representation of the upper and lower visual field. Moreover, we used the technique of fMRI adaptation to determine whether neurons in LO code for both the upper and lower contralateral quadrants. We found that even when visual stimulus locations are equivalent across comparisons, the LO was more sensitive to location changes that crossed hemifields than location changes within a hemifield. These results suggested that within high-tier visual areas the increasing integration of visual field information is a two-stage process. The upper and lower visual representations are combined first, in LO, then the left and right representations. Furthermore, these results provided evidence for a neural mechanism to explain behavioral findings of greater integration within than between hemifields.