Early visual areas (V1, V2, V3/VP, V4v) contain representations of the contralateral hemifield within each hemisphere. Little is known about the role of the visual hemifields along the visuo-spatial attention processing hierarchy. It is hypothesized that attentional information processing is more efficient across the hemifields (known as bilateral field advantage) and that the integration of information is greater within one hemifield as compared with across the hemifields. Using functional magnetic resonance imaging we examined the effect of distance and hemifield on parallel attentional processing in the early visual areas (V1–V4v) at individually mapped retinotopic locations aligned adjacently or separately within or across the hemifields. We found that the bilateral field advantage in parallel attentional processing over separated attended locations can be assigned, at least partly, to differences in distractor position integration in early visual areas. These results provide evidence for a greater integration of locations between two attended locations within one hemifield than across both hemifields. This nicely correlates with behavioral findings of a bilateral field advantage in parallel attentional processing (when distractors in between cannot be excluded) and a unilateral field advantage if attention has to be shifted across separated locations (when locations in between were integrated).