Posterior cortical volume changes and abnormal visuomotor performance are present in patients with Huntington's disease (HD). However, it is unclear whether posterior cortical volume loss contributes to abnormal neural activity, and whether activity changes predict cognitive dysfunction. Using magnetic resonance imaging (MRI), we investigated brain structure and visual network activity at rest in patients with early HD (n = 20) and healthy controls (n = 20). The symbol digit modalities test (SDMT) and subtests of the Visual Object and Space Perception Battery were completed offline. For functional MRI data, a group independent component analysis was used. Voxel-based morphometry was employed to assess regional brain atrophy, and ‘biological parametric mapping’ analyses were included to investigate the impact of atrophy on neural activity. Patients showed significantly worse visuomotor and visual object performance than controls. Structural analyses confirmed occipitotemporal atrophy. In patients and controls, two spatiotemporally distinct visual systems were identified. Patients showed decreased activity in the left fusiform cortex, and increased left cerebellar activity. These findings remained stable after correction for brain atrophy. Lower fusiform cortex activity was associated with lower SDMT performance and with higher disease burden scores. These associations were absent when cerebellar function was related to task performance and disease burden. The results of this study suggest that regionally specific functional abnormalities of the visual system can account for the worse visuomotor cognition in HD patients. However, occipital volume changes cannot sufficiently explain abnormal neural function in these patients.