Patients treated for bilateral congenital cataract are later impaired on several hallmarks of adults' expertise with upright faces but report no problem with remembering faces. Here, we provide the first formal data on their face memory. We compared 12 adults with a history of visual deprivation from bilateral congenital cataracts to 24 age-matched controls with normal vision on their ability to recognize famous and recently learned faces, and on their subjective impression of their face memory. Bilateral congenital cataract patients demonstrated a prosopagnosic-like deficit, being slower and less accurate in recognizing both famous faces and recently learned faces, despite not differing on most questions about their impression of their face memory. Patients' results on three perceptual tasks (the composite face effect, the Benton test of recognizing faces through a change in point of view, and the Jane test of sensitivity to feature spacing) were also not correlated with their face memory deficits. These results suggest that early visual input is necessary not only for perceptual expertise in differentiating among unfamiliar upright faces, but also for normal accuracy in remembering the identity of individual faces. © 2012 Wiley Periodicals, Inc. Dev Psychobiol 56: 96–108, 2014.