Animal models agree that the perirhinal cortex plays a critical role in object recognition memory, but qualitative aspects of this mnemonic function are still debated. A recent model claims that the perirhinal cortex is required to recognize the novelty of confusable distractor stimuli, and that damage here results in an increased propensity to judge confusable novel objects as familiar (i.e., false positives). We tested this model in healthy participants and patients with varying degrees of perirhinal cortex damage, i.e., amnestic mild cognitive impairment and very early Alzheimer's disease (AD), with a recognition memory task with confusable and less confusable realistic object pictures, and from whom we acquired high-resolution anatomic MRI scans. Logistic mixed-model behavioral analyses revealed that both patient groups committed more false positives with confusable than less confusable distractors, whereas healthy participants performed comparably in both conditions. A voxel-based morphometry analysis demonstrated that this effect was associated with atrophy of the anteromedial temporal lobe, including the perirhinal cortex. These findings suggest that also the human perirhinal cortex recognizes the novelty of confusable objects, consistent with its border position between the hierarchical visual object processing and medial temporal lobe memory systems, and explains why AD patients exhibit a heightened propensity to commit false positive responses with inherently confusable stimuli. © The Authors. Hippocampus Published by Wiley Periodicals, Inc.