Memory loss resulting from damage to the medial temporal lobes (MTL) is traditionally considered to reflect damage to a dedicated, exclusive memory system. Recent work, however, has suggested that damage to one MTL structure, the perirhinal cortex (PRC), compromises complex object representations that are necessary for both memory and perception. These representations are thought to be critical in shielding against the interference caused by a stream of visually similar input. In this study, we administered a complex object discrimination task to two memory-impaired populations thought to have brain damage that includes the PRC [patients diagnosed with amnestic mild cognitive impairment (MCI), and older adults at risk for MCI], as well as age-matched controls. Importantly, we carefully manipulated the level of interference: in the High Interference condition, participants completed a block of consecutive perceptually similar complex object discriminations, whereas in the Low Interference condition, we interspersed perceptually dissimilar objects such that there was less buildup of visual interference. We found that both memory-impaired populations were impaired on the High Interference condition compared with controls, but critically, by reducing the degree of perceptual interference, we were largely able to improve their performance. These findings, when taken together with convergent evidence from animals with selective PRC lesions and amnesic patients with focal damage to the PRC, provide support for a representational-hierarchical model of PRC function and suggest that memory loss following PRC damage may reflect a heightened vulnerability to perceptual interference. © 2012 Wiley Periodicals, Inc.