Despite its small size, the perirhinal cortex (PRh) plays a central role in understanding the cerebral cortex, vision, and memory; it figures in discussions of cognitive capacities as diverse as object perception, semantic knowledge, feelings of familiarity, and conscious recollection. Two conceptual constructs have encompassed PRh. The current orthodoxy incorporates PRh within the medial temporal lobe (MTL) as a memory area; an alternative considers PRh to be a sensory area with a role in both perception and memory. A historical perspective provides insight into both these ideas. PRh came to be included in the MTL because of two accidents of history. In evolutionary history, the hippocampus migrated from its ancestral situation as medial cortex into the temporal lobe; in the history of neuropsychology, a “memory system” that originally consisted of the amygdala and hippocampus came to include PRh. These two histories explain why a part of the sensory neocortex, PRh, entered into the conceptual construct called the MTL. They also explain why some experimental results seem to exclude a perceptual function for this sensory area, while others embrace perception. The exclusion of perceptual functions results from a history of categorizing tasks as perceptual or mnemonic, often on inadequate grounds. By exploring the role of PRh in encoding, representing, and retrieving stimulus information, it can be understood as a part of the sensory neocortex, one that has the same relationship with the hippocampus as do other parts of the neocortex that evolved at about the same time. © 2012 Wiley Periodicals, Inc.