The medial temporal lobes (MTL) and frontal cortex have been shown to subserve memory processes. Neurodegenerative diseases, such as Alzheimer's disease (AD), disrupt the neuronal networks that underlie memory processing. The ε4 allele of the apolipoprotein E gene is a genetic risk factor for AD and is associated with decrements in memory and in olfactory function. The present study utilized EQS, a structural equation modeling software program, to examine differences in the neuronal networks between non-demented ε4 carriers and ε4 noncarriers during a cross-modal olfactory recognition memory paradigm. Prior to fMRI scanning, participants were presented with 16 odors. During two scans, participants discriminated between names of odors presented before scanning (targets) or not presented (foils). The results indicate significant connections between bilateral frontal lobes and MTL for ε4 carriers when they misidentified a foil as a target. When ε4 noncarriers correctly identified a target, there were greater associations between the amygdala, MTL, and right frontal lobe; these associations also modeled the brain's response when ε4 noncarriers misidentified a foil as a target. During memory retrieval, affective cues may facilitate retrieval in ε4 noncarriers relative to ε4 carriers. Last, no model was found that best represented the functional network used by ε4 carriers when they correctly identified a target, which may reflect variability of neuronal recruitment within this population. Hum Brain Mapp, 2013. © 2011 Wiley Periodicals, Inc.