Understanding the minds of others is one of the great challenges humans face. Accordingly, much work in cognitive neuroscience has explored the brain systems engaged when perceivers share and make inferences about the internal states of social targets. These studies, however, typically use divergent and highly simplified stimuli and methods and as a consequence have produced largely non-overlapping sets of results and artificially constrained theories about the processes involved in perceivers’ abilities to understand targets. Here we suggest that these difficulties may stem from two main sources: the lack of meaningful behavioral data about the brain bases of perceivers’ accuracy in inferring target states and qualitative differences between the social stimuli used in neuroimaging paradigms and the social information perceivers encounter in the real world. We advocate more focus on studies of naturalistic social cognition, which could overcome these limitations and complement current approaches, and discuss work in our laboratory that has demonstrated the feasibility and utility of such a focus. Finally, we discuss the relevance of naturalistic social cognition to diagnosing and treating autism spectrum disorder. Overall, using naturalistic paradigms in neuroimaging will be critical to modeling the way the brain actually understands other minds.