Many studies attest to the critical role of affect in the condemnation of harmful actions, but few attempt to identify the precise representations underlying this affective response. We propose a distinction between two potential sources of affect: an aversion to the negative outcomes of an action versus an aversion grounded in the action itself. Whereas previous models have focused on outcome-oriented processes (e.g. empathy and victim perspective-taking), we argue that moral judgment is also strongly influenced by action-based aversions. Specifically, we propose that individuals engage in a process of ‘evaluative simulation’ when judging others, imagining how much it would bother them to perform the same action. Furthermore, we present evidence that this aversion can be based in superficial sensory or motor properties of the action. We consider how such ‘action aversions’ might be acquired, and we highlight important areas for future research.