Judgments of offensiveness and accountability of hate speech as a function of contextual factors of the speech and characteristics of the observers were examined. A sample of 212 college students and 53 community participants responded to 12 scenarios describing incidents of hate speech. The within-subject variables manipulated in the scenarios were the target of the speech (ethnic groups, women, and gays), the publicness of the speech, and the behavioral response of the target. Ethnic speech was rated more offensive than gender-or gay-targeted speech; public speech was rated more offensive and more accountable than private speech; and public speech was rated more offensive and accountable when a response occurred and private speech was rated more offensive when a response did not occur. The gender and ethnicity of the raters moderated the effects of the experimental variables, as well as showing main effects. The findings of this study suggest that responses to hate speech are complex and contextual.