On Different Sides: Investigating the Persuasive Effects of Anger Expression in Political News Messages

Anger expression is increasingly prevalent in political news messages. However, the persuasive effects of expressing anger in a political context have received scant attention from researchers. We conducted two experiments to investigate the hypothesis that anger expression is detrimental to persuasion because it runs counter to well-established social norms for the polite expression of opinions. We created political news messages including a persuasive appeal by a politician that was supported either with an expression of anger or with an expression of nonemotional disagreement. The results of Experiment 1 (N = 120) showed that anger messages were perceived as less appropriate than control messages, and that politicians expressing anger were perceived as less likable and less competent than politicians who disagreed in nonemotional terms. In Experiment 2 (N = 1,005), the negative effects of anger expression on perceived likability and competence were replicated. Also in line with Experiment 1, anger messages were perceived as less appropriate, but this time only for those with negative a priori attitudes toward the advocated position. In contrast, those with positive a priori positions toward the advocated position perceived anger messages as more appropriate than the control messages.

anger" (Hackman, 2016), Senator Bernie Sanders proclaimed "Americans have a right to be angry" (Sanders, 2016). Arguably, anger was one of the dominant themes of the election. But does anger expression actually help political actors to get their point across?
Despite the prevalence of anger expression in today's politics, no research to date has investigated this issue. In political psychology, there is a strong tradition of research on emotions. Yet a lot of this research centers on the emotions of voters (e.g., Marcus & Mackuen, 1993;Marcus, Neuman, William, & MacKuen, 2000). Research in this tradition for instance investigates how politicians can effectively appeal to voters' emotions (Brader, 2005(Brader, , 2006 or how such emotions influence relevant voter behavior (e.g., Valentino, Brader, Groenendyk, Gregorowicz, & Hutchings, 2011). In the present research, however, we focused on emotions as they are expressed by political actors in mediated messages. Specifically, we focused on the expression of anger.
Interestingly, a close look at the social-psychological literature on emotion expression reveals that predicting the persuasive effects of anger expression is not entirely straightforward. Rather, the literature suggests that these effects are contingent on several factors and that an especially important factor is the perceived appropriateness of the emotion expression (Van Kleef, 2009). Emotion expression may be persuasive to the extent that it is perceived as appropriate, but it may backfire to the extent that it is perceived as inappropriate. In the present research, we expected anger expression in political communication to be perceived as inappropriate by most individuals and therefore to backfire as a persuasive strategy, but we also suspected that such a negative effect of anger expression is contingent on factors related to the receiver and the message. In Experiment 1, evidence was found for the notion that anger expression in a political persuasive message is considered inappropriate. In Experiment 2, results were more complicated, with evidence that anger is perceived as inappropriate among those participants whose prior attitudes are not in line with the message, but appropriate among those participants whose prior attitudes are in line with the message.

Anger Expression
The idea that some other has harmed our concerns or obstructed our goal pursuit is considered to be at the center of anger (Frijda, 2007;Lazarus, 1991). The behavioral tendencies that accompany anger are mostly aimed at undoing the harm and/or removing the obstacle (Frijda, 2007). Importantly, anger also has an interpersonal function. Humans use facial expressions, posture, tone of voice, and a range of other nonverbal signs to communicate their emotions to each other (e.g., Cordaro, Keltner, Tshering, Wangchuk, & Flynn, 2016;Schyns, Petro, & Smith, 2009), a skill they share with other primates (Parr, Preuschoft, & de Waal, 2002). In addition, humans have the ability to state their emotions verbally. In American politics today, emotional expression, but specifically anger expression, looms large. In addition to the examples of President Trump and Senator Sanders cited above, one can think of "the very angry tea party" (Bernstein, 2010) or of Senator Cory Booker saying that he is "seething with anger" (Hains, 2018). Examples are not hard to come by.
Why would a political actor want to communicate anger? In an interpersonal context, people express anger at others in order to signal to these others that they should change their behavior (Fischer & Roseman, 2007;Lelieveld, Van Dijk, Van Beest, Steinel, & Van Kleef, 2011). Similarly, in political communication, anger expression is often used to signal that something is wrong and should be changed. Anger can be leveled at a specific opponent or more broadly at a certain state of affairs. The opponent is rebuked; the situation decried.
When anger expression is leveled at a specific political opponent, it constitutes an instance of political "negativity." Negativity generally refers to political speech that attacks a political opponent rather than promotes the desired candidate (Fowler & Ridout, 2012;Lau, Sigelman, & Rovner, 2007). In that sense, anger expression is always negative, but not all negativity contains anger. Likewise, there are differences between anger expression and the concept of "incivility." Incivility refers to political speech in which the opponent is derided, insulted, or in which his or her legitimacy is denied (Coe, Kenski, & Rains, 2014;Massaro & Stryker, 2012). While a lot of uncivil political discourse is angry in tone, incivility is not always angry; derision, for instance, can communicate contempt rather than anger. Likewise, anger expression is not uncivil per se. Although many instances of anger expression may be considered socially inappropriate (see below), in some cases anger may be considered morally justified (Rawls, 1971, p. 533) and an important driver of change (J. Van Doorn, Zeelenberg, & Breugelmans, 2014).
In sum, in the present research, we investigated the persuasive effects of anger expression. This phenomenon can be distinguished from concepts that have previously been investigated in political psychology, such as negativity and incivility. We worked within a psychological tradition in which it is assumed that humans are generally quite adept at communicating and interpreting emotions (Collignon et al., 2008). "Anger expression" by a political actor, as we use the term, is simply this person communicating a state of anger to the audience.

The Social Functional Perspective on Emotions
According to the social functional perspective on emotions (e.g., Elfenbein, 2007;Van Kleef, 2009), emotions contain valuable information about the feelings and intentions of the sender of the emotion, which can have consequences for the behavior of the receiver. When expressing an emotion, an inferential process is triggered in which the receiver deduces information from the emotional expression. These inferences often take the form of "reverse appraisals" (Hareli & Hess, 2010), whereby receivers make inferences about the causes underlying the emotion and adjust their own attitudes and behavior accordingly (Van Kleef, Anastasopoulou, & Nijstad, 2010). When this happens, effective social influence takes place.
The literature also suggests, however, that emotion expression can backfire as a strategy for influence when it is likely to be considered as socially inappropriate. The "emotions as social influence" (EASI) theory, developed by Van Kleef and colleagues (Van Kleef, 2009;Van Kleef, Van Doorn, Heerdink, & Koning, 2011), suggests that an "inferential" reaction such as described above is likely when the emotion expression is considered appropriate, but that an "affective" reaction is likely when the emotion expression is considered inappropriate. In such cases, receivers do not think about the sender's reasons for his or her emotion, but instead "mirror" the emotion that is expressed. In the case of anger, this will likely lead to negative impressions about the sender (Clark & Taraban, 1991).
What determines whether anger expression will be perceived as appropriate or inappropriate? It is likely that characteristics of the receiver and the message contrive to make anger expression more or less appropriate to the individual receiver. For instance, individual differences such as need for social harmony have been shown to affect the perceived acceptability of anger (Suls, Martin, & David, 1998). Characteristics of the anger expression itself are also important. Steinel and colleagues (2008) showed that expressing anger in negotiations was perceived as more acceptable when it was directed at the target's offer than when it was directed at the target personally. These factors can be seen as the moderators that determine the effect of anger expression on perceived appropriateness.
Perceived appropriateness itself is a judgment by the receiver and therefore the consequence of the anger expression. It can best be seen as a mediator of anger expression's persuasive effect. After all, people expect others to conform to social norms (Chekroun & Brauer, 2002;Schachter, 1951;Szczurek, Monin, & Gross, 2012). In a persuasive context, the likely result of flouting social norms is that the sender is perceived negatively, hindering his or her ability to change the receiver's mind. Indeed, studies in advertising have shown that "offensive" advertising (Beard, 2008;Prendergast, Cheung, & West, 2008) results in negative perceptions of the advertising (Ketelaar, van Hemmen, & Anschutz, 2012) and the advertised product (Chan, Li, Diehl, & Terlutter, 2007). As such, it can be expected that perceived inappropriateness will result in lower levels of persuasion.
In the present study, we expected that anger expression will on average lead to lower levels of perceived appropriateness than nonemotional disagreement and that perceived appropriateness influences persuasive outcomes (mediation), a hypothesis that we investigated in Experiment 1. We also expected that what is appropriate to some receivers may be inappropriate to others. In other words, the effect of anger expression on perceived appropriateness is contingent on several factors related to the individual and the message (moderation). We investigated this latter possibility in Experiment 2.

The Inappropriateness of Anger
Expressions of emotion are subject to socially learned norms. These norms are sometimes called "display rules" (Ekman & Oster, 1979), informing individuals whether and how to express emotions in social interactions. Because of the danger of anger for relationships (Averill, 1982;Saarni & Von Salisch, 1993;Von Salisch & Vogelgesang, 2005), anger expression may be subject to especially strict display rules (Geddes & Callister, 2007). Indeed, there is evidence that people regularly suppress anger in order to maintain healthy and successful social relationships (Ravid, Rafaeli, & Grandey, 2010;Tangney et al., 1996). This is also the case in politics and media. Mutz and Reeves (2005) suggest that, when it comes to rhetorical style, many people seem to hold politicians to similar standards that govern day-to-day interpersonal communication (Brooks & Geer, 2007;Fridkin & Kenney, 2008;Mutz & Reeves, 2005). Messages containing incivility are usually seen as less fair, less informative, and less important than civil messages (Brooks & Geer, 2007). In fact, a "backlash" effect of negative or uncivil messages sometimes occurs in this literature (Fridkin & Kenney, 2008;Lau & Pomper, 2004). Of course, there is a difference between incivility and negativity on the one hand, and anger on the other hand, as we have discussed above. But the general conclusion from the incivility/negativity literature is that people are not very accepting of political messages that do not abide by socially accepted norms for polite discourse, and this is likely to extend to anger expression (Ridout & Searles, 2011).
Two previous studies seemed to find a persuasive effect of anger expression. In one study (Van Kleef, van den Berg, & Heerdink, 2015, Study 4), it was found that being exposed to a message containing anger directed at Greenpeace resulted in less positive attitudes towards Greenpeace than being exposed to a message containing happiness about Greenpeace. However, by contrasting expressions of anger with expressions of happiness, the expression of emotion in this study (happiness versus anger) was confounded with the valence of the judgment (positive versus negative). A second study (Van Kleef et al., 2015, Study 5) circumvented this problem by using expressions of anger and happiness to argue for the same position. Thus, all messages argued for the same position, using expressions of happiness about the position or anger about an alternative position as the experimental manipulation. This study yielded evidence for a persuasive advantage of anger expression over expressions of happiness, although the advantage was confined to participants with high levels of "need for structure." But other empirical research suggests that anger expression is likely to be ineffective as a social influence strategy. One study investigated the effects of anger and disappointment on compliance with requests. The results showed that pairing a request with disappointment increased compliance, but pairing it with anger did not. In fact, participants in this study perceived anger as inappropriate, which reduced compliance (E. A. Van Doorn, van Kleef, & van der Pligt, 2015). Another set of studies found similar results: In a negotiation context, anger expression was perceived as inappropriate, and angry negotiators were perceived as selfish, leading to lower willingness to cooperate (Yip & Schweinsberg, 2017).
All the evidence considered, we venture that, although circumstances can arise which make the expression of anger appropriate , the scope for appropriate anger expression is likely to be limited in most contexts (Lindebaum & Fielden, 2011). We therefore predicted that: H1: Expressing anger in a news media context is perceived as less socially appropriate than expressing nonemotional disagreement.
The consequences of this may be important. According to Kelley's (1973) attribution model, behavior that violates existing social norms is more likely to be ascribed to internal causes (i.e., disposition) than to external causes (i.e., circumstances). Thus, observers are more likely to ascribe anger to the person's disposition, rather than engage in "reverse appraisals." If this is the case, the likely result of expressing anger is that the sender is perceived negatively, hindering his or her ability to change the receiver's mind.
H2: Expressing anger in a news media context leads to more negative attitudes towards the position of the sender than expressing nonemotional disagreement and to lower levels of willingness to act Furthermore, we expect that perceived inappropriateness reduces the likelihood of successful persuasion: Persuasive messages will be less successful to the extent that they are perceived as inappropriate. We hypothesized that perceived appropriateness mediates the predicted negative effect of anger expression: H3: Perceived appropriateness mediates the effects of anger expression on attitude and willingness to act.
To corroborate the notion that anger expression will be attributed to internal rather than to external characteristics, we also investigated the effects of anger expression on perceptions of perceived likability and competence of the sender. Several preliminary findings are in line with the notion that anger expression results in negative evaluations of the sender. Research on anger expression in a persuasive context has shown that anger expression can lead to perceptions that the sender is unfriendly and cold (Kim & Niederdeppe, 2014;Schaubroeck & Shao, 2012, Study 2;Tiedens, 2001). We therefore expected that: H4: Expressing anger in a news media context leads to decreased perceptions of the sender as likable as compared with expressing nonemotional disagreement Previous research has also shown that people expressing anger are sometimes seen as competent (Tiedens, 2001). In one instance, however, this effect was only found for men who expressed anger, with the results suggesting the opposite effect for angry women (Brescoll & Uhlmann, 2008). In other work, the effect was not replicated at all (Kim & Niederdeppe, 2014). Finally, one study found a negative relation between frequency of anger expression by managers in a professional context and perceptions of leader competence (Schaubroeck & Shao, 2012, Study 2). Because previous findings are not so straightforward with regard to perceived competence, we formulated a research question rather than a hypothesis: RQ1: What is the effect of anger expression in a news media context on the perceived competence of the sender?

The Present Research
We conducted two experiments, in which we exposed participants to fabricated news articles that quoted a politician who made a persuasive argument. The (fictional) politician argued against the Transatlantic Trade and Investment Partnership (TTIP), a much debated planned free-trade agreement between the European Union and the United States. The news articles were manipulated such that this "sender" expressed either anger or nonemotional disagreement. Experiment 1 was conducted in a controlled laboratory environment, using 120 undergraduate students as the research sample. Experiment 2 was conducted online, enabling us to recruit a larger (N = 1,005) and more representative sample. This allowed us to investigate the role of potential moderators. Experiments 1 and 2 were conducted simultaneously, in October-December 2016. The data for Experiment 1 was analyzed first, hence the order of presentation in the present article. Experiment 2 was preregistered at https://osf.io/kwv7p/.

Design and Participants
Experiment 1 had a one-factorial design with two conditions: anger expression versus expression of nonemotional disagreement. Each participant was randomly assigned to read one of two news articles. A total number of 119 undergraduate students at a Dutch university (21.0% male) participated in Experiment 1. Their age ranged between 18 and 63 years old (M = 23.53; SD = 6.91). There were no significant differences between conditions in terms of gender, χ 2 (1)= 0.05, p = .82, age, t(116) = 0.60, p = .55, or any other variable assessed at baseline, ps > .17.

Procedure
Participants were invited into our lab and seated in individual soundproof booths where they took part in the study, in exchange for €5. They received a paper-and-pencil questionnaire containing all materials and measures. The participants were exposed to a fabricated news article featuring a political dispute which included our manipulations. The posttest questionnaire assessed perceived appropriateness of the sender's comments and the dependent variables of the study: perceived appropriateness, attitude, willingness to act, perceived likability, and perceived competence. After taking part in the study, participants were thanked and debriefed.

Stimulus Materials
The fabricated news article featured a dispute about the "Transatlantic Trade and Investment Partnership" (TTIP). In the article, Cecilia Malmström, the European Union's Commissioner for Trade, was described as espousing the advantages of TTIP. Subsequently, a (fictional) Danish member of the European Parliament (hereafter called "the sender") was quoted as either disagreeing with Malmström's remarks in nonemotional terms (disagreement condition) or as being angry about them (anger condition). TTIP was chosen as a context for our manipulations because the topic of free trade seems to elicit strong opinions and angry reactions from many people, both in mass and social media (Euronews, 2016). Resistance to free trade has been an important rallying cry for Donald Trump in his 2016 presidential campaign (Broad, 2017). In the Netherlands, TTIP specifically has frequently been the target of public anger (AD, 2016). Thus, TTIP constituted an ecologically valid topic.
To manipulate anger expression, the sender was described as either "furious" about Malmström's praise for TTIP or as "not in agreement" with it. He called TTIP either "a disgrace" or "not a good idea" and argued that TTIP should be "thrown in the rubbish bin" or that it "should not be signed." The arguments that the sender used to support his opinion did not differ between conditions. In all conditions, the sender supported his opinion with the argument that free trade is bad for workers, as they have to compete with workers abroad. This argument was chosen because it has lately become a highly prominent argument in discussions about free trade, both in the United States and in Europe (Walker, 2017). It also has the benefit that it is not without empirical merit, as recent research in economics has suggested that low-and middle-wage workers in developed economies do indeed suffer from free trade (Autor, Dorn, & Hanson, 2016). Thus, in the anger (nonemotional disagreement) condition, the sender was quoted as saying: "TTIP is a disgrace. (TTIP is not a good idea.) [Workers] will have to compete with other workers abroad. Especially people with a weak position on the labor market will suffer the consequences." 1 Outcome Measures Perceived appropriateness. Perceived appropriateness was assessed with four items asking participants to indicate the extent to which they thought the sender's response was appropriate, the extent to which they thought the response fitted the topic at hand, the extent to which they thought the response was fitting for a news article, and the extent to which they thought the response was understandable, all on a 7-point scale (1 = not at all; 7 = very much). In a factor analysis (KMO = .74), the Scree test yielded a single factor, explaining 57.95% of variance. Therefore, a single perceived appropriateness score was calculated by averaging all four items (M = 4.77; SD = 1.05; α = .75).
Attitude. Following previous research (Calanchini, Moons, & Mackie, 2016), we assessed participants' attitude towards TTIP. On a 7-point scale, ranging from 1 (totally disagree) to 7 (totally agree), participants indicated to what extent they agreed with the items "TTIP should not be signed"; "It is a bad idea to sign TTIP"; "There are good arguments in favor of TTIP"; and "The European Commission is right to want to sign TTIP." The scores on the last two items were reversed, so that higher scores indicated increased agreement with the sender's persuasive appeal (see Calanchini et al., 2016). In a factor analysis (KMO = .76), the Scree test yielded a single factor, explaining 73.51% of variance. Therefore, all items were averaged into an attitude index (M = 3.68; SD = 1.15; Cronbach's α =.88).
Willingness to act. To assess willingness to influence the outcome of political deliberations, 14 items asked participants to indicate the likelihood that they would engage in distinct behaviors that could potentially influence the decision-making process (1 = highly unlikely; 7 = highly likely). For this measure, previous measurement instruments of political participation (Bakker & de Vreese, 2011;Moeller, de Vreese, Esser, & Kunz, 2014) were reworded to fit our purposes (cf. Peng, Lee, & Heeter, 2010). Specifically, seven items dealt with online acts of political participation (Moeller et al., 2014), such as posting an anti-TTIP message or video on a social network site, and argued against TTIP in an online discussion. Seven items dealt with off-line acts of political participation (Bakker & de Vreese, 2011), such as participating in anti-TTIP demonstrations and arguing against TTIP during a debate or lecture. In a factor analysis (KMO = .89), the Scree test yielded a single factor, explaining 52.91% of variance. Therefore, all items were averaged into a single willingness score (M = 1.72; SD = 0.73; Cronbach's α =.91).
Perceived likability and perceived competence of the sender. We assessed participants' ratings of likability and competence with regard to the sender. In line with previous research (Kim & Niederdeppe, 2014), perceived likability was assessed with four items, in which participants were asked to indicate on a 7-point scale (1 = not at all; 7 = very much) how warm, nice, kind, and friendly they perceived the sender to be. In a factor analysis (KMO = .78), the Scree test yielded a single factor, explaining 82.39% of variance. Therefore, a mean score of the four items was calculated (M = 3.58; SD = 1.32, α = .93).
In line with previous research (Fridkin & Kenney, 2011), perceived competence was assessed with 10 items that asked participants to use a 7-point scale (1 = not at all; 7 = very much) to indicate how rational, sensible, intelligent, calculating, good at his job, knowledgeable, down-to-earth, professional, business-like, and no-nonsense they perceived the sender to be. In a factor analysis (KMO = .87), the Scree test yielded a single factor, explaining 53.05% of variance. Therefore, a mean score was calculated, such that higher scores indicate greater perceived competence (M = 4.26; SD = 1.04; α = .89).

Data Analyses
We used t-tests to investigate differences in means between conditions for perceived appropriateness, attitude, willingness to act, and perceived competence. Hedges g was used as the effect-size measure. Power calculations using GPower (Erdfelder, Faul, & Buchner, 1996) revealed that, given a medium-sized effect for the difference between the two conditions of Cohen's f = 0.30, a sample of N = 120 would result in a power of β = 0.93. Hayes' (2012) PROCESS macro for SPSS was used to test for mediation. κ 2 was used as an effect size measure for the mediation. In addition to the variables mentioned above, we assessed a limited number of other variables, at both pretest and at posttest. Analyses pertaining to those variables are not described here for reasons of brevity. All data for Experiment 1 as well as all questionnaires and the complete stimulus materials can be found at https://osf.io/kwv7p/.

Perceived Appropriateness
The analyses for Hypothesis 1 revealed a significant effect of the anger manipulation on perceived appropriateness, t(116) = 2.69, p = .008, Hedges g = −0.50, with lower levels of perceived appropriateness in the anger condition versus the nonemotional disagreement condition (see Table 1).

Persuasion
With regard to attitude, the results showed no support for Hypothesis 2. There was no significant main effect of the anger manipulation, t(117) = −0.38, p = .703, g = 0.06 (see Table 1 for the means and standard deviations). With regard to willingness to act, no support for Hypothesis 2 was found. There was no substantial main effect of the anger manipulation, t(115) = 0.35, p = .726, g = −0.07.

Mediation
Hypothesis 3 stated an indirect effect of anger expression on persuasion through perceived appropriateness. Although there was no total effect of anger expression on persuasion, it is still possible that there is an indirect effect of anger expression on persuasion through perceived appropriateness (Hayes, 2009). With regard to attitude, the results of a first analysis showed that anger versus  09. All in all, support for Hypothesis 3 was found with regard to willingness to act.

Perceived Likability and Competence
The analyses revealed that the anger manipulation affected participants' perceptions of sender likability. There was a significant effect of the anger manipulation, t(118) = 3.83, p < .001, g = −0.70, in which the angry sender was seen as less likable than the disagreeing sender (see Table 1). Competence also showed a significant effect of the anger manipulation, t(118) = 3.18, p = .002, g = −0.58, in which the angry sender was seen as less competent than the sender expressing disagreement.

Discussion
In Experiment 1, anger expression was considered less socially appropriate than nonemotional agreement, and perceived inappropriateness contributed to more negative attitudes and lower levels of willingness to act. The total effect of anger expression on attitude and willingness to act was not significant, however, and as such no evidence of a veritable backlash effect of anger was found. However, Experiment 1 did reveal that a sender who expressed anger was seen as less likable and less competent than a sender expressing nonemotional disagreement.
But can anger expression be persuasive under different circumstances? So far, we have assumed that anger expression is generally inappropriate in the context of political news, but perhaps characteristics of the message and/or the recipients can influence this. As we have stated in the introduction, contextual factors likely determine the extent to which anger expression is considered appropriate (Brescoll & Uhlmann, 2008;Schaubroeck & Shao, 2012;Van Kleef et al., 2011). As such, Experiment 1's focus on main effects of anger expression is a limitation of the study. Therefore, we turned to the investigation of potential moderators of the persuasive effects of anger expression in Experiment 2.

EXPERIMENT 2
In Experiment 2, we investigate moderators of the anger expression effect. We focused both on characteristics of the message and characteristics of the recipient as moderators of the effects of anger expression. With regard to characteristics of the message, it is notable that the messages we used in Experiment 1 considered TTIP in terms of a rational discussion of its likely consequences. That is, the sender expressing either disagreement or anger justified his position with arguments that could be considered part of a more or less rational discourse on economic policy. Arguably, the message employed what is known as an "issue frame" (Brooks & Geer, 2007). Economic debates, however, can also be framed in terms of antagonism between groups, for instance when topics are described as a struggle between selfish politicians versus the common people or between "Wall Street" and "Main Street" (e.g., Barofsky, 2012). In the context of such antagonism (Gross & Brewer, 2007), anger expression may have a positive persuasive effect. At least one previous study's results are suggestive of this. This study (Brescoll & Uhlmann, 2008, Study 3) showed that providing a justification for anger expression can limit negative effects. As such, when anger expression is "backed up" with assertions of wrongdoing, it may seem the appropriate reaction and may be more persuasive than the expression of mere disagreement. In Experiment 2, therefore, we hypothesized that: H5: Issue versus antagonism framing moderates the persuasive effect of anger expression, such that expressing anger-as compared to expressing disagreement-will have a negative effect on perceived appropriateness, attitudes, and willingness to act when messages employ issue frames, but a positive effect when they employ antagonism frames.
With regard to characteristics of the recipients, we investigated whether initial attitudes could moderate the effect of anger expression. One could expect anger expression to be more persuasive when receivers have a priori negative attitudes towards the topic. For this reason, we chose to assess likely predictors of the recipients' position towards TTIP.
A recent study investigating predictors of support for TTIP in 28 EU member states found that attitude toward globalization, attitude towards the EU and satisfaction with democracy on the national level were strong predictors of support for TTIP (Steiner, 2016). As such, we assessed these variables in Experiment 2 and investigated whether they moderated the effect of anger expression on perceived appropriateness, attitude, and willingness to act. It could be expected that anger expression would have a positive persuasive effect for recipients with negative attitudes towards globalization, negative attitudes towards the EU and low levels of satisfaction with democracy on the national level, as this profile has been shown to be associated with opposition to TTIP (Steiner, 2016) and is thus in line with the sender's position. We formulated the following Research Question: 2 RQ2: Do the effects of anger expression versus nonemotional disagreement on perceived appropriateness, attitude, and willingness to act depend on recipients' a priori attitudes towards the sender's position?
As can be seen at https://osf.io/kwv7p/, Hypothesis 5 constituted a preregistered hypothesis, to be tested using confirmatory data analyses. For Research Question 2, exploratory data analyses were planned (see below).

Procedure
A short questionnaire first assessed gender, age, education, family income, and employment status. Next, in a pretest questionnaire, attitude towards globalization, attitude towards the EU, and satisfaction with democracy were assessed as potential moderators of the effects of antagonism and anger. The participants were then exposed to a fabricated news article, which included our manipulations. After this, a questionnaire assessed the dependent variables: perceived appropriateness, attitude, willingness to act, perceived likability, and perceived competence. After taking part in the study, participants were debriefed and informed of the true purpose of the study.
Pretest Questionnaire Support for globalization was assessed with a single item, taken from previous research (Mansfield & Mutz, 2009;Steiner, 2016), asking participants to indicate agreement with the statement "Globalization is an opportunity for economic growth," on a scale ranging from totally disagree (1) to totally agree (7) (M = 4.98; SD = 1.13). Attitude towards the EU was assessed with a single item, taken from previous research (Steiner, 2016), asking participants to answer the question "In general, does the EU conjure up for you a positive or negative image?" on a scale ranging from very negative (1) to very positive (7) (M = 4.03; SD = 1.46). Satisfaction with democracy was assessed with a single item, taken from previous research (Steiner, 2016), asking participants to answer the question "To what extent are you satisfied with the way democracy in the Netherlands works?" on a scale ranging from not satisfied at all (1) to very satisfied (7) (M = 4.26; SD = 1.47).

Stimulus Materials
We used the same fabricated news article as in Experiment 1. However, the sender's argument was framed differently in the issue versus the antagonism conditions. In the issue-framing condition, the sender pointed out that free trade hurts workers, especially those with a tenuous position on the labor market (see Experiment 1).The fabricated news article was written in such a way that the sender invoked more or less rational arguments about the disadvantages of TTIP. In the antagonism framing condition, however, the sender cast TTIP in terms of a struggle between the European Union and large companies versus the common people. Importantly, the sender alleged that the European Commission deliberately hurts the interest of workers, suggesting that the European Union just "shoves these people aside." In the antagonism-framing condition, then, the sender's comments were written in such a way to portray wrongdoing on behalf of the European Union. For instance, in the anger (nonemotional disagreement) condition, the sender was quoted as saying: "TTIP is a disgrace (TTIP is not a good idea).
[Workers] will be the victim, because they will have to compete with other workers abroad. Especially people with a weak position on the labour market are being had." Outcome Measures Perceived appropriateness. Perceived appropriateness was assessed with the same four items as in Study 1. In a factor analysis (KMO = .82), the Scree test yielded a single factor, explaining 72.04% of variance. Therefore, a single perceived appropriateness score was calculated by averaging all four items (M = 5.01; SD = 1.10; α = .87).
Attitude. Participants' attitude towards TTIP was assessed in the same manner as in Study 1. In a factor analysis (KMO = .72), the Scree test yielded a single factor, explaining 67.31% of variance. Therefore, all items were averaged into an attitude index (M = 4.41; SD = 1.27; Cronbach's α =.84).
Willingness to act. Willingness to influence the outcome of political deliberations was assessed in the same manner as in Study 1. In a factor analysis (KMO = .96), the Scree test yielded a single factor, explaining 63.52% of variance. Therefore, all items were averaged into an attitude index (M = 2.67; SD = 1.36; Cronbach's α =.95).
Perceived likability and perceived competence of the sender. We assessed participants' ratings of likability and competence with regard to the sender in the same manner as in Study 1. For perceived likability, the Scree test in a factor analysis (KMO = .81) yielded a single factor, explaining 80.99% of variance. Therefore, a mean score of the four items was calculated (M = 4.14; SD = 1.17, α = .92). For perceived competence, the Scree test of a factor analysis (KMO = .94) yielded a single factor, explaining 61.06% of variance. Therefore, a mean score was calculated, such that higher scores indicate greater perceived competence (M = 4.62; SD = 1.03; α = .92).

Data Analyses
We performed two sets of analyses; confirmatory analyses and exploratory analyses. In the confirmatory analyses, we tested our hypotheses, using two-way ANOVAs to analyze the effects of issue versus antagonism framing, anger expression, and the interaction between issue versus antagonism framing and anger expression. Partial eta squared was used as the effect-size measure. In addition, we conducted multiple linear regression analyses, in which we controlled for all control variables. The semipartial correlation was used as the effect size measure for individual effects. Power calculations using GPower revealed that, given a medium-sized effect for the interaction between framing and anger expression of Cohen's f = 0.25, a sample of N = 1,000 would result in a power of β = 0.99.
In the exploratory data analyses, we tested the moderating effect of attitude towards globalization, attitude towards the EU, and satisfaction with democracy (RQ2) on the influence of anger expression on perceived appropriateness, attitude, and willingness to act. To protect ourselves from an inflated chance of Type I error, we used the "split-samples method" for the exploratory analyses (Fafchamps & Labonne, 2016). In the split-sample method, the data is randomly split in a training data set and a test data set. Analyses are performed on the training sample, and in case of a significant effect, this effect is tested again in the testing sample. An interpretation of this method is that it allows us to perform a direct replication of our work within the same dataset (Fafchamps & Labonne, 2016). Even though only half of the sample was used in both data sets, statistical power to detect a medium-sized interaction effect of Cohen's f = 0.25 was high, estimated at β = 0.99 with the help of GPower.
In addition to the variables mentioned above, we assessed a limited number of additional variables. Analyses pertaining to those variables are not described here for reasons of brevity. All data for Experiment 2 as well as all questionnaires and the complete stimulus materials can be found at https://osf.io/kwv7p/.

Perceived Appropriateness
The results revealed no significant effect of the interaction between issue versus framing and anger expression, F(1, 997) = 0.51, p = .476, η p 2 = .00. Also, there were no main effects of anger expression, F(1, 997) = 0.19, p = .666, η p 2 = .00, and issue versus antagonism framing, F(1, 997) = 1.06, p = .303, η p 2 = .00. Means and standard deviations can be seen in Table 3. Table 4 reveals that these results are similar when controlling for all covariates in a linear regression analysis.

Persuasion
The results for attitude were similar, with no significant effects for the interaction, F(1, 1001) = 1.04, p = .309, η p 2 = .00, the main effect of anger expression, F(1, 1001) = 1.94, p = .164, η p 2 = .00, or the main effect of issue versus antagonism framing, F(1, 1001) = 3.22, p = .073, η p 2 = .00. For willingness to act, no significant effect of the interaction was found, F(1, 998) = 0.70, p = .404, η p 2 = .00, nor did the main effects of anger expression, F(1, 998) = 0.46, p = .497, η p 2 = .00, and issue versus antagonism framing, F(1, 998) = 1.97, p = .161, η p 2 = .00, have a significant contribution to the prediction of willingness to act. Thus, Hypothesis 5 could not be confirmed. Table 4 reveals that these results are similar when controlling for all control variables. The only departure from the ANOVA results is that the effect of issue versus antagonism framing on attitude becomes significant in the linear regression analysis, with antagonism framing resulting in more positive attitudes towards the sender's position than issue framing.

Perceived Likability and Competence
With regard to the effects of our manipulations on perceived likability, the results revealed no significant effect of the interaction between issue versus framing and anger expression, F(1, 1001) = 1.46, p = .227, η p 2 = .00. 3 There was also no main effects of issue versus antagonism framing, F(1, 1001) = 2.53, p = .112, η p 2 = .00. Anger expression did affect perceived likability, however, F(1, 1001) = 11.89, p = .001, η p 2 = .01, with higher perceived likability in the disagreement conditions than in the anger conditions (see Table 3). Table 4 reveals that this result is similar when controlling for all control variables. 3 In the preregistration of Experiment 2, we designated the analyses of the effects of our manipulations on perceived likability and competence as exploratory. However, after finding the significant effect of anger expression on perceived likability and competence in Experiment 1, we were confident enough to test this effect again in Experiment 2 as a part of the confirmatory data analyses. Employing the split samples procedure designated for the exploratory analyses yielded the same conclusion. This was the only departure from our intentions as detailed in the preregistration. .07 Note. Support glob = support for globalization; Attitude EU = Attitude towards the EU; Satisfact democr = Satisfaction with democracy; Significant effects at p With regards to perceived competence, the results revealed no significant effect of the interaction between issue versus framing and anger expression, F(1, 1001) = 1.20, p = .274, η p 2 = .00. There was also no main effect of issue versus antagonism framing, F(1, 1001) = 0.68, p = .410, η p 2 = .01. Anger expression did affect perceived competence, however, F(1, 1001) = 9.76, p = .002, η p 2 = .01, with higher perceived competence in the disagreement conditions than in the anger conditions (see Table 3). This result is similar when controlling for all control variables (Table 4).

Exploratory Analyses
In the training data set, we tested moderators of the effects of our experimental manipulations on perceived appropriateness, attitude, and willingness to act. Only when a variable significantly moderated the effect of anger expression in the training data set, did we investigate this interaction again in the test data set. Using this procedure, we found that attitude towards globalization, p = .023, attitude towards the EU, p = .005, and satisfaction with democracy, p = .049, significantly moderated the effect of anger expression on perceived appropriateness in the training data set. However, only attitude towards the EU significantly moderated the effect of anger expression on perceived appropriateness in the test data set, B = −0.08, t(373) = −1.97, p = .050, r semi-partial = −.10. Investigation of the simple slopes (in the total sample) revealed that, for participants with positive attitudes towards the EU (one SD above the mean), anger expression resulted in lower levels of perceived appropriateness than nonemotional disagreement, B = −0.16, t(997) = −3.32 p = .001, r semi-partial = −.10. For participants with negative attitudes towards the EU (one SD below the mean), there was a significant positive effect of anger expression, B = 0.13, t(997) = 2.67 p = .008, r semi-partial = .08. No other moderation effects were found.
Exploratory mediation analyses. The finding from Experiment 1 that anger expression leads to lower perceived appropriateness and that lower perceived appropriateness leads to lower levels of persuasion was not replicated in Experiment 2. The exploratory analyses, however, revealed an interaction between anger expression and attitude towards the EU for perceived appropriateness. Although the total effects of this interaction on attitude and willingness to act were not significant, it is still possible that there is an indirect effect on attitude and willingness to act through perceived appropriateness that is dependent on attitude towards the EU (Hayes, 2009). To explore this possibility, we took the entire sample of participants and used PROCESS to estimate a mediated moderation model, in which attitude towards the EU moderated the effect of anger expression on perceived appropriateness, and perceived appropriateness mediated an effect on attitude and willingness to act.
The results showed that there was a significant effect of the interaction between anger expression and attitude towards the EU on perceived appropriateness ( Here as well, the negative indirect effect of anger expression through perceived appropriateness on willingness to act found in Experiment 1 was replicated, but only for those with positive attitudes towards the EU. For those with negative attitudes towards the EU, a positive indirect effect of anger expression was found.

General Discussion
Anger is a prominent feature of political news messages. The consequences of this phenomenon for the effectiveness of political messages remain underexplored, however. We therefore investigated the effects of anger expression in a political news context, using fabricated news articles in which a fictional politician expressed either anger or nonemotional disagreement in a persuasive message. In Experiment 1, participants rated anger expression as less appropriate than nonemotional disagreement. In addition, a politician expressing anger was seen as less likable and less competent. In Experiment 2, the negative effects of anger expression on perceived likability and competence were replicated. The negative effect of anger expression on perceived appropriateness was only found for participants who may have been particularly unreceptive to the politician's appeal, while a positive effect of anger expression on perceived appropriateness was found for participants who may have been particularly receptive to the politician's appeal. In both experiments, there was an indirect effect of anger expression, via perceived appropriateness, on both our persuasive outcome measures: attitude and willingness to act, although in Experiment 2 this indirect effect was moderated by receptiveness to the appeal. Contrary to our expectations, however, the total effects of anger expression on attitude and willingness to act were not significant in both studies.
So how should we interpret these findings? It should be noted that our issue versus antagonism framing did not affect the effects of anger expression in Experiment 2, but that attitudes towards the EU did. We expected that anger expression would be more persuasive when it was justified with claims of wrongdoing than when it was not. Such a justification could potentially draw receivers' attention away from possible internal attributions for the anger expression (i.e., the sender's angry disposition) and direct their attention to the external causes of the anger, that is, the alleged wrongdoing. There was no evidence that this was the case, however. Nevertheless, it would be too early to conclude that perceptions of antagonism/harm of concern do not have consequences for the effects of anger expression. In this regard, it is notable that attitude towards the EU did moderate the effect of anger expression on perceived appropriateness in Experiment 2.
We included attitude towards the EU as it has been shown to be a major predictor of support for TTIP (Steiner, 2016). As such, it could be concluded that a priori attitudes towards the sender's position influence the persuasive effects of anger expression. Two other determinants of support for TTIP (attitude towards globalization and satisfaction with democracy) had no moderating effect, however, so it is hard to draw definitive conclusions on the role of a priori attitudes. It is likely, however, that those with negative attitudes towards the EU, besides from being less likely to support TTIP, were also more likely to harbor feelings of antagonism towards the EU. To the extent that a negative attitude towards the EU is the result of feelings of antagonism or suspicion, our assessment of attitude towards the EU may have successfully tapped into a feeling that our manipulation of antagonism framing failed to affect: the sense that the individual and the EU are "on different sides." Future studies could investigate the factors that determine the effects of anger expression in political communication. As mentioned in the introduction, several studies have found negative effects of anger expression in situations of social influence (E. A. Van Doorn et al., 2015;Yip & Schweinsberg, 2017), but two previous studies found evidence suggesting that anger can be persuasive in some circumstances (Van Kleef et al.,Study 4 and 5). In Experiment 2, anger expression was deemed inappropriate by those with positive attitudes towards the EU, but appropriate by those with negative attitudes towards the EU. This lends support to the notion that emotion expression can be effective as a strategy for social influence in situations in which it is likely to be deemed appropriate, but it may backfire in situations in which it is likely to be deemed inappropriate (Van Kleef, 2009). It is worth investigating which other characteristics of the receiver and the message can make anger expression appropriate and persuasive, or inappropriate and not persuasive.
One question that may be asked is whether our materials also manipulated negativity or incivility, rather than only anger expression. In the introduction, we argued that there is a distinction between anger expression and negativity and between anger expression and incivility. But does this distinction hold up in light of our results? With regard to negativity, it is important to note that the control condition employed political speech in which the sender disagreed with the European Union and argued against TTIP. Strictly speaking, this in itself constitutes negativity in the sense that the sender focused on the downsides of others' policies, rather than on his own plans or solutions. The fact that our anger manipulation nevertheless affected several of our outcomes suggests that anger expression can have effects over and above negativity per se. With regard to incivility, the issue is somewhat more complex. If incivility is defined as political speech that does not confirm to social conventions about the polite expression of opinions, then the present research suggests that anger expression can be perceived as uncivil. But there can still be many contexts in which anger expression will not be perceived as inappropriate or uncivil, but rather completely morally justified (the moderating role of attitudes toward the EU in Experiment 2 hints at this). As such, we propose that our conceptual distinction between anger expression and incivility still holds. The present research contributes to the literature by showing that anger expression can be perceived as inappropriate and lead to perceptions of the angry person as not particularly likable or competent, but that this may depend on characteristics of the receiver.
Another thing to note is that the design of the present research precludes us from determining with certainty whether it was anger per se that drove the effects or whether emotionality more generally is responsible for the observed effects. We venture, however, that it is likely that anger drove the effects, as anger expression may be subject to strict display rules. Furthermore, emotional appeals are extremely common in political communication (Brader, 2005(Brader, , 2006. Previous research has focused specifically on enthusiasm, hope, and fear as predominant emotions in political advertising. Although this may be contingent on the specific context, there is no indication that such emotions are generally perceived as inappropriate. The (qualified) effect of our anger manipulation on perceived appropriateness is therefore likely due to anger. Still, the question of whether additional factors were manipulated is complicated. We did not include confound checks in the present research, and so we cannot be sure that the anger expression manipulation did not affect participants' experiences in some unforeseen way. Future research should include assessments of participants' emotions and a wider range of perceptions of the sender to investigate whether anger expression influences these factors.
In addition, it should be noted that our sample was Dutch, while the (fictional) sender of the persuasive appeal was Danish. Persuasive appeals are generally more powerful when they come from someone who is perceived as similar (Howard & Kerin, 2011;Pandelaere, Briers, Dewitte, & Warlop, 2010), and perhaps this is especially likely to be the case for a persuasive appeal that includes emotional expression. As such, the persuasive effect of anger expression may have been stronger if the sender would have been Dutch, or if the sender was more similar to the participants in some other way. Future research should investigate this.
Another thing to note is that no overall effects of anger expression on attitude and willingness to act were found in the present research. Thus, the negative indirect effect of anger expression through appropriateness in Experiment 1 did not result in a total effect of anger expression on attitude and willingness to act. Similarly, in Experiment 2, the interaction between anger expression and receptiveness had a significant effect on perceived appropriateness, but not on attitude or willingness to act. One explanation for this could be the existence of a separate psychological process, affecting persuasion in a positive way and counteracting the negative effects of anger expression via perceived inappropriateness. An important factor could be perceptions of the sender as dominant, high status, and/or an effective leader. People often attribute dominance and high status to those who express anger (Tiedens, Ellsworth, & Mesquita, 2000). Potentially, politicians could take advantage of this, expressing anger in order to be seen as a dominant, effective leader (Tiedens, 2001). In the present research, increased perceptions of dominance or high status as a result of anger expression may have contributed to persuasion in a positive way, thereby annulling the negative effects of perceived inappropriateness (see also Van 't Riet, Schaap, & Kleemans, 2018, Study 4).
A final thing to note is the fact that effect sizes in Experiment 2 were small. As such, it would be overconfident to proclaim that our results can be useful to explain voters' perceptions of specific politicians, let alone election results. For the study of anger expression in political messages, however, they are an important starting point. Based on literature regarding "display rules" (Von Salisch & Vogelgesang, 2005) and on previous findings (E. A. Van Doorn et al., 2015;Yip & Schweinsberg, 2017), we theorized that anger expression is generally considered inappropriate in a political news context, making effective persuasion less likely and leading to negative perceptions of the sender. The results partly confirmed these expectations, suggesting that a negative effect is possible, but likely contingent on receivers' a priori attitudes.
The rather muted (and qualified) effects of anger expression found in the present research may seem surprising, given the increasing prevalence of anger expression in politics and political news (Sobieraj & Berry, 2011). Possibly, political actors do not use anger expression as a way to persuade undecided voters, but rather as a way to motivate and energize their supporters. The moderating role of a priori attitudes found in Experiment 2 suggests as much. Corroborating evidence comes from an experimental study that found a strong effect of an anger-inducement manipulation on a measure of political participation (Valentino et al., 2011, Study 1, see also Marcus et al., 2000;Ridout & Searles, 2011). However, the present results suggest that expressing anger can be a double-edged sword. While it can be effective as a strategy to energize the base, it risks alienating members of the broader general public.