Is Nonverbal Behavior During Conversation Related to Perceived Fluency?

As an important dimension of second language (L2) speaking, perceived fluency refers to a listener’s judgment of the smoothness in an L2 speaker’s oral performance. Researchers have explored the relationship between perceived fluency and various metrics of speaker utterances such as articulation rate, pause duration, and repair frequency; however, previous research has not examined the potential role of nonverbal behaviors in fluency perception. Therefore, for this study, we extracted conversations from an existing corpus and coded video-recordings of each conversation for six categories of speakers’ nonverbal behavior: (a) head movements, (b) eye movements, (c) eyebrow movements, (d) displays of positive emotion (smiling, laughter), (e) beat gestures, and (f) non-beat gestures. In correlational analyses, we examined associations between speakers’ nonverbal behaviors and their ratings of each other’s fluency (operationalized as speech flow). Our findings revealed that eyebrow movements and displays of positive emotion had a positive relationship with perceived fluency, while non-beat gestures showed a negative association. We discuss implications of our findings for L2 pedagogy.


INTRODUCTION
Second language (L2) students typically aim to be effective language users, which involves learning both verbal and nonverbal aspects of communication.One aspect of effective language use is fluency, which Segalowitz (2010) described in three ways.The first is cognitive fluency, which is efficiency in a speaker's operation of underlying production processes, for example, at the level of planning, monitoring, and executing an utterance.The second is utterance fluency, which refers to the observable speech features produced by a speaker, such as their pauses or repetitions.Finally, defined as "a judgment made about speakers based on impressions drawn from their speech" (Segalowitz, 2010, p. 48), perceived fluency concerns how a speaker's speech, such as its fluidity or smoothness, impacts the listener.Our focus is on perceived fluency and its previously underexplored link with L2 speakers' nonverbal behavior.
When it comes to observable dimensions of L2 performance that contribute to a speaker's perceived fluency, listeners tend to primarily rely on temporal dimensions of speech such as articulation rate, pausing, and repair in the form of repetitions and self-corrections (Bosker et al., 2013;Kahng, 2018;Saito et al., 2018;Williams & Korko, 2019).Other linguistic dimensions that underpin perceived fluency include grammar and pronunciation, where speakers' morphological and syllable structure errors are associated with lower perceived fluency for listeners (Rossiter, 2009;Suzuki & Kormos, 2020).In terms of the relative weight of various speech characteristics, according to a recent meta-analysis (Suzuki et al., 2022), listeners most strongly associate perceived fluency with articulation speed and pause frequency as opposed to other linguistic dimensions.Considering that up to 60% of variance in perceived fluency in Suzuki et al.'s extensive metadata was unexplained through measures of speaking speed, pausing, and repair, both L2 students and their teachers might find it useful to know which other aspects of communication are tied to perceived fluency, making a speaker appear more or less fluent to listeners.
Considering the tight coordination between speech and various body signals in the form of gesture, torso movement, and eye gaze (Holler, 2022), a speaker's perceived fluency might be associated with various nonverbal behaviors.For example, a speaker might use a round movement of a forearm with an index finger pointing downward when searching for and retrieving a lexical item to describe a cake (Kendon, 1980).In this case, the speaker's reliance on gesture for word retrieval might be a sign of word-finding difficulty (Krauss et al., 2000), which would translate into observable dysfluency phenomena, including hesitations and pauses, all contributing to a decrease in perceived fluency.Alternatively, speakers use various gestures (e.g., hand, head, shoulder, eyebrow, or finger movements) to demarcate the beginning and ends of meaningful informational chunks or phrase groups in their speech (Kita, 2000).These behaviors might simplify speech segmentation for listeners and emphasize particularly important content (Drijvers & Özyürek, 2017(Drijvers & Özyürek, , 2020;;Hardison, 2018), with a positive impact on perceived fluency.According to yet another perspective, the visual information available through facial expressions (e.g., smiling, frowning) and gestures might evoke visuospatial imagery for observers, and this additional detail may increase perceived quality of a speaker's speech (Freedman, 1977), including perceived fluency.Put simply, speakers' use of nonverbal behaviors may be associated with how listeners perceive their fluency.
Given that there are no studies known to us that investigate the relationship between nonverbal behavior and perceived fluency, we conducted an exploratory, corpus-based study targeting this issue.Because nonverbal behaviors occur most naturally in interaction rather than in monologic performances, we specifically explored this relationship in L2 conversations.In addition, rather than ask external raters such as teachers, naïve listeners, or trained assessors to provide perceived fluency ratings, we asked the conversational partners to evaluate each other's perceived fluency, assuming that interlocutor perceptions of each other's fluency can impact their interaction.Our study was guided by the following exploratory question: Is there a relationship between the frequency and type of L2 speakers' nonverbal behaviors and their perceived fluency, as evaluated by their interaction partner?

Participants and Data Collection
As part of a larger corpus of interactions involving L2 English university students (McDonough & Trofimovich, 2019), we selected conversations between 40 students (20 females, 20 males), with a balanced distribution of students' self-reported genders across conversations (6 femalefemale, 6 male-male, 8 female-male).These students (Mage = 24.1 years, SD = 4.1) were enrolled in undergraduate (n = 28) and graduate (n = 12) degree programs at English-medium universities in Montreal, Canada.They had been paired with students from different first language (L1) backgrounds, the majority being L1 speakers of Mandarin, Farsi, and Spanish, to talk about three topics for 10 minutes per topic: Moving to Montreal, personal close-call experiences, and an academic research discussion.We focused on their conversations during the moving to Montreal task in which they described challenges they experienced and how to overcome those challenges.They carried out this task first, which meant that they had no prior knowledge of or perceptions about each other.Compared to the close-call task (where students exchanged monologic narratives) and the academic discussion (where they compared their understanding of two research texts on the same topic), the moving to Montreal task was the most interactive conversation with the closest resemblance to everyday conversation.The task encouraged interlocutors to share personal (and often emotional) experiences as recently arrived international students who had to overcome similar challenges while settling in and adjusting to a new environment.After the task, each student used a 100-point sliding scale to rate various dimensions of their own and their partner's performance.The key rating for this study is the partner rating of perceived fluency as flow, which was defined as the ability to speak with ease and fluidity and without many pauses and hesitations.Their conversations were audio-and video-recorded.

Data Analysis
Video recordings of all interactions were analyzed through a bottom-up process to identify speakers' nonverbal behaviors.The initial coding yielded a comprehensive behavior set which included eyebrow raises and frowns (for eyebrow movement), eye gaze directed upward, downward, or aside (for eye movement), smiling and laughter (for displays of positive emotion), head nods, tilts, and shakes (for head movement), and various types of gesture.After consulting prior literature (e.g., Kita, 2000;McNeill, 1992), including our own work (e.g., McDonough et al., 2022), larger categories were combined under six main themes through iterative coding, for example, with head nods and head shakes combined under the category of head movements and eyebrow raises and frowns contributing to a single eyebrow movement count.Thus, as shown in Table 1, the six final categories were head, eye, and eyebrow movements, displays of positive emotion, and two types of hand gestures (beat and non-beat).Beat gestures included hand movements that followed the stress and rhythmic patterns of a speaker's utterance.Non-beat gestures encompassed the combined count of iconic, metaphoric, and deictic gestures because individual incidence of these gesture subtypes was too small to be considered in separate categories.Nonverbal behavior while listening was not included in the analysis because it rarely occurred.To check the reliability of our coding, we trained a research assistant in the coding categories, after which she coded 20% of the videos independently.The two-way mixed intraclass correlation coefficients were as follows: head movements (.89), eye movements (.93), eyebrow movements (.93), displays of positive emotion (.96), non-beat gestures (.91), and beat gestures (.77).To account for variation in the frequency of occurrence, the sum in each behavior category was normalized by dividing it by each student's total word count multiplied by 100, which resulted in a frequency of each coded category per 100 spoken words.French and I did the same thing Note.P5 refers to Participants 5; /---/ denotes unintelligible speech.

FINDINGS
First, we totaled how often the behaviors occurred (sum and normed mean) and compiled the partner flow ratings, both of which are provided in Table 2. Eye movements were the most frequent (approximately 3 instances per 100 spoken words), followed by beat gestures (about 1.4 instances per 100 spoken words).Then, head movements, eyebrow movements, and displays of positive emotion showed similar frequency rates (at the rate of about 1 instance per 100 spoken words), while non-beat gestures did not occur very often (with about 1 instance observed every 400 words).For perceived fluency, students gave their partner a mean flow rating of 81.43 out of 100 (SD = 12.64).To explore the relationship between the nonverbal behaviors and the flow ratings, we obtained non-parametric, rank-ordered (Spearman) correlation coefficients because five of the six nonverbal behavior counts (except eye movements) were non-normally distributed (p < .029according to Shapiro-Wilks tests).To interpret the relationships, we applied benchmarks from applied linguistics research (Plonsky & Oswald, 2014) that describe correlation coefficients as weak (.25), medium (.40), and large (.60).Therefore, we considered as meaningful only the associations that reached or surpassed the benchmark for a weak association (.25).As shown in Table 3, non-beat gestures had a weak (yet non-significant) negative relationship with flow ratings, which means that as non-beat gestures decreased, flow ratings increased.In contrast, there were weak-to-medium positive relationships between flow ratings and displays of positive emotion and between flow ratings and eyebrow movements.As eyebrow movements and displays of positive emotion increased, so did flow ratings.

CONCLUSION
Our findings showed that nonverbal behaviors during speaking had both positive and negative associations with perceived fluency.For the negative relationship between non-beat gestures and perceived fluency, conversational partners may have interpreted gestures as dysfluency signals.
For instance, non-beat gestures may have occurred as speakers had trouble retrieving a word (Krauss et al., 2000).Alternatively, non-beat gestures may have occurred with mid-clause pauses, which are particularly detrimental to perceived fluency (Kahng, 2018).In these cases, gestures provide visual cues to the interlocutor that a speaker has experienced difficulty, for instance, with retrieving a lexical item, assembling a grammatically appropriate utterance, or executing a smooth articulation plan.This is a novel finding, because a speaker's use of non-beat gestures has been previously shown to contribute positively to word-level intelligibility (Drijvers & Özyürek, 2017) and utterance-level comprehension (Sueyoshi & Hardison, 2005).While nonbeat gestures may aid the interlocutor in extracting the meaning content of an utterance, the same gestures might simultaneously signal that a speaker is experiencing dysfluency.In contrast, both eyebrow movements and displays of positive emotion had positive relationships with perceived fluency.As far as eyebrow raises and frowns are concerned, these facial cues may have highlighted speech prosody for the interlocutor by demarcating phrase boundaries or important speech content (Pelachaud et al., 1996).Put differently, eyebrow movements may have enhanced prosodic cues to speech segmentation and comprehension for the interlocutor (Krahmer & Swerts, 2007), thus contributing to speakers being perceived as more fluent.In terms of displays of positive emotion, which had the strongest relationship with perceived fluency, our findings extend previous work where laughter was found to facilitate the flow of interaction and to maintain the interest of a conversation partner (Vettin & Todt, 2004).
As shown here, displays of positive emotion, including laughter, might similarly enhance the perception of a speaker's conversational fluency.Smiling and laughter are also more common among low-versus high-anxious L2 speakers (Gregersen, 2005), and speakers may smile or laugh to show that they are engaged and comfortable interacting with their partner (Hardison, 2018), so these signs were likely interpreted by the interlocutor as indicators of efficient, fluid, comfortable speech flow.
Our findings must be interpreted in light of several limitations.First, our study relied on an existing dataset, which means that our analyses and interpretations are limited to observations of speaker performances in a set task.Many behaviors such as pursed lips or face frowns were not attested, while the occurrence of others (e.g., non-beat gestures to illustrate concepts) may have been minimized by task demands, which prioritized sharing of common experience rather than exchanging of novel information.Second, our speaker sample was too small to conduct finer grained analyses of perceived fluency by speakers' self-reported gender or to examine coordination of nonverbal behaviors (and its association with perceived fluency) across conversation partners.Finally, as pointed out by external reviewers, our dataset did not allow us to explore interesting questions about speakers' use of nonverbal behaviors and various aspects of cognitive fluency (e.g., a speaker looking away from the interlocutor as a sign of a wordfinding difficulty) and utterance fluency (e.g., a head nod or a gesture co-occurring with a midclause pause).These and other interesting questions await future work.
Although the relationships were not strong, the findings point to the importance of discussing nonverbal behaviors with students as they can potentially affect how they are perceived by their conversational partners.For example, L2 teachers can introduce different nonverbal behaviors and discuss various ways in which they can be interpreted by different interlocutors (e.g., who share or do not share cultural knowledge) and in different contexts (e.g., in an interview vs. a class presentation).Illustrating the behaviors associated with perceived fluency can raise students' awareness about which types of behavior to avoid or practice (for a teacher-oriented guide to nonverbal behaviors, see Gregersen, 2007).For example, excessive hand gestures, averted eye contact, or crossed arms may demonstrate a lack of confidence or defensiveness, whereas laughter may project the feelings of interest and comfort.Therefore, students can learn, practice, and transfer these skills to other contexts, such as speaking tests, presentations, and interactions outside the classroom.In sum, helping students become aware of the important role played by nonverbal behavior in conversation can help them successfully communicate.

Table 1
Definitions and Examples of Nonverbal Behavior

Table 2 .
Descriptive Statistics for Fluency (Rating on a 100-point Scale) and Nonverbal Behaviors (Mean Rate per 100 Spoken Words)

Table 3
Spearman Correlation Coefficients and Associated p-Values (in Parentheses) for Speakers'