An empirical investigation on the benefits of gamification in communication within university development teams

The field of software management recognizes communication as a fundamental aspect, which has garnered significant attention in recent years. In response, scholars have advocated for the implementation of innovative pedagogical approaches to foster students' engagement in communication. One such approach is the utilization of gamification, the integration of game design elements into nongame contexts, which has been demonstrated to be an effective teaching technique across various fields of education. The aim of this study was to investigate the impact of gamification on in‐team communication quality within the context of software development. To this end, an experimental design was employed with four teams of third‐year students enrolled in software development courses. The results of the study indicate that the introduction of gamification significantly enhances communication quality among the students. Furthermore, our findings indicate that a specific gamification strategy, feedback, plays a crucial role in guiding the students toward desired communication behaviors. On the basis of the results of this study, it can be concluded that gamification presents a promising approach to enhance communication quality within software development teams.


| INTRODUCTION
The practice of software management is a critical aspect of software engineering, as highlighted by Bacharach and Aiken [3] and Cordero and Farris [15].Communication is a vital component of software management, playing a crucial role in the success of team software development processes [66].Communication is also a central tenet of all collaboration practices and processes, as it contributes to building trust among team members, establishing a shared understanding of task responsibilities [51,66], and facilitating the sharing of knowledge and information among team members [26].Effective communication can also help prevent conflicts and achieve [60].Therefore, it is extremely important to be able to communicate effectively within the development team.
However, literature in the field of software engineering suggests that communication within development teams can pose significant challenges.This challenge was particularly conspicuous during the COVID-19 pandemic [48].These challenges include geographical distance that limits face-to-face communication, time zone differences that prevent synchronous communication, language, and cultural differences that lead to misunderstandings, professional knowledge differences that cause misunderstandings in requirements [32], inadequate communication channels that obstruct knowledge sharing [60], and insufficient communication skills that hinder the generation of high-quality software engineering documents [44].
The use of gamification as a technique for improving user's performance has garnered attention in various fields, including education [24], healthcare [25], tourism [63], and business [31].In recent years, gamification has gained popularity in educational settings as a means of supporting and motivating students to enhance their learning outcomes [34].Despite the growing interest in gamification, limited research has been conducted on the application of gamification in software engineering education with a focus on areas, such as requirement documentation [46], development process management [40], and software validation and verification.However, there is still a need to examine the impact of gamification on communication within software development teams.
Several challenges and limitations exist in the application of gamification strategies to improve communication within development teams.First, the empirical evidence of the effectiveness of gamification strategies remains limited [20,45].Second, the control group and treatment group must be properly constructed to accurately reflect the impact of gamification [13,20,40].Additionally, multiple game elements are often studied in unison, making it challenging to isolate the impact of individual elements [13,40,41,46].For example, feedback, which is one of the most used gamification strategies, is worth to explore as it might be a hint to guide participation in a desired direction [43].Finally, while the impact of gamification on communication quality may be established, questions still remain regarding the sustainability of this impact [33].Further research is necessary to examine the longterm effects of gamification on communication quality.
We aim to address the research questions, "Can gamification improve students' communication quality within development teams?" "Does use a specific gamification strategy, feedback, guides participation toward to desired behaviors?""Does gamification effect endure after withdrawing?"To address these research questions, we conducted an experiment with four agile software development teams comprised of 21 students each, all enrolled in the same software development module.Our experiment was designed to assess the effectiveness of gamification strategies, specifically focusing on the role of feedback in guiding participation toward desired behaviors.Furthermore, we explored the sustainability of the impact of gamification on communication quality.To better understand the effectiveness of gamification, we perform an empirical analysis (bivariate analysis), and distribute the questionnaire to students, such questions including: "Do you more involved in the team when the gamification system appeared?""Do you more engage in communication?" and so forth.
Our contribution to the current literature is the provision of empirical evidence that the implementation of appropriate gamification strategies leads to an improvement in the quality of communication.Specifically, we found that the gamification strategy of feedback can effectively direct participation toward desired behaviors and that the impact of gamification endures after its removal.Additionally, based on previous research, we synthesize the key characteristics of high-quality communication and identify areas in the communication process that can be facilitated through gamification.Furthermore, we formulate various gamification strategies and associate them with relevant behaviors.To further validate our findings, we also present graphical evidence and distribute a questionnaire to the student participants.
This paper proceeds as follows: Section 2 documents the related work and hypotheses development, and Section 3 introduces the experimental setting.Section 4 reports methodology.The results are presented in Section 5, and the validity is discussed in Section 6.Finally, this study concludes with a discussion in Section 7.

| RELATED WORK AND HYPOTHESES DEVELOPMENT
Software development is a complex process that involves the integration of various areas of knowledge [22], and effective communication plays a crucial role in this regard.Communication has been recognized as a critical aspect of software management, and has been linked to improved project performance [5,12,57,59].

| Characteristics of communication
In the literature, communication has been defined by several characteristics, including frequency, formalization, structure, and openness of information exchange [29].The frequency of communication refers to the REN and BARRETT | 1809 amount of interaction between team members, which is positively associated with the quality of communication, such as the time spent on communication.The degree of formalization, which encompasses both formal and informal communication, reflects the level of spontaneity in communication among team members.Formal communication involves a significant amount of preparation and planning, such as scheduled meetings and written status reports, while informal communication includes spontaneously initiated contacts, such as messages, short phone calls, and quick emails.The communication structure pertains to the ability of team members to communicate directly with all other members of the team.

| Distance of communication
Another concept used to describe the quality of communication is distance, which has been empirically established in the field of software engineering [4,5,7,27,36].Distance refers to the difference in position or level between entities, and it has been found to impact the workload of communication within software development teams and increase the likelihood of communication gaps [7].The distance can take various forms, including geographical, organizational, psychological, and cognitive distances [7].
Geographical distance refers to the physical distance between team members, while organizational distance is characterized by differences in rank or position within an organizational structure.Psychological distance refers to the perceived difficulty of communication between team members, and cognitive distance encompasses the differences in knowledge, understanding, and other forms of cognition between team members.

| Improving communication
Prior research has established that communication traits can be altered through a variety of behaviors, thereby enhancing the quality of communication [29].
Regarding the communication characteristics part, for instance, increasing communication frequency has been shown to enhance communication quality [50].The formalization of informal communication can lead to improved team performance, as it enables team members to more efficiently and effectively share, discuss, and evaluate their ideas and contributions [35,50].This approach is deemed particularly effective, as informal communication serves as the primary coordination mechanism within the team [21,38].By establishing structures that facilitate direct communication among team members, the quality of communication can be enhanced as exchanging information through intermediaries (e.g., team leaders) is more timeintensive and may result in incorrect transmissions [23,50].Additionally, enhancing the openness of information exchange can facilitate the integration of the knowledge and experience of team members on shared tasks, which in turn can benefit communication [23,50].
In terms of communication distance, the reduction of distance has been found to be a key factor in promoting the quality of team communication.For example, bringing developers from different locations and levels within an organization together to mitigate geographical and organizational distance [6].

| Challenge in communication
Nevertheless, the aforementioned behaviors, which have the potential to enhance the caliber of communication, present a formidable challenge to implement, for example, insufficient communication among team members due to the synchronous communication barrier, limited face-to-face communication, and misunderstandings regarding requirements obstacles to knowledge sharing [32,60].Another reason that people are reluctant to engage in these behaviors may be owing to the dearth of motivation in embracing them [54].
In recent years, the gamification approach has been introduced as a potential means of changing behaviors and providing motivation.The implementation of this technology presents an opportunity to explore new and innovative ways of addressing the impact of distances and characteristics on team communication.

| Gamification
The utilization of gamification has garnered significant attention in recent years and has been implemented across a wide range of domains, including education, healthcare, tourism, and business [24,25,31,63].Within the field of computer science, gamification is defined as the incorporation of game design elements into a nongame setting [19].Gamification utilizes gamification strategies, which are a set of rules designed to alter human behavior, attitudes, and motivation, to attain improved outcomes [19,25].
The utilization of gamification in software engineering has witnessed an upsurge in recent years [9,40].In the aspect of development practice, for instance, Prause and Jarke [52] proposed that gamification can enhance coding conventions, while Towey et al. [62] suggested that incorporating gamification into the software development process can yield benefits in software testing.Additionally, Manohar et al. noted an increase in the pool of professionals with contemporary software verification and validation knowledge and skills as a result of gamification.
Moreover, scholars have begun to focus on the application of gamification in software management practice, including boosting team performance through improved collaboration [49], incentivizing team members to complete their timesheets, and facilitating proper documentation execution [46].Despite these efforts, there has been limited research into the application of gamification strategies in communication practice in software engineering [22,24], even though gamification has demonstrated efficacy in promoting communication efficiency in the business [64] and tourism domain [58].For example, previous research in the business domain has explored the relationship between game elements, instructional design, and business communication.In the tourism sector, studies have evaluated the potential of gamification in online tourism marketing, indicating that it can effectively boost customer loyalty, brand awareness, online engagement, and revenue for successful online advertising activities.
Therefore, our first hypothesis emerges from the above arguments: H1: Gamification improves students' communication efficiency within a development team.

| Further exploration of gamification
The implementation of appropriate gamification strategies is essential for the successful application of gamification to the practice of software engineering [53].Furthermore, the corresponding strategies must be further investigated.According to the previous literature, the three most commonly used gamification strategies are points, leaderboards, and feedback [13,20,37,47,55].Recent studies have demonstrated that the introduction of these strategies can result in changes in motivation and behavior [17,30].For example, students may modify their behavior to acquire more points or a higher position on the leaderboard.In addition to points and leaderboards, which are essential components of gamification strategies [25], feedback is also a popular gamification strategy [14].Previous research has theoretically suggested that feedback can be seen as a hint, which guides participation in activities [43].However, an empirical validation of the effectiveness of feedback as a gamification strategy is still lacking.Therefore, a question arises that do feedback guide participation toward specific behaviors?H2: Gamification strategy feedback guides participation toward desired behaviors.
The sustainability of gamification in the software engineering field remains an understudied area.To date, research has primarily focused on the sustainability of gamification in contexts outside of software engineering, such as classroom engagement [61], taking quizzes [56], and reading behaviors [39].These studies have shown that the effects of gamification persist even after the withdrawal of the gamification elements.However, the sustainability of gamification effects in the software engineering field requires further investigation [33].Given that gamification has been shown to play a role in enhancing communication efficiency, it remains an open question as to whether the effects of gamification persist beyond the presence of gamification elements.

| SETUP OF THE EXPERIMENT
We conducted a controlled experiment involving 84 third-year undergraduate students from an Irish university.The experiment was conducted over a period of 38 days during the academic year of 2022.Among the participants, there were 63 males and 21 females, all within the age range of 20-23.The majority of the students, approximately 64 individuals (76%), had English as their native language.The students were randomly divided into four development teams (A, B, C, and D), each consisting of 21 students.All participants engaged in group-based collaboration and received the module through a blended format of online and inperson delivery.To ensure that exogenous variables did not interfere with our results, participants possessed similar programming experiences and academic backgrounds, as they were all enrolled in the same module (Software Design and Implementation) within the same major and university and received instruction from the same instructor.Additionally, it is noteworthy that none of the students dropped out or withdrew from the experiment during the course of the study.
Data was collected via Discord, a social communication platform that was already familiar to all students. 1 Each student was assigned group tasks that were released 1 Discord is an American VoIP and instant messaging social platform.Users have the ability to communicate with voice calls, video calls, text messaging, media, and files in private chats or as part of communities called "servers."See: https://discord.com.

REN and BARRETT
| 1811 in week 7 and the experiment lasted approximately 6 weeks (38 days).The experiment ended 1 week before the task deadline to eliminate any confounding factors that could arise from the pressure of an impending deadline [16].The objective of the experiment was for the students to develop a sophisticated application for the Android system, which refers to a software program that exhibits a high level of complexity, advanced features, and refined user experience on the Android platform.
In accordance with the recommendation by Callan et al. [11], to effectively use gamification in educational settings, the students' participation must be entirely voluntary [11].Therefore, the experiment was completely voluntary and did not offer any grading rewards, allowing students to exercise their autonomy in participating in the experiment.
In this study, three gamification strategies were adopted: points, leaderboard, and feedback.The experiment was divided into three parts to test the hypotheses.Part one followed the Pretest-Posttest Control Group Design and involved only Groups A and B. During the first 18 days, the students in Groups A and B did not receive any gamification treatment, serving as the control group.On day 19, the students in Groups A and B were introduced to the gamification strategies and instructions on how to "play."From days 19 to 32, the students in Groups A and B received the gamification treatment every 2 days, which served as the treatment group.Gamification strategies were communicated to the students through private messages on the Discord platform.These messages included information on their points, an explanation of how points were calculated, and their position on the leaderboard (as depicted in Table 1).
In the second part of the experiment, we aimed to examine the effect of a specific gamification strategy, feedback, on the students' behavior.As previously discussed, the gamification strategies utilized in the study were points, leaderboard, and feedback.To increase the number of participants in the experiment, we designate Groups A and B as the control group in Experiment part 2. This decision is based on the implementation of gamification strategies, specifically providing points and a leaderboard, for the control group in this phase.It is noteworthy that these gamification strategies mirror those employed for the treatment group in Experiment part 1.Consequently, Groups A and B can be regarded as the control group in Experiment part 2.
We designated Groups A and B as the control group for the period from day 25 to 32, while Groups C and D were designated as the control group from day 19 to 24.The control group received gamification treatment consisting of points and leaderboards, administered every 2 days.The treatment group, consisting of Groups C and D from day 25 to 32, received additional strategy, feedback, administered every 2 days (as depicted in Table 2).
To investigate the persistence of gamification effects beyond the treatment period, we randomly selected Group B for analysis.During the period from day 19 to 32, the students in Group B received a gamification treatment, consisting of the implementation of points and leaderboard, on a bidaily basis.Subsequently, from day 33 to 38, the delivery of the gamification treatment was discontinued, as documented in Table 3.
Regarding the changing personal relationship among the students, in our study, the participants consisted of students from various majors, including Computer Science, Information System, Computer Engineering, and so forth.They were randomly assigned to different groups, suggesting that they did not have pre-existing strong personal relationships.Moreover, we observed that the students did not engage in any postproject discussions or chats on the Discord platform.On the basis of these factors, we believe that their relationships did not significantly change or develop into strong personal connections.

| METHODOLOGY
In this study, we examine the effectiveness of gamification on communication practice using graphical evidence and bivariate analysis.To this end, a sample of 84 observations was collected from Discord.To test our first hypothesis (H1), that gamification has an effect on communication, we utilized an indicator variable to represent gamification as the independent variable, and communication variables based on communication characteristics and distance theory, as the dependent variables.For our second hypothesis (H2), that feedback guides participation toward specific behaviors, we used feedback as the independent variables and communication proxies as the dependent variables.For our third hypothesis (H3), that gamification treatment (Sustain) has an effect on communication, we employed an indicator variable for the gamification treatment as the independent variable, and communication as the dependent variable.Additionally, a questionnaire was used to assess the subjective perceptions of the students.

| Measurement of communication
By employing a variety of behaviors to modify communication characteristics and decrease the distance, the quality of communication can be improved [7,29].Due to the close relationship between communication characteristics and the concept of distance, we have classified the targeted behaviors aimed at improving communication quality into five categories: frequency and formalization, structural and organizational distance, openness of information exchange and cognitive distance, psychological distance, and geographical distance.
In this study, we used six variables to reflect different aspects of communication quality in virtual teams. 2 First, we use the Number of Messages (NMs) to reflect the frequency and formalization.Frequency refers to the amount of communication between team members, and increasing communication can help address the issue of insufficient communication [28].Thus, a higher value of NM indicates higher frequency and better communication quality.We focus only on informal communication in terms of formality, represented by messages in Discord.
Second, we create a variable, the NMs on the Proper Channel (PC), to represent the structure and organizational distance.Structure refers to direct communication between team members without intermediaries, and reducing organizational distance requires communication within the same hierarchy of units and departments.PC reflects whether students communicate directly with the right people and avoid intermediaries.A higher value of PC indicates more communication within the same hierarchy, and better structure and lower organizational distance.
To assess the substantive quality of the messages exchanged within a team, it is imperative to consider factors beyond the mere frequency of message exchange [11].To this end, we have developed a variable, referred to as the Number of Times a Team Member Shares Their Work or Knowledge with the Team (SK), to reflect the concepts of openness of information exchange and cognitive distance.These concepts focus on the differences in knowledge, understanding, and other cognitive factors between team members.A higher value of SK indicates lower cognitive differences between team members, thereby reflecting improved communication quality.
To represent the concept of psychological distance, which is the degree of subjective effort a team member perceives it takes to communicate with another, we have constructed two variables.The first variable is the number of replies to messages (RM), which reflects the timely response to previous messages.The second variable is the NMs for sharing or answering questions (SA), which reflects the purpose of the communication being to ask or answer questions.A higher value of these two variables indicates a lower psychological distance, as the team members are working together on a common goal or task, and this has been shown to reduce psychological distance [7].
Fifth, the concept of geographical distance, which pertains to the physical separation between team members, can be reduced through in-person meetings.However, due to the pandemic, video meetings have been considered as an alternative, and all meetings are online.To quantify the frequency of video meetings, we have constructed a variable, "Number of Meeting Times (MT)," where a higher value of MT reflects a lower level of geographical distance between team members.The summary of the measurement of communication is shown in Table 4.

| Gamification design
In software engineering, the purpose of gamification is not simply to entertain engineers, but to serve as a tool to influence their behavior [19].Our objective is to evaluate the impact of gamification strategies on students' communication quality.The pivotal aspect of gamification design is the selection of appropriate strategies [53].Previous literature suggests that incorporating points, leaderboards, and rewards can effectively modify user behavior.One possible explanation is that students are more likely to engage in behaviors that are incentivized by the researcher, leading to a shift in their current behavior.In this paper, such behaviors refer to encouraging students to communicate in a manner that enhances communication quality and addresses previously defined challenges.To address Hypothesis 2, we also include feedback as an additional gamification strategy.As a result, in the construction of gamification strategies, points, leaderboards, and feedback are selected as crucial components.
Points and Leaderboard: The gamification rules are designed to give users points, which help them to improve their position on the leaderboard.Points are calculated based on whether students follow the behaviors that can improve the communication quality.The first part of the gamification rules are as below: -Frequency and formalization: 1. Creating one message on Discord (NM), the student will get 1 point.
-Structure and organizational distance: 2. If the message is created in the PC, the student will get an extra 1 point.
-The openness of the information exchange and cognitive distance: 3. If the student shares his or her work or knowledge with the team (SK), he or she will get 10 points every time.
-Psychological distance: 4. If the student replies to the message (RM), he or she will get an extra 1 point.
5. If the student share or answer a question (SA), he or she will get an extra 1 point.
-Geographical distance: 6.If the student attends a video meeting (MT), he or she will get 3 points (maximum two times per week).
To support students' self-coordination, they are notified of their position on the leaderboard, and they receive additional leaderboard bonus points based on their position.The rule for getting the bonus is as below: 7. The highest ranking each time will earn an extra 15 points, and the second highest will get an extra 14 points, and so forth.
Feedback: To encourage students to share knowledge, we set the desired behaviors are introducing personal work to the team (SK), RM, and sharing or answering any questions (SA).The feedback is a suggestion about how to get more points based on desired behaviors, such as "for improving your gamification score in the following, I suggest that you can try to introduce your work to your team and share or answer any questions."

| Measurement of gamification
Measuring gamification effectiveness is the key part of our research design.We treat gamification intervention as a "shock" during the experiment.We construct three independent variables, which are dummy variables, to reflect gamification's impact on communication for a different part.
In the first part, we construct the first independent variable, Gamification.Gamification is used to indicate that before and after receiving gamification treatment, which is 1 means after receiving gamification treatment, otherwise 0. This variable can help determine the effect of the gamification intervention.
In the second part, we construct the second independent variable Feedback, which compares the impact on communication between receiving feedback or not.This variable can help examine whether the continued addition of "feedback" to the previous gamification strategies would have a corresponding impact on communication.
In the third part, we generate the third independent variable Continue to reflect whether we continue to

| Questionnaire design
To further analyze the gamification influence on communication, an online questionnaire has been used to gain opinions of people engaged in realizing projects, and we adopt a cross-sectional design for the survey.The questionnaire included 6 questions and has been completed by 43 respondents (84 students in total), who belong to different groups.All the questions are close-ended questions, which can provide standardized answers.Each respondent marked their view of the question on a five-point Likert scale (not at all, not really, moderately well, well, and very well).The five-point Likert scales are a common rating format for surveys.
Respondents rank quality from high to low or best to worst using five levels [1].Here are the questions: Q1-Do you more involved in the team when the gamification system appeared?Q2-Do you more engage in communication?Q3-Do you feel the gamification strategy is fair?Q4-Do you easily understand how the point is computed?
Q5-Do you think this activity has positive motivating effects on communication in your team?Q6-Do you think this activity helps you to do additional effort in communication?

| RESULT
To ascertain whether gamification affects communication quality, we present graphical evidence to reflect the trend.Subsequently, we utilize bivariate analysis to investigate the correlations between gamification strategies and communication quality.

| Gamification and communication quality
In Figures 1 and 2, the first part of the experiment is depicted and spans from day 1 to 32, with 16 increments on the x-axis.This means that each value on the x-axis represents the average of 2 days for a particular variable.For instance, the value of 2 (x = 2) is the average of the variable's value between days 3 and 4. When x exceeds 10, teams A and B receive the gamification treatment.To effectively showcase the values of different dependent variables in the teams, two steps were taken to process the data.First, the mean value of each variable for the entire experiment period was calculated for each team.Second, the ratio between the value of the variable on a specific day and the average was determined.Thus, in the figures, a value greater than 1 on the y-axis indicates that the value of that variable on that day is higher than the average for the entire cycle.For example, the average value of NM during the experiment period is 11.0 and the average value of NM between days 21 and 22 (x = 11) is 16.5, yielding a value of 1.5 (16.5/11.0)when x equals 11 in the figure .In both figures, the higher values of each variable after x exceeds 10 suggest that teams A and B have improved communication quality after the gamification intervention.For example, in Figure 1, when x equals 12, the value of NM on the y-axis is 2.29, indicating that the average NMs between days 23 and 24 were 2.29 times the average NMs from days 1 to 32.The comparison of communication quality before and after the gamification intervention provides evidence of the effectiveness of gamification strategies, supporting Hypothesis 1 that gamification can enhance and sustain students' communication quality.
F I G U R E 1 Effect of gamification on communication quality in Group A. MT, number of meeting times; NM, number of messages; PC, proper channel; RM, number of replies to messages; SA, sharing or answering questions; SK, shares their work or knowledge.The objective of the bivariate test in Table 5 is to provide a more in-depth evaluation of the impact of gamification on communication quality at the individual level.The bivariate analysis compares the means of the key variables of interest between two groups [2].If the p value is found to be significant at the designated threshold (.01, .05,or .1 are commonly used), then it provides evidence to reject the null hypothesis of equal means in the two groups, suggesting a statistically significant difference between the means of the two groups.On the other hand, if the p value associated with the t test is greater than the threshold, then the null hypothesis is not rejected, indicating no significant difference between the two groups [8].
The impact of gamification on communication quality was evaluated through the examination of six communication variables (NM, PC, SK, RM, SA, and MT) over the course of the experiment (from day 1 to 32).The results of the bivariate analysis, presented in Table 5, indicate that students who received gamification treatment (Gamification = 1) demonstrated significantly higher levels of communication, as evidenced by an increase in message generation by 370% and question sharing/ answering by 1000%.These findings support the hypothesis that gamification leads to an improvement in communication quality.The mean difference test conducted between teams with and without gamification treatment produced a highly significant p value of less than .01,further strengthening the conclusion that gamification has a significant impact on communication quality.In conclusion, the bivariate results provide strong evidence in support of H1, which posits that gamification strategies can enhance communication quality.

| Examine effect of gamification strategy "feedback"
In this section, we examine Hypothesis 2 (H2): that a specific gamification strategy, namely, Feedback, will guide participation toward desired behaviors.To assess the impact of the Feedback on the desired behaviors, we compare the average values of the team's SK, RM, and SA before and after the Feedback introduction, as displayed in Figure 3.The period for this section spans from day 19 to 32, with the x-axis representing every 2 days.Teams C and D received Feedback as an additional gamification strategy on day 25 (x = 4).For example, when x equals 2, the value is the average of the variables (SK, RM, and SA) MT, number of meeting times; NM, number of messages; PC, proper channel; RM, number of replies to messages; SA, sharing or answering questions; SK, shares their work or knowledge.between days 3 and 4. From day 19 to 24 (from x = 1 to 3), the values of the variables (SK, RM, and SA) remain relatively constant, while the values demonstrate superior performance after the Feedback was introduced (from x = 4), suggesting that the Feedback had a positive effect in guiding participation toward desired behaviors.
To examine H2, we also compare the average value of the team's SK, RM, and SA between the treatment (Groups C and D) and the control group (Groups A and B) in Figure 4. From day 25 to 32, the treatment group receives Feedback as an additional gamification strategy, and it earns incremental value (SK, RM, and SA) over the control group.
We conducted a bivariate analysis (Table 6) to compare the mean of key variables of interest between groups at the individual level.We examined the effect of Feedback on the differences in behavior between those who applied it and those who did not.The results in Table 5 showed that Feedback guided participants toward desired behaviors, including increased openness to information exchange and decreased cognitive and psychological distance.For instance, the table reported bivariate results indicating that students with Feedback generated significantly more messages about sharing knowledge (SK: 319% increase), replying messages (RM: 179% increase), and asking or answering questions (SA: 255% increase).Behaviors that were not encouraged by Feedback (e.g., MT) did not show statistical differences.The increase in NM and PC can be attributed to the behavioral modifications induced by Feedback, such as students sharing their knowledge more actively.Taken together, our results confirmed the findings of figures, and thus, H2, which states that a particular gamification strategy (Feedback) guides participation toward desired behaviors.F I G U R E 4 Performance between treatment groups and control groups.RM, number of replies to messages; SA, sharing or answering questions; SK, shares their work or knowledge.

| Examine sustainability of effect of gamification
In this section, we evaluate Hypothesis 3 (H3) which posits the persistence of the gamification effect following the cessation of the intervention.Table 7 employs bivariate analysis to assess the durability of the gamification effectiveness beyond the intervention period.Our analysis reveals that all p values, with the exception of SK, are greater than .1,indicating a lack of significant difference in communication quality following the withdrawal of the gamification intervention.These findings provide empirical support for the conclusion that the gamification effect endures even after the intervention has been withdrawn.
The t test analysis revealed that in the study, terms with a p value less than .1 were considered to have a significant difference, indicating the presence of meaningful findings.Furthermore, these significant differences corresponded to Cohen's g effect sizes greater than 0.5, suggesting medium or large effects.On the other hand, for the terms that did not reach statistical significance, the effect sizes were consistently less than 0.2, indicating small effects or negligible impact.These results underscore the importance of considering both statistical significance and effect sizes when interpreting the findings, as it allows for a comprehensive understanding of the magnitude and practical significance of the observed effects.

| Survey of gamification effectiveness on communication
Table 8 reports the findings of a survey aimed at evaluating the impact of gamification on communication quality, based on six questions.The results indicate a general agreement among the majority of students, with a small minority of dissenting opinions.The survey instrument employed a single-item rating scale, ranging from 1 to 5, rather than a multiple-choice format.As depicted in Table 8, the scores on the questions were found to be distributed similarly.Questions 1 and 2 elicit subjective evaluations from students, indicating a perceived increase in engagement in communication due to gamification.Questions 3 and 4 elicit opinions on the fairness of the gamification strategy.Particularly, questions 5 and 6 elicit beliefs that gamification enhances team communication.In this section, we address the potential threats to the validity of our study, prioritizing them in accordance with the guidelines set forth by Wohlin et al. [65].
Internal validity refers to the possibility of extraneous variables affecting causality beyond the researchers' awareness.External validity pertains to the extent to which the results of an experimental study can be generalized to other settings beyond the current context, such as industrial settings.Construct validity relates to the generalizability of the experimental results to other theoretical and conceptual frameworks.Conclusion validity pertains to the potential issues that could compromise the accuracy of drawing conclusions about the relationship between the treatment and the outcome of the experiment.

| Internal validity
Researchers proposed six potential threats to internal validity: history, maturation, testing, instrumentation, statistical regression, and selection.Starting from our sample selection, to control for other factors that might impact communication quality, and since most are unobservable, such as different personalities due to the different backgrounds, therefore all the groups are constructed randomly from the same module.The threat of history was addressed by collecting data within the same time frame, while the threat of maturation was reduced by the limited duration of the project (38 days), which is unlikely to result in substantial changes in participants' personalities.Further, the testing threat is posed when participants do the pretest before the experiment starts.In the second part of the experiment, although students got some experience with gamification from the first part, we introduced the additional gamification element "feedback," which is different from the previous gamification strategies (points and leaderboards).Thus, this may weaken the testing threat.The instrumentation threat could be eliminated since all data is collected from Discord and online questionnaires.

| External validity
The external validity of the study is limited by its generalizability to industry settings [10].The potential threat of interaction between selection and treatment was considered, as the participants in the study may not be representative of a larger project group.However, the group size of 21 students was deemed appropriate for the complexity of the task.The threat of interaction between setting and treatment was mitigated by utilizing the upto-date communication tool Discord.The potential threat of interaction between history and treatment was addressed by administering the questionnaire after the completion of the project, to eliminate any potential influence.

| Construct validity
Inadequate preoperational explication of constructs: The measurement construction of communication is based on transparent, previous definitions that were argued and tested.
Evaluation apprehension: This study is not exposed to this threat because the students are not evaluated based on the results they obtained in the experiment.

| Conclusion validity
The statistical power of the test is not a threat due to the sample size.We do not consider fishing to be a possible threat.Data analysis follows the same standard to limit fostering for specific outcomes.The error rate, or significance level, was not modified because the actual data analysis was carried out by just one researcher.Given that benchmarked treatments were given to all subjects at the same time, the treatment implementation was reliable.Theoretically, due to the population being selected from undergraduate students with similar backgrounds in the subjects, heterogeneity should be reduced.

| DISCUSSION
The present study was motivated by recent literature highlighting the importance of communication in software development [22], leading to the question of how to improve its quality.In light of the use of gamification to enhance communication efficiency in the business sector [64], the research question was framed as follows: can gamification improve communication quality in a software development team?The results of the study support this hypothesis, indicating that gamification can indeed improve communication quality.Additionally, the study provides empirical evidence of the impact of feedback on communication, demonstrating that feedback can steer participants toward desired behaviors.Furthermore, the effects of gamification were found to endure even after its withdrawal.
The study has several practical implications.First, it sheds new light on the impact of communication quality and how it can be improved through gamification strategies.Additionally, the results emphasize the significance of feedback in enhancing knowledge sharing.The study also offers implications for software engineering education, as it suggests that gamification can be used to improve students' communication quality and that the effects persist even after its withdrawal.Lastly, the study's approach can easily be generalized and applied to all Discord users.
However, it should be noted that the findings of the study are applicable to student and novice developers and may not necessarily generalize to senior engineers.Thus, a potential direction for future work would be to expand the scope of the study to include experienced engineers and large projects.The study can also compare with similar students from the previous year.Studying the changes in student relationships during the experiment can enhance the accuracy of measuring the effects of gamification.Furthermore, comparing the communication quality of students who were reluctant to participate in the experiment and those who volunteered to do so allows for a better understanding of the role that gamification plays in this.Additionally, future studies could explore the role of gamification in other software management practices, such as requirement documentation management [17, 18, 20, 21, 26-28, 42, 49, 57, 60, 65].

F I G U R E 3
Feedback impact on knowledge sharing.RM, number of replies to messages; SA, sharing or answering questions; SK, shares their work or knowledge.
T A B L E 4 Summary of measurement of communication.
Bivariate analysis of gamification impact on communication quality.
Abbreviations: MT, number of meeting times; NM, number of messages; PC, proper channel; RM, number of replies to messages; SA, sharing or answering questions; SK, shares their work or knowledge.
T A B L E 6 Bivariate analysis of feedback impact on knowledge sharing.
Abbreviations: MT, number of meeting times; NM, number of messages; PC, proper channel; RM, number of replies to messages; SA, sharing or answering questions; SK, shares their work or knowledge.
T A B L E 8 Questionnaire result.