Modelling human emotions for tactical decision-making games

Authors


  • Gillian Visschedijk, Anja van der Hulst and Nathalie Vink work at the Departments “Training & Performance Innovations” and “Military Operations,” TNO Defence, Security and Safety. They specialise in serious games for all kinds of military training. Ard Lazonder and Henny Leemkuil work at the Department of Instructional Technology, University of Twente. They specialise in technology-enhanced learning and have a particular interest in learning with games and simulations.

Ms Gillian C. Visschedijk, TNO Defence, Security and Safety, P.O. Box 23, 3769 ZG Soesterberg, The Netherlands. Email: gillian.visschedijk@tno.nl

Abstract

Abstract

The training of tactical decision making increasingly occurs through serious computer games. A challenging aspect of designing such games is the modelling of human emotions. Two studies were performed to investigate the relation between fidelity and human emotion recognition in virtual human characters. Study 1 compared five versions of a virtual character that expressed emotions through different combinations of posture, facial expression, and tone of voice. Results showed that emotion recognition was best when all three behavioural cues were present; posture + face and posture + tone of voice were joint second best. In study 2, these three versions were supplemented with contextual information. Cross-variant comparisons yielded marginal differences in emotion recognition and no differences in tactical decision making. Together, these findings suggest that the combination of posture with either facial expression or tone of voice is sufficient to ensure recognition of human emotions in tactical decision-making games.

Practitioner Notes

What is already known about this topic

  • • Tactical decision making often involves the recognition of human emotions.
  • • Humans exhibit their emotions through facial expressions, body movement and posture and tone of voice.
  • • Some of these behavioural cues are more predominant than others.
  • • Environmental cues convey important additional information to help recognise or infer emotional states.

What this paper adds

  • • Not all three behavioural cues need to be present to recognise human emotions in a known context.
  • • The combination of posture with either facial expression or tone of voice is sufficient to recognise human emotions.
  • • Both cue combinations lead to equally high recognition rates and qualitatively comparable tactical decisions.

Implications for practice and/or policy

  • • Designers of tactical decision-making games can lower the quality of the game characters' facial expressions or omit their vocalisations.
  • • As high-end graphics involve high development costs, the former option seems the most cost-effective.

Ancillary