Playing the piano requires one to precisely position one's hand in order to strike particular combinations of keys at specific moments in time. This paper presents the first system for automatically generating three-dimensional animations of piano performance, given an input midi music file. A graph theory-based motion planning method is used to decide which set of fingers should strike the piano keys for each chord. As the progression of the music is anticipated, the positions of unused fingers are calculated to make possible efficient fingering of future notes. Initial key poses of the hands, including those for complex piano techniques such as crossovers and arpeggio, are determined on the basis of the finger positions and piano theory. An optimization method is used to refine these poses, producing a natural and minimal energy pose sequence. Motion transitions between poses are generated using a combination of sampled piano playing motion and music features, allowing the system to support different playing styles. Our approach is validated through direct comparison with actual piano playing and simulation of a complete music piece requiring various playing skills. Extensions of our system are discussed.Copyright © 2012 John Wiley & Sons, Ltd.

This paper presents the first system for automatically generating three-dimensional animations of piano performance, given an input midi music file. A graph theory-based motion planning method is used to decide which set of fingers should strike the piano keys for each chord. As the progression of the music is anticipated, the positions of unused fingers are calculated to make possible efficient fingering of future notes. The system supports automatically generating animation for different complex piano techniques such as finger crossovers and arpeggio.

Developable surfaces have been extensively studied in computer graphics because they are involved in a large body of applications. This type of surfaces has also been used in computer vision and document processing in the context of three-dimensional (3D) reconstruction for book digitization and augmented reality. Indeed, the shape of a smoothly deformed piece of paper can be very well modeled by a developable surface. Most of the existing developable surface parameterizations do not handle boundaries or are driven by overly large parameter sets. These two characteristics become issues in the context of developable surface reconstruction from real observations. Our main contribution is a generative model of bounded developable surfaces that solves these two issues. Our model is governed by intuitive parameters whose number depends on the actual deformation and including the “flat shape boundary”. A vast majority of the existing image-based paper 3D reconstruction methods either require a tightly controlled environment or restricts the set of possible deformations. We propose an algorithm for reconstructing our model's parameters from a general smooth 3D surface interpolating a sparse cloud of 3D points. The latter is assumed to be reconstructed from images of a static piece of paper or any other developable surface. Our 3D reconstruction method is well adapted to the use of keypoint matches over multiple images. In this context, the initial 3D point cloud is reconstructed by structure-from-motion for which mature and reliable algorithms now exist and the thin-plate spline is used as a general smooth surface model. After initialization, our model's parameters are refined with model-based bundle adjustment. We experimentally validated our model and 3D reconstruction algorithm for shape capture and augmented reality on seven real datasets. The first six datasets consist of multiple images or videos and a sparse set of 3D points obtained by structure-from-motion. The last dataset is a dense 3D point cloud acquired by structured light. Our implementation has been made publicly available on the authors' web home pages. Copyright © 2012 John Wiley & Sons, Ltd.

This paper presents two main contributions. First, a computational model of inextensible paper-like surfaces. Second, an algorithm that estimates this model's parameters from multiple images.

We present a crowd model informed by common ground theory to accommodate high-level socially aware behavioral realism of characters in crowd simulations. In our approach, group members maintain group cohesiveness by communicating and adapting their behaviors to each other. The resulting character behaviors in animations form a consequential chain interpreted as a coherent story by observers. We demonstrate that our model produces more believable animations from the viewpoint of human observers through a series of user studies. Copyright © 2013 John Wiley & Sons, Ltd.

We present a crowd model informed by common ground theory to accommodate high-level socially aware behavioral realism of characters in crowd simulations. In our approach, group members maintain group cohesiveness by communicating and adapting their behaviors to each other. In the course of social interaction, agents present gestures or other behavioral cues according to their communicative purposes. We demonstrate that our model produces more believable animations from the viewpoint of human observers through a series of user studies.

This paper presents a model for simulating realistic crowd behaviors at low computation cost. The proposed model is inspired by video data. In our approach, we first classify the crowd into two categories: main and background characters. Whether the agents are main characters or not is influenced by two factors, one is the agent's trajectories and the other one is the change of the environment. In the second stage, we adopt two approaches to simulate the behaviors of main and background characters. Main characters are intelligent agents with the perception, the memory, the planning, and the psychology so that they can make decisions themselves. Background characters are informed of the behavior options for execution by the “smart environment.” Finally, we simulate the road-crossing scenario in a three-dimensional virtual environment. The experimental results demonstrate that our approach not only well reflects the characteristics of agent behaviors but also reduces the computation complexity of simulating realistic crowd. Copyright © 2013 John Wiley & Sons, Ltd.

This paper presents a model for simulating realistic crowd behaviors at low computation cost. In our approach, we classify the crowd into two categories based on agent's trajectories extracted from the video data and the change of the environment: main characters and background characters. We adopt two approaches to simulate the behaviors of main characters and background characters to improve the realism of the scenario and guarantee the simulation rate.

This paper proposes a collision avoidance behavior model for crowd simulation based on psychological findings of human behaviors such as gaze movement angle (GMA), side stepping, gait motion, and personal reaction bubble to have better results in crowd simulation. By calculating the GMA between agents, collision can be predicted and avoided without knowing the exact trajectories of the agents. The proposed model consists of four phases: (1) GMA-based collision prediction for mid/long range by using speed-variant information process space, (2) collision avoidance steering, (3) gait-based locomotion generation, and (4) space keeping based on personal reaction bubble. The effectiveness of the proposed speed-variant information process space was tested on various types of agent flows with different densities. The total loss of kinetic energy accumulated during an agent's movement and the ratio of the length of the path actually traveled to the length of the original path are used as key metrics to figure out the features between the different types of flows. Finally, examples of tuning the parameters with well-known fundamental diagrams are presented. Copyright © 2013 John Wiley & Sons, Ltd.

This paper proposes a collision avoidance behavior model for crowd simulation based on psychological findings of human behaviors such as gaze-movement angle (GMA), side stepping, gait motion, and personal reaction bubble (PRB) to get better results in crowd simulation. The total loss of kinetic energy accumulated during an agent's movement and the ratio of the length of the path actually traveled to the length of the original path are used as key metrics.

One of the major obstacles in incompressible fluid simulations is the projection step that enforces zero divergence of the velocity field. We propose a novel heterogeneous CPU–GPU parallel multigrid Poisson solver that decomposes the high-frequency components of the residual field using a wavelet decomposition and conducts an additional smoothing process on them, using the CPU, while the GPU is performing projection at the coarsest level. In example animations of smoke and turbulent flow with thermal buoyancy, this additional smoothing improves the accuracy of the parallel multigrid Poisson solver in a single multigrid cycle and reduces the number of multigrid cycles required to reach a specified accuracy. Copyright © 2013 John Wiley & Sons, Ltd.

We propose a novel heterogeneous CPU-GPU parallel multigrid Poisson solver that decomposes the high-frequency components of the residual field using a wavelet decomposition and conducts an additional smoothing on the CPU while the GPU handles the coarsest level evaluation. We demonstrate the efficiency of our solver with animations of smoke and turbulent flow with thermal buoyancy.

We propose a method for handling elastic solids in smoothed particle hydrodynamics fluids. Our approach samples triangulated surfaces of solids using boundary particles. To prevent fluid particle tunneling in case of large expansions, additional boundary particles are adaptively generated to prevent gaps and undesired leakage. Furthermore, as an object compresses, particles are adaptively removed to avoid unnecessary computations. We demonstrate that our approach produces plausible interactions of smoothed particle hydrodynamics fluids with both slowly and rapidly deforming solids. Copyright © 2013 John Wiley & Sons, Ltd.

We propose a method for handling elastic solids in smoothed particle hydrodynamics fluids. Our approach samples triangulated surfaces of solids using boundary particles. To prevent fluid particle tunneling in case of large expansions, additional boundary particles are adaptively generated to prevent gaps and undesired leakage. Furthermore, as an object compresses, particles are adaptively removed to avoid unnecessary computations. We demonstrate that our approach produces plausible interactions of smoothed particle hydrodynamics fluids with both slowly and rapidly deforming solids.

We present a rigid-motion-inspired method for animating liquid characters in this paper. Our method allows an animator to control the motion of liquid characters with motion capture data that is widely used in rigid body animation. It animates the most visual interesting part of liquid character, that is, to preserve character's shape as well as produce enough liquid details. To this end, we build a two-layer model to represent the character by two coaxial layers: the rigid kernel and the liquid shell. Different control paradigms are used for the two layers instead of applying homogeneous force that is common in previous approaches. By embedding the control algorithm to the Navier–Stokes equations, we compute the fluid velocity that drives the motion of the liquid character. Results show that the method is easy and intuitive to use while incurring little additional cost.Copyright © 2013 John Wiley & Sons, Ltd.

We present a rigid-motion-inspired method for animating liquid characters. Our method allows an animator to control the motion of liquid characters with motion capture data. It animates the most visual interesting part of liquid character, that is, to preserve character's shape as well as produce enough liquid details. The method is easy and intuitive to use while incurring little additional cost.

This paper presents a hybrid animation approach to the flexible and rapid crack simulation of brittle material. At the physical level, the local stress tensors induced by collision are analyzed by using the smoothed particle hydrodynamics (SPH) formulation. Specifically, in order to determine the internal stress when rigid bodies collide with each other or neighboring environments, we treat all of them as completely rigid body that has infinite stiffness and then evaluate virtual displacement for colliding particles. At the geometric level, in order to faithfully maintain the fracture interface during the crack simulation, we utilize an efficient shape representation of solid based on the tetrahedral decomposition of the original solid geometry. This novel hybrid approach resorts to local particle models, whose goal is to avoid heavy computational burden during crack interface updating and topological changing, and meanwhile, it facilitates the user-initiated interactive control during the crack generation and propagation. Our animation experiments demonstrate the effectiveness of our novel particle-based method to simulate the crack of brittle material. Copyright © 2013 John Wiley & Sons, Ltd.

A novel hybrid particle-based animation approach is presented to the flexible and rapid crack simulation of brittle material. A smoothed particle hydrodynamics formulation is adapted to solve the linear elastic mechanics for fracture animation, the hybrid sampling approach and effective shape representation scheme low computation burden, and has an advantage in mesh processing and crack propagation.

We introduce novel approaches of intuitive and easy-to-use realistic fire animation, starting from real-life fire by image-based techniques and statistical analysis. The results can be utilized as a pre-rendered sequence of images in video games, motion graphics, and cinematic visual effects. Instead of physics-based simulation, we employ an example-based principal component analysis and introduce “EigenFires.” We visualize the main features of various fire samples to analyze their tracks and synthesize a new fire by combining various fire samples, recorded with high frame rates, in order to edit given sequences of fire animations. For this purpose, we present how to recognize similarity of the shapes of fire in order to change the pattern from one style of fire to another distinct style of fire procedurally. Our techniques require very little parameter tuning, compared with conventional physically based fire synthesis, video textures, and dynamic textures. A similar level of visually pleasing compressed fire is also easily produced by using principal component analysis techniques. Copyright © 2013 John Wiley & Sons, Ltd.

We propose novel approaches of generating realistic fire animation, using image-based techniques, statistical analysis, and motion transitions. By introducing “EigenFires,” we visualize the main features of fire, compress it, and even synthesize a new fire from our database.

This study proposes a novel procedural modeling method using convolution sums of divisor functions to model a variety of natural trees in a virtual ecosystem efficiently. The basic structure of the modeling method defines the growth grammar, including the branch propagation, a growth pattern of branches and leaves, and a process of growth deformation for various tree generation. Here, the proposed procedural method for trees is to utilize convolution sums of divisor functions as a novel approach. The structure of convolution sums has branch propagation of a uniform pattern, which is controllable, so that it is efficient for real-time virtual ecosystem construction. Furthermore, it can process changes of environment factors or growth deformation for various and unique tree generation simply through the properties of divisor functions. Finally, an experiment is performed in order to evaluate our proposed modeling method whether it can generate natural and various tree models, and a real-time virtual ecosystem of a large area where a variety of trees are presented using the modeling method can be constructed efficiently.Copyright © 2013 John Wiley & Sons, Ltd.

To model various natural trees growing in a virtual ecosystem, a new procedural tree modeling method was proposed on the basis of the convolution sums of divisor functions. For the efficient construction of a real-time virtual ecosystem, a growth pattern, which is based on the convolution sums of divisor functions, was defined to have branch propagation of a uniform pattern. Also, the growth grammar and procedural tree modeling method, by which processes such as branch propagation, a growth pattern of branches and leaves, and growth deformation for various tree generations controlled intuitively and conveniently, were designed and evaluated via the experiment.

This paper proposes a method for simulating the dynamics of porous deformable shells in the presence of water that floats on the surface or is absorbed into the interior. The proposed method enables various effects such as surface flow, capillary flow involving absorption and saturation of water, changes of the material properties caused by water saturation, and the deformable body dynamics including tearing. The experiments demonstrate that the proposed method produces promising results.Copyright © 2013 John Wiley & Sons, Ltd.

This paper proposes a method for simulating the dynamics of porous deformable shells in the presence of water that floats on the surface or is absorbed into the interior. The proposed method enables various effects such as surface flow, capillary flow involving absorption and saturation of water, changes of the material properties caused by water saturation, and the deformable body dynamics including tearing.

This paper proposes a novel video-based method to transfer the illumination from a single reference facial performance video to a target one taken under nearly uniform illumination. We first filter the key frames of the reference and the target face videos with an edge-preserving filter. Then, the illumination component of reference key frame is extracted through dividing the filtered reference key frames by the corresponding filtered target key frames in skin region. The differences in non-skin region caused by different expressions between the reference and target face may bring about artifacts. Therefore, we interpolate the illumination component of the non-skin region by that of the surrounded skin region to ensure the spatial smoothness and consistency. After that, the illumination components of key frames are propagated to non-key frames to ensure the temporal consistency between the two adjacent frames. We obtain convincing results by transferring the illumination effects of a single reference facial performance video to a target one with the spatial and temporal consistencies preserved. Copyright © 2013 John Wiley & Sons, Ltd.

This paper proposes a novel video-based method to transfer the illumination from a single reference facial performance video to a target one taken under nearly uniform illumination. We use an edge-preserving filter and illumination component interpolation in non-skin region to ensure the spatial smoothness and consistency of illumination component. The illumination components of key frames are propagated to non-key frames to ensure the temporal consistency between the two adjacent frames of illumination component.

Because existing image abstraction systems can hardly incorporate with the changing light, we present an integrated image abstraction and relighting rendering system, which is based on a salient edge-guided luminance field optimization approach. For an input image, we first adopt a sparsity prior-based illumination decomposition method to remove its original illumination and have an intrinsic image. Meanwhile, we iteratively extract the salient edge inside by employing a message-passing strategy. Then, we simplify this image with a proposed salient edge-guided image abstraction optimization algorithm in the luminance field. Finally, we put forward a salient edge-guided image relighting optimization method to simulate the effect of dynamic lighting along different directions. Experiment results show that our system can artistically adjust the illumination of the abstracted image and makes it more vivid. Copyright © 2013 John Wiley & Sons, Ltd.

We present an integrated image abstraction and relighting rendering system to incorporate the dynamic lighting effects into the abstracted images in an artistic style. It carries out prior-based image illumination decomposition, message-passing-based salient edge extraction, salient edge-guided image abstraction and relighting in order. Experiment results show that our system can artistically adjust the illumination of the abstracted image and makes it more vivid.

Simulating realistic makeup effects is one of the important research issues in the 3D facial animation and cosmetic industry. Existing approaches based on image processing techniques, such as warping and blending, have been mostly applied to transfer one's makeup to another's. Although these approaches are intuitive and need only makeup images, they have some drawbacks, for example, distorted shapes and fixed viewing and lighting conditions. In this paper, we propose an integrated approach, which combines the Kubelka–Munk model and a screen-space skin rendering approach, to simulate 3D makeup effects. The Kubelka–Munk model is used to compute total transmittance when light passes through cosmetic layers, whereas the screen-space translucent rendering approach simulates the subsurface scattering effects inside human skin. The parameters of Kubelka–Munk model are obtained by measuring the optical properties of different cosmetic materials, such as foundations, blushes, and lipsticks. Our results demonstrate that the proposed approach is able to render realistic cosmetic effects on human facial models, and different cosmetic materials and styles can be flexibly applied and simulated in real time. Copyright © 2013 John Wiley & Sons, Ltd.

In this paper, we propose an integrated approach, which combines the Kubelka–Munk model and a screen-space skin rendering approach, to simulate 3D makeup effects. The Kubelka–Munk model is used to compute total transmittance when light passes through cosmetic layers, whereas the screen-space translucent rendering approach simulates the subsurface scattering effects inside human skin. The parameters of Kubelka–Munk model are obtained by measuring the optical properties of different cosmetic materials, such as foundations, blushes, and lipsticks.

Modern virtual environments can contain a variety of characters and traversable regions. Each character may have different preferences for the traversable region types. Pedestrians may prefer to walk on sidewalks, but they may occasionally need to traverse roads and dirt paths. By contrast, wild animals might try to stay in forest areas, but they are able to leave their protective environment when necessary. This paper presents a novel path planning method named Modified Indicative Routes and Navigation (MIRAN) that takes a character's region preferences into account. Given an indicative route as a rough estimation of a character's preferred route, MIRAN efficiently computes a visually convincing path that is smooth, keeps clearance from obstacles, avoids unnecessary detours, and allows local changes to avoid other characters. To the best of our knowledge, MIRAN is the first path planning method that supports the aforementioned features while using an exact representation of the navigable space. Experiments show that with our approach, a wide range of different character behaviors can be simulated. It also overcomes problems that occur in previous path planning methods such as the Indicative Route Method. The resulting paths are well suited for real-time simulations and gaming applications. Copyright © 2013 John Wiley & Sons, Ltd.

Modern virtual environments can contain a variety of characters and traversable regions. Each character may have different preferences for the traversable region types. We present a novel path planning method name MIRAN that computes visually convincing paths while taking a character's region preferences into account.

We present a new method to create realistic paint simulation, utilizing the characteristics of paint, such as fluidity, diffusion, and absorption. We treat the painting elements separately as pigment, binder, solvent, and paper. Adopting smoothed-particle hydrodynamics including a consideration of viscoelastic movement, we simulate the fluid motion of the paint and the solvent. To handle the diffusion of the pigment in the solvent, we utilize the mass transfer method. Following Fick's law, the concentration of pigment changes and each pigment particle is diffused to the neighborhood accordingly. As time elapses, the binder and the solvent are absorbed, and for the most part, the pigment remains on the paper. The Lucas–Washburn equation determines the distance of absorption. The examples show that our approach can effectively generate various types of painting. Copyright © 2013 John Wiley & Sons, Ltd.

We present a new method to create realistic paint simulation, utilizing the characteristics of paint, such as fluidity, diffusion, and absorption. Adopting smoothed particle hydrodynamics including a consideration of viscoelastic movement, we simulate the fluid motion of the paint and the solvent. To handle the diffusion of the pigment in the solvent, we utilize Fick's law. As time elapses, the Lucas-Washburn equation determines the distance of absorption of the binder and the solvent.

We present an automatic, efficient, and simple technique to create pencil drawing animation, starting from a video. We generate pencil drawing from a source frame based on stroke modeling, specifying the properties of strokes, in combination with layered lines, flow-guided difference-of-Gaussian (DoG) filter to several layers. Generated pencil strokes are translated and rotated because of forces exerted from the sequential frames using rigid body dynamics. Linear and angular forces acting on strokes are calculated according to the temporally filtered per-pixel optical flow vectors. Our framework effectively generates the coherent animation of pencil strokes preserving the structured appearance of charcoal or pastel, which is difficult to achieve with previous-abstraction based non-photorealistic animation. Moreover, our stroke simulation step is suitable for animating the different styles of strokes, such as oil painting and watercolor, and can be efficiently implemented to produce animation in relatively inexpensive manner. Copyright © 2013 John Wiley & Sons, Ltd.

We present an automatic, efficient, and simple technique to create pencil drawing animation, starting from a video. We combine pencil drawing stylization and rigid body dynamics framework to translate and rotate strokes because of temporally filtered per-pixel optical flow vectors. Our framework effectively generates the coherent animation of pencil strokes preserving the structured appearance of charcoal or pastel, which is difficult to achieve with previous abstraction-based non-photorealistic animation.

We present a real-time algorithm for realistically deforming 3D human blood vessels, while automatically detecting and avoiding interference among a large number of blood vessels under deformation. Sweep surfaces are employed for this purpose. Using a dynamic bounding volume hierarchy, specially designed for sweep surfaces, we support collision detection and other related geometric computations in real time. Copyright © 2013 John Wiley & Sons, Ltd.

Using a dynamic bounding volume hierarchy for sweep surfaces, we present a real-time algorithm for realistically deforming 3D human blood vessels, while automatically detecting and avoiding interference among a large number of blood vessels under deformation.

Our goal is to create socially responsible agents, either robots or virtual humans. In this paper, we present an integration of emotions, attachment, and learning in emotional decision-making to achieve this goal. Based on emerging psychological theories, we aim at building human-like emotional decision-making, where emotions play a central role in selecting the next action to be performed by the agent. Here, we present our own approach for emotion appraisal where we use emotional attachment as an important impulse for determining the intensities of emotions. Emotions in their turn are used to calculate the emotional attachment toward the users and for learning to predict future consequences. We report on the results of a simulation evaluation where we assess the influence of emotions, attachment, and learning on decision-making. It is our strong belief that by giving an agent the ability to have emotions and to feel empathy and emotional attachment toward others, we will ultimately give this agent the ability to learn and improve its social behavior skills through interactions with the users and through user feedback. Copyright © 2013 John Wiley & Sons, Ltd.

This paper has the creation of socially responsible agents as its goal. On the basis of emerging psychological theories, it presents an integration of emotions, attachment, and learning in emotional decision making where emotions play a central role in the decision making. It also presents an approach for emotion appraisal where emotional attachment is used in determining the intensities of emotions. Emotions in their turn are used to calculate the emotional attachment toward the users and for learning to predict future consequences.

The use of polite agents is a new approach in order to improve efficiency and naturalism in navigation for player characters in crowded virtual worlds. This paper aims to model the politeness of virtual humans using logic-based approaches, subject to theory of politeness decomposed of conventional and interpersonal politeness. To do so, we propose a high-level agent architecture combined with normative framework to model and reason about ‘polite’ behaviours in social situations. With this architecture, we demonstrate (i) specifying polite behaviours as a form of social norms; (ii) generating polite behaviours using social reasoning technique; (iii) deliberation with such norms in belief–desire–intention agents; and (iv) realising physical actions based on the decision. Implementation for social reasoning is achieved by InstAL, based on the semantics of answer set programming. Using experiments with simple collision avoidance model, we show the effectiveness of polite behaviour in navigation designed by such architecture, as well as the adequacy of this architecture for modelling theory of politeness in all circumstances. Copyright © 2013 John Wiley & Sons, Ltd.

This paper aims to model the politeness of virtual humans in social situations using logic-based approaches, through the high-level agent architecture combined with normative framework being capable of social reasoning. Using experiments with simple collision avoidance model, we show the effectiveness of polite behaviour in navigation designed by our approach and the adequacy of this architecture for modelling theory of politeness in all circumstances.

For a virtual training system, how to effectively and quickly generate training scenarios has become a challenging issue. A scenario generation system is needed to produce scenarios that can meet different objectives and at the same time be customized for individuals. In this paper, we introduce a scenario generation framework for mission-based virtual training, which aims to generate scenarios from both trainer and trainee's perspective. The framework allows a trainer to direct the scenario generation process, so that the generated scenarios reflect the trainer's preferences over different mission objectives. It also considers how the scenarios could adapt to different trainees’ skill levels. The representation of scenario beat is proposed, and the scenario generation process adopts a combinatorial optimization approach generating the sequence of scenario beats. The efficacy of the proposed framework is demonstrated through an empirical study of human players in a simple food distribution mission game. The results show that a trainee can achieve better performance improvement when playing the customized scenarios tailored to the trainee's skill level as compared with the uncustomized scenarios. Copyright © 2013 John Wiley & Sons, Ltd.

For a virtual training system, how to effectively and quickly generate training scenarios has become a challenging issue. In this paper, we introduce a scenario generation framework for mission-based virtual training, which aims to generate scenarios from both trainer and trainee's perspective. The framework is designed to generate scenarios that can reflect the trainer's preferences over different mission objectives and adapt to different trainees' skill levels.

Human body movements and postures carry emotion-specific information. On the basis of this motivation, the objective of this study is to analyze this information in the spatial and temporal structure of the motion capture data and extract features that are indicative of certain emotions in terms of affective state descriptors. Our contribution comprises identifying the directly or indirectly related descriptors to emotion classification in human motion and conducting a comprehensive analysis of these descriptors (features) that fall into three different categories: posture descriptors, dynamic descriptors, and frequency-based descriptors in order to measure their performance with respect to predicting the affective state of an input motion. The classification results demonstrate that no single category is sufficient by itself; the best prediction performance is achieved when all categories are combined. Copyright © 2013 John Wiley & Sons, Ltd.

The objective of this study is to analyze human body movements and postures in the spatial and temporal structure of the motion capture data and extract features that are indicative of certain emotions in terms of affective state descriptors. Our contribution comprises identifying the directly or indirectly related descriptors to emotion classification in human motion and conducting a comprehensive analysis of these descriptors (features) that fall into three different categories: posture descriptors, dynamic descriptors, and frequency-based descriptors.

We describe a compression method for three-dimensional animation sequences that has notable advantages over existing techniques. We first aggregate the frame data by similarity and reorganize them into clusters, which results in the sequence split into several motion fragments of varying lengths. To minimize the number of clusters and obtain optimal clustering, we perform frame alignment, which eliminates the “global” rigid transformation from each frame data and use only “pose” when evaluating the similarity between frames. We then apply principal component analysis for each cluster, from which we get coordinates of corresponding frames in a reduced dimension. Because similar frames are considered, the number of coefficients required for each frame becomes smaller; thus, we obtain better dimension reduction for a given reconstruction error. Further, we perform intracluster compression based on linear coding. Because every motion fragment presents similar frames, conventional linear predictive coding can be replaced by key frame-based linear coding to achieve minimal reconstruction error. Results show that our method can obtain a high compression ratio, with a limited reconstruction error. Copyright © 2013 John Wiley & Sons, Ltd.

We describe a compression method for 3D animated mesh sequences that has notable advantages over existing techniques. The key ideas of this method is to cluster the animation frames according to their pose similarity and compress each cluster by using principal component analysis in search of a smaller number of Eigen basis.

This paper presents a novel framework for garment grading. In CG, an extensive amount of study has been carried out to clothe human characters, but little attention has been taken to the grading problem itself. For the development of a grading technique, we obtained the insight from the process of drawing the patternmaking draft (sloper) in the clothing field. Noting that the draft can be completely determined by supplying the primary body sizes, we abstract the draft construction process as a computer procedure, which we call the parametrized draft. With the parametrized draft, we develop a grading method based on the draft-space warping, which takes three steps: (i) draft-space encoding, (ii) target draft construction, then (iii) draft-space decoding. The proposed grading method can be performed instantly for any given body without calling for the user's intervention. With experimental results, we show that the new grading framework can bring an improvement to garment grading. Copyright © 2013 John Wiley & Sons, Ltd.

Draft space warping is the fast and automatic method for garment grading. For the development of a grading technique, we introduced retargeting technique, which is widely used in the computer graphics field and obtained the insight from the process of drawing the patternmaking draft (sloper) in the clothing field. Our approach can minimize designer's specialized know-how and save performing time for the grading of real garment and virtual garment.

We present a method to make an accurate and dense reconstruction from the input of video captured by a free moving handheld camera in real time. By the method firstly, the positions of the camera and sparse 3D points are estimated by simultaneous localization mapping. Then the depth maps of selected reference frames are computed from corresponding camera bundles. Lastly a novel linear algorithm is also proposed to integrate all the depth maps into dense meshes partially. The main contributions of this paper are in the following points: the reference frames and corresponding camera bundles are able to be selected automatically, then accurate and smooth depth maps are generated in real time, and the depth maps are merged into a dense mesh by using a linear algorithm based on the error clouds optimization. Our algorithm is implemented on dual CPU and graphics processing unit in a parallel framework for improving the performance. Copyright © 2013 John Wiley & Sons, Ltd.

The live reconstruction system is designed in a parallel frame as the work pipeline such as the following: live camera tracking is firstly implemented by real-time structure from motion algorithm. Then accurate depth maps are generated from selected frame bundles with three restrictions. Finally, depth maps are fused into a dense mesh by a linear algorithm.

With the explosive growth of motion capture data, it becomes very imperative in animation production to have an efficient search engine to retrieve motions from large motion repository. However, because of the high dimension of data space and complexity of matching methods, most of the existing approaches cannot return the result in real time. This paper proposes a high level semantic feature in a low dimensional space to represent the essential characteristic of different motion classes. On the basis of the statistic training of Gauss Mixture Model, this feature can effectively achieve motion matching on both global clip level and local frame level. Experiment results show that our approach can retrieve similar motions with rankings from large motion database in real-time and also can make motion annotation automatically on the fly. Copyright © 2013 John Wiley & Sons, Ltd.

We propose a high-level semantic feature in a low-dimensional space to represent the essential characteristic of different motion classes. On the basis of the statistic training of Gauss Mixture Model, this feature can effectively achieve motion matching on both global clip level and local frame level. Experimental results show that our approach can retrieve similar motions with rankings from large motion database in real-time and also can make motion annotation automatically on the fly.

We present an intuitive and interactive approach for motion editing through space–time constraints on positions. Given an input motion of an elastic body, our approach enables the user to interactively edit node positions in order to alter and fine-tune the motion. We formulate our motion editing as an optimization problem with dynamics constraints to enforce a physically plausible result. Through linearization of the editing around the input trajectory, we simplify this constrained optimal control problem into an unconstrained quadratic optimization. The optimal motion thus becomes the solution of a dense linear system, which we solve efficiently by applying the adjoint method in each iteration of a conjugate gradient solver. We demonstrate the efficiency and quality of our motion editing technique on a series of examples. Copyright © 2013 John Wiley & Sons, Ltd.

Given an input motion of an elastic body, our approach enables the user to interactively edit node positions in order to alter and fine-tune the motion. When the user edits the motion through position constraints, our system produces a new animation at interactive rates in which the constraints are visually met. The rope bridge in the input animation is displayed with textures, whereas the one in the output animation is displayed in green.

Motion editing requires the preservation of spatial and temporal information of the motion. During editing, this information should be preserved at best. We propose a new representation of the motion based on the Laplacian expression of a 3D+t graph: the set of connected graphs given by the skeleton over time. Through this Laplacian representation of the motion, we propose an application that allows an easy and interactive editing, correction, or retargeting of a motion. The new created motion is the result of the combination of two minimizations, linear and non-linear: the first penalizes the difference of energy between the Laplacian coordinates from an animation to the desired one. The other one preserves the length of segments. Using several examples, we demonstrate the benefits of our method and in particularly the preservation of the spatiotemporal properties of the motion in an interactive context. Copyright © 2013 John Wiley & Sons, Ltd.

This paper proposes a new representation of motion based on the Laplacian expression of a 3D+t graph: the set of connected graphs given by the skeleton over time. Our approach enables an easy and interactive editing, correction, or retargeting of motion. Using several examples, we demonstrate the benefits of our method and in particularly the preservation of the spatiotemporal properties of the motion in an interactive context.

Improvements in ways of game controlling in recent years yielded higher level of interaction. Release of motion controller devices changed conventional ways of controlling games that have been used so far. Microsoft ®; Kinect™ (Microsoft Corporation, WA, USA) recognizes motions of the players as game controlling inputs. Although touchless interaction is perceived to be attractive, games mimicking real life activities may benefit from hand-held tangible objects for the player to get more involved into game. In this study, a tangible gameplay interaction method is developed using Microsoft ®; Kinect™ that senses hand-held objects with their dimensions and incorporates them into gameplay. Proposed algorithm is implemented on an experimental game, and a user study is performed to measure effects of tangible interaction on Kinect gameplay experience. Results revealed that an improved gameplay with more natural and accurate motion controlling is achieved. Copyright © 2013 John Wiley & Sons, Ltd.

In this study, a tangible gameplay interaction method is developed using Microsoft Kinect that senses hand-held objects with their dimensions and incorporates them into gameplay. Proposed algorithm is implemented on an experimental game, and a user study is performed to measure effects of tangible interaction on Kinect gameplay experience. Results revealed that an improved gameplay with more natural and accurate motion controlling is achieved.