The SIJ Transactions on Computer Science Engineering & its Applications (CSEA), Vol. 2, No.

3, May 2014
ISSN: 2321-2381 2014 | Published by The Standard International Journals (The SIJ) 93



AbstractMost Kinect games have only few reacting motions, and may not feel realistic enough for the
reactions, so we propose an interactive system to simulate real-time motions of bipedal characters for one-on-
one fighting games. The primary contribution of the proposed system is that we dynamically generate
believable reacting motions according to different fighting actions and hit parts of a body instead of solely
playing pre-captured or hand-crafted motions from an example motion dataset. First, Kinect skeletal tracking is
employed to capture a users fighting motion for attacking his\her game opponent. Second, we apply the
principal component analysis to reduce the dimensionality of the fighting motions such that we can obtain a
feature vector for determining what a fighting pattern is and how to attack the game opponent. In addition, we
apply Mahalanobis distances to increase the accuracy of Kinect motions comparison. In the last step, a
proportional-derivative control is exerted to track the pre-prepared example motion for reacting properly when
the opponent is hit by the fighting pattern. Accordingly, a support vector machine is employed offline to
classify the example motions automatically. Experimental results show that our system is feasible and flexible
to provide dynamic reacting motions in fighting games.
KeywordsCharacter Animation; Feature Extraction; Motion Capture; Motion Classification; Skeletal
Tracking.
AbbreviationsPrincipal Component Analysis (PCA); Proportional-Derivative (PD); Self-Organizing Map
(SOM); Support Vector Machine (SVM).

I. INTRODUCTION
game is a type of playing activity; theoretically,
people like to play all types of games due to the
inherent human desire. According to the common
definition, playing and pretending are two essential elements
of play games. High reactions and good hand-eye
coordination are fundamental features that an action game
needs to possess. Fighting games are a type of action game
where game characters fight each other through a hand-to-
hand combat. Most design innovation in fighting games
focuses on developments in game characters actions and
reactions. In particular, the strategy to control attacking and
defending players is an interesting issue for most game
developers over the past few years. The straightforward
method is to employ a one-on-one strategy in which two
opposite fighters take their turns to be attacking and
defending actions for taking place several rounds.
In this paper, we devise a novel framework and make use
of the proposed framework to implement a 3D one-on-one
fighting game with Kinect. Kinect skeletal tracking, principal
component analysis (PCA), support vector machine (SVM),
and Proportional-Derivative (PD) control are exerted to build
the game framework. Regarding the conventional motion
sensing games, they take advantage of Kinect device to track
a real players skeleton for directly animating the captured
skeletal motions on the screen without accomplishing motion
recognition. On the other hand, even though Kinect is capable
of tracking two humanoid skeletons simultaneously, it will be
an amazing phenomenon to see that two players fight each
other in real-time while directly using Kinect to design a
fighting game without any design adaptation. Moreover, most
fighting games adopt pre-captured motion data to be served
as the reacting motions of a player. The reacting motions of
the character cannot vary with the attacking power and the hit
part of the body due to the fact that game developers could
not prepare all possible reacting motions of a character while
designing a game. Obviously, we can see that all reacting
motions of the characters are restricted no matter how a
player attacks his\her opponent.
Kinect is an easy-to-use motion sensing device with the
affordable price. In addition to adopt Kinect to track a
A
*Associate Professor, Department of Computer Science and Information Engineering, National Chiayi University, Chiayi, TAIWAN.
E-Mail: tclu{at}mail{dot}ncyu{dot}edu{dot}tw
**Graduate Student, Department of Computer Science and Information Engineering, National Chiayi University, Chiayi, TAIWAN.
***Graduate Student, Department of Computer Science and Information Engineering, National Chiayi University, Chiayi, TAIWAN.
****Ph.D. Student, Department of Computer Science and Information Engineering, National Chiayi University, Chiayi, TAIWAN.
Tainchi Lu*, Yuchen Chen**, Jiayi Li*** & Minchih Tsai****
Simulating Bipedal Character Motions
for Kinect Games
The SIJ Transactions on Computer Science Engineering & its Applications (CSEA), Vol. 2, No. 3, May 2014
ISSN: 2321-2381 2014 | Published by The Standard International Journals (The SIJ) 94
players attacking motion, we designate a standard attacking
motion offline and then use PCA to extract a feature vector
from the attacking motion data for comparing with each other
in real time. Consequently, we can confirm whether the
attacking motion is valid or not at a time. This is the
difference between the proposed system and the conventional
interactive games with Kinect. For the purpose of providing
dynamic and believable reacting motions, we do not just
animate the attacked character by using the designated pre-
captured reacting motions. Automatic motion classification
works are essential. SVM is adopted to classify the numerous
pre-captured reacting motions that are captured by utilizing
the motion capture system into different motion categories.
The motion classification is an automatic process without any
user tedious intervention. Therefore, the proposed system can
provide a one-to-one motion correspondence with the
opponents attack. This is another innovation of the proposed
system. After confirming which type of the attack is and how
to hit the attacked character, we tackle the physics-based
reacting problem by applying a PD control to joints of a
character. Relying on the PD control, the attacked character
can physically react to the attacking perturbation and
gradually recovery from the perturbation by tracking the
designated reference motion. The main contribution of the
proposed system is to combine kinematics with physics
models to simulate the natural-looking reacting motions.
The remainder of the paper is organized as follows.
Section 2 addresses the related studies about Kinect, PCA,
SVM, and PD control servo. In Section 3, we present the
adopted technologies in detail. Experimental results are
shown in Section 4 for demonstrating the effectiveness of the
proposed method. Section 5 concludes the paper and
mentions some future directions.
II. RELATED WORKS
With the rapid growth of entertainment technologies, a wide
variety of wearable, wired or wireless controllers have been
developed to be worked well for facilitating the game play.
Kinect is a motion sensing input device that is developed by
Microsoft for Xbox 360 game consoles. Game users need not
to hold any body control device for interacting with a game;
in contrast, they simply use their body motions and voice to
control the game. Kinect comprises 3D depth sensors, RGB
cameras, and multi-array microphones that can provide the
full-body motion capture and voice recognition capabilities.
In this paper, we take advantage of Kinect to capture depth
and skeleton information of a bipedal character.
Principal Component Analysis (PCA) is an effective
technique to perform dimensionality reduction [Wheatland et
al., 2013]. Lim et al., (2005) proposed a framework for robot
movement coordination and learning, and they adopted PCA
to extract the joint trajectory basis functions from human
motion capture data. Tang et al., (2006) presented a method
to simulate interactive falling behaviors and they used PCA
on captured motions to reduce the joint dimensionality in
distance calculation.
Support Vector Machine (SVM) is a part of machine
learning algorithm and it is a useful technique to quickly
classify online the physical motions for various purposes.
SVM needs a set of training data to build a model for
predicting the expected class of motion data. Zordan et al.,
(2007) proposed an approach that computes fast dynamic
response to unanticipated interactions and they used SVM for
data classification by selecting among a set of possible
reaction strategies.
Self-Organizing Map (SOM) is a type of Artificial
Neural Network (ANN) that is trained using unsupervised
learning to produce a low-dimensional (typically two-
dimensional), and this makes SOM useful for visualizing
low-dimensional views of high-dimensional data, and
convenient for observing and analyzing data.
Proportional-Derivative (PD) control has been used
successfully for the purpose of tracking reference motions
stored in a database while simulating characters current
motions in different environments [Yin et al., 2007]; in
addition, a PD controller has been also applied to work on
characters with disturbances for keeping them in balance
[Abe & Popovi, 2006; Sok et al., 2007]. Conventionally, this
controller is used to employ in robotics, and Zordan &
Hodgins (1999) further took the controller into account to
properly implement it in the field of computer graphics and
animation. Now most of the studies about response motions
take good advantage of a PD controller with an optimization
method to search for target motions in the next time step [Yin
et al., 2003; Abe et al., 2007; Silva et al., 2008]. In particular,
using a PD controller needs to specify exact parameters that
could be manually configured by using biometrics or
experimental experience.
III. SYSTEM OVERVIEW
We show the block diagram of the proposed system in figure
1 and then give an overview for each system component in
this subsection. In the part of Kinect skeletal tracking, a game
player poses the body and waves the hands to pretend to fight
his\her opponent; meanwhile, Kinect will track and extract
his\her continuous motion data. PCA is adopted to
accomplish the feature extraction such that we can obtain a
feature vector to stand for the high-dimensional motion data.
A standard fighting pose has been defined in advance to
compare with the tracked motion data. As a result, we can
confirm that the fighting action is a valid attacking motion or
not. Concerning the part of the reference motion
classification, the reference motion data are captured by using
a motion capture system from a real actor and are stored in a
motion dataset. The reference motion data will be
automatically segmented into different motion categories by
means of applying SVM. After we verify the attack invoking
by the players opponent is a valid fighting action, the motion
correspondence will be carried out to pick up the most
correlative pre-captured motion to serve as the target
reference motion. In the case of a successful hit, a PD control
will be employed to track the target reference motion for
The SIJ Transactions on Computer Science Engineering & its Applications (CSEA), Vol. 2, No. 3, May 2014
ISSN: 2321-2381 2014 | Published by The Standard International Journals (The SIJ) 95
properly reacting; otherwise, the player will be still animated
by using the original reference motion, such as idle or
defensive motion. Because the system is a one-on-one
fighting game, the player gets his\her turn to attack his\her
opponent in the next round.

Figure 1: The Block Diagram of the Proposed System
IV. SKELETAL TRACKING AND FEATURE
EXTRACTION
In this subsection, we first describe how to use Kinect to
accomplish a skeletal tracking and then apply PCA to obtain
a feature extraction. As shown in Figure 2, we specify a
skeletal model of a character to be corresponded to the
default skeleton format defined by Kinect. First, image,
depth, and skeleton data streams of a game player are
continuously captured by using Kinect where the frame rate
was often set to a value of 30 frames per second. In addition,
a standard attacking motion will be specified in advance in
order to compare with the on-line captured motion. As a
result, the game player will be demanded to imitate the
specified attacking motion; for example, the game player lifts
up his\her hands with his\her face facing upwards and he\she
puts down his\her hands as if throwing an energy ball. After
calculating standard deviations between the standard motion
and the captured motion, we can allow a valid attacking
motion of the game player to fight his\her opponent in terms
of the different orientation, strength, velocity, and
acceleration of an attacking action.

Figure 2: A Skeletal Structure of a Character
The second step is to apply PCA to extract a low-
dimensional feature vector from the on-line captured motion
to avoid high-dimensional data computation and to further
verify the motion accuracy. We refer to Smith (2002) to build
a PCA model that is applied to accomplish the dimensionality
reduction for the sake of extracting a motion feature vector.
Suppose that there is a configuration vector X of the
character
T
n
x x x X ) , , , (
2 1
. The mean vector X is
calculated and we must subtract the mean from each of the
data dimension. The formula of calculating a variance in one
dimension is:
) 1 (
) (
1
2
2

n
X X
s
n
i
i

(1)
The next step is to calculate the covariance matrix
x
C
between
i
Dim and
j
Dim . It is defined as:
)) , cov( , (
, , j i j i j i x
Dim Dim c c C ,
) 1 (
) )( (
) , cov(
1

n
X X X X
Dim Dim
n
i
j j i i
j i

(2)
where
j i
c
,
represents the covariance between
i
X and
j
X .
After obtaining the covariance matrix, we calculate the
eigenvectors and eigenvalues of the covariance matrix.
v v A ,
0 ) ( I A Det 0 ,
(3)
where is an eigenvalue of a square matrix. From the
eigenvectors and eigenvalues, we can choose principal
components and form a feature vector to stand for the original
high-dimensional motion data with or without leading to loss
of some information.
, =

1

(4)
, =

=1
(5)
Based on Lu et al., (2014), our new dimension space is
composed of eigenvalues and eigenvectors. In comparison
with the above study, we use Mahalanobis distances in (4) to
determine the similarity between two covariance matrices,
where x is the observation, c is the sample data and is the
The SIJ Transactions on Computer Science Engineering & its Applications (CSEA), Vol. 2, No. 3, May 2014
ISSN: 2321-2381 2014 | Published by The Standard International Journals (The SIJ) 96
covariance matrix of the sample data, and then we sum up
each observation values in (5), to determine the dissimilarity
of the sample data, where there are k observations.
V. REACTION DETERMINATION AND
MOTION TRACKING
In order to pursue approaching trajectories from reference
motion data in real time, a PD controller was applied here to
yield appropriate joint torque on a character. In each time
step, we generate extra torque by means of a PD controller to
be as closely to the example motion as possible. An
illustration of applying an external force to a rigid body is
shown in figure 3. The formula of a PD control servo is
defined as:
=

(6)
In (6),

and

are the proportional gain and derivative

gain, and are the joint angles and joint velocities,

and

are the desired joint angles and joint velocities, and is

the joint control torque. The differences,

and

,
will give the tracking directions and magnitudes from the
current poses to the desired poses. An illustration of applying
an external force to a rigid body is shown in Figure 3. The
gains will control the weights for the produced torque.
Concerning an articulated rigid body, we need further take
moment of inertia
c
I into account and the equation is defined
as:
=

(7)
where
c
I is the total accumulated moment of inertia with
respect to all linked joints. In practice, we suppose that
d
K is
about
p
k 2 and then we add
c
I 2 to the second term of
(7).

Figure 3: An Illustration of Applying an External Force to a Rigid
Body
VI. IMPLEMENTATION AND RESULTS
In this section we describe how to systematically implement
the proposed system and show the experimental results to
verify the framework. We use Unity to serve as our game
development system and take advantage of Kinect to extract
game players attacking motions. With respect to example
motions, META GypsyGyro-18motion capture suit was used
to capture real motions from a real actor for building a
reacting motion data set. We make use of Libsvm [Chang &
Lin, 2011] and Scilab [Scilab Enterprises, 2013] to
accomplish off-line SVM classification and PCA verification.
On the other hand, singular value decomposition (SVD) is
employed to provide the on-line eigenvalues and eigenvectors
calculation for dynamically obtain a feature vector after
capturing a players attacking motion. Figure 4 shows the
result the SOM weighting position of a 418 * 57 matrix
sample data, which includes three motions for this type of
motion. Figure 5 shows the sample hits of the data.
While adopting Kinect to track the skeletal motions of
the game player, we initially set the frame rate to a value of
30 frames per second and capture the motions for two
seconds. Consequently, there are total 60 frames within the
captured motion. We then sample the 60-frames motion every
four frames and acquire 15 samples. In addition, our scheme
is to ignore some of the joints that have less importance in
motion tracking and the number of joint dimensions which is
from 46 to 64. After applying PCA, we reduce the number of
joint dimensionality from 46 to 4 and 64 to 5 dimensions with
about 5% of data loss for our motion information. Table 1
lists the parameters used in the tracking examples. Figure 6
shows the result of performing PCA, and we choose the four
highest eigenvalues for the left figure, and five highest
eigenvalues for the right figure to standard for the original
captured motion. A screenshot of the proposed game system
is shown in figure 7. One game player tries to attack another
player through an energy ball.









Table 1: Parameters used in the Tracking Examples
Body Head Left Upper Arm Left Fore Arm Left Hand Right Upper Arm Right Fore Arm Right Hand Spine
K
p
150 300 200 50 300 200 50 3000
K
d
15 30 20 5 30 20 5 300
The SIJ Transactions on Computer Science Engineering & its Applications (CSEA), Vol. 2, No. 3, May 2014
ISSN: 2321-2381 2014 | Published by The Standard International Journals (The SIJ) 97

Figure 4: Weight Positions of SOM Results

Figure 5: Sample Hits of the SOM Results


Figure 6: The Result of Performing PCA for Obtaining the Feature
Vector


Figure 7: A Screenshot of the Proposed Game System. One Game
Player Tries to Attack another Player through an Energy Ball
VII. CONCLUSIONS AND FUTURE WORK
In this paper, we have presented a method that adopts Kinect,
PCA, SVM, and PD control to simulate real-time reacting
motions for one-on-one fighting game. The key insight
behind our approach is that we adopt physical-based model to
incorporate with kinematics-based model. However, we did
not come up with optimization criteria but give an easy-to-
implement and practical solution for Kinect games. The idea
that we give is very intuitive and efficient; that is, PCA was
used to reduce motion dimensionality for obtain a low-
dimensional feature vector, SVM was applied to perform
motion classification automatically and accurately, and PD
control was employed to track an example motion by yielding
appropriate joint torque on a character.
For future research, we would like to explore more
attacking motions to provide combo attacks and define a
strategy to try to efficiently defend an opponents fighting
action. Moreover, PD servos typically produce unstable
control forces, and we will aim to use more stable PD
controllers to replace the current one.
ACKNOWLEDGMENT
This work is supported in part by the National Science
Council at Taiwan, R.O.C., under the project grant numbers
NSC102-2221-E-415-021.
The SIJ Transactions on Computer Science Engineering & its Applications (CSEA), Vol. 2, No. 3, May 2014
ISSN: 2321-2381 2014 | Published by The Standard International Journals (The SIJ) 98
REFERENCES
[1] V.B. Zordan & J.K. Hodgins (1999), Tracking and Modifying
Upper-Body Human Motion Data with Dynamic Simulation,
Proceedings of the Computer Animation and Simulation, Pp.
1322.
[2] I.L. Smith (2002), A Tutorial on Principal Component
Analysis, Retrieved Nov. 2013, from
http://www.cs.otago.ac.nz/cosc453/student_tutorials/principal_
components.pdf.
[3] K.K. Yin, M.B. Cline & D.K. Pai (2003), Motion Perturbation
based on Simple Neuromotor Control Models, Proceedings of
the 11th Pacific Conference on Computer Graphics and
Applications, Pp. 445449.
[4] B. Lim, S. Ra & F.C. Park (2005), Movement Primitives,
Principal Component Analysis, and the Efficient Generation of
Natural Motions, Proceedings of the 2005 IEEE International
Conference Robotics and Automation, Pp. 46304635.
[5] N. Tang, Z. Pan, L. Zheng & M. Zhang (2006), Interactive
Generation of Falling Motions, Journal of Visualization and
Computer Animation, Vol. 17, No. 34, Pp. 271279.
[6] Y. Abe & J. Popovi (2006), Interactive Animation of
Dynamic Manipulation, Proceedings of the 2006 ACM
SIGGRAPH/Eurographics Symposium on Computer Animation,
Pp. 195204.
[7] V. Zordan, A. Macchietto, M. Soriano & C.C. Wu (2007),
Interactive Dynamic Response for Games, Proceedings of
the 2007 ACM SIGGRAPH Symposium on Video Games, Pp.
914.
[8] K.W. Sok, M. Kim & J. Lee (2007), Simulating Biped
Behaviors from Human Motion Data, ACM Transactions on
Graphics, Vol. 26, No. 3.
[9] K.K. Yin, K. Loken & M.v.d. Panne (2007), SIMBICON:
Simple Biped Locomotion Control, ACM Transactions on
Graphics, Vol. 26, No. 3, Pp. 110.
[10] Y. Abe, M.d. Silva & J. Popovi (2007), Multiobjective
Control with Frictional Contacts, Proceedings of the 2007
ACM SIGGRAPH/Eurographics Symposium on Computer
Animation, Pp. 249258.
[11] M.d. Silva, Y. Abe & J. Popovi (2008), Interactive
Simulation of Stylized Human Locomotion, ACM
Transactions on Graphics, Vol. 27, No. 3, Pp. 110.
[12] C.C. Chang & C.J. Lin (2011), A Library for Support Vector
Machines, ACM Transactions on Intelligent Systems and
Technology, Vol. 2, No. 3.
[13] Scilab Enterprises (2013), Scilab for Very Beginners,
Retrieved Nov. 2013, from
http://www.scilab.org/content/view/full/849.
[14] N. Wheatland, S. Jrg & V. Zordan (2013), Automatic Hand-
Over Animation using Principle Component Analysis,
Proceedings of the Motion on Games, Pp. 175180.
[15] T.C. Lu, Y.C. Chen, C.Y. Li & M.C. Tsai (2014), Real-time
Reacting Motion Simulation for One-on-One Fighting Games,
Proceedings of 2014 Annual Conference on Engineering
&Information Technology (ACEAIT).
Tainchi Lu received his Ph.D. degree in
computer science and engineering from
National Sun Yat-sen University, Kaohsiung,
Taiwan, R.O.C., in 1999. Since 2006, he has
been an associate professor in the Dept. of
Computer Science and Information
Engineering at National Chiayi University,
Chiayi, Taiwan, R.O.C. His current research
interests are in the areas of computer graphics,
computer animation, and interactive multimedia.
Yuchen Chen received his B.S. degree from
National Chiayi University (NCYU) of
computer science of information engineering.
He is a graduate student in computer science
of information engineering at NCYU. His
research interest includes motion clustering
and pattern recognition.

Jiayi Li received his B.S. degree from I-Shou
University of computer science of information
engineering. Now, he is a graduate student in
computer science of information engineering
at National Chiayi University. His research
interest includes human and machine
interaction and motion data analysis.

Minchih Tsai received his masters degree
from National Chiayi University (NCYU) of
computer science of information engineering.
He is a Ph.D. student in computer science of
information engineering at NCYU. His
research includes computer graphics and
physics-based character animation.