Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
1 Introduction
The most precise measurements of human motion are obtained by motion cap-
ture systems. During an acquisition markers attached on human body are tracked
by set of calibrated cameras. Their 3D coordinates are reconstructed, which
allows to determine orientations of bone segments. Thus, skeleton based motion
capture data are represented by time series of poses described by joints rotations
as well as global location and orientation of a skeleton.
In baseline approach of motion analysis Euler angles are utilized for cod-
ing 3D rotations of human joints. They contain data of three successive basic
c Springer Nature Switzerland AG 2019
N. T. Nguyen et al. (Eds.): ACIIDS 2019, LNAI 11432, pp. 567–578, 2019.
https://doi.org/10.1007/978-3-030-14802-7_49
568 A. Świtoński et al.
rotations performed around axes of local coordinate systems. Euler angles are
intuitive, especially in case of medical diagnosis. They can correspond to flex-
ion/extention, abduction/adduction and rotation of bone segments which are
commonly analyzed in medical assessment of motion data. However, processing
of rotations described by three clearly uncorrelated values is also troublesome.
It can lead to improper results, because there is much more difficult to take into
account mutual relationships between basic rotations in such the case. Further-
more, Euler angles suffer gimbal lock problem.
It justifies application of unit quaternions which are broadly used in computer
graphics, also in motion processing and analysis. Quaternions provide compact
angle-axis representation of 3D rotations performed by α angle about fixed axis
described by vector u = (u1 , u2 , u3 ).
α α
q = cos + (u1 · i + u2 · j + u3 · k) · sin (1)
2 2
There is an algebra defined for quaternions with selected operators corre-
sponding to transformations of 3D rotations. Moreover, they are efficient in
integration over time. However, the most important advantage of unit quater-
nions over Euler angles is related to the simultaneous and coherent processing
of complete rotational data. It can have an impact on obtained results.
Segmentation is process of data division into disjoint multiply partitions
which collectively cover them. It simplifies the representation – data became
easier for interpretation, further processing and recognition. In case of motion
data segmentation can be useful in detection of short lasting specific movements
or phases as well as in generic feature extraction for the sake of classification
problems.
The current paper concerns issues of segmentation of motion data. On the
basis of our previous experiences on gait identification [13,14] a new method
which utilizes unit quaternions and watershed transform is introduced.
The following sections are devoted to: a short review of state of the arts
methods focused on segmentation of motion data, explanation of different vari-
ants of watershed transform used in computer vision as well as descriptions of
the proposed methods and obtained preliminary results.
2 Related Work
There are numerous studies on segmentation of motion capture data. Barbič et al.
[4] introduced three highly cited approaches. In the first one, on the basis of selected
number of principal components reconstruction of consecutive motion frames is
carried on. Boundaries between detected segments are denoted by time instants
of sudden increase of reconstruction error. In another variant Probabilistic Prin-
cipal Components Analysis is used to model the first K frames of time series. It
allows to assess fitness of frames K+1 through K+T to estimated Gaussian dis-
tribution. Mahalanobis distance is employed to distinguish new partitions. In the
third approach Gaussian Mixture Models are utilized to cluster entire sequence.
Quaternion Watershed Transform in Segmentation of Motion Capture Data 569
Time series are divided into time instants of consecutive frames belonging to dif-
ferent Gaussian distributions. In all methods unit quaternion are employed to rep-
resent rotational data. However in fact they only form four dimensional vectors.
Thus, it is difficult to point any advantages over the case of Euler angle usage.
Technique proposed by Bouchard and Badler [8] employs neural networks
trained on the basis manual segmentation and input attributes corresponding
to specified kinematic features of selected markers. To reduce oversegmentation,
temporary aggregation of detected boundaries in local windows of 25 s width is
carried on. The peaks are determined and they point proper segments. Super-
vised machine learning is also applied by Arikan et al. [2] – poses represented
by coordinate vector are classified as belonging to selected simple motions e.g.
running, walking. Support Vector Machines are used. The coordinate vector con-
tains joints positions in one second local windows centered in reference to the
classified pose.
The method which utilizes unit quaternions and unsupervised learning is
proposed by Wu et al. [16]. Joint rotations are transformed into tangent spaces
on the basis of logarithm operator. It allows to calculate similarity matrix by
using baseline Euclidean metric. Further, poses are clustered by the normalized
cut model and the weighted kernel k-means. In the postprocessing filtering which
removes some isolated short segments is carried on. Finally, so called category
strings containing sequences of subsequent segments are formed and analyzed to
reduce number of obtained partitions.
Xie et al. [17] transforms motion capture data into low dimensional embed-
ding on the basis of Laplacian Eigenmap algorithm. The proper segmentation
is realized by extreme tracking of amplitudes of characteristic curve. Quite sim-
ilar method is presented by Yang et al. [18]. In successive steps dimensionality
of pose space is reduced by Principal Component Analysis, velocity vectors are
calculated and clustered according to cosine distance. In recently proposed app-
roach Arn et al. [3] employ generalized curvature analysis for curves residing in
n-dimensional pose space.
Lin and Kulic [12] for initial partitioning analyze velocity features such as
velocity peaks and zero velocity crossings. In the second stage Hidden Markov
Models are employed to precisely modify segments from the initially identified
candidates. In another segmentation approach Yang et al. [19] utilize Fisher’s
optimal partitioning algorithm for multidimensional time series transformed into
matrix form.
There are also examples of proposed segmentation methods targeted to
motion data registered by inertial measurement units [1,11].
Summarizing, the problem of motion data segmentation is broadly studied in
the literature, there are plenty of techniques introduced. The most of the appli-
cations are related to detection of selected activities in long lasting recordings.
However, there is no one general method applicable to all problems. Moreover,
some of them require training data and other are only semiautomatic. Further-
more, there is no study strictly focused on gait analysis. Therefore, conducting
research on gait segmentation is justified. Still new techniques and applications
can be proposed.
570 A. Świtoński et al.
3 Watershed Transform
The watershed transform originally introduced by Bouchard and Badler [5] has
still numerous applications in 2D and 3D image segmentation [9,10,15]. The
transform determines boundaries between segments in locations with the high-
est gradient magnitude. The markers of successive segments refer to homoge-
neous regions which are pointed by local minima of the gradient magnitude. It
is analogous to flooding of 3D surface – barriers are built in places where water
of different catchment basins is met.
In the implementation of watershed transform priority queues are utilized [7].
The priorities relate to gradient magnitudes of image pixels. In the initial step
markers are extracted, labeled and inserted to the queue. In a baseline approach
markers correspond to regional minima of the gradient. Further, the process is
iterated until the queue is not empty. In the single iteration, the first element x
from the queue is extracted and every, so far unlabelled neighbor of x is assigned
with the same label as x and inserted to the queue.
To segment motion data on the basis of dynamics of joint and body move-
ments a method to approximate angular velocities has to be applied. Simi-
larly to [13] differential filtering of time domain is chosen. In case of Euler
angles Δ(anglest , anglet−1 ) defined in Eq. (4) for rotations from consecutive
time instants t and t − 1 is calculated:
Segmentation concerns main cycles containing two middle adjacent steps per-
formed by left and right lower limbs and analyzes only rotational data. To extract
main cycles from gait sequences tracking of extremes of interaankle distances is
574 A. Świtoński et al.
0.45
60 Gradient magnitude
0.4 Watershed lines
Catchment basins
50
0.35
Gradient magnitude
40 0.3
OY [in]
0.25
30
0.2
20
0.15
10 0.1
0.05
0
0
-40 -30 -20 -10 0 10 20 30 40 20 40 60 80 100
OZ [in] Time [10ms]
0.14
40
0.12
OY [in]
30 0.1
20 0.08
0.06
10
0.04
0 0.02
0
-40 -20 0 20 40 20 40 60 80 100
OZ [in] Time [10ms]
Gradient magnitude
0.2 Watershed lines
Catchment basins
Gradient magnitude
0.15
0.1
0.05
0
20 40 60 80 100
Time [10ms]
Fig. 5. Watershed segmentation of randomly selected gait instance (Color figure online)
0.45
Gradient magnitude
60
Watershed lines
0.4
Catchment basins
50
0.35
Gradient magnitude
40 0.3
OY [in]
0.25
30
0.2
20
0.15
10
0.1
0 0.05
0
-40 -20 0 20 40 20 40 60 80 100
OZ [in] Time [10ms]
Gradient magnitude
60
0.2 Watershed lines
Catchment basins
50
Gradient magnitude
40 0.15
OY [in]
30
0.1
20
10
0.05
0
-40 -20 0 20 40 20 40 60 80 100
OZ [in] Time [10ms]
60 Gradient magnitude
0.2 Watershed lines
Catchment basins
50
Gradient magnitude
40 0.15
OY [in]
30
0.1
20
10
0.05
0
-40 -20 0 20 40 20 40 60 80 100
OZ [in] Time [10ms]
Fig. 6. Hierarchical watershed segmentation of gait instance from Fig. 5 (Color figure
online)
0.45
Gradient magnitude
60 Watershed lines
0.4
Markers
50 0.35
Gradient magnitude
40 0.3
OY [in]
0.25
30
0.2
20
0.15
10
0.1
0 0.05
0
-40 -20 0 20 40 20 40 60 80 100
OZ [in] Time [10ms]
Fig. 7. Marker controlled watershed gait segmentation of gait instance from Fig. 5
represented by complete rotational data (Color figure online)
References
1. Aoki, T., Venture, G., Kulic, D.: Segmentation of human body movement using
inertial measurement unit. In: IEEE International Conference on Systems, Man,
and Cybernetics (SMC), pp. 1181–1186. IEEE (2013)
2. Arikan, O., Forsyth, D.A., O’Brien, J.F.: Motion synthesis from annotations. In:
ACM Transactions on Graphics (TOG), vol. 22, pp. 402–408. ACM (2003)
3. Arn, R., Narayana, P., Draper, B., Emerson, T., Kirby, M., Peterson, C.: Motion
segmentation via generalized curvatures. IEEE Trans. Pattern Anal. Mach. Intell.
(2018)
4. Barbič, J., Safonova, A., Pan, J.Y., Faloutsos, C., Hodgins, J.K., Pollard, N.S.: Seg-
menting motion capture data into distinct behaviors. In: Proceedings of Graphics
Interface 2004, pp. 185–194. Canadian Human-Computer Communications Society
(2004)
578 A. Świtoński et al.
5. Beucher, S.: Use of watersheds in contour detection. In: Proceedings of the Inter-
national Workshop on Image Processing. CCETT (1979)
6. Beucher, S.: Watershed, hierarchical segmentation and waterfall algorithm. In:
Serra, J., Soille, P. (eds.) Mathematical Morphology and Its Applications to Image
Processing, pp. 69–76. Springer, Dordrecht (1994). https://doi.org/10.1007/978-
94-011-1040-2 10
7. Beucher, S., Meyer, F.: The Morphological Approach to Segmentation: The Water-
shed Transformation, Mathematical Morphology in Image Processing, pp. 433–481.
Marcel Dekker Inc., New York (1992)
8. Bouchard, D., Badler, N.: Semantic segmentation of motion capture using Laban
Movement Analysis. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Kar-
pouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 37–44. Springer,
Heidelberg (2007). https://doi.org/10.1007/978-3-540-74997-4 4
9. Ciecholewski, M.: River channel segmentation in polarimetric SAR images: water-
shed transform combined with average contrast maximisation. Expert Syst. Appl.
82, 196–215 (2017)
10. Huynh, H.T., Le-Trong, N., Oto, A., Suzuki, K., et al.: Fully automated MR liver
volumetry using watershed segmentation coupled with active contouring. Int. J.
Comput. Assist. Radiol. Surg. 12(2), 235–243 (2017)
11. Lin, J.F.S., Joukov, V., Kulic, D.: Human motion segmentation by data point
classification. In: 36th Annual International Conference of the IEEE Engineering
in Medicine and Biology Society (EMBC), pp. 9–13. IEEE (2014)
12. Lin, J.F.S., Kulic, D.: Online segmentation of human motion for automated reha-
bilitation exercise analysis. IEEE Trans. Neural Syst. Rehabil. Eng. 22(1), 168–180
(2014)
13. Switonski, A., Josinski, H., Wojciechowski, K.: Dynamic time warping in classi-
fication and selection of motion capture data. In: Multidimensional Systems and
Signal Processing, pp. 1–32 (2018)
14. Szczesna, A., Switonski, A., Slupik, J., Zghidi, H., Josinski, H., Wojciechowski,
K.: Quaternion lifting scheme applied to the classification of motion data. Inf. Sci.
(2018)
15. Wisaeng, K., Sa-ngiamvibool, W.: Automatic detection and recognition of optic
disk with maker-controlled watershed segmentation and mathematical morphology
in color retinal images. Soft Comput. 22, 6329 (2018). https://doi.org/10.1007/
s00500-017-2681-9
16. Wu, Z., Liu, W., Xing, W.: A novel method for human motion capture
data segmentation. In: IEEE 15th International Conference on Dependable,
Autonomic and Secure Computing, 15th International Conference on Perva-
sive Intelligence & Computing, 3rd International Conference on Big Data
Intelligence and Computing and Cyber Science and Technology Congress
(DASC/PiCom/DataCom/CyberSciTech), pp. 780–787. IEEE (2017)
17. Xie, X., Liu, R., Zhou, D., Wei, X., Zhang, Q.: Segmentation of human motion
capture data based on Laplasse eigenmaps. In: Chen, H., Zeng, D.D., Karahanna,
E., Bardhan, I. (eds.) ICSH 2017. LNCS, vol. 10347, pp. 134–145. Springer, Cham
(2017). https://doi.org/10.1007/978-3-319-67964-8 13
18. Yang, Y., Chen, J., Liu, Z., Zhan, Y., Wang, X.: Low level segmentation of motion
capture data based on hierarchical clustering with cosine distance. Int. J. Database
Theor. Appl. 8(4), 231–240 (2015)
19. Yang, Y., Shum, H.P., Aslam, N., Zeng, L.: Temporal clustering of motion capture
data with optimal partitioning. In: Proceedings of the 15th ACM SIGGRAPH
Conference on Virtual-Reality Continuum and Its Applications in Industry-Volume
1, pp. 479–482. ACM (2016)