Poster: Improving Motor Rehabilitation Process through a Natural
Interaction Based System Using Kinect Sensor
Alana Da Gama¹, Thiago Chaves², Lucas Figueiredo³, Veronica Teichrieb
Voxar Labs at Center of Informatics of the Federal University of Pernambuco
ABSTRACT
In general, the motor rehabilitation process can take advantage of
natural interaction based systems, including measurements from
patient performance to track its evolution during time and therapy
direction. Thus, the aim of this research is the analysis of the use
of Kinect sensor as interaction support tool for rehabilitation
systems. The Kinect sensor gives three-dimensional information
about the user body, recognizing skeleton and joint positions,
however does not provide the detection of the body specific
movements. This way, the correct description of a rehabilitation
movement (shoulder abduction, for instance) was implemented in
a system prototype. A scoring mechanism was also developed in
order to measure the patient performance, as well as to stimulate
his improvement by displaying a positive feedback.
KEYWORDS: Rehabilitation, Natural Interaction, Body Tracking,
Kinect.
INDEX TERMS: H.5.1 [Information Interfaces and Presentation]:
Multimedia Information Systems - Artificial, augmented, and
virtual realities; I.3.6 [Computer Graphics]: Interaction
Techniques; I.5.5 [Pattern Recognition] Implementation -
Interactive Systems
1 INTRODUCTION
Different kinds of rehabilitation can be improved with
interactive systems such as Virtual Reality (VR), Augmented
Reality (AR) based systems, including orthopedic, and cognitive
treatments [1]. These platforms have brought revealing
advantages, in relation to conventional therapies, in order to
achieve rehabilitation goals. The use of technology can motivate
the patient and promote entertainment, making the repetitive
motor control practice agreeable, as well as diverting patient
attention from its pain. Therewith, the adhesion of patients to
treatment and conscience of its activity is showed to be
significantly increased [2].
In both VR and AR applications, interaction is a fundamental
characteristic. These systems enable the user to interact with real
and/or virtual environments and through these interactive tools it
is possible to control stimulus and perform measurements. This
way, rehabilitation process can be optimized and patients’
accession improved [1].
Moreover, different tools can be used in VR and AR
applications, including sensors with the intent to improve
interaction [3]. Microsoft recently launched the Kinect, which
enables three-dimensional perception from captured images [4-5].
¹aefg@cin.ufpe.br
²tmc2@cin.ufpe.br
³lsf@cin.ufpe.br
4
vt@cin.ufpe.br
IEEE Symposium on 3D User Interfaces 2012
4-5 March, Orange County, CA, USA
978-1-4673-1205-9/12/$31.00 ©2012 IEEE
4
This way, in order to create a rehabilitation system, this study
proposes the use of a natural interface, enabling a body motion
interaction with movement freedom, aiming to improve the
exercise execution by the patient. Moreover, the use of the
proposed tool extends the supervision of the exercise and the
assistance to correct it, providing orientations and stimulus.
Shoulder abduction movement, due to its frequent use in
rehabilitation exercises, was used as case study to evaluate the
first system prototype.
2 REHABILITATION SUPPORT SYSTEM
The system introduced in this poster consists of a method for
movement recognition, focused on the guidance and correction of
the movement in execution, during motor rehabilitation therapies.
This orientation can be useful to home care therapies, where
physiotherapist has to be far from patient and it has to execute
rehabilitation program alone.
The Kinect sensor is capable of detecting the user body position
without any wearable device or calibration, and few
computational resources are necessary. Through the skeleton
tracking provided by the Kinect sensor it is possible to perform the
needed analysis to recognize and correct movement [5].
For rehabilitation support system to work properly it is
necessary to recognize a movement which should be executed
correctly during therapy. This project was undertaken to recognize
the shoulder abduction movement, largely used in shoulder
rehabilitation, where the patient arm performs a movement from
the position attached to the body (with the hand touching the hip)
to a position away from it, creating an angle between trunk and
arm at the frontal plane (aligned to trunk).
For complete shoulder abduction description, the shoulder and
elbow angles need to be computed. The reference vectors of the
shoulder angle are obtained from the shoulder-elbow and
shoulder-hip pairs of points. In turn, the elbow angle is computed
from the elbow-shoulder and elbow-hand vectors. Each angle was
obtained from arc cosine of the dot product between the arm and
trunk vectors, for shoulder abduction, divided by theirs norms. For
elbow angle the same was computed for arm and forearm vectors .
Additionally, to guarantee that the movement is being executed
within the correct plane (independently of user orientation in
relation to the sensor), it is computed the angle between the arm
vector and the normal vector of the target plane, being this last
one obtained from thorax points.
The correctness of the movement execution in a rehabilitation
process is essential for the treatment efficacy. Due to this, the
system is programmed to punctuate whenever the user executes
the movement correctly: i. shoulder abduction angle >= 90
degrees; ii. elbow angle >= 160 degrees; iii. angle between the
arm and the normal vector of the frontal plane >= 80 and <= 100
degrees; iv. right and left shoulder height (Y coordinate) must be
similar, to control trunk compensations; v. actual abduction angle
must be higher than it was before; vi. return to rest position to
continue.
145
 During rehabilitation process corporal conscience of correct
movement is important and can be facilitated by feedback
directing treatment according to goals [3]. This way, aiming to
provide a more efficient tool, a feedback system was
implemented, taking into account the following considerations.
Visual feedback is shown in Figure 1 and Figure 2. The score is
increased each time that movement is executed correctly. For each
five points a congratulation message is given. When movement is
finished correctly a message indicates to return the arm to initial
position (Figure 1c). Movement correction is also enable
describing which the movement is wrong marking it on body and
saying how to perform it correctly (Figure 2).To guide the
movement an additional feedback is provided through a target
showing where the arm should attach to complete the movement
(Figure 1). Additionally a movement status bar is presented that is
loaded gradually accordingly to the movement route (0 to 90
degrees) (Figure 1).
Figure 1. Correctly executed movement
Figure 2. Wrong movements executed: a) anterior elevation; b)
posterior elevation; c) shoulder abduction with flexed elbow d)
reverse movement (up to down) e - f) course deviation; g - h)
postural compensation (trunk lateral inclination)
146
3 RESULTS AND DISCUSSION
The developed prototype, using Kinect as a natural interaction
tool for rehabilitation proposes, showed to be responsive to users’
movements (including little ones) and effective on the evaluation
of the movement correctness and indication aiming its adjustment.
These characteristics can be used to improve AR and VR
technologies on motor rehabilitation, optimizing treatment.
During execution of 50 correct movements all were recognized
correctly and punctuated, including user at seated position and at
different angles in relation to sensor. None of the 60 wrong
exercises were recognized as correct, including postural
compensation which is the most common compensation found on
rehabilitation processes.
Due to the different prototype applicability, as traumatic,
neurologic or geriatric therapies, user tests were applied to three
different populations: three physiotherapy professionals, in order
to gather technical opinion about the system and the application
itself, four adults, to evaluate general usability and interaction,
and three elderly subjects who are members of geriatric
physiotherapy groups, due to the wide actuation of motor
rehabilitation on this population, with a total of ten subjects.
Using system, from ten subjects, eight learned to execute the
correct movement with some prototype help and the others two
got it right since the first execution, without any need of external
guidance. The usability questionnaire used in the tests is scored
from one to five. The negative aspects found were mainly related
to the letters size, information clarity and the stimulus (2.77, 3.75
and 3.47 points respectively). The positive evaluations were most
about the user satisfaction, motivation, the system easiness (4.67,
4.67, 4.64 points respectively).
User evaluation showed that the correction and guidance
provided by the system were executed with efficacy. However,
some interface improvements are needed in order to achieve a
better usability for the application. Moreover, visual feedback
supplied by the system favored interaction and promoted a better
execution of the exercise.
This work introduced a movement recognition method
developed using Kinect sensor information. The implemented
prototype showed efficacy when detecting correct therapeutic
exercises, avoiding wrong movements during the rehabilitation,
this way preventing lesions and optimizing the treatment. The
prototype demonstrated levels of precision and sensibility which
enable the adaptation for physical limited subjects.
REFERENCES
[1] H. Sveistrup, "Motor rehabilitation using virtual
reality," Journal of NeuroEngineering and
Rehabilitation, vol. 10, pp. 1-10, 2004.
[2] E Richard, et al., "Augmented Reality for Rehabilitation
of Cognitive Disable Children: A preliminary Study," in
Virtual Rehabilitation 2007, 2007, pp. 102-108.
[3] J W Buker, et al., "Augmented Reality Games for
Upper-Limb Stroke Rehabilitation," in 2nd
International Conference on Games and Virtual Worlds
for Serious Applications 2010, 2010.
[4] J. Shotton, et al., "Real-time human pose recognition in
parts from single depth images," in Computer Vision
and Pattern Recognition (CVPR), 2011 IEEE
Conference on, 2011, pp. 1297-1304.
[5] PrimeSense. (2011, OpenNI: User Guide. Available:
http://www.openni.org/Documentation.aspx