Proceedings of the 26th Annual International Conference of the IEEE EMBS
San Francisco, CA, USA • September 1-5, 2004
REAL-TIME HEAD MOVEMENT SYSTEM AND EMBEDDED LINUX
IMPLEMENTATION FOR THE CONTROL OF POWER WHEELCHAIRS
H.T. Nguyen, L.M. King, G. Knight
Key University Research Centre for Health Technologies, Faculty of Engineering,
University of Technology, Sydney, NSW, AUSTRALIA
Abstract –- Mobility has become very important for our quality of
life. A loss of mobility due to an injury is usually accompanied by a
loss of self-confidence. For many individuals, independent mobility is
an important aspect of self-esteem. Head movement is a natural form
of pointing and can be used to directly replace the joystick whilst still
allowing for similar control. Through the use of embedded LINUX
and artificial intelligence, a hands-free head movement wheelchair
controller has been designed and implemented successfully. This
system provides for severely disabled users an effective power
wheelchair control method with improved posture, ease of use and
attractiveness.
Keywords – Head movement, power wheelchair control, neural
network, embedded system.
I. INTRODUCTION
The number of people who need mobility assistance is
regularly increasing, not only as a result of longer life
expectancy, but also due to improved resuscitation techniques
following perinatal, automobile or sports accidents.
Conventional electrical wheelchairs are not always sufficient
to compensate for mobility disabilities: serious spasticity
(cerebral palsy), excessive weak residual physical capacities
(tetraplegia), or some cognitive impairments (head trauma)
exclude or hamper their use. For many, even the operation of
powered wheelchairs is a difficult and demanding task.
To overcome the problems associated with joystick control, a
new variety of interfaces have been designed to aid in the use of
the joystick. This includes repositioning or adding mechanical
attachments to the end of the joystick so that it can be operated
by a person’s elbow or chin. Other replacements for the joystick
include eyewink control [1] and voice control [2-3].
One of the hands-free technologies we have focused over in
the past eight years is head movement. Head movement is a
natural form of pointing and can be used to directly replace the
joystick whilst still allowing for similar control. This method of
wheelchair control has been used previously by sufferers of high
level spinal lesions and motor neurone disease.
One head movement-sensing device (the Oxford Optical
Pointer) was developed by Tew [4]. This device used a photo
quadrant sensor which is made up of four photodiodes fabricated
on one substrate. As a light spot from an infrared emitting diode
passed over the detector, the relative proportion of photocurrents
from each quadrant was used to determine the light source
position. Recently, Coyle [5] developed a system which uses a
combination of technologies: standard hand controlled joystick,
ultrasonic non-contact head controller and a small vocabulary
voice recognition system.
0-7803-8439-3/04/$20.00©2004 IEEE
In 1998, we developed an alternative telemetric head
movement device for the control of powered wheelchairs. In
contrast to Coyle’s system, we used a tilt sensor and wireless
technology [6]. This system had many benefits for the disabled
wheelchair operator: visual benefit (no wiring, small motion
detector), operator mobility and posture (full mobility, upright
posture), individual training and ease of installation [7-8].
In this paper, we describe the development of a real-time head
movement system and an embedded Linux implementation for
the control of power wheelchairs. The integration of an
embedded system will have a number of positive aspects for the
user such as freeing up the space directly in front of the user and
creating a more natural environment for the user of the
wheelchair. This system has been designed to enable disabled
people to control a powered wheelchair easily, accurately and
comfortably.
II. METHODOLOGY
A. Real-Time Head Movement System
An Analogue Devices ADXL-202EB232 is used as the head
movement transducer. It contains an ADXL-202JQC tilt sensor, a
16C63 Microchip microcontroller, an AD666 for voltage
regulation of the RS-232 request-to-send (RTS) signal, an
ADM202 for the RS-232 interface, and a crystal. The serial
communication protocol uses a 9600 baud rate and an 8N1
parity. The sensor outputs this data as a pulse width modulated
signal and this signal is transmitted via a serial RS232 connection
on request, the request being the reception of the ASCII character
“G”.
Head movements are detected by tipping the head in a
nodding action for the desired direction. This movement is
sampled upon a request from a computer or an embedded system,
maintained at a rate of approximately 20 Hz.
The data obtained from the head movement sensor contains
offsets, and is normalised to remove these effects. In addition, an
adaptive base level implementation is used to allow the sensor
position on the head to be moved without requiring recalibration
of the zero level.
Various trials have been performed to evaluate the possibility
of a future technique using an adaptive structure to handle
changes to the system environment. They are implemented in
Labview and an example can be seen in Figure 1. Normalised
head nods in the forward, backward, left and right positions are
recorded using the data-logging feature of the developed
software. For this implementation, a real-time neural network is
designed to classify intended commands.
4892

Fig. 1. Labview interface for the head movement system
B. Embedded System
The embedded system contains three layers: the hardware
layer, software layer and the control layer. The hardware layer is
responsible for the acquisition of input data, facilitation of the
software and control algorithms and the configuration of output
control signals. The software enables the hardware and control
layers to interact and control the wheelchair. The control layer is
a neural network which takes the input data from the hardware
layer, processes it and controls the motors on the wheelchair.
Figure 2 below outlines this structure.
Control Layer
Software Layer
Hardware Layer
Input Outputs
Fig. 2. Embedded system layered structure
The hardware layer uses DIMM-PC based hardware, in which
a single DIMM module provides CPU, system memory, flash
hard disk and other I/O capabilities, all powered from a single
+5V supply with a very low power usage. With all of this
hardware contained on a 68x40mm board, it is an ideal solution
for the embedded application outlined in this paper. The DIMM
PC used is an AMD Élan SC520-I operating at 133MHz with
32MB of system memory and 32MB of flash hard disk. The
primary advantage of using this technology over other embedded
solutions is the reduction in development time due to its x86
architecture and the power and availability of the development
tools available. Figure 3 outlines the hardware layer, showing its
data inputs and control signal outputs.
4893
Speed
Bluetooth DIMM- Output
telemetry PC
Direction
Output
LCD
Input Display Output
Fig. 3. Hardware layer module
Using both the DIMM-PC module and a DIMM-COMBO
module, which has additional serial interfaces and an LCD
output, we are able to meet all of the I/O requirements of the
figure above. The serial to Bluetooth interface connects directly
to the UART of the DIMM-PC, and both the speed and direction
output are connected to another two UARTs on the device, which
are in turn converted to the 0 to +5V output range via a serial
digital to analogue converter. The LCD display device is
controlled by the DIMM-COMBO module and is mounted above
the embedded circuitry which is in turn mounted on the arm rest
of the wheelchair, making visibility straightforward for the user.
One of the main advantages of using an x86 based
architecture for our embedded system is that we can make use of
powerful development tools and operating systems to increase
the efficiency of the development process. For this embedded
system, White Dwarf Linux 1.11 has been chosen as the
operating system, since it is specifically designed for use on the
DIMM-PC and as such, has been configured and optimised for
the embedded design. This installation has been further
optimised and as such all non-essential services, such as web and
file severing, has been disabled.
In addition to the operating system, the software layer is
responsible for facilitating control of the wheelchair. These
modules include control of the I/O functionality under Linux,
demodulation of the transmitted telemetry and pre-processing of
the demodulated data for the control layer. Additionally, error
checking, normalisation and other processing must also be
handled in this layer. These components have been developed in
C, due to the availability of open source tools.
The structure for the neural network is a multilayered neural
network with a single hidden layer. The input layer consists of
63 neurons, with the first 31 being the input from the
forward/backward transducer, the next 31 being the input from
the left/right transducer and an augmentation input, -1. The
output layer consists of five neurons corresponding to each of the
five commands the network is to classify, which are forward,
back, left, right and stop (which is a sideways shake of the head).
The hidden layer consists of 25 neurons, giving a smooth cycle
error for all commands plus one fixed input bias neuron.
Therefore two weight matrices exist for this neural network; they
are the hidden layer weight matrix W (25×63) and the output
layer weight matrix W (5×26).
 The network requires three exclusive data sets in order to be
trained; these are the training set, the verification set and the test
set. For the network shown in this paper samples from five
people were used to populate the three data sets. The training set
comprised of 16 sets of each command class, i.e. 16 x 5 = 80
samples which are in turn recycled during the training cycle. The
verification set contains 4 sets of each command class, i.e. 20
samples. The total number of samples used in the training of this
network is 217, giving quite a large number of samples and
exposing the training of the network to a variety of
interpretations of the commands.
III. RESULTS
The success of this system depends heavily on integrity of the
training set data as well as the topology of network chosen.
Using the topology outlined above, supervised learning was
undertaken using the delta learning rule and a static learning
constant, whereby the network learns the correct response by
comparing the targeted response with the response of the system
and updating the weights.
The cycle error plot of both the training and validation set
allow us to judge the success of the neural network and these
have been shown in Figures 4 and 5. Figure 4 shows the training
set cycle error, which has a smooth curve with no sharp changes
indicating the correct number of hidden layer neurons have been
used. One class has a slower error decent than the others, and
this is the stop class which is to be expected since it is the most
complicated signal which its horizontal shaking action. One
aspect which is interesting is that the shape of the cycle error is
generally the same as the other classes, indicating that the stop
command is being trained like the others only over more cycles.
Cycle Error - Training Set
35
30
25
20
Error
15
10
5
0
5 10 15
Cycles
20 25
Fig. 4. Training set cycle error
Figure 5 shows the cycle error of the validation set which has
a number of interesting points. Firstly, for every class except
stop (which again has the slowest descending error) the cycle
errors are identical and do not have an early minimum, indicating
there is no early stopping point and that these classes have not
been overtrained. The stop command however does have a
minimum, although this is a local minimum and the error over
the long term does settle to a value approximating zero and again
this suggests that the network is not being over trained.
Cycle Error - Verification Set
0.7
0.6
0.5
0.4
Error
0.3
0.2
0.1
0
5 10 15
Cycles
20 25 30
Fig. 5. Validation set cycle error
After training and validation, a final weight vector was
generated and applied to the test set, thereby demonstrating the
overall system performance. The final system was able to
correctly classify 95.8% of the signals fed to the network.
IV. CONCLUSION
This project has used DIMM-PC technology or the control of a
power wheelchair using head movement technology. It has
achieved this by developing an entire system to capture, sample
and train a neural network in the classification of head
movements via a tilt sensor. This project has also dealt with
issues relating to the design and manufacture of embedded
hardware, embedded Linux and embedded graphical user
interfaces. An operational embedded system has been created
which successfully interfaces with the head movement sensor. A
real-time head movement classifier has been developed and
tested to above 90% correct classification and the embedded GUI
has been demonstrated operational.
REFERENCES
30
[1] E.E. Crisman, A. Loomis, R. Shaw, Z. Laszewski, “Using
the Eye Wink Control Interface to Control a Powered
Wheelchair,” Proceedings of the Annual International
Conference of the IEEE Engineering in Medicine and
Biology Society, 13:1821-1822, 1991.
[2] M. Mazo, F.J. Rodriguez, J.L. Lazoro, J. Urena, J.C.
Garcia, E. Santiso, P.A. Revenga, “Electronic Control of a
Wheelchair guided by voice commands,” Control Eng.
Practice, 3: 665-674, 1995.
4894
[3] R.C. Simpson, S.P. Levine, “Voice Control of a Powered
Wheelchair”, IEEE Trans. Neural Systems and
Rehabilitation Engineering, Vol. 10, No. 2, pp. 122-125,
June 2002.
[4] A.I. Tew, “The Oxford Optical Pointer: A Direction-
sensing Device with Proportional Electric Output,” Med. &
Biol. Eng. & Comp., 26: 68-74, 1988.
[5] E.D. Coyle, “Electronic Wheelchair Controller Designed
for Operating by Hand-Operated Joystick, Ultrasonic Non-
Contact Head Control and Utterance from a Small Word-
Command Vocabulary,” IEE Colloquium (Digest), 55: 3/1-
3/4, 1995.
[6] T. Joseph and H.T. Nguyen, "Neural Network Control of
Wheelchairs using Telemetric Head Movement",
Proceedings of the IEEE Engineering in Medicine and
Biology Society, Hong Kong, 2731-2733, 1998.
[7] H. Nguyen, G. Knight, R. Ekanayake, "Telemetric Head
Movement Control of Powered Wheelchair," World
Congress on Medical Physics and Biomedical Engineering,
Sydney, 24-29 August 2003, Paper 2516 (CDROM).
[8] P.B. Taylor, H.T. Nguyen, "Performance of a Head-
Movement Interface for Wheelchair Control", Proceedings
of the IEEE Engineering in Medicine and Biology Society,
Mexico, 1590-1593, 2003.

4895