Multisensor Fusion and Integration

1.0 INTRODUCTION

Sensor is a device that detects or senses the value or changes of value of the
variable being measured. The term sensor some times is used instead of the term
detector, primary element or transducer.

The fusion of information from sensors with different physical characteristics,

such as light, sound, etc enhances the understanding of our surroundings and provides
the basis for planning, decision making, and control of autonomous and intelligent
machines.

Data fusion techniques combine data from multiple sensors, and related
information from associated databases, to achieve improved accuracies and more
specific inferences than could be achieved by the use of a single sensor alone. The
concept of multisensor data fusion is hardly new. Humans and animals have evolved
the capability to use multiple senses to improve their ability to survive. For example,
it may not be possible to assess the quality of an edible substance based solely on the
sense of vision or touch, but evaluation of edibility maybe achieved using a
combination of sight, touch, smell, and taste. Similarly, while one is unable to see
around comers or through vegetation, the sense of hearing can provide advanced
warning of impending dangers. Thus multisensory data fusion is naturally performed
by animals and humans to achieve more accurate assessment of the surrounding
environment and identification of threats, thereby improving their chances of
survival.

SCT College of Engineering 1

 Multisensor Fusion and Integration

2.0 SENSORS EVOLUTION

A sensor is a device that responds to some external stimuli and then provides
some useful output. With the concept of input and output, one can begin to
understand how sensors play a critical role in both closed and open loops.

One problem is that sensors have not been specified. In other words they tend
to respond variety of stimuli applied on it without being able to differentiate one from
another. Nevertheless, sensors and sensor technology are necessary ingredients in any
control type application. Without the feedback from the environment that sensors
provide, the system has no data or reference points, and thus no way of understanding
what is right or wrong with its various elements.

Sensors are so important in automated manufacturing particularly in robotics.

Automated manufacturing is essentially the procedure of removing human element as
possible from the manufacturing process. Sensors in the condition measurement
category sense various types of inputs, condition, or properties to help monitor and
predict the performance of a machine or system.

SCT College of Engineering 2

 Multisensor Fusion and Integration

3.0 SENSOR AND SENSOR TECHNOLOGY

IN THE PAST

The earliest example of sensors is not inanimate devices but living organisms.
A more recent example of living organisms used in the early days of coal mining in
the United States and Europe.

Robots must have the ability to sense and discriminate between objects. They
must then be able to pick up these objects, position them properly and work with
them without damaging or destroying them.

Intelligent system equipped with multiple sensors can interact with and
operate in an unstructured environment without complete control of a human
operator. Due to the fact that the system is operating in a totally unknown
environment, a system may lack of sufficient knowledge concerning the state of the
outside world. Storing large amounts of data may not be feasible. Considering the
dynamically change world and unforeseen events, it is usually difficult to know the
state of the world. Sensors can allow a system to learn the state of the world as
needed and to cautiously update its own model of the world.

SCT College of Engineering 3

 Multisensor Fusion and Integration

4.0 SENSORS PRINCIPLES

A sensor is defined as a measurement device which can detect characteristics of

an object through some form of interaction with them. It is a device that measures a
physical quantity and converts it into a signal which can be read by an observer or by
an instrument. For example, a mercury-in-glass thermometer converts the measured
temperature into expansion and contraction of a liquid which can be read on a
calibrated glass tube. A thermocouple converts temperature to an output voltage which
can be read by a voltmeter. For accuracy, all sensors need to be calibrated against
known standards.

A good sensor obeys the following rules:

• Is sensitive to the measured property

• Is insensitive to any other property
• Does not influence the measured property

Ideal sensors are designed to be linear. The output signal of such a sensor is
linearly proportional to the value of the measured property. The sensitivity is then
defined as the ratio between output signal and measured property. For example, if a
sensor measures temperature and has a voltage output, the sensitivity is a constant
with the unit [V/K]; this sensor is linear because the ratio is constant at all points of
measurement.

Sensors can be classified into two categories:

Contact and noncontact.

SCT College of Engineering 4

 Multisensor Fusion and Integration

A contact sensor measure the response of a target to some form of physical

contact .this group of sensors responds to touch, force ,torque,pressure,temperature or
electrical quantities.

A noncontact type sensor measures the response brought by some form of

electromagnetic radiation. This group of sensors responds to light, x-ray, acoustic,
electric or magnetic radiation.

5.0 IDEAL SENSOR

An ideal sensor should possess the following properties
1. Continuous operation without effecting the measurand.
2. Appropriate sensitivity and selectivity.
3. Fast and predictable response.
4. Reversible behavior.
5. High signal to noise ratio.
6. Compact
7. Immunity to environment.
8. Easy to calibrate.

5.1 SENSOR DEVIATIONS

If the sensor is not ideal, several types of deviations can be observed:

• The sensitivity may in practice differ from the value specified. This is called a
sensitivity error, but the sensor is still linear.
• Since the range of the output signal is always limited, the output signal will
eventually reach a minimum or maximum when the measured property

SCT College of Engineering 5

 Multisensor Fusion and Integration

exceeds the limits. The full scale range defines the maximum and minimum
values of the measured property.
• If the output signal is not zero when the measured property is zero, the sensor
has an offset or bias. This is defined as the output of the sensor at zero input.
• If the sensitivity is not constant over the range of the sensor, this is called
nonlinearity. Usually this is defined by the amount the output differs from ideal
behavior over the full range of the sensor, often noted as a percentage of the
full range.
• If the deviation is caused by a rapid change of the measured property over
time, there is a dynamic error. Often, this behavior is described with a bode plot
showing sensitivity error and phase shift as function of the frequency of a
periodic input signal.
• If the output signal slowly changes independent of the measured property, this
is defined as drift (telecommunication).
• Long term drift usually indicates a slow degradation of sensor properties over a
long period of time.
• Noise is a random deviation of the signal that varies in time.
• Hysteresis is an error caused by when the measured property reverses
direction, but there is some finite lag in time for the sensor to respond,
creating a different offset error in one direction than in the other.
• If the sensor has a digital output, the output is essentially an approximation of
the measured property. The approximation error is also called digitization
error.
• If the signal is monitored digitally, limitation of the sampling frequency also
can cause a dynamic error.
• The sensor may to some extent be sensitive to properties other than the
property being measured. For example, most sensors are influenced by the
temperature of their environment.

All these deviations can be classified as systematic errors or random errors. Systematic
errors can sometimes be compensated for by means of some kind of calibration

SCT College of Engineering 6

 Multisensor Fusion and Integration

strategy. Noise is a random error that can be reduced by signal processing, such as
filtering, usually at the expense of the dynamic behavior of the sensor.

Resolution

The resolution of a sensor is the smallest change it can detect in the quantity that it is
measuring. Often in a digital display, the least significant digit will fluctuate,
indicating that changes of that magnitude are only just resolved. The resolution is
related to the precision with which the measurement is made. For example, a scanning
tunneling probe (a fine tip near a surface collects an electron tunnelling current) can
resolve atoms and molecules.

6.0 MULTISENSOR FUSION AND INTEGRATION

Multisensor integration is the synergistic use of the information provided by

multiple sensory devices to assist in the accomplishment of a task by a system.Multi-
sensor integration is still a matter of research in many areas. A major aim was to
create a system, which is open for any kind and any number of sensors while
providing a united programming interface. Manipulation primitives constitute such an
interface that enables programmers to specify sensor-guided and sensorguarded
motion commands in an intuitive way.The complete system has been implemented as
prototype for industrial use, i.e. all results are practical ones and not only simulation
results.

Multisensor fusion refers to any stage in the integration process where there is
an actual combination of different sources of sensory information into one
representational format. Sensor fusion is the combining of sensory data or data derived
from sensory data from disparate sources such that the resulting information is in

SCT College of Engineering 7

 Multisensor Fusion and Integration

some sense better than would be possible when these sources were used individually.
The term better in that case can mean more accurate, more complete, or more
dependable, or refer to the result of an emerging view, such as stereoscopic vision
(calculation of depth information by combining two-dimensional images from two
cameras at slightly different viewpoints).

The data sources for a fusion process are not specified to originate from identical
sensors. One can distinguish direct fusion, indirect fusion and fusion of the outputs of
the former two. Direct fusion is the fusion of sensor data from a set of heterogeneous
or homogeneous sensors, soft sensors, and history values of sensor data, while indirect
fusion uses information sources like a priori knowledge about the environment and
human input.

6.1 MULTISENSOR INTEGRATION

The diagram represents multisensor integration as being a composite of

basic functions. A group of n sensors provide input to the integration process. In
order for the data from each sensor to be used for integration, it must first be
effectively modelled. A sensor model represents the uncertainty and error in the data
from each sensor and provides a measure of its quality that can be 7used by the
subsequent integration functions.

After the data from each sensor has been modelled, it can be integrated into
the operation of the system in accord with three different types of sensory processing:
fusion, seperate operation, and guiding or cueing.

SCT College of Engineering 8

 Multisensor Fusion and Integration

Sensor registration refers to any of the means used to make data from each
sensor commensurate in both its spatial and temporal dimensions. If the data provided
by a sensor is significantly different from that provided by any other sensors in the
system, its influence on the operation of the sensors might be indirect. The separate
operation of such a sensor will influence the other sensors indirectly through the
effects he sensor has on the system controller and the world model. A guiding or
cueing type sensory processing refers to the situation where the data from one sensor
is used to guide or cue the operation of other sensors.

The results of sensory processing functions serve as inputs to the world model
.a world model is used to store information concerning any possible state of the
environment that the system is expected to be operating in. A world model can
include both a priori information and recently acquired sensory information. High
level reasoning processes can use the world model to make inferences that can be
used to detect subsequent processing of the sensory information and the operation of
the system controller.

Sensor selection refers to any means used to select the most appropriate
configuration of sensors among the sensors available to the system.

SCT College of Engineering 9

 Multisensor Fusion and Integration

6.2 MULTISENSOR FUSION

The fusion of data or information from multiple sensors or a single sensor

over time can takes place at different levels of representation.

The different levels of multisensor fusion can be used to provide information

to a system that can be used for a variety of purposes.eg signal level fusion can be
used in real time application and can be considered as just an additional step in the
overall processing of the signals, pixel level fusion can be used to improve the
performance of many image processing tasks like segmentation ,and feature and
symbol level fusion can be used to provide an object recognition system with
additional features that can be used to increase its recognition capabilities.

SCT College of Engineering 10

 Multisensor Fusion and Integration

7.0 APPLICATIONS OF MULTISENSOR FUSION AND

INTEGRATION

In recent years, benefits of multisensor fusion have motivated research in a variety of

application area as follows….

7.1 Robotics

Robots with multisensor fusion and integration enhance their flexibility and
productivity in industrial application such as material handling, part fabrication,
inspection and assembly.

Mobile robot present one of the most important application areas for
multisensor fusion and integration .When operating in an uncertain or unknown
environment, integrating and tuning data from multiple sensors enable mobile robots
to achieve quick perception for navigation and obstacle avoidance.

Marge mobile robot equipped with multiple sensors.perception, position

location, obstacle avoidance vehicle control, path planning, and learning are
necessary functions for an autonomous mobile robot.

Honda humanoid robot is equipped with an inclination sensor that consists of

three accelerometer and three angular rate sensors. each foot and wrist is equipped
with a six axis force sensor and the robot head contains four video

SCT College of Engineering 11

 Multisensor Fusion and Integration

cameras.multisensor fusion and integration of vision ,tactile,thermal,range,laser radar,

and forward looking infrared sensors play a very important role for robotic system.

Fig 1: Honda humanoid robot

SCT College of Engineering 12

 Multisensor Fusion and Integration

Fig 2: Anthrobot five-fingered robotic hand holding an object in the field of-view of a
fixed camera.

Fig 3:MARGE mobile robot with a variety of sensors

SCT College of Engineering 13
 Multisensor Fusion and Integration

7.2 Military application

It is used in the area of intelligent analysis, situation assessment, force

command and control, avionics, and electronic warfare. It is employed for tracking
targets such as missiles, aircrafts and submarines.

7.3 Remote sensing

Application of remote sensing include monitoring climate, environment, water

sources, soil and agriculture as well as discovering natural sources and fighting the
important of illegal drugs. Fusing or integrating the data from passive multispectral
sensors and active radar sensors is necessary for extracting useful information from
satellite or airborne imaginary.

7.4 Biomedical application

Multisensor fusion technique to enhance automatic cardiac rhythm monitoring

by integrating electrocardiogram and hemodynamic signals. Redundant and
complementary information from the fusion process can improve the performance
and robustness for the detection of cardiac events including the ventricular activity
and the atria activity.

7.5 Transportation system

Transportation system such as automatic train control system, intelligent

vehicle and high way system, GSP based vehicle system, and navigation air craft
SCT College of Engineering 14
 Multisensor Fusion and Integration

landing tracking system utilize multisensor fusion technique to increase the

reliability, safety, and efficiency.

Table 1: Defense applications

Table 2: Non Defense applications

SCT College of Engineering 15

 Multisensor Fusion and Integration

8.0 FUTURE RESEARCH DIRECTIONS

The current state of the art in multisensor fusion is in continuous

development. there are therefore, promising future research areas the encompass
multilevel sensor fusion ,sensor fault detection, micro sensors and smart sensors, and
adaptive multisensor fusion as follows.

8.1 Multilevel sensor fusion

Single level sensor fusion limits the capacity and robustness of a system, due
to the weakness in uncertainity, missing observation, and incompleteness of a single
sensor.therfore there is a clear need to integrate and fuse multisensor data for

SCT College of Engineering 16

 Multisensor Fusion and Integration

advanced system with high robustness and flexibility and the multilevel sensor fusion
system is needed in advanced system.

There are different levels, low level fusion methods can fuse the multisensor
data, and medium level fusion methods can fuse data and feature to obtain fused
feature or decision. High level fusion methods can fuse feature and decision to obtain
the final decision.

8.2 Fault detection

Fault detection has become a critical aspect of advanced fusion system design.
Failures normally produce a change in the system dynamics and pose a significant
risk. There are many innovative methods have been accomplished.

8.3 Micro sensors and smart sensors

Successful application of a sensor depends on sensor performance, cost and

reliability.

However, a large sensor may have excellent operating characteristics but its
marketability is severely limited by its size. Reducing the size of a sensor often
increases its applicability through the following.

1 lower weight and greater portability

2 lower manufacturing cost and fewer materials

3 wider range of application.

SCT College of Engineering 17
 Multisensor Fusion and Integration

Clearly, fewer materials are needed to manufacture a small sensor but the cost
of materials processing is often a more significant factor. The revolution and
semiconductor technology have enabled us to produce small reliable processors in the
form of integrated circuits. The microelectronic applications have led to a
considerable demand for small sensors or micro sensors that can fully exploit the
benefits of IC technology. Smart sensors can integrate main processing, hardware and
software. According to the definition proposed by Breckenridge and Husson, a smart
sensor must possess three features

The ability to

Perform logical computable functions

Communicate with one or more other devices and

Make a decision using logic or fuzzy sensor data

8.4 Adaptive multisensor fusion

In general, multisensor fusion requires exact information about the sensed

environment.however, in the real world, precise information about the sensed
environment is scare and the sensors are not always perfectly functional.therfore a
robust algorithm in the presence of various forms of uncertainty is necessary.

Researchers have developed adaptive multisensor fusion algorithm to address

uncertainties associated with imperfect sensors.

SCT College of Engineering 18

 Multisensor Fusion and Integration

9.0 CONCLUSION

Sensors play an n important role in our everyday life because we have a need
to gather information and process it for some tasks. Successful application of sensor
depends on sensor performance, cost and reliability.

The paradigm of multisensor fusion and integration as well as fusion

techniques and sensor technologies are used in micro sensor based application in
robotics, defense, remotesensing, equipment monitoring, biomedical engineering and
transportation systems. Some directions for future research in multisensor fusion and
integration target micro sensors and adaptive fusion techniques. This may be of
interest to researches and engineers attempting to study the rapidly evolving field of
multisensor fusion and integration.

SCT College of Engineering 19

 Multisensor Fusion and Integration

10.0 REFERENCE

1. Ren.C.Luo, Fellow, IEEE Chin Chen Yih and Kuo Lan Su “Multisensor
Fusion And Integration: Approaches, Applications, and Future Research
Directions”, IEEE Sensors Journal, Vol 2 ,No 2 April 2002 pp 107-118

2. Encyclopedia of instrumentation and control pp 610

3. Paul Champan, “Sensors Evolution”, International Encyclopedia of robotics

Application and Automation,vol 3 pp 1505- 1516

4. M . Rahimi and P.A Hancock, “Sensors, Integration”, International

Encyclopedia of Robotics application
& Automation Vol 3 pp 1523- 1531

5. Kevin Hartwig, “Sensors,Principles”, International Encycloprdia of

Robotics Application and Automation, Vol 3 pp 1532-1536

SCT College of Engineering 20

 Multisensor Fusion and Integration

ABSTRACT

Multisensor fusion and integration is a rapidly evolving research area.

Multisensor fusion and integration refers to the combination of sensory data from
multiple sensors to provide more accurate and reliable information. It requires
interdisciplinary knowledge in control theory, signal processing, artificial
intelligence, probability and statistics, etc. The advantages gained through the use of
redundant, complementary, or more timely information in a system can provide more
reliable and accurate information. This paper provides an overview of current sensor
technologies and describes the paradigm of multisensor fusion and integration as well
as fusion techniques at different fusion levels. Applications of multisensor fusion in
robotics, biomedical system, equipment monitoring, remote sensing, and
transportation system are also discussed. Finally, future research directions of
multisensor fusion technology including microsensors, smart sensors, and adaptive
fusion techniques are presented

SCT College of Engineering 21

 Multisensor Fusion and Integration

SCT College of Engineering 22