Sensig Kap1 2

Sensor Signal Processing
Introduction

Prof. Dr.-Ing. Andreas König
Lehrstuhl Integrierte Sensorsysteme
FB Elektrotechnik und Informationstechnik

Technische Universität Kaiserslautern
Fall Semester 2005
© Andreas König Slide1-1

Brief Course Profile Introduction
Course contents: Basics of sensor signal processing and analysis, feature

computation, dimensionality reduction, classification, and optimization
Lecture: Tuesday 10:00 – 11:30, 23-188
Lab: 12/425 and 21/260, time to be announced
Level: Diploma, Master, and PhD students
Course materials:
Lecture slides for download as pdf-documents
(http://www.eit.uni-kl.de/ ) Lehre/Sensorsignalverarbeitung
Examination: oral (presentation & discussion of semester project)
Consultations: (koenig@eit.uni-kl.de)
Recommended text books and references:
1. R. Hoffmann, Signalanalyse und Erkennung, Springer 1998, ISBN 3-540-63443-6
2. S. Haykin, Neural Networks – A Comprehensive Foundation, Prentice Hall, 1998,
ISBN 0132733501
3. R. Duda, P. Hart, D. Stork, Pattern Classification, Wiley, 2000, ISBN 0471056693
1
Sensor System Applications Introduction
Inspection Man-Machine
Interface
Automotive
IES/IMEMS
IES/IMEMS
Applications
Applications
Biometry Robot
Vision

Sensor System Applications Introduction
Architectural Study of an Over-Take-Monitor:
¾ Driver assistance system

¾ High real-time requirements
¾ Complex scene
¾ Harsh illumination conditions
¾ Tough invariance requirements
¾ Compatibel with car installation
¾ IES/IMEMS with HW-friendly
operators required
¾ Topic of industrial interest !
2
Plethora of Sensors Introduction
Bosch GmbH & Goodyear

Bosch GmbH
Exhaust gas
temperature sensor
MAZeT JENCOLOUR ICs
Various
characteristics/
DMS electrical interfaces
Robot
Piezo-Foiles
Pulse, blodd pressure, Gassensor (FhG IPM)

air flow QinetiQ Ltd Bosch GmbH
Monolithic or Microsystem/MEMS Integration

Integration Technologies Introduction
Single- and multilayer boards (SMD)
© EPCOS AG
© PHYTEC AG
3
Miniaturized LTCC-Modules
© EPCOS AG

Multichip-Modul (MCM)
© EPCOS AG
4
Sensor System Architecture Introduction
Signal processing
and recognition
FPGA/
FPGA/ Memory
Memory
ASIC
ASIC
Intelligent camera
with CMOS-sensor
Processing
Processing
Sensors
Sensors ADC
ADC DAC
DAC Actuators
Actuators
Unit
Unit
Human
HumanIfc
Ifc Diagnostics
Diagnostics Aux.
Aux.Systems
Systems
SoC
SoC
Fingertip
sensor & recognition
Microsystem
Microsystem

Natural vs.Technical Sensing Systems Introduction
Intriguing Properties of Human Brain and Nervous System Inspire Computing:
¾ Neural systems offer attractive

capabilities superior to
conventional computing
¾ Massive parallelism
¾ Invariant signal processing
¾ Learning & adaptation
¾ Robustness & fault-tolerance
Adapted from [Zell 94]
Adapted from [Zell 94]
due to self-repair capability
Sensing, Recognition,
Understanding, Acting
Processing chain in conventional sensing & recognition systems:
Feature- Dimension Classifier Classification

Sensing
computation reduction Training/Test results
5
Design of Recognition System Introduction
¾ Application-Specific Acquisition & Processing of Multisensor Data
Tongue Nostrils
Senses of Living Beings
Eye
Sense of Taste Olfaction
Sense of Touch Heat
Audition Vision Receptors
Technical Models
Sensor Technology MEMS X-Ray UV Visual IR Radar
Light
Microelectronic 380 nm 780 nm
Most Complex Sense: Vision

¾ Application-Specific Acquisition & Processing of Multisensor Data
Night-Vision in Automotive Applications*:
¾ Information unique to visual/IR domain

¾ Appropriate Sensor Fusion required
¾ Embedded real-time capable solution
mandatory
*Courtesy BMW AG, Munich, T. Weidner, H. Hahn
6
¾ Application-Specific Expenditure of Resources
Conjunctiva Pupil
Cornea
Iris
Ciliar muscle
Lens
Retina
Optic nerve
Fovea
Cross Sections
Fly´s Eye Human Eye
¾ Technical spectrum: Photoelectric barrier to special purpose camera

¾ Options: Sensor/spatial resolution, rate, color, stereo (depth), motion, IR

Cost
Cost
(low/high
(low/highvol.)
vol.)
Size,
Size,weight,
weight, Reliability,
Reliability,
power
power Safety,
Safety,(FT)
(FT)
Multiobjective
Multiobjective
Time-to-market Design
DesignOptimization
Optimization
Time-to-market Real-time
Real-time
Flexibility,
Flexibility, Performance
Performance
Adaptivity
Adaptivity (Recognition)
(Recognition)
¾ Appropriate methodology and flow for viable & feasible design mandatory
¾ Approach: Bio-inspired adaptive circuits & systems (integrated HW/SW)
7
Challenges of Intelligent System Design:
¾ Requires substantial engineering effort

¾ Dominantly manual and intuitive task
¾ Time, labour, and cost intensive
¾ Strong diversity of available methods & tools
(from signal processing to Soft-Computing)
¾ Requires experienced and qualified staff
¾ State-of-the-art: IC design 20 years ago
¾ Design style: Full-Custom
¾ DA: Only emerging for visual inspection
¾ No interface to standard chip design hierarchy

Typical Image Processing and Recognition System Design:
Image ROI- & Feature Feature

Selection & Classification
Acquisition Computation
Extraction
Configuration
Observations
¾ DA required: Toward semi-custom or system synthesis design style

¾ Learning approach draws from multiple sources of bio-inspiration
¾ Vision: From optimization/learning in the design phase to on-line
adaptation & self-repair
8
Feedback of Constraints
Intelligent
IntelligentSystem
System ES/MEMS
ES/MEMS
Design
DesignFramework
Framework Design
DesignFramework
Framework
Feedforward of feasible behavioral IS
¾ Linking the Frameworks of IS & ES design using multiobjective optimization

¾ Merging SW & HW development by chosen information processing paradigm
¾ Long term: Migrate from design-time to run-time optimization/adaptation

¾ Holistic Modelling and Design Methodology
Optimization Loop Evaluator HW Constraints

(Constraints)
Sample Design Design Constructor

Sets with Space Specification Structure Design Instance
Class affiliat. Explorer (Parameters) Synthesis
Pareto optimal
Parameter Parameterization design
Parametr. of Structure
Ranges Design Intra/Intermodule
Architecture Space Optimization
Framework
¾Application-specific design
A priori Method Library:
knowledge
¾Learning Paradigm: Learning
Image/Signal Proc.
FS Dim. Reduction from examples or experience
Classification ¾Constraint feedback, e.g.,
numerical resolution/bits
9
QuickCog Environment:
¾ Fast & consistent design
¾ Assessment and optimization
¾ Intra/inter level optimization
¾ Holistic modelling and

simulation
¾ Opportunistic & parsimonious
¾ DR (AFS) salience:
physical savings !

Application Examples Introduction
¾ Automatic Visual Consumption Control

Application-specific resolution
determined by methodology
Visual consumption registration &

interpretation in an embedded vision
system (SoC)
10
Template
Templatematching
matchingapproach
approach
applied
applied in twoindustrial
in two industrial
feasibility
feasibilitystudies
studies

Eye-Tracker for 3D-Display Control (TU Dresden, CS, AI Inst. D4D-Group):
¾ To achieve 3D-perception for system user, eye positions must be tracked

¾ Embedded solution required
¾ Stereo CCDs and TI 320C82 on dedicated board & SW by D4D-Group
¾ CCD properties not satisfactory for harsh illumination conditions (game halls)
¾ Low-power, low-cost deeply embedded solution aspired
¾ Higher performance aspired: recognition/tracking, multiple users
11
Eye-Tracker Research Vehicle:

¾ Monofocal modelling
¾ HDR CMOS-Sensor with integrated
signal processing
¾ Low-power analog/mixed design
¾ Close relation to automotive tasks !
Selective Face/Eye Feature

Attention Location List
Difference Image Features: Feature

Symmetry Gabor jets Tracking
Multi-scale Proc. Loc. autocorr.
Loc. orient.
Classification: 3D-Display
RNN, RBF, Control

Subtask: Eye-Shape Classification:
Gabor-Jets AFS Verilog-A for DFW II

RNN-Classifier with constrained Manhattan Metric
12
¾ Eye-Shape Classification Using Gabor Jets:
¾ AFS chooses feature 2, 4, 6, 7, 8, and 12 from 12-D Gabor-Jet

¾ Recognition results of 100% (one pattern very close to error)
¾ These features were used for RNN classifier implementation work

Eye-Tracker Results: Eye-shapes
¾ SFS best/stable solution 1,000

0,980
¾ Generalization is affected by 0,960

0,940
0,920
dimensionality reduction 0,900

0,880
0,860
¾ SFS solution for HW-design 0,840
0,820
0,800
0,780
Train
Reduced feature space
FSSPEA (10)
Test
FSSPEA (50)
PM
SBS
SFS
ORG
Method Train Test Features

ORG 0,858374 0,953647 12
SFS 0,94704 0,989362 6
SBS 0,93842 0,953647 6
PM 0,93842 0,896657 4
FSSPEA (50) 0,95567 0,896657 3
FSSPEA (10) 0,93842 0,896657 4
13
¾ Real-World-Problem: Medical Tube Classification (M. Eberhardt)
¾ Medical Laboratory Robot DAVID:

Instance
Instance Optimization
Optimization ¾ Task: Tubes sorting & decapping
Parameter, Results ¾ Multiple installation sites in Europe
Structure
Design Phase
Sensor -
data,
Features
Data
Data •Compensation Target
Target
base
base values
values
¾ Machine-In-the-Loop-Learning
¾ General system development
¾ Instance training for compensation
of nonidealities & deviations

¾ DAVID generic training system:
14
¾ Feature Space and Classification for Medical Tube Recognition
¾ Geometric feature computation

¾ Resulting feature space obtained from
image sensor & features computation:
Error due
to loose
label

¾ DAVID generic test system:
¾ The generic recognition system performs well

¾ Site-specific variations affect performance !
¾ MILL concept compensates these deviations
¾ Training limited to types actually to be recognized !
¾ Aging effects can also be compensated by retraining
15
¾ A training assistant guides through

the calibration/training procedure
¾ Trays with known tube types are fed
in the robot for sample set creation
Result:
2k-5k tubes/day
30 sites in Europe

¾ Wireless color sensor system and its application
Spectral response curves for

MAZeT color sensors in TO5 package,
MCS3AT (top) and MCS3BT /
from left to right : MCS3AT, MCS3BT,
MCSi (bottom) (© MAZeT Data
MCSi
Sheet MCS3AT/BT)
16
¾ Xbow Mica2 and Mica2Dot Hardware

¾ MAZeT color sensors were employed with 4-channel transimpe-

dance amplifier:
U − I in ⋅ R f if U ref ≥ I in ⋅ R f
U out (I in ) =  ref
 0 else (1)
¾ Extension board to MICA2Dot motes with ATMEL Atmega 128 µC

& wireless com. by CHIPCON CC1000 fm-transc.-chip (433 MHz)
17
¾ Wireless color sensor system: Schematic and Board
¾ Photo current to
voltage conversion:
R4
U ref = U bat (2)
R3 + R4
R4
U out = U batt − I in R f
R3 + R4
(3)
1024
U batt = 0.6V ⋅ (4)
ADCbatt

¾ Wireless color sensor system: Base Station and Color Sensor Module
¾ Simple SW for sensor readout based

on TinyOS and Linux/Cygwin) on
PC
¾ The achieved wireless color sensor
module was investigated for color
registration & classification
¾ Goal: functional validation &
feasibility experiments
18
¾ Functional Validation: Flourescent Lamp Measurement
20 ms
¾ Response of Color Sensor to Illumination by a Flourescent Lamp. Left:

Test Setup, Right: Green Channel Amplifier Output

¾ Functional Validation: Tri-Color-Source Measurement
0,9
0,8
normalized color intensity
0,7
0,6
0,5
0,4
0,3
0,2
0,1
0
0 30 60 90 120 150 180 210 240 270 300
time / s Red Green Blue
19
¾ Color classification:
Paper Strip with Eight Sample Colors
¾ Two series of 100 repeated measurements for each color (class),

were made with the MCS3BT and used in NN-classification
(QuickCog)
¾ Results confirm feasibility of wireless sensor module for classification


¾ Alternative tube classification by wireless color sensor module:
20
¾ Achieved Medical Tube Classification Rates

¾ Two series of 100 measurements for each of seven tube types
(restricted problem) and an eight uncapped tube were recorded
¾ Result: recognition rate of 99.875% (type 6 to type 2 confusion)

¾ Mechatronic Application: Stall-Margin-Indicator (SMI) for aircraft

jet engines (TU Darmstadt, FG Flugantriebe)
21

22

QuickCog-Module
23
¾ QuickCog SMI Classification Module:

V isor,1
ii Weight
Watcher,1
ii
RN N (R),1
Konfusions C hart,1
M atrix,1
i
Input i
(Datei),1
ii
i N ormalisiere 353
356 355
(T),1 165
164
166155
156
153 296
325294
295 282
340
293
339 357351
354
163
167
208 154
2 274
152
09
275276
211
240 328
281292
338
337
283
284 317
332
277
326
335
341
316
297370
327
331
336
334
314
329375
352371
318 373
104 161
157
160
222
158
159 151
224221
273
201
225191
223
190324
250
210 307
304
182
343
342
303
212 348
308
347
280
306
233
322
321
239
192
298231
234
305
346 291
319
287
349
285
330
315
333
286
290
372
374
103 137 206
162
2204
07189
220
215
188
267 302
238
177
266 183
323
258
241
309
262
181
345 350
232
344279
313
RN N (L),1 8481
80 111
124
125
105
102
114
121
117
106
120
1118
123
101 15
119 140
139
136
144 205202 249
176
214
219
2
203
217 184
261
172
213
263
272
200
252
237
265
264
216
18
248 259
230
173
175
242
271
187
178
185310
270
179
180
197
254
255
299
174
300
235
320
236
301
194
193
243
253
244
260 288
229289
311
195
278
83
82 79
9485
86
77
87 122
78112113107
109
110
76
141
138
116
147
145
126
146
127
150
108143
149
132
130
134
135
133
131
129
142
169
168
170198
199
171
268
2452196
186
251
247
228
269
312
256
257
46 358 369
368367
62
63
91
990
929995
98
38996 148128 227359 360 366362
60
61
59 64
56
65
55
100
9788 226 361 365 363
ii 12
13
14
15
11
1
1722
6
24 20
23
21
12519 41 33
8 31
32
30
29
6653
54
58 72
57
7471
52
75
70
5173 364
10
79125
684 40 34
26
3828
35
27 669
67 8
3 424339
4437 449
48
3645
50
476
i
Input
(Datei),2
Speichern Speichern
(Datei),1 (Datei),2
i i

QuickCog Demonstration Introduction
¾ Several data sets have been acquired for the SMI task from the
described setup
¾ They have been archived as classification benchmark data similar to
other repository data
¾ In QuickCog SMI a training and a test module has been established
for the SMI classification problem
¾ Feature data from FFT spectra with 24-dimensions, 375 samples, and
4 classes for four operating regions are employed
¾ Due to the significant dynamic range, normalization has been applied
before classification
¾ QuickCog demonstration for the SMI partial system
24
Main Objectives Introduction
¾ Introduce Master and PhD students to the state-of-the-art of

sensory systems and the underlying information processing
requirements
¾ Familiarize with advanced information processing methods for
the design of intelligent systems
¾ Develop the skills to use methods from signal processing,
statistics, artificial neural networks, evolutionary computation,
fuzzy and hybrid systems to process sensor signals
¾ Extend these basic skills to systematically combine methods to
design and validate a complete sensor signal processing system
¾ Tackle applications from Mechatronics to Visual Inspection
¾ Develop the ability to evolve and extend a basic system
implementation for modified requirements
¾ Provide understanding of the interaction of real-world
implementations with the chosen algorithmic architecture
¾ Introduce to adaptation techniques for compensation of
(dynamic) interactions and influences (drift, aging, etc.)

Introduction
Course Contents:
1. Introduction
2. Signal Processing and Analysis
3. Feature Computation
4. Cluster Analysis
5. Dimensionality Reduction Techniques
6. Data Visualization & Analysis
7. Classification Techniques
8. Sensor Fusion
9. Systematic Design of Sensor Systems
10. Outlook
25
Introduction

Recommended Textbooks and complementary Readings:
1. R. Hoffmann, Signalanalyse und Erkennung, Springer 1998, ISBN

3-540-63443-6
2. S. Haykin, Neural Networks – A Comprehensive Foundation,
Prentice Hall, 1998, ISBN 0132733501
3. R. Duda, P. Hart, D. Stork, Pattern Classification, Wiley, 2000,
ISBN 0471056693
4. K. Fukunaga, Introduction to Statistical Pattern Recognition,
Academic Press, 1990, ISBN 0122698517
5. H.-R. Tränkler, E. Obermeier (Hrsg.), Sensortechnik – Handbuch
für Praxis und Wissenschaft, Springer, 1998
6. H. Ahlers (Hrsg.), Multisensorikpraxis, Springer, 1997
7. R. Schalkoff, Digital Image Processing and Computer Vision, John
Wiley & Sons, 1989.
List will be extended during the course presentation


Introduction

Software for hands-on experience of course contents:
1. QuickCog: Image Processing and Recognition System Design

Environment (Evaluationversion): http://www.eit.uni-kl.de/
Research QuickCog (Visual Programming of Image Processing and
Recognition Systems, Sample-Set oriented Learning)
2. Matlab, standard in signal processing and engineering applications,
class-room licence in Lab 12/425 available,
http://www.mathworks.com
3. GNU Octave (free Matlab compatible program)
http://www.octave.org
List will be extended during the course presentation
26

Sensig Kap1 2

Caricato da

Informazioni sul documento

Titolo originale

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

Sensig Kap1 2

Caricato da

Copyright:

Formati disponibili

Sensor Signal Processing

Sensor Signal Processing

FB Elektrotechnik und Informationstechnik

Fall Semester 2005

© Andreas König Slide1-1

Sensor Signal Processing

Course contents: Basics of sensor signal processing and analysis, feature

© Andreas König Slide1-3

Sensor Signal Processing

Architectural Study of an Over-Take-Monitor:

¾ Driver assistance system

© Andreas König Slide1-4

Bosch GmbH & Goodyear

Pulse, blodd pressure, Gassensor (FhG IPM)

Monolithic or Microsystem/MEMS Integration

© Andreas König Slide1-5

Sensor Signal Processing

Single- and multilayer boards (SMD)

© Andreas König Slide1-6

© Andreas König Slide1-7

Sensor Signal Processing

© Andreas König Slide1-8

© Andreas König Slide1-9

Sensor Signal Processing

¾ Neural systems offer attractive

Processing chain in conventional sensing & recognition systems:

Feature- Dimension Classifier Classification

© Andreas König Slide1-10

¾ Application-Specific Acquisition & Processing of Multisensor Data

Most Complex Sense: Vision

© Andreas König Slide1-11

Sensor Signal Processing

¾ Application-Specific Acquisition & Processing of Multisensor Data

Night-Vision in Automotive Applications*:

¾ Information unique to visual/IR domain

¾ Application-Specific Expenditure of Resources

¾ Technical spectrum: Photoelectric barrier to special purpose camera

© Andreas König Slide1-13

Sensor Signal Processing

© Andreas König Slide1-14

Challenges of Intelligent System Design:

¾ Requires substantial engineering effort

© Andreas König Slide1-15

Sensor Signal Processing

Typical Image Processing and Recognition System Design:

Image ROI- & Feature Feature

¾ DA required: Toward semi-custom or system synthesis design style

© Andreas König Slide1-16

Feedforward of feasible behavioral IS

¾ Linking the Frameworks of IS & ES design using multiobjective optimization

© Andreas König Slide1-17

Sensor Signal Processing

¾ Holistic Modelling and Design Methodology

Optimization Loop Evaluator HW Constraints

Sample Design Design Constructor

© Andreas König Slide1-18

¾ Assessment and optimization

¾ Intra/inter level optimization

¾ Holistic modelling and

¾ Opportunistic & parsimonious

© Andreas König Slide1-19

Sensor Signal Processing

¾ Automatic Visual Consumption Control