Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
BlueEyes
Human-Operator Monitoring System
Team members Pawe Kowalik Piotr Kubiaczyk Krystian Nowak Stanisaw Osiski Tomasz Puak Jan Kniat, Ph. D.
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
Team mentor
BlueEyes
Human Operator Monitoring System
Agenda
Motivation What is BlueEyes (and what is not) ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
BlueEyes
Human Operator Monitoring System
Motivation
Human error - a frequent reason of catastrophes and ecological disasters temporal indisposition weariness oversight Monitoring of conscious brain involvement Capable sensors available Lack of complex solution exploiting wireless link technology, database systems and AI algorithms
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
BlueEyes
Human Operator Monitoring System
What is BlueEyes ?
The complex solution for human-operator monitoring: conscious brain involvement monitoring triggering user-defined actions data recording
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
BlueEyes
Human Operator Monitoring System
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
BlueEyes
Human Operator Monitoring System
BlueEyes - benefits
Prevention from dangerous incidents Minimization of ecological consequences financial loss a threat to a human life The reconstruction of the course of operators work
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
BlueEyes
Human Operator Monitoring System
Physiological foundations
Human brain
Preconscious brain Conscious brain
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
alogical automatic
BlueEyes
Human Operator Monitoring System
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
Exploratory mode
Planning mode
BlueEyes
Human Operator Monitoring System
Eye movements
Retina - central vision (macula)
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
BlueEyes
Human Operator Monitoring System
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
Jazz Multisensor
BlueEyes
Human Operator Monitoring System
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
BlueEyes
Human Operator Monitoring System
Physiological foundations
System design System implementation
Future improvements
Project work summary System demonstration
BlueEyes
Human Operator Monitoring System
Design objectives
A mobile data acquisition device
Physiological foundations
System design Objectives System overview DAU CSU System implementation Future improvements Project work summary System demonstration
A reliable real-time buffering, processing and recording A clear visualization interface Mechanisms for creating custom data analyzers To ensure a possibility of distributing the processing among a number of computers
BlueEyes
Human Operator Monitoring System
System overview
Data Acquisition Unit Central System Unit
Physiological foundations
System design Objectives System overview DAU CSU System implementation Future improvements Project work summary System demonstration
Bluetooth
Bluetooth
Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system.
BlueEyes
Human Operator Monitoring System
DAU - features
Data Acquisition Unit
Physiological foundations
System design Objectives System overview DAU CSU System implementation Future improvements Project work summary System demonstration
Bluetooth
Lightweight Runs on batteries - low power consumption Easy to use - does not disturb the operator working ID cards for operator authorization Voice transmission using hardware PCM codec
BlueEyes
Human Operator Monitoring System
CSU - features
Connection management Data processing Visualization Data recording
Central System Unit
Physiological foundations
System design Objectives System overview DAU CSU System implementation Future improvements Project work summary System demonstration
Bluetooth
BlueEyes
Human Operator Monitoring System
Physiological foundations
System design Objectives System overview DAU CSU System implementation
Eye movement velocity 250 200 150 100 50 0 -50 -100 -150 -200 -250 0 8000
saccade
Differentiation
16000
Time [ms]
24000
32000
BlueEyes
Human Operator Monitoring System
Physiological foundations
System design Objectives System overview DAU CSU System implementation Future improvements Project work summary System demonstration
BlueEyes
Human Operator Monitoring System
Data security
Only registered mobile devices can connect to the system Bluetooth connection authentication Bluetooth connection encryption Access rights restrictions Personal and physiological data encryption
Physiological foundations
System design Objectives System overview DAU CSU System implementation Future improvements Project work summary System demonstration
BlueEyes
Human Operator Monitoring System
Implementation - DAU
Made from scratch by the team
Physiological foundations
System design System DAU CSU Future improvements Project work summary
implementation
System demonstration
Bluetooth module inside Jazz sensor interface Microphone and earphone sockets 5-key keyboard Two lines, 8 characters LCD Powered from 6 AA batteries
BlueEyes
Human Operator Monitoring System
Implementation - DAU
Physiological foundations
System design System DAU CSU Future improvements Project work summary
implementation
System demonstration
BlueEyes
Human Operator Monitoring System
Implementation - CSU
Standard MDI user interface Custom-built Bluetooth communication routines Intel USB camera ID card programmer Developed using Visual C++ 6.0 Deployment environment: Windows 2000, MS SQL Server
Physiological foundations
System design System DAU CSU Future improvements Project work summary
implementation
System demonstration
BlueEyes
Human Operator Monitoring System
Future improvements
DAU small CMOS camera to monitor the operators point of gaze single PCB (SMD technlogy) low voltage ICs - LiIO batteries power CSU data mining algorithms advanced database encryption using e.g. AES algorithm
Physiological foundations
System design System implementation
System demonstration
Future improvements Project work summary
BlueEyes
Human Operator Monitoring System
Physiological foundations
System design System implementation
System demonstration
Future improvements Project work summary
BlueEyes
Human Operator Monitoring System
Physiological foundations
System design System implementation
BlueEyes
Human-Operator Monitoring System
Future improvements
Project work summary System demonstration
System demonstration
BlueEyes
Human Operator Monitoring System
Physiological foundations
System design System implementation
System demonstration
Future improvements Project work summary
BlueEyes
Human-Operator Monitoring System
BlueEyes
Human Operator Monitoring System
DAU - components
microphone earphone
MC 145483 PCM codec Jazz Multisensor Beeper LCD display LED indicators
Bluetooth module
MAX232
BlueEyes
ID card interface
ID card
Simple keyboard
UART
BlueEyes
Human Operator Monitoring System
Bluetooth
... ...
Bluetooth
Initializes Bluetooth Data demultiplexer Data producer queues modules Inquires for new Separated Raw Data devices in range streams data Operator Buffer Manager Alerts Establishes Alert buffer connections ... Supports operator ... roaming Feeds Operator Raw Data Operator Buffer Manager with Manager Alerts Alert buffer physiological data
BlueEyes
Human Operator Monitoring System
N
Horizontal Acceleration variance high
N
YES
Y
NO
No answer
BlueEyes
Human Operator Monitoring System
...
BlueEyes
Human Operator Monitoring System
Data queue
Data queue
Data consumer
Data Data consumer & Processing consumer producer Data queue
BlueEyes
Human Operator Monitoring System
...
OM
OM
Alerts
Data Analysis
Visualization Module
Data Logger
Processed data