Gesture Recognition Technology

Computer Science & Engineering Department Institute Of Aeronautical Engineering By, A.PRASAD 07951A0551

CONTENTS
Introduction WORKING Input devices Gesture Typologies Gestural control devices Hand Gesture Recognition Haptics & Data gloves Facial Gesture Recognition Microsoft's XBOX & Xbox 360 Uses Challenges Conclusion

Introduction
HCI ?????

What is Gesture?

Gesture Recognition ? Gesture recognition is the mathematical interpretation of

a human gesture by a computing device.

NEED ?
building a rich bridge between machines and humans.

PUI?
Gesture recognition + facial recognition + voice recognition + eye tracking + lip movement recognition = perceptual user

interface kinesics ?
The study of gestures and other nonverbal types of communication

WORKING

Input devices
Requirement: Have ability to track a person's movements and determine what gestures they are Performing.

Some Tools: Stereo cameras Controller-based gestures Single camera Depth-aware cameras

Gesture Typologies :
Arbitrary gestures

Mimetic gestures
deictic gestures

Gestural control devices:

1) Control of Memory and Display

Architecture for Gestural Control of Memory and Display

2) Local Control of Actuated Mechanisms.

3) Remote Control of Actuated Mechanisms:

Telerobotics

Hand Gesture Recognition:

Controlling the computer through hand movement, without touching a device

CASE STUDY: Gesture-based Remote control for TV

Haptics & Data gloves:

Data gloves provide very accurate measurements of hand-shape But are cumbersome to wear Expensive

continuous gesture:

A continuous gesture creates a trajectory in feature space

We can project a new image onto the trajectory

Facial Gesture Recognition

Anger

Fear

Disgust

Happy

Sad

Surprise

Example Case Study: Opera Face Gestures

Microsoft's XBOX & xbox 360:

Xbox is the first game console without a controller. The system has a 3D camera that maps the exacts position of your hands, your fingers, your feet, your header, your nose, everything in a 3D map. This allows you to control the game with only your body, in great detail, and no controller needed. Furthermore, it recognises voice and faces and supports complex video chat.
As of January 2011, there are over 50 million Xbox 360 consoles worldwide

Human Computer Interface using Gesture

Replace mouse and keyboard Pointing gestures Navigate in a virtual environment Pick up and manipulate virtual objects Interact with a 3D world No physical contact with computer Communicate at a distance

USES
Remote control Sign language recognition For socially assistive robotics Control through facial gestures--- Eye tracking Immersive game technology Virtual controllers Affective computing

Challenges
Problems

in image processing

accuracy and usefulness of gesture recognition software Issues with viability of the technology to general usage camera's resolution and quality

robust computer vision methods are also required Lighting, distance from the camera Gorilla arm problem
"Gorilla arm" was a side-effect that destroyed vertically-oriented touch-screens as a mainstream input technology despite a promising start in the early 1980s.

Conclusion:
Goal??
needs considerable improvement in image processing techniques technical challenges social challenges. many features but needs high speed processors can completely replace Mouse, Keyboard and all other conventional input devices

way to interact with 3D world.

We can navigate in a virtual environment