Real time static and dynamic hand gesture recognition system

By

Anam Arif
Munazza Javed
Malik Ibrahim
M Shabaz Swati

Under the supervision of

Dr Mushtaq Ali

A report submitted in partial fulfillment of the

requirements for the degree of
Bachelors of Science in Computer Science

Department of Information Technology

Hazara University, Mansehra
2018

i
 Dedication
This thesis is dedicated to our

Dear Parents

And

Respectable Teachers

ii
 CERTIFICATION

It is certified that the thesis

Entitled: ………………………………………………………………….

Submitted by: ……………………………………………………………

For the award of degree of Bachelors of Science in

Computer Science

of Hazara University Mansehra, is a bonafide account of the work carried out by

him/her in this department under our supervision.

Supervisor:

Head of Department:

External examiner:

iii
 ACKNOWLEDGMENT

We are thankful to Almighty ALLAH who is Merciful, Beneficent and Gracious, who has
provided the opportunity for leading the project on this path and gave the courage and strength to
complete this hard work. There has been imbuement of many people towards the completion of
this research. We forward the sincere apologies to all those whom we may not succeed to
mention.

We feel highly privileged in taking the opportunity to thanks our supervisor

Dr.Mushtaq Ali, whose wisdom and guidance made it possible for us to accomplish the project
successfully.

We also extend our thanks to Dr. Noor-ul-Amin, Head of Department of IT, for his co-
operating, warm and helping attitude towards studies.

Our deepest appreciation goes to Mam Sumera Sattar for her additional support in each
and every step of our project.

We are deeply thankful to our Dear parents, their prayers became the stepping-stones
towards the completion of our project.

Anam Arif

Malik Ibrahim

Munazza Javed

M Shabaz Swati

iv
 CONTENTS

1. INTRODUCTION 1
1.1 Historical development of hand gesture recognition 1
1.1.1 History of data glove 2
1.1.2 Vision based hand gesture recognition 6
1.2 Related work in gesture recognition 6
1.3 Objective of the project 7
1.3.1 Introduction of the project 7
1.3.2 Primary objective 7
1.3.3 Secondary objective 7
1.4 Limitations of the project 7
2. LITERATURE REVIEW 8
2.1 Introduction 8
2.2 Gesture definition 9
2.3 Hand gestures 9
2.3.1 Static gestures 9
2.3.2 Dynamic gestures 10
2.4 The basics of gesture recognition 10
2.5 Applications of hand gesture recognition 11
3. IMAGE PROCESSING AND NEURAL NETWORK 15
3.1 Image 15
3.2 Image creation 15
3.3 Image sensors 16
3.4 Image processing 16
3.5 Proposed combinational features for classification 20
3.5.1 structural features based on region properties 20
3.6 Artificial neural networks 21
3.7 Neural networks architectures 22
3.7.1 Single layer neural network 22
3.7.2 Multilayer neural network 22
3.8 Further classification on the basis of learning paradigms 23
3.8.1 learning paradigms 23
3.8.1.1 Supervised learning 23
3.8.1.2 Unsupervised learning 24
3.9 Advantages of neural computing 24
3.10 Summary 24

v
4. DOMAIN METHODOLOGIES 26
4.1 Digital image processing 26
4.1.1 Applications 26
4.2 Matlab 26
4.2.1 Variables 27
4.2.2 Functions and scripts 27
4.2.3 Matlab operators 27
4.2.4 Graphics 30
4.2.5 Programming constructs 30
4.2.6 Global variables 31
4.2.7 Debugging 31
4.2.8 Other features 31
4.3 The Matlab language 32
4.4 Publishing results 34
4.5 Deploying applications 34
5. SYSTEM OVERVIEW 35
5.1 The system flow 35
6. EXISTING SYSTEMS 48
6.1 Vision based solutions 48
6.2 Sensor based solutions 48
6.3 Comparison and analysis 49
7. GRAPHICAL USER INTERFACE 51
8. CONCLUSION AND FUTURE ASPECTS 57
9. REFERENCES 58

vi
 LIST OF FIGURES

1.1 Artistic impression of a sensor glove that places sensors on finger joints 2
1.2 MIT Acceleglove with its multiple sensors 3
1.3 Cyber Glove III for motion capture 3
1.4 CyberGlove II 4
1.5 5DT Motion Capture Glove and Sensor Glove Ultra.(Left) Current version,
(right) early version 4
1.6 X-IST Data Glove 5
1.7 P5 glove 5
2.1 Vitarka Mudra (Rose, 1919) 8
2.2 Hand gestures 10
2.3 A generic gesture recognition system 11
2.4 A Gestural Interface to Virtual Environments (O'Hagan, Zelinsky, &
Rougeaux, 2002) 12
2.5 The ASL sign language 13
2.6 Menu Items are displayed in a pie shape; the thumb is extended to Switch
from Draw Mode to Menu Mode [37] 13
2.7 Human –Robot Interaction [40] 14
3.1 Single layer neural network 22
3.2 Multilayer neural network 23
5.1 Hand segmentation 36
5.2 K curvature method 38
5.3 The flow of k curvature algorithm 39
5.4 Flow chart of static hand gesture recognition system 41
5.5 Flow chart of dynamic hand gesture recognition system 43, 44
5.6 Flow chart of virtual mouse 46
7.1 The guide tool window 52

vii
 LIST OF TABLES

4.1 Matrix operators 28

4.2 Matrix functions 28, 29
4.3 Math functions 29, 30
4.4 Programming constructs 30, 31
6.1 Comparison and analysis of existing hand gesture recognition system 49

viii
 Abstract

Real-time, static and dynamic hand gesture recognition affords users the ability to interact
with computers in more natural and intuitive ways. The gesture can be used to communicate
much more information by itself compared to computer mice, joysticks, etc. allowing a greater
number of possibilities for computer interaction.
The purpose of this work was to design, develop and study a practical framework for real-time
gesture recognition that can be used in a variety of human-computer interaction applications.
The system include the processes of image acquisition, hand segmentation, feature of
hand extraction, gesture recognition etc.

ix