International Journal of Emerging Technology in Computer Science & Electronics (IJETCSE)

ISSN: 0976-1353 Volume 21 Issue 4 – APRIL 2016



SIGN DETECTION FOR DUMB PEOPLE

P. Arunagiri#1 , B. Hariharan*2, R. Ilammathiazhagan*3, Kiran Joy Pynadath*4, S. Narendhiran*5
#
A.P, Electronics & Communication Engineering, Sri Manakula Vinayagar Engg. college, Puducherry, India
*
B.Tech, Electronics & Communication Engineering, Sri Manakula Vinayagar Engg. college, Puducherry, India

realizing a natural interaction between humans and computers

Abstract— In this paper, a gesture based device is proposed
for dumb person to communicate with normal person. Generally
dumb people use sign language for communication, but they find
difficulty in communicating with others who don’t understand
sign language. So there is a barrier in communication between
these two communities. This work aims to lower this barrier in
communication. The main aim of the proposed project is to
develop a cost effective system which can give voice to voiceless
people with the help of Smart Gloves. With the proposed work,
American sign language is converted into text and speech using
flex sensor and TTS module. The flex sensor values are
converted to text by microcontroller and displayed in LCD. It
means that using smart gloves communication will not be a
barrier between two different communities.
Index Terms— American Sign Language, Flex Sensor, Glove,
LCD, Microcontroller and TTS module.
I. INTRODUCTION
Our environment is continuously evolving with new
computerized and electronic systems that are changing the
way we live and interact with others. Electronics become
smart, and sensors are everywhere, thereby enabling the
development of devices that can analyze, recognize and
interpret our actions. While most deaf individuals with
various degrees of deafness can lip-read and understand
someone speaking, they almost always communicate with
sign language. This language is usually unknown by normal
people. A typical situation is a person (student) who wants to
use sign language to give a presentation to other students who
do not understand the sign language. To enable this type of
communication, we have built an experimental system for the
automatic translation of sign language into text and speech.
II. LITERATURE SURVEY
In the recent years, there has been tremendous research on
the hand sign recognition. The technology of gesture
recognition is divided into two categories-
2.1 Vision Based
In vision-based methods, computer camera is the input
device for observing the information of hands or fingers [1].
The Vision Based methods require only a camera, thus
without the use of any extra devices. These systems tend to
complement biological vision by describing artificial vision
systems that are implemented in software and/or hardware.
This poses a challenging problem as these systems need to be
background invariant, lighting insensitive, person and camera
independent to achieve real time performance. Moreover,
such systems must be optimized to meet the requirements,
including accuracy and robustness.
Vision based analysis, is based on the way human beings
perceive information about their surroundings, yet it is
probably the most difficult to implement in a satisfactory way.
Several different approaches have been tested so far.
 One is to build a three-dimensional model of the human
hand. The model is matched to images of the hand by one
or more cameras, and parameters corresponding to palm
orientation and joint angles are estimated. These
parameters are then used to perform gesture
classification.
 Second one to capture the image using a camera then
extract some feature and those features are used as input
in a classification algorithm for classification.
2.2 Glove-based
In glove based systems, data gloves are used which can
archive the accurate positions of hand gestures as its positions
are directly measured. The Data-Glove based methods use
sensor devices for digitizing hand and finger motions into
multi-parametric data. The extra sensors make it easy to
collect hand configuration and movement [2]. However, the
devices are quite expensive and bring much cumbersome
experience to the users some of the earlier gesture recognition
systems attempted to identify gestures using glove-based
devices that would measure the position and joint angles of
the hand. However, these devices are very cumbersome and
usually have many cables connected to a computer. This has
brought forth the motivation of using non-intrusive,
vision-based approaches for recognizing gestures Also the
sensors used for the detection of the sign language and the
gesture recognition in the system that are available in the
market are quite costly [3]. In computer recognition of spoken
language, speech data is captured using a microphone
connected to an ADC.

112
 International Journal of Emerging Technology in Computer Science & Electronics (IJETCSE)
ISSN: 0976-1353 Volume 21 Issue 4 – APRIL 2016

Similarly a data-capturing device is also required in order

to recognize sign language, in this case measuring the position
and movement of the signer's hands.

2.3 Comparison between vision-based hand gesture

recognition and gloves- based hand gesture recognition
In case of vision- based gesture recognition systems, a lot
of the digital signal processing has to be done. Also there
require large programming. Because of this the response of
the system is quite slow. Also the electric, magnetic fields,
Fig.2. Variation of resistance with degree in bend
and any other disturbances may affect the performance of the
system. In case of glove-based sign language recognition The flex sensor is made of carbon element. The
system that is available in the market lot of the hardware is resistance value of flex sensor varies on bending. As the flex
required. Large numbers of cables have to be connected to the sensor is bended, the resistance rises up to 30- 40 kilo ohms at
computers for monitoring the data [4]. Hence the systems approximately 90 degrees. The dimensions of the sensor are
require lot of space. Also the system is not handy. 0.2 inches wide, 2.5 inches long and 0.019 inches thick.

III. BLOCK REPRESENTATION

Flex Sensor

Display Fig.3. Equivalent circuit of flex sensor

Unit
Arduino Uno The output voltage is unwavering based on the
Hand
Gestures equation Vin * R1 / (R1 + R2), where R1 is the other input
Speech
Synthesis resistor to the non-inverting terminal. Using the voltage
divider concept the output voltage is determined and it ranges
from 1.30V to 2.8V. When bending angle of flex sensor
increases, resistance also increases. And when resistance
Accelerometer Audio increases, the output voltage from flex sensor will decrease.
Sensor guidance
3.2. Accelerometer
system
Accelerometers are used for tilt sensing. They
measure both static and dynamic acceleration. ADXL103
measuring system has shown in Fig.4. The tip product of the
Fig.1. Block diagram of system measuring system is provided to 3rd module, which
incorporates pipeline structure of 2 ADC. There‟s a technical
issue at this stage of the project, that the analog output of the
Description of each block is given as follows:
measuring system as shown in Fig.4, which ranges from 1.5
3.1. Flex sensor volts to 3.5 volts. Additionally, a digital 8-bit output the
The Flex sensors are normally glued to the glove using systems, becomes terribly sensitive.
either a needle or a thread. It generally requires a 5V Input
power supply and Output between 0V and 5V, the resistivity
altering with the sensors degree of bend and the voltage
output changing consequently. The sensors are usually
connected to the microcontroller via three pin connectors
namely ground, live, and output. The device can turn on the
sensor from off mode, enabling it to power down when not in
use and greatly decreasing power consumption.

113
 International Journal of Emerging Technology in Computer Science & Electronics (IJETCSE)
ISSN: 0976-1353 Volume 21 Issue 4 – APRIL 2016
3.5. Text to speech module
The converter consists of text to speech module that allows
the device to translate the gesture. The Output text of Atmega
328 is converted to sound by TTS. The TTS module used is
Sunrom 1231. It is a 28 pin DIP, 8 bit microprocessor
Programmed with 600 letters to sound rules. The built-in
Algorithm allows real time translation of English ASCII
Characters to allophone addresses. This is used along with
Speaker to facilitate text to speech conversion. The technique
behind TTS is that it accepts serial Data ASCII characters and
translates to syllabic sounds. Sunrom 1231 generates an audio
signal using five sine-synthesis Generators for the allophones.
Fig.4. Block diagram of accelerometer sensor The voice output is through a speaker.

If a lot of sensitive system is employed then there's a large

modification within the digital output with the little tilt of the IV. RESULT AND ANALYSIS
script that is troublesome to do with. The system has Flex sensor for detecting the hand motions.
The sensor values are stored on the microcontroller unit.
3.3. Arduino Uno Based on the hand motions the stored outputs are displayed on
Arduino is an open source platform based on simple the LCD and also played through the speaker. Embedded C
microcontroller board. The controller used in the device has language is used to write the microcontroller program.
built atmega328 in it. Atmega328 has 32KB on-chip flash ARDUINO IDE software is used to develop the system. This
memory for storing codes of which 2KB used for boot loader. system detects the hand motions in few seconds and hence
It also includes a 2KB of SRAM and 1KB of EEPROM. The more accuracy.
program that is developed is to be stored on the flash memory
of the Controller. The Arduino software also includes a serial
Monitor, that allows data to be sent to or from the Arduino
board.
The Arduino Uno board receives input from flex sensor.
For each gesture, there exists a particular resistance with
respect to initial values. This comparison produces a
character. With the help of the program, character is
converted into ASCII codes. These codes are sent to the LCD
module, along with the character sent to the TTS module.

3.4. LCD
A LCD consists of two sheets of polarising material with a
liquid crystal solution between them and does not emit light.
The LCD panels are back-lit to make them easier to view in Fig.5. Smart Gloves and its assembled circuit
bright environments, provide a higher resolution, are slim in
construction and light in weight, and have a low power  Disabled use these gloves to convert sign performed by
consumption. A 16x2 LCD means it can display 16 characters them into text.
per line and there are 2 such lines. In this LCD each character
 From the convenience of simple flex sensors, a user is able
is displayed in 5x7 pixel matrix. This LCD has two registers,
to interact with others in more comfortable and easier
namely, Command and Data. The command register stores the
manner. This makes it possible for the user to not only
command instructions given to the LCD. A command is an
interact with their community, but with others also and
instruction given to LCD to do a predefined task like
they can also live a normal life.
initializing it, clearing its screen, setting the cursor position,
controlling display.

114
 International Journal of Emerging Technology in Computer Science & Electronics (IJETCSE)
ISSN: 0976-1353 Volume 21 Issue 4 – APRIL 2016
7. M. Kekre, and K. Vaishali, “Speaker identification by using
vector quantization,” IEEE Transactions on mechatronics, Vol.
15, No. 1, pp. 1034-1620, 2010.

8. H. Mark, Rebecca, O. Peter, “A voice-input voice-output

communication aid for people with severe speech impairment”,
IEEE Transactions on neural systems and rehabilitation
engineering, Vol. 21, No. 1, pp. 1534-4320,2013.

9. B. Oyaaran, “Vision based sign language recognition modelling

and recognizing signs with manual and non-manual
components”, Inter-national journal of computer science and
engineering, Vol. 12, No. Paper-back, 03 pp. 378-1695, 2008.

10. J. Ravikiran, M. Kavi, M. Suhas, R. Dheeraj, S. Sudheender, and

V. Nitin, “Finger detection for sign language recognition”,
Proceedings of the international multi conference of engineers
and computer scientists, Vol. 39, No. 12, pp. 334-620, 2009.

Fig.6. Text being displayed in response of Sign language.

 In Fig.6 when user made alphabet „E„ using sign language

wearing smart gloves, then with the help of flex sensors
and microcontroller it displayed using the LCD.

V. CONCLUSION
Smart Gloves is proposed to bridge the barrier of
communication between disabled person and normal person.
Sign language is the only medium for deaf and dumb persons
to share their feeling or thoughts with other but their
communication is restricted to other disabled person as
normal cannot understand what they wants to say. Using smart
gloves fitted with flex sensors helps them to convert sign
language in text and also given to speakers. With the help of
Smart gloves the gap or bridge between two different
communities can be filled up.

REFERENCES
1. Shraddha, K. Surendra, “A Communication System for Deaf and
Dumb People” IJSR Journal,Vol. 4, No.9, 2015.

2. Ruize Xu, Shengli Zhou and Wen J. Li, “Mems Accelerometer

Based Nonspecific-User Hand Gesture Recognition” IEEE
Sensors Journal, Vol. 12, No. 5, May 2012.

3. D. Gowri, D. Vidhubala, “Sign language Recognition for deaf

and dumb people” IJRET Sensors Journal, Vol. 3, No. 7, May
2014.

4. S. Amarjot, K. Devinder, S. Phani, K. Srikrishna, and Niraj, A.

“An intelligent multi-gesture spotting robot to assist persons with
disabilities”, International journal of computer theory and
engineering, Vol. 4, No. 6, pp. 1290-1420,2012.

5. F. Alois, R. Stefan, H. Clemens, and R. Martin. “Orientation

sensing for gesture-based interaction with smart artefacts”, IEEE
Transactions on audio, speech, and language processing, Vol.28,
No. 8, pp. 1434-1520, 2007.

6. C. Jean, and B. Peter, “Recognition of Arm Gestures Using

Multiple Orientation Sensors: Gesture Classification”, IEEE
Intelligent transportation systems conference on electronics, Vol.
13, No. 1, pp. 334-520,2004.

115