SWE1011-SOFT COMPUTING

TAMIL HANDWRITTEN CHARACTER RECOGNITION USING

CONVOLUTIONAL NEURAL NETWORK

PROJECT REPORT

M.TECH SOFTWARE ENGINEERING (INTEGRATED 5 YEARS)

T.SHARMILA-15MIS0099
M.KRISHNAVENI - 15MIS0339
A.VISHNU PRIYA-15MIS0347

SUBMITTED TO
INSTRUCTOR - Prof. SENTHIL KUMAR
SCHOOL OF INFORMATION TECHNOLOGY AND ENGINEERING (SITE)

1
 TABLE OF CONTENTS

1. Title………………………………………………………………….……………………3

2. Abstract…………………………………………………………………………………..3

3. Introduction……………………………………………………………………………...3

4. Related Works…………………………….……………………………………..………4

5. Architecture………………….…………………………………………………………..6

6. Convolutional neural network………………….………………….……………..…….7

7. Methodology….……………………………….…………………………………...…….8

8. Result and Discussion…………………………………………………………………...9

9. Conclusion……………………………………………………………………………....12

10. References………………………………………………………………………………13

2
 TAMIL HANDWRITTEN CHARCTER
RECOGNITION USING CONVOLUTIONAL
NEURAL NETWORK

ABSTRACT:
The process which involves the conversion of characters that are handwritten into digital or
printed characters is known as handwritten character recognition. This paper provides a
unique and sophisticated approach that extracts and recognizes the text that is handwritten in
Tamil language. The whole system is being implemented in MATLAB tool and a GUI
(Graphical user interface) is created for the same. This system uses Convolutional neural
networks which is one of the widely recognized techniques for the classification of inputs
derived from images. Once the image that’s tested is uploaded, it is preprocessed and the
characters are extracted and after classification, the printed or digital format of the text is
displayed in the GUI that is exclusively created for implementing this system. This
symbolizes the process that the handwritten characters are recognized by the software very
well and thus resulting in the achievement of our objective. In this paper, the various aspects
that comprises of and the methodology related to the handwritten character recognition will
be discussed in detail in the forthcoming topics.

INTRODUCTION:
The Tamil language is one of the most primitive and vast language in south India particulary
in Tamil nadu and the other surrounding regions. Tamil language is also one of the 22 official
languages of India. It is also provided for the students of Tamil nadu as a medium of
education. Moreover, Tamil language is the only language that satisfies the 11 criteria
specified for a language to be declared as a classical language. Hence it became the only
classical language among the other Indian languages recognized by UNESCO. The Tamil
script consists of 31 independent letters and 216 letters that are combinant additionally which
sums up to 247 combinations in total that can be used to write in this language.
Handwritten character recognition has been known to be a challenging area for the
researchers. The degree of difficulty for recognizing characters in each language varies
drastically because of the variations in writing style, format, orientation of letters etc. This is
particularly difficult in Tamil language because it contains a lot of loops, curves and circles
or holes. These issues combine to form a major challenge for the researchers to use this
language. Hence only a very little amount of research work is done in the recognition of
Tamil characters or scripts when compared to other languages.
We have used Matlab here as it contains the image processing toolbox that greatly provide
the necessary tools for us implement this system. The first step in our system is acquiring the
images by writing the Tamil characters and taking a picture of that and store them in jpeg
format for further processing and the process is referred as image acquisition. The second
step is the preprocessing in which the image is taken and resized and converted from rgb

3
image to grey image which is then subjected to binarization where the grey scale image is
converted to binary image by using the threshold process. Then the characters are extracted
separately which then undergoes classification using the convolution neural networks and the
characters that are recognized are displayed in the interface created for the users to access the
system in MATLAB. These topics will be discussed in detail as you proceed further.

RELATED WORKS:
K.Verma et.al.[1] have proposed a system that uses convolutional neural networks to
recognize the characters in hindi which is a language that is considered official to be spoken
in India. The dataset is collected for the purpose and the characters are preprocessed and
extracted followed by the application of the convolutional neural network. The CNN is
applied using two models such as Model A and Model B using two kinds of optimizers such
as SGD optimizer and Adam optimizer applied over different category of experiments. Then
the loss and accuracy values are recorded and the result is found to that the models work best
under Adam optimizer than the SGD optimizer. The influential parameters turned out to be
dropout, beta value, learning and the size of the filter.
K.Duraisamy et.al.[2] have proposed a system that uses Support Vector Machine to
recognize the characters written in tamil language. The dataset is gathered here by making
different people to write tamil letters on A4 sheets which is then scanned and preprocessed to
increase the quality of the image. The features are then extracted based on the densities of the
pixels which is calculated for 64 distinct zones.This is calculated for both the testing and
training data. The training data is trained using the SVM and then tested to record the
accuracy of the recognition of tamil characters which is found to be 82.04%. The paper
promises that the accuracy can further be improved when combining the multiple classifiers.
J.Venkatesh et.al.[3] have proposed a system that utilizes Kohonon’s self-organizing
maps(SOM) to recognize the characters that are handwritten in tamil language. The dataset is
collected which undergoes scanning, preprocessing, segmentation, extraction of features
followed by classification using SOM. The learning algorithm is efficiently applied to the
training data to train and then it is subjected to testing. Then a mapping called Unicode
mapping is done where the characters that are recognized are mapped with the corresponding
characters in the Unicode table. Then the respective Unicode fonts are displayed in the
picture box.
F.F.Neural et.al.[4] compares the process of recognition of handwritten tamil characters using
Wavelet transform and Zernike moments. They have implemented a system which uses 100
samples of the dataset collected from different people. The dataset is preprocessed and the
noise is reduced to improve the image quality. Then the features are extracted by two
methods such as Wavelet transforms and Zernike moments. Then the classification of the
characters in tamil is done by using the feed forward neural network. The performance and
accuracy of the Wavelet transforms and Zernike moments are compared and due to the image
normalization, computing becomes difficult in Zernike moments.
C. Boufenr et.al[5] they proposed a deep learning Convolution Neural Network (CNN) for
Arabic hand written character recognition. We have three different way of Convolution
Neural Network they are training the CNN, transferring learning strategies to before trained

4
datasets, and final one is fine tuned the weight of CNN. In that they have used a transfer
learning strategies for a trained model of two largest datasets. The name of the two large
datasets are OIHACDB and AHCD. They train and test the dataset using CNN the first layer
of CNN is convolution layer, then max pooling layer, then fully connected layer, and then
final layer is soft-max layer. We have different CNNs in that they proposed the AlexNet
CNN. In OIHACDB dataset they have 30000 character images as an instance for 40 classes
while experiment those images they got 99.33% accuracy. Where as in another dataset they
have 16800 character images for 28 different classes in that they gave 13440 images for
training and remaining for testing while experiment they combine the first 28 character
images of first dataset with the new dataset data set and test the images by doing this they got
99.98% accuracy. By analysing this we get to know that if we give more dataset for training
you will get more accuracy.
J. Gu et.al [6] in this paper they have explained an advance layers, features in each layer of
Convolution Neural Network. In that they deeply explain about the advance components of
Convolution Neural Network. The three components in CNN are convolution layer, pooling
layer and activation function. And also they explained about the different aspects in CNN.
They were loss function, optimization, fast processing, application, and regularization. And
also the different type of CNN architectures was explained they are AlexNet, GoogleNet,
ResNet and VGGNet. By concluding this they said that while do the implementation using
convolution neural network requires more human effect to create dataset. While
implementing you have to provide more training dataset then only the accuracy will be more.
Tong He et.al [7] here they proposed image classification for text component and not text
components using Convolution Neural Network. To extract the text region in a particular
image they proposed a novel text attentional convolution neural network. To get better result
they trained the CNN using multiple datasets, binary text, non-text component, text region
mask and character label for classification. This supervised training will provide you to
predict the ambiguous image and text this will increase the robustness against the background
pixels. And they used a Maximally Stable External Region (MSER) method to predict the
text in the image which has complicated backgrounds that is real world images. Examples are
text image in banners, photos, traffic control boards etc. By applied this method they got 57%
accuracy. Because here the real world image has complicated background that why it is hard
to detect the text from the image.
Dr. P. Banumathi et. al [8] in this paper the authors discussed Tamil handwritten character
recognition using gradient feature and multi – layer neural network. In this paper, the input
image scanned and pre-processed done by removing noise, brightened and binarized. And
the image is decomposed into individual characters in segmentation phase and the 21 features
are extracted using character geometry method and gradient features method. Finally the
features are feed to multi-layer perception in the classification phase and the character was
predicted.
Surya Nath R S et. al [9] in this paper they conducted the review in the field of handwritten
character recognition and various techniques were used in each steps. For pre-processing we
can use binarization, noise removal, skeletonization and normalization. In segmentation
phase the word image spilt into individual characters and the features are extracted through
various techniques such as Contourlet transform, Gabor filter, Direactional Features, Gradient

5
and courvature and etc.,. The final phase is classification where the test characters are
classified and for classification the following methods can be used, SVM (Support Vector
Machine), Multilayer perceptron, Decision tree, Genetic algorithm and etc.,. Form these
techniques, contourlet transform and feed forward neural network gives maximum accuracy
rate 97.3 %. And it says that the accuracy is based on feature extraction and classification.
Tiji M jose et. al [10] in this paper the authors has proposed back-propagation neural network
for classifying the handwritten character which is given as test image. In pre-processing the
image converted into gray image, binarized the image and thinning image. Using wavelet
decomposition method, the features are extracted with different wavelet functionalities such
as ‘db1’ and ‘db2’, in feature extraction phase. And three layer feed forward back
propagation neural network is used to classify and recognition the test image with sigmoid
activation function. For different wavelet families they got different accuracy. From this
process they have obtained maximum accuracy rate as 89% in level 4 db2.

ARCHITECTURE
In this section, we described about architecture of the proposed system. Tamil handwritten
character recognition consist scanning image, pre-processing, segmentation, classification
and recognition of the character. Fig 1, is the block diagram for Tamil handwritten character
recognition architecture.
Scanning Image:
In this phase, the scanned image is selected for testing and the image will be classified in the
final phase. The scanned image may be a document or paragraph or sentence or word or
character. The image can be JPG, PNG or BMP and some picture format.
Pre-processing:
The next phase is pre-processing. The scanned image may not be a perfect image for
classification or feature extraction. It may contain some noise which cause wrong prediction
in future. To prevent this problem we need to pre-process the image. So, first the character
in the image is darkened by increasing the pixel values which value is less than 200. And
then binarize the image for next phase.
Segmentation:
Segmentation is the important step in image processing. In Tamil handwritten character
recognition, we need to extract each character separately for classification because the image
may be paragraph or sentence. Based on the area or size of the character and connected lines
of the character, we extract the characters separately. And the characters are resized into 50 x
50 images for classification.
Classification:
In classification phase, the segmented characters are classified into different classes. We
used Convolutional Neural Network (CNN) for classification. The CNN contain four steps
which are Convolution layer, ReLU layer (activation function), Pooling layer and fully
connected layer. The extracted characters went through these four stages and the values will

6
be stored in the network in training time. While testing, the test image also went through
these stages and the final value is compare with training value and testing values and the class
label is predicted.

CONVOLUTIONAL NEURAL NETWORK

Convolutional neural network is deep learning method which is used to classification and
prediction in machine learning fields. This is used for mostly in images related works. It is
help to produce the fully connected network when processing with image. Fig. 1 is the basic
block diagram of convolutional neural network.
i. Convolutional layer

In this layer, first select the size of the feature matrix, number of features matrix and input
image. Multiply the feature matrix with each image pixels and add the value. Take the
average of the sum. Apply this process to the image for number of feature matrix. Based on
number of feature matrix the accuracy may vary.
ii. ReLU layer

ReLU layer is the activation function layer which is used to remove the negative value in the
image. If the pixel value is less than zero then the pixel value converted into zero. If the
pixel value is greater than or equal to zero then the pixel value remain same. Apply this
activation function to every feature matrix output.

 0 if x  0
f ( x)  
 x if x  0
iii. Pooling layer

Pooling layer is used to shrink the image. Mostly 2 x 2 or 3 x 3 stride or matrix is used in
pooling layer. First select the stride size to shrink. Take the maximum value in 2 stride keep
it in new matric then move the stride throughout the feature matric. These three layers will
process iteratively until the feature matric is reduced to 2 x 2 matric. Once the feature matric
is obtain the size 2 x 2 move to fully connected layer.
iv. Fully connected layer

In fully connected layer take the 2 x 2 matric and list the pixel value in the stack of array.
This stack will act as neural network for the convolutional neural network classification. This
stack is used to predict the class label of the test images.
v. Soft-max layer

In soft-max layer, the prediction will be done. The test image will pass through all the layer
of convolutional neural network and produce output as fully connected layer or stack. Soft-
max layer will sum up all the stack value of training image and the testing image. Finally it
will compare testing image value with training image values. If any training value is similar
to testing value then it will predict the class label accordingly.

7
METHODOLOGY
Training Phase:
In training phase, Tamil handwritten character dataset loaded into the convolutional neural
network for training the network. Tamil character recognition is done by MATLAB
software. MATLAB is easy to work and interact. And we used image processing toolbox
and deep learning toolbox for this project. The dataset contain 124 classes which are the
main character of Tamil language and each class contain 15 images, total 1,860 images.
These images are pre-processed and stored as individual characters based on the class name.
Dataset loaded into the MATLAB using imageDatastore method. And dataset are given to
CNN to train a network. CNN contain four layers which are discussed above. The image
size is 50 x 50 matrices. Filter size is 5 x 5 matrices and number of filter is 256. In pooling
layer we used 2 x 2 matrices to shrink the image. Image shrink process are done iteratively
up to we got 2 x 2 matrices. Once the shrink is done then the image pixels are converted into
single array. Once the training is done, we can use the network for testing.
Testing phase:
The image loaded in to the MATLAB for testing. First step, remove the noise in the picture
by removing noise pixels. And binarize the image for further process.
The image may be a sentence or word, so segment the image into individual character by
separating character based on connected lines and size of the character. And resize the
separated character as 50 x 50 matrices for classification.
Test images are loaded for classification and prediction. Images converted to 2 x 2 matrices
using iterative process of convolutional layer, ReLU layer and pooling layers. Once the
image converted to 2 x 2 matrices then the image matrix convert to one dimensional array in
fully connected layer. In soft-max layer, testing value is compared with training value and
the class label is predicted. Based on the label value the Tamil character will be printed in
the output.

Fig. 1 Block diagram for convolutional neural network

8
 Fig. 3 Architecture

Fig. 2 Flow chart for Tamil handwritten character

recognition

RESULT AND DISCUSSION

Fig. 4 shows the GUI of Tamil handwritten character recognition. Frist we need to train the
network for classification. The training process may take several minutes to train the
network.
Once the training is finished we need to select the test image. Fig. 5 show the test image
which selected for testing. The word is written in paper and scanned for testing. This image
may contain some noise, so, pre-process the image using noise removing technique and
binarizing the image and the image is shown in Fig.6.
After pre-processing the scanned image is segmented in to individual characters for
classification. The extracted character is shown in Fig. 7. The extracted characters are
classified by convolutional neural network which is already trained in the earlier stage.
Based on the classification result or label the digital characters are printed in the output box
which is shown in Fig. 9.
From this we can say that, the CNN accuracy will increase if we increase the training data.

9
Fig. 4 GUI for Tamil handwritten character recognition

Fig. 5 Select scanned image for testing

10
 Fig. 6 Pre-processing the test image

Fig. 7 Extract individual characters from test image

11
 Fig. 8 predicted output for test image

CONCLUSION:
Although this system is implemented in other algorithms and technologies, in this paper
we have provided a simple yet nobel approach using the convolutional neural networks to
implement this system explaining the underlying process to reach our objective. This system
requires less cost and maintenance to implement. It has showed great results and proved to be
efficient. In future, we can increase the dataset of images and can improve the performance
and accuracy to near 100%.

12
REFERENCES:

[1] K. Verma and M. Singh, “Hindi Handwritten Character Recognition using

Convolutional
Neural Network,” no. 6, pp. 909–914, 2018.

[2] N. S. Æ. K. Duraiswamy, “A novel SVM-based handwritten Tamil character

recognition system,” pp. 173–180, 2010.

[3] J. Venkatesh and C. Sureshkumar, “Tamil Handwritten Character Recognition Using

Kohonon’s Self Organizing Map,” vol. 9, no. 12, pp. 156–161, 2009.

[4] F. F. Neural, F. Forward, and E. Approach, “A Comparative Study for Handwritten

Tamil Character Recognition using Wavelet Transform and Zernike Moments,” vol. 2,
no. 4, 2014.

[5] Chaouki Boufenar, Adlen Kerboua, Mohamed Batouche, “ Investigation on deep

learning for off-line handwritten Arabic character recognition ”, Science Direct, 13
November 2017.

[6] Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai,
Ting Liu, Xingxing Wang, Gang Wang, Jianfei Cai, Tsuhan Chen, “ Recent advances
in convolution neural network ”, Science Direct, 2018.

[7] Tong He, Weilin Huang, Member, IEEE, Yu Qiao, Senior Member, IEEE, and Jian
Yao, Senior Member, IEEE, “ Text-Attentinal Convolution Neural Network for Scene
Text Detection “, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 25,
JUNE 2016.

[8] Dr.P.Banumathi, Mr. M.Ashraf, Mr. K.Gokul Kumar and Mr. G.Ragul, “ TAMIL
HANDWRITTEN CHARACTER RECOGNITION BASED ON GRADIENT
FEATURES “, International Journal of Recent Innovation in Engineering and
Research, 2017.

[9] Surya Nath R S, Afseena S, “ Handwritten Character Recognition – A Review “,

International Journal of Scientific and Research Publication, Volume 5, Issue 3, March
2015.

[10] Tiji M Jose, Amitabh Wahi, “ Recognition of Tamil Handwritten Character using
Daubechies Wavelet Transforms and Feed – Forward Backpropagation Network “,
International Journal of Computer Application, Volume 64, February 2013.

13