313 A-2

ANN-Based Handwritten Character Recognition

Huang Hanmin Huang Xiyue Zhang Ping Chai yi Shi Weiren
The institute of Automation, Chongqing University
Chongqing City, 400034,P.R.China

ABSTRACT
Based on arriticial neural network, digital image processing, and features extraction theory, authors analyzed
BP network's a<fect, h e n presented its improving solutions. In this paper, a new kind of handwritten character
system has be2n consrructed. Referring to shortcoming of traditional BP algorithm, a modified learning factor
with adaptation is introduced, and a bizarre sample feature database is constructed for speeding up modified BP
learning and clssification. Experimental shown that the modified BP neural network algorithm (three layers
forward, no fedback) can be used in handwritten character recognition, and satisfactory results have been
obtained.

KEYWORDS
Artificial Nezal Ken\vrk; Image processing. BP
Algorithm. Han5vrincn Character recognition
As to hidden layers, so far. there is no theoreiical
criterion on how to choose hidden layers riurilbei.
Improper hidden layers may result iffyhe fiilure
in whole network learning.

1. INTRODPCTIOS
Since 1940's. based on the theor) of investisating
human brain srrucxx. researchers have been
studying. simuixion c?f brain struc;ur!i::.i'i. In this
model: there 2 5 ~ 1u s ~ f i i lfeatures such as highly
parallel fundaztntal . excellent fault tolerance,
linking thinkii.; , ziaptation, self-learning. BP
network, as a represznrarive, has of good selflearning, p o ~ ~ f u discriminating
l
ability, fault
tolerance. and rcjustness. which caa be implemented
by non-liner fucxion s! stem from inpui io output.
In this paper.
f o c u on esploring modified BP
network systex. whic3 can be used in character
recognition, MzLily Emphzsizing on improving the
algorithm, feaillres extraction, samples learning
database.

As stated as above mentioned, it is necessai.? to

improving the BP algorithm for handirritten charezier
recognition. There have existed many soIuti,ms in
literatures.
( 1 ) Adding Momentum Item Method

An additional momentum item is added :o crizinzl

gradient (equals to the varying amount of last i t eigh:
value), for filtering out high frequeiit co;npona~tsin
the curve surface of function, in order to avoid !he
oscillation during learning process. The adjustment
formula:

2. Improvement of algorithm
In the pre\ious qplicationj, traditional EP
algorithm has some disadvantages as f d l o w :
Basically, B? algo+tmi can be consider as LkfS,
the algorith .use3 in multi-layers f--wc:d
network is non-liner optimization. It is ceitain
that the a l p i t h m s i s u local iiiiniiilization. If
the traditioml algorithm is used in complex
problems, caybe it will make the iearning
procedure fdure.
While leamkg, tht smziiei tiie erroi b:.comtl.s ,
especially z-riving minimi.im poirir. i h r iower
The convergrnt s p z d becomes, as :I< rcsul;. !he
efficiency a h becczw l ~ ~ v e r .

i.!;
/! is momen!iim coefficient and

,L[

<!

iteration.
Although this method can smooth 1carni::g !,I :)ct'ss.
unfortunately, at the same tiinc, !kc lilctl?i>d a:x ;:!so
bring
the learning procedxe enter in:o iocni
mininiuin area, which makes 5 1 learfiiiig ic!si iis

PR0001-3/99/0000-1177 $4.00

SICE '99 July 28-30, Morioka

- 1177 -

01999 SICE

sample subset, until to include all the samples. In

such case, we can improve the learning speed, the
key problem is choose the representative samples.
(2) Learning Factor kkthod with A d a p t a t i ~ n ~ ~ ~ . ~ ~ ~
(6) Skip Read
Learning factor 17 usually is called step. In the
During sample learning, some samples are
standard BP algorithm, normally, 1j7 is between 0 to 1.
approaching
the criteria which held a minimum error.
The large the learning factor r] becomes, the faster
In this case, if we continuously calculate forward and
the learning convergent speed is. But in this situation,
feedback parameters, it will be non-sense due to its
it will result in no convergence in learning; otherwise,
very
little contributions to network, even bringing
if q is set too small, it will result in increasing
disadvantage. In order to avoid this situation, it is
learning time. which makes learning speed very
necessary that the minimum error parameter must be
slowly. In this paper, an adaptive, changeable
set up. If the error produced by any sample is less
learning factor is presented for improving network
than the minimum error parameter, then all the
convergent feature.
feedback calculations in this sample will be
terminated.
a
Considering continuousness of network variation,
during next learning stage, it will neglect all of
=
forward calculation; instead, adding very little value
I
(3
to existing error values as new error value .
This procedure is called " skip read". Skip read
a is Control constant: n is number of iteration.
processing can continuous until the learning error
Changeable Iexning factor may have better learning
exceeds defined values, then modify the weisht
convergent fez:llres than fixed step learning factor.
values.
However, how to choose and adopt the change step is
In our applications, all of methods have been
exploring subject. depending on certain situations. In
adopted, especially for method. ( 3 ) , (4), (j), (61,
this paper, four 9 e s of modified BP algorithms are
which can speed up BP network learning and arrive a
presented as fo!iows
good convergence.
(3) Re-Initializxion
3. Feature Encode
Variations of :xitial \ alues \vi11 have great effect on
Inputs of network are chosen from N dimensiona;
learning con\-rgence. For threshold and weight
vectors, the network can classify these vectors after
values, normally, there has a tendency to choose
learning. Many inputs can be mapped directly from
smaller randor. data firstly, and to approach to a
character image pixels in the pre-literatures. For
minimum pokt secondly. While occurring no
example, for 32x24 pixels, it needs 768 input points
convergence or error out of requirement in the
and hidden layers and output points, which will take
learning process. it is not urgent to adjust learning
up about one thousand net points. Of course, all of
factor q or mor:-ntum coefficient ,g in stead, we can
the calculations are of huge , IOW efficiency .In this
reset initial v a i q beginning at new point to iterate.
paper, a feature encode is adopted.
Afier several =eking. it is possible to obtain the
The feature extraction and fezture encode actually
optimal learnins solution.
are feature compression, for example, as to width
feature, it will need 3 bits to encode, which represent
(4) Weight Valc-s Random Alteration
1-8 times width information referring to basic width.
Another example is cross point feature. Fig. I is cross
If network convergznt speed is very slow or
point feature encoding diagram.
network occurs oscillation during learning process.
possibly, it m a x :adient algorithm has entered
x I x I x I x
local minimum trap. In such circumstances, it will
adopt supervisor to interrupt the network by adapting
-.-<
10% random adjustment !o help network jum?ing out
the trap.
NO of Cross Point(2 bits) P O ~of. Cross Point( 16bits)
function. Adjusting the threshold value also brings
the similar consequence.

Cfi2(II)

I-..

(5) Hierarchy Laming

Fig. 1 Cross Point Feature Encoding

Usually, there is plenIy of redundant information in

the learning samples. Firstly, we choose sub class sei
to learn whick c m represem character in most
efficient way. After arriving discrimination ability in
some
degree . then we can further adopt larger

No. of Cross Point: Crossing strokes numbers,

such as beginning point, end point, three cross poitit,
four cross point in the character inizgc, gccerally
speaking, the No. of cross point is less than 4

1178 -

Fos. of Cross Point: describing cross point location

by usi!ig coorlinates. Detailed method is to divide
character image become 4x4 subareas, each area is
represented by 1 bit position as indicated as Fig 1.
when cross poin! occurs in any position in character
Image, it can De encoded by using one of 16 bits. For
three kinds of cross-points, it needs (1 6+2) x3=54
bits to 5ncode.

4. Bizarre Sample Feature Database

Although 'Ae adopted above solutions to speed up
network learning, and to get rid of local minimum
tiap. there stil! exists 1% of samples which do not
approach convergence. As to these samples, we call
them bizarre samples. These Samples are very
harmful to n-work learning that are not only
decreasing network learning speed, but also
damaging the weight values which had learnt from
c?:her samples. even \vi11 result in all system lose of
its conrroi.
We introduxd a bizarre samples database to
cvercol-ne abo:.e defect. By using bizarre feature
darabaje, these zanip1-s can br filtered out before the
samples are sent to network system. Detailed
sohtions are described as follows
While leaning several time, these samples still
have not arri\.ed con\.srgence. The samples will be
sent to dataixse and formed an Index. During
cnrrfing out rxognicion, if any character has same
features as Ind-s's in the database, the system will
directly check- the Index to get correct recognition,
instead of inptlt the feature into network. By using
this method, it \vi11 sa\.e plenty of time, and improve
recognizing acarac)..
In our imp!.-mentarion. among 1000 learning
samples, there only about IO samples. or only IO
records in the dxabas?. For every character, it needs
150 feature point-vectors, plus additional supervision
signals, totall? tvill take up 20 bytes. As to such a
database, index is a time-saving.

5. Application

( I ) Fig. 2 is student mark

Chongqing Univxsity. In
recognized is written-hand
numerals can be send to
automatically.

form used in the

this form, to be
characters. These
university database

Fig.2 a example of handwritten characters

recognition in students mark form
Example of Retrieval Form of Students Mark Jointed
Southwes! Computer Industry Ltd.. School of
Information. Chongqing University developed AI
handwritten character automatic entry and
recognition system, which based on neural network
recognition. This system has the function of form
automatic processing. handwritten character
recognition and data management.

i-ienry
David
Helen

i Reading I listening
I 7i 1
81

72

S2

writing
85

sj

grammar
90
91

1
87

1 Martin I

I Rafter 1

ilios

76
91

83
90

52

91

93
67
88

78
83
84

Fig. 3 Retrieval form from Fig2

The sample for learning is scanned to computer
through flat scanner: forming binary image, carrying
out document analysis, filtering out form skeleton
(horizontal lines and vertical lines), extracting convex,
concave, skeleton, cross point features. Encoding to
130 binary feature vectors. The feature vectors are
sent to network to learn, to recognize through
randomly sampling. The recognition accuracy can
arrive 99%, which had tested more than ten thousand
characters.
Fig. 3 is retrieval form from Fig. 2 , the recognized
characters can be sent to student database in
university administration center to accomplish data
entry, data statistics, form printing.

6. Conclusions
In this paper, the BP network used in optical
chatter recognition has been investigated, which has
following properties:
BP network learning effect is greatly affected by
how to choose the learning samples. As the set of

- 1179 -

network parameters, Authors proposed a modified BP

network learning criterion, as well as network
parameters. It is reasonable,for practical application.
(1) As to shortcoming of existed BP algorithm,
authors presented four improving solutions,
which have effects on BP algorithm.
( 2 ) Authors Constructed sample features database in
order to avoid network intempting, increase
network effectiveness
(3) Authors introduced form analysis and data
automatic entry system, which
makes
handwritten character into database become
possible.
(4) The experiments have shown that modified three
layers BP net work with above properties can
realize written-hand character recognition.

Reference
Huang Desuang, Neural Network and Pattern
Recognition theory, Electronic Press in China,
1996.
FU Jia zhong, Computerized
Character
Recognition Technique. Qisian Press in China.
1994
Jack S.N. Jean and Jin Wang LVeight Smoothing
to Improve Network Generalization IEEE Tians
On Neural Networks VoI.6 No.5: pp752-763,
Sept 1994
Dimitris A. Karras and Stavaros J. Perantonis A n
Efficient Constrained Training Algcrithm for
feedforword Networks. IEEE Trans On Neural
Networks Vo1.6 No.6 pp1420-1434 Nov. 1995s.
Ergenzinger and E.Thomsen. An Accelerated
Learning Algorithm for Multi-layer Perceptrons
Optimization Layer by Layer, IEEE Trans On
Neural Networks, Vo1.6. No. I , pp3 1-43,
Jan.1995
Guo Fanxian. Scanner principle and development
applications, QingHua University Press. 1996

- 1180 -