Int J Speech Technol (2006) 9: 133150
DOI 10.1007/s10772-008-9009-1
Arabic speech recognition using SPHINX engine
Hussein Hyassat Raed Abu Zitar
Received: 1 October 2008 / Accepted: 9 October 2008 / Published online: 28 October 2008
Springer Science+Business Media, LLC 2008
Abstract Although the Arab world has an estimated
number of 250 million Arabic speakers, there has
been little research on Arabic speech recognition when
compared to other languages of similar importance
(e.g. Mandarin). Due to the lack of diacritic Ara-
bic text and the lack of Pronunciation Dictionary
(PD), most of previous work on Arabic Automatic
Speech Recognition has been concentrated on devel-
oping recognizers using Romanized characters i.e. let
the system recognizes the Arabic word as an English
one, then map it to Arabic word from lookup table that
maps the Arabic word to its Romanized pronunciation.
In this work, we introduce the first SPHINX-
IV-based Arabic recognizer and propose an auto-
matic toolkit, which is capable of producing (PD)
for both Holly Quraan and standard Arabic lan-
guage. Three corpuses are completely developed in
this work, namely the Holly Quraan Corpus HQC-1
about 18.5 hours, the command and control corpus
CAC-1 about 1.5 hours and Arabic digits corpus ADC
less than one hour of speech. The building process is
H. Hyassat
Arab Academy of Business and Financial Sciences,
Amman, Jordan
R. Abu Zitar ()
School of Computing and Engineering, New York Institute
of Technology, Amman, Jordan
e-mail: rzitar@nyit.edu
completely described. Fully diacritic Arabic transcrip-
tions, for all the three corpuses were developed too.
SPHINX-IV engine was customized and trained,
for both the language model and the lexicon modules
shown in the frame work architecture block diagram
on next page.
Using the three mentioned corpuses; the (PD) de-
veloped by our automatic tool with the transcripts,
SPHINX-IV engine is trained and tuned in order to
develop three acoustic models, one for each corpus.
Training is based on an HMM model that is built on
statistical information and random variables distribu-
tions extracted from the training data itself. New algo-
rithm is proposed to add unlabeled data to the training
corpus in order to increase the corpus size. This algo-
rithm is based on Neural Network confidence scorer
and then is used to annotate the decoded speech in or-
der to decide whether the proposed transcript is ac-
cepted and can be added to the seed corpus or not.
The model parameters were fine-tuned using simu-
lated annealing algorithm; optimum values were tested
and reported. Our major contribution is mainly using
the open source SPHINX-IV model in Arabic speech
recognition by building our own language and acoustic
models without Romanization for the Arabic speech.
The system is fine-tuned and data are refined for train-
ing and validation. Optimum values for number of
Gaussian mixtures distributions and number of states
in HMMs have been found according to specified per-
formance measures. Optimum values for confidence
134
scores were found for the training data. Although
much more work need to be done to complete the
work with this size, we consider the corpus used in our
system is enough to validate our approach. SPHINX
has never been used before in this manner for Arabic
speech recognition. The work is an invitation for all
open source speech recognition developers and groups
to take over and capitalize on what we have started.
Keywords SPHINX engine Pronunciation
Dictionary Diacritic Arabic
1 Introduction
Large Vocabulary Continuous Speech Recognizers
(LVCSR) are commercially available from different
vendors. Along with this increased availability comes
the demand for recognizers in many different lan-
guages that often were not focused on the speech
recognition research. So far, Arabic language is one
of these languages. With the increasing role of com-
puters in our life, there is a desire to communicate
with them naturally. Speech processing by computer
provides one vehicle for natural communication be-
tween man and machine. Interactive networks provide
easy access to a wealth of information and services
that will fundamentally affect how people work, play
and conduct their daily affairs.
The average citizen needs to communicate with
these networks using natural communication skills us-
ing everyday devices, such as telephones either mo-
bile or fixed and televisions. Without fundamental ad-
vances in user-centered interfaces, a large portion of
society will be prevented from participating in the info
Int J Speech Technol (2006) 9: 133150
era, resulting in further stratification of society and
tragic loss in human potential. Automatic Speech
Recognition (ASR) field is one of these interfaces,
which witnesses over the last decade enormous
progress, and can be reliably done on large vocab-
ularies, on continuous speech and speaker indepen-
dently. The word error rate of these recognizers under
special conditions often is below 10 percent (Pallet
et al. 1999) and for general purposes Large Vocabulary
Continuous Speech Recognizers (LVCSR) the best
word error rates were as high as 23.9% (Rosti 2004;
Hain et al. 2003) for English language.
With an estimated number of 250 million native
speakers, Arabic language is the sixth most widely
spoken language in the world. But research on ASR
for Arabic is too limited compared to other languages
with similar importance like mandarin (Kirchhoff et al.
2002).
Most of previous work on Arabic ASR aims at
developing recognizer, for either Modern Standard
Arabic (MSA) or Egyptian Colloquial Arabic (ECA).
Some results of Word Error Rates (WER) obtained
from both MSA and ECA are shown in Table 1.
From Table 1 we see that the performance of the
Arabic ASR for ECA are very poor compared to other
ASR for other languages like English, this result is an-
other motivation for this research.
Most previous works on Arabic ASR was done
by training the system using two formats, either the
Romanized format or standard Arabic script with-
out Romanized transcript. Arabic ASR are concen-
trated on developing recognizers for Modern Standard
Arabic (MSA), which is a formal linguistic standard
used throughout the Arabic-speaking world and is em-
ployed in the media (e.g. broadcast news), lectures,
courtrooms, etc. or colloquial Arabic Language.
Int J Speech Technol (2006) 9: 133150
Table 1 WER (%) Arabic language type Year
obtained for both MSA and
ECA
MSA 1997
ECA 96/97
ECA 2002
SPHINX-IV engine will be customized in this re-
search. SPHINX-IV is an open source speech recog-
nition engine built for research purposes by speech re-
search group at Carnegie Mellon University (CMU)
(CMU SPHINX Open Source Speech Recognition En-
gines 2007; Huang et al. 2003). Many theses around
the world tackled SPHINX for Speech Recognition
(Rosti 2004; Nedel 2004; Doh 2000; Ohshima 1993;
Raj 2000; Huerta 2000; Rozzi 1991; Liu 1994; Gou-
va 1996; Seltzer 2000; Siegler 1999), but not for
Arabic. Reasons for selecting this engine will be pre-
sented latter. SPHINX-IV architecture consists of se-
ries of processes independent of each other as it will be
shown in next sections. Each block in this figure rep-
resents a series of process, independent of each other.
2 Review of speech recognition engines
In this section some of the famous speech recogni-
tion engines are reviewed, SPHINX, Hidden Markov
Toolkit (HTk) and Center for Spoken Language Un-
derstanding Toolkit (CSLU).
2.1 SPHINX engine
SPHINX is a large-vocabulary, speaker-independent,
Hidden Markov Model (HMM)-based continuous
speech recognition system. SPHINX was developed
at CMU in 1988 (Russell et al. 1995; Christensen
1996; Rabiner and Juang 1993) and was one of the
first systems to demonstrate the feasibility of accu-
rate, speaker-independent, large-vocabulary continu-
ous speech recognition. SPHINX-II (Russell et al.
1995) was one of the first systems to employ semi-
continuous HMMs.
SPHINX is a collection of several ASR; it was
created in collaboration between the SPHINX group
at CMU, Sun Microsystems Laboratories, Mitsubishi
Electric Research Labs (MERL), and Hewlett Packard
(HP), with contributions from the University of Cal-
ifornia at Santa Cruz (UCSC) and the Massachusetts
135
Word error rate (WER) Reference
1520% Billa et al. (2002a, 2002b)
6156% Kirchhoff et al. (2002), Zavagliakos
et al. (1998)
55.154.9% Kirchhoff et al. (2002)
Institute of Technology (MIT). The current working
engines of SPHINX are SPHINX I, II, III, IV and
pocket SPHINX, in addition to these engines SPHINX
has one trainer, this trainer is capable to produce
acoustic model, this model can be used in all SPHINX
version except SPHINX-I, every SPHINX engine has
its unique characteristics and usage.
2.2 Hidden Markov Model Toolkit (HTK)
S. Young, presented a frame work for the HTK TOOL
KIT (Hermansky 1990) he stated that The Hidden
Markov Model Toolkit (HTK) is a portable toolkit
for building and manipulating hidden Markov mod-
els. HTK is primarily used for speech recognition re-
search although it has been used for numerous other
applications. It was originally developed at the Ma-
chine Intelligence Laboratory (formerly known as the
Speech Vision and Robotics Group) of the Cambridge
University Engineering Department (CUED) where it
has been used to build large vocabulary speech recog-
nition systems. It consists of a set of library modules
and a set of more than 20 tools. A HTK-based recog-
nizer was included in both the ARPA September 1992
Resource Management Evaluation and the November
1993 Wall Street Journal CSR Evaluation, where in
both cases performance was comparable with the sys-
tems developed by the main ARPA contractors.
In the year 1999, the current version of HTK was
V2.2 and all rights to HTK rested with Entropic. At
this time Entropics major business focus was voice-
enabling the Web and Microsoft purchased Entropic in
November 1999. Recently Microsoft decided to make
the core HTK toolkit available again and licensed the
software back to researchers and academic usage, so
that it could distribute and develop the software for
these purposes.
2.3 Hybrid systems
A. Ganapathiraju et al. described the use of a powerful
machine learning scheme, Support Vector Machines
136
(SVM) (Lee et al. 1990), within the framework of Hid-
den Markov Model (HMM) based speech recognition.
They developed the hybrid SVM/HMM system based
on their public domain toolkit. The hybrid system has
been evaluated on the OGI Alpha-digits corpus and
performs at 11.6% WER, as compared to 12.7% with
a triphone mixture-Gaussian HMM system, while us-
ing only a fifth of the training data used by triphone
system. Several important issues that arise out of the
nature of SVM classifiers have been addressed.
3 Arabic language speech recognition research
Katrin Kirchhoff et al. on their project at the 2002
Johns Hopkins Summer Workshop (Kirchhoff et al.
2002), which focused on the recognition of dialectal
Arabic. Three problems were addressed:
1. The lack of short vowels and other pronunciation
information in Arabic texts.
2. The morphological complexity of Arabic.
3. The discrepancies between dialectal and formal
Arabic.
They used the only standardized corpus of dialectal
Arabic currently available (2002), the LDC Call Home
(CH) corpus of ECA. The corpus is accompanied by
transcriptions in two formats: standard Arabic script
without diacritics and a Romanized version, which
is close to a phonemic transcription. Example of the
Romanized form used in their experiments is shown
in Table 2, they stated that Romanized Arabic is un-
natural and difficult to read for native speakers; more-
over, script-based recognizers (where acoustic models
are trained on graphemes rather than phonemes) have
performed well on Arabic ASR tasks in the past.
3.1 Automatic Romanizing Tool (ART)
Once Katrin Kirchhoff et al. evaluate their system,
a WER of 59.9% and 55.8% is obtained (evaluated
Table 2 ECA transliterated and Romanized sentence represen-
tations (Kirchhoff et al. 2002)
ECA
Transliterated script AlHmd llh kwlsB w Antl Azlk
Romanized word forms llHamdulillA kuwayyisaB wi inti
izzayik
Int J Speech Technol (2006) 9: 133150
against the script and Romanized transcriptions, re-
spectively), they concluded that it would be advan-
tageous to have large amount of Romanized training
data for the development of future Arabic ASR sys-
tems, and focused on building an (ART), rather than
try to explore the reasons behind these results.
In my opinion the real reasons for this result is
that, it is unfair to compare these two systems, as
they are totally two different systems. The first one
was trained on standard Arabic script without diacrit-
ics, while the other was trained using Romanized tran-
scription which includes vowels information, doing so
you fool the system by hiding important information
like the short vowels in the former, while this infor-
mation is presented in the later. Due this reason, the
fact that Romanized Arabic is unnatural and difficult
to read for native speakers and the failure of using
out-of-corpus data that have proved successful in other
languagesaccording to Katrin Kirchhoff et al., we
think that, research for Arabic ASR should be done
on original fully or partially diacritized Arabic cor-
pus not Romanized and to start developing (APDT),
rather than developing ART as stated by Sir Thomas
Elliot where he stated that If physicians be angry, that
I have written physics in English, let them remember
that the Greeks wrote in Greek, the Romans in Latin,
Avicenna and the other in Arabic, which were their
own and proper maternal tongues (CMU SPHINX
trainer 2008).
Modular recurrent Elman neural networks
(MRENN) for Arabic isolated speech recognition is
implemented (Young 1994). This is a special kind of a
recurrent network. The Elman network, originally de-
veloped for speech recognition, is a two-layer network
in which the hidden layer is recurrent. The inputs to
the hidden layer are the present inputs and the outputs
of the hidden layer are saved from the previous time-
step in buffers called context units. Their work is a
duplicate of a previous work done by (Ganapathiraju
et al. 2000) but for English language. They described
a novel method of using recurrent neural networks
(RNN) for isolated word recognition. Each word in the
target vocabulary is modeled by a fully connected re-
current network. To recognize an input utterance, the
best matching word is determined based on its tem-
poral output response. The system is trained in two
stages. First, the RNN speech models are trained inde-
pendently to capture the essential static and temporal
characteristics of individual words. This is performed
Int J Speech Technol (2006) 9: 133150
by using an iterative re-segmentation training algo-
rithm which gives the optimal phonetic segmentation
automatically for each training utterance. The second-
stage involves mutually discriminative training among
the RNN speech models aiming at minimizing the
probability of misclassification. M. M. El Choubassi
et al. used a separate Elman network for each word
in the vocabulary set, although they obtained promis-
ing results (accuracy 87%) for their very small iso-
lated system, but this approach may be suitable for
isolated very small vocabulary size and not suitable
for (LVCSR), a memory and performance problems
will be faced. In Baugh and Cable (1978), a method
for automatically segmenting Holly Quraanic Arabic
is presented; a linguistic segmentation estimate was
used when the recognition failed to provide a strong
classification. In El Choubassi et al. (2003), authors
presented work for Data Recording, Transcription,
and Speech Recognition for Egyptian colloquial, their
work is based on Romanized transcription of Arabic,
table below represent below represent the Romanized
symbols used in their work. They stated that Roman-
ized Grapheme-to-Phoneme tool for Standard Arabic
(collected in Tunisia and Palestine) already developed
at CMU. They use Egypt Call Home data and pronun-
ciation dictionaries of (LDC).
4 Pronunciation Dictionaries
(PD) is a human-generated or machine table of pos-
sible words and their permissible phonetic sequences
-their pronunciations-. Since there are many possi-
ble sequences of phonetic units that do not comprise
actual words the lexical model prevents many pho-
netic sequences from being explored during recogni-
tion process (Lee et al. 1998).
Often the creation of a (PD) is not a trivial task.
It can be created manually by a human expert in the
modeled language. But especially with large vocabu-
lary recognizers in which we deal with ten of thou-
sands of words. This approach can be very expensive
and time consuming and therefore it is often not a fea-
sible option. So the process has to be at least in part be
automatized. There are many researches on develop-
ing (PD) for ASR, in this section we will review some
of these researches.
137
4.1 Automatic generation of a Pronunciation
Dictionary
In Essa (1998), it is stated that it has been con-
firmed that an appropriate (PD) constructed by hand
or by a rule-based system improves recognition per-
formance. But such dictionaries require time and ex-
pertise to construct. Since relation between grapheme
to phoneme is not direct, the researchers proposed
a method for automatically generating of a (PD) for
Japanese language, based on a pronunciation neural
network that is able to predict plausible pronunciations
from the canonical pronunciation. They use a multi-
layer perceptron to predict alternative pronunciation
from the canonical pronunciations based on maximum
output of the network.
In Schultz (2002), authors use a statistical proce-
dure to determine the phonetic realization of phonemes
base forms -expected pronunciation-. Taking account
of lexical stress and word boundary information, they
generated statistics for phonemes in word base form
from a phonetically labeled speech corpus. The esti-
mates derived from this corpus were then used to gen-
erate pronunciation networks from base forms in the
(DARPA) Resource Management (RM) task. A sig-
nificant improvement in recognition accuracy was ob-
tained on (RM) task using the pronunciation networks
thus derived, relative to the base form pronunciations.
In Fukada et al. (1999), authors described the proce-
dure for generating the phoneme sequence automati-
cally using their general purpose phonetic front end. In
order to generate a pronunciation string for each word,
a neural network first assigned a score fro each of 39
phonemes to each 6 msec frame in the word. Then a
Viterbi search finds the best scoring sequence. They
found that the difference between systems using net-
works derived from hand labels and those using ma-
chine labels is not significant, which means that they
recommend the use of automatic generated (PD).
In Fukada et al. (1999), authors stated that sev-
eral approaches have been adopted over the years
for grapheme-to-phone conversion for European Por-
tuguese: hand derived rules, neural networks, classifi-
cation and regression trees, etc. their first approach to
grapheme-to-phone conversion was a rule-based sys-
tem with about 200 rules. Later, this rule-based ap-
proach was compared with a neural net approach. Fi-
nally they described the development of a grapheme-
to-phone conversion module based on Weighted Finite
138
State Transducers. They investigated both the use of
knowledge-based and data-driven approaches.
4.2 Arabic speech sounds and properties
Arabic is a Semitic language, and it is one of the old-
est languages in the world today. It is the fifth widely
used language nowadays. Arabic alphabets are used in
several languages, such as Persian and Urdu (Hiyassat
et al. 2005) Arabic linguistics came into being in the
eighth century with the beginning of the expansion of
Islam. This early start can be explained in terms of the
tremendous need felt by the members of the new com-
munity to know the language of the Holly Quraan,
which had become the official language of the young
Islamic state (Al-Zabibi 1990).
Arabic linguistics exerted huge effort, explaining
linguistic rules and Arabic grammar; however, this lin-
guist did not last long especially in the information era
(Alghamdi et al. 2004).
The relative regularity of the syntax presents some
advantages for its formalization. In addition, the Ara-
bic language has the following characteristic: from
one root the derivational and inflectional systems are
able to produce a large number of words, or lexical
forms, each of which has specific patterns and seman-
tics. In a certain sense, the Arabic language seems bet-
ter suited for computers than English or French (Hadj-
Salah 1983).
Contemporary Standard Arabic, a modernized ver-
sion of classical Arabic, is the language commonly
in use in all Arab speaking lands today. It is the lan-
guage of science and learning, of literature and the
theater, and of the press, radio and television. Notwith-
standing the unanimous acceptability of Contempo-
rary standard Arabic and its general adoption as the
common medium of communication throughout the
Arab world, it is not every day speech of the people
(Alghamdi et al. 2004).
4.3 Grapheme based Pronunciation Dictionary for
Arabic
Grapheme-to-phoneme conversion is an important
prerequisite for many applications involving speech
synthesis and recognition (Lee et al. 1998). For ASR
this process is important in developing the (PD), nor-
mally as mentioned earlier this (PD) is hand crafted.
In this section, a thorough description of grapheme
Int J Speech Technol (2006) 9: 133150
Based (PD) will be presented, first the importance of
PD for ASR will be described, then rules of phonolog-
ical Arabic system will be presented followed by Or-
thographic to Phonetic Transcription description. The
section will be concluded by description of the gener-
ation of (PD) for both MSA and Holly Quraan Large
Vocabulary Continuous Speech Recognizers (LVCSR)
are commercially available from different vendors.
Along with this increased availability comes the de-
mand for recognizers in many different languages that
often were not focused enough for the speech recog-
nition research so far. It is estimated that today as
much as four to six thousand different languages exist
(Alghamdi 2001). Therefore, over the last period in-
creased thought has been given to creating methods for
automating the design of speech recognition systems
for new languages while making use of the knowledge
that has been gathered from already studied languages.
One of the core components of a speech recognition
system is the PD. The main purpose of it is to map the
orthographic representation of a word to its pronunci-
ation. The search space of the recognizer is the (PD)
(Andersen et al. 1996). The performance of a recogni-
tion system depends on the choice of subunits and the
accuracy of the PD. An accurate mapping of the or-
thographic representation of a word onto a subunit se-
quence is important to ensure recognition quality, oth-
erwise the acoustic models trained with the wrong data
or during decoding the calculation of the scores for a
hypothesis is falsified by applying the wrong models
(Schultz 2002; Schultz et al. 2004).
The PD lists the most likely pronunciation or cita-
tion form of all words that are contained in the speech
corpus. The pronunciation of the corpus can range
from very simple and achievable with automatic pro-
cedures to very complex and time-consuming (Fukada
et al. 1999).
The creation of a PD is not a trivial task as men-
tioned earlier and the process has to be at least in part
be automated. With sufficient knowledge of the target
language, one can try to build a set of rules that map
the orthography of a word to its pronunciation. For
some languages this might work very well, for others
this might be almost impossible. Arabic language is
an example of languages with a very close grapheme
to phoneme relation (Hadj-Salah 1983). Thus compar-
atively few rules suffice to build a PD containing the
canonical information.
Int J Speech Technol (2006) 9: 133150
4.4 Automatic versus Hand Crafted Pronunciation
Dictionaries
The recognition quality is maintained by maintaining
the quality of the PD with which it maps the orthog-
raphy of a word to the way it is pronounced by the
speakers. Best dictionaries such as CMU-dict (pronun-
ciation dictionary created by Carnage Mellon Univer-
sity) are usually achieved with hand-crafted dictionar-
ies (Fukada et al. 1999; Killer et al. 2003). However,
manually created dictionaries require an expert in the
target language (Killer et al. 2004). However; this is
a time consuming and costly approach, especially for
large vocabulary speech recognition. If no language
expert knowledge is available or affordable, methods
are needed to automate the PD creation process. Sev-
eral different methods have been introduced over time.
Most of them are based on the conversion of the ortho-
graphic transcription to a phonetic one, using either
rule based (Killer et al. 2003) or statistical approaches
(Killer et al. 2004).
In order to reduce, both the cost and time required
to develop LVCSR systems, the problem of creating
(PD) must be solved. In the following sections devel-
oping automatic tool for (PD) for Standard Arabic lan-
guage will be described.
4.5 Segmenting Arabic utterance
The first basic rule that operates in the phonological
system of Arabic without exception is that the number
of syllables in an utterance is equal to the number of
vowels. The issue, then, is not the number of syllables
in an utterance, since this is automatic, but rather the
boundaries that are signaled either by zero, one or two
consonants (Alghamdi et al. 2004).
The second basic rule of Arabic phonology is that
the onset of the syllable equals the beginning of an ut-
terance. Thus, both can begin with a single consonant
example the first phoneme is consonant n
and the second is the short vowel ] ], (Alghamdi
et al. 2004).
The third rule is that the coda of the syllable is iden-
tical with the end of an utterance, coinciding infinitely
with the codas of the six syllable types previously dis-
cussed. Accordingly, syllables in Arabic can be either
open or closed, i.e., they can end in one or two conso-
nants, respectively.
Clearly, then, one should use the three rules just
stated to begin the process of segmentation in Arabic.
139
When properly applied, these rules enable one to seg-
ment almost any utterance in Arabic correctly and eas-
ily, for they make the division between the coda and
the onset of nearly all contiguous syllables clear-cut.
4.6 Orthographic to phonetic transcription
Conversion of Arabic phonetic script into rules is one
of the major obstacles facing the researchers on Ara-
bic text to speech systems and speech recognition. Al-
though Arabic is one of the oldest languages that its
sounds and phonological rules were extensively stud-
ied and documented (more than 12 centuries ago) (Al-
ghamdi et al. 2004), these valuable studies need to be
compiled from scattered literatures and formulated in
a modern mathematical frame work. The aim of this
section is to formulate the relation between grapheme
to phoneme relationship for Arabic.
Arabic language is an algorithmic language, at least
from the phonology, writing and derivatives point of
view, for example no law can explain the pronuncia-
tion of g in English in the following words laugh,
through, good and geography, while Arabic language
has direct grapheme to phoneme mapping for most
grapheme. In general, Arabic text with diacritics is
pronounced as it is written using certain rules. Con-
trary to English, Arabic does not have words with dif-
ferent orthographic forms and the same pronunciation.
There are sixteen essential rules in orthographic
to phonetic transcription (Al-Zabibi 1990; Hadj-Salah
1983).1 These rules are:
1. The sokon sign ( ), is not symbol of any phoneme,
but it is meaning is this consonant is followed
by another consonant, without intermediate vowel,
(i.e. if it exist or not it will not affect the pronun-
ciation of the consonant itself). Example this
means that will be pronounced as is without
introducing any vowles.
2. The after group waw as in is not
pronounced.
3. Pharyngealization (emphasis): There are Pharyn-
gealized consonants in standard Arabic were you
stress the consonant when pronouncing it.
Example is the word ( ) count the sign here,
used as stress when the is pronounced .
1 URL:http://www.phonetik.unimuenchen.de/Forschung/BITS/
TP1/Cookbook/node145.html (2006).
140
Table 3 Pronunciation
rules for laam ( )
Int J Speech Technol (2006) 9: 133150
Pronunciation rule Successor letter
Moonlam ( ) pronounced if
it is followed by these letters
Sunlam ( ) assimilated if it is
followed by these letters

4. The pronunciation of alef is totally dependent on Table 4 Pronunciation

examples
the successors characters as follows:
a. Not pronounced if followed by two consonant
as in (in the school), this pronounced
as . 10. The pause: an utterance in Arabic is never termi-
b. Pronounced if it is part of the laam of definite nated by a short vowel this means that the short
article as in the will be pronounced vowel of the last word of the sentence is not pro-
as follows . nounced.
c. Pronounced as the short vowel , if it is the 11. The rule for pronouncing the three Tanween
first of a verb, with the third character of it has double diacritic signs ( ) namely ( , , ) this di-
the short vowel as in , in this case it acritics are pronounced as N ( ) as in it
is pronounced as .
is pronounced as if it is not finally or
d. If the above rules did not apply, then the alef
pronounced as otherwise.
pronounced as the short vowel , as in is
12. The lengthening alef , as in , is pronounced
pronounced as .
5. The alef almaqsorah , its predecessor is always as .
the short vowel as in , it is pronounced as 13. If the predecessor of the vowel waw , is the
short vowel as in , then it is pronounced
.
as .
6. Feminine Taa ( ) as in which is
used in Arabic at the end of a noun to modify its 14. If the predecessor of vowel Yaa is the short
gender from masculine to feminine if the word con- vowel as in , then the Yaa is pronounced
taining feminine Taa found as the last word in sen- as .
tence then the Taa pronounced as Haa other- 15. If there is three successive consonant, as in
wise it is pronounced as . , then short vowel is introduced as
7. The letter laam ( ) in ( ) is the Laam of the def- .
inite article, prefixed to nouns; they are added to 16. The laam ( ) always in traqeeq except in the pro-
the structure of the word. There are two types nunciation of the name of Allah ( almighty )
of Laam The moonlam Alqmar ( ) pronounced or it is tafkheem if comes in at the begin-
a l q a m r and the sunlam Alshams ning of the utterance or its predecessor either one
( ) pronounced Ashmes the Laam of the two short vowels , . Example it is pro-
is not pronounced her. The letter Laam in ( ) ei- nounced respectively.
ther pronounced or assimilated depending on the
successor character as shown in Table 3.
4.7 Generation of Pronunciation Dictionaries
8. The HamzGlottal-(( ) ):
The Hamza is pronounced when it comes after
Pronunciation dictionaries are built by splitting a word
a pause or at the beginning of an utterance. But it
into its graphemes which are used as subunits. For ex-
is not pronounced in all other cases as shown in
ample the word would be simply followed by
Table 4.
its fragmentation into subunits (graphemes) as in the
9. The rules of pronunciation two successive words
when the last character of the first and the first char- following example . This is a very
acter of the second word are not vowelized. The easy and fast approach to produce pronunciation dic-
general rule is that the short vowel \i\ should be in- tionaries. The questions arise how well graphemes are
troduced after the last character of the first word. suited as subunits, to what extent are they inferior to
Int J Speech Technol (2006) 9: 133150
phonemes or do they perform comparably well, how
do we cluster graphemes into poly graphemes, how do
we generate the questions to built up the decision tree?
Apart from dialectic Arabic, there are two kinds of
pronunciations for Arabic language, the MSA pronun-
ciation and The Holy Quraan pronunciation (Baugh
and Cable 1978). The standard Arabic pronunciation
is governed by the above rules mentioned earlier in
Sect. 4.5, while The Holly Quraan pronunciation is
governed by what is called Tajweed rules which will
be described in Sect. 5.3. The proposed (PD) deals
with both of these pronunciations.
5 Experimental environment
In this section the development of Arabic Corpuses
and Baseline System used in experimenting the (PD)
developed will be presented. Namely the Holly
Quraan Corpus (HQC-1), Command And Control
Corpus (CAC-1) and Arabic Digits Corpus (ADC).
The focus of our research is on developing these cor-
puses in order to facilitate testing the (PD) already
developed. Selecting the SPHINX-IV engine, which
is built on open architecture, will make the results we
present independent of the specific recognition engine
used. The particular aspects of the speech databases
are presented to provide the reader with useful context
information for interpreting our results and to provide
other researchers with enough information to repeat
and validate our experiments.
5.1 Arabic Corpus and Baseline System
Most of the research done on SPHINX-IV used ei-
ther Wall Street Journal (s3-94, s0-94), and/or Re-
source Management Corpus (RM) (Rosti 2004; CMU
SPHINX Open Source Speech Recognition Engines
2007; Huang et al. 2003; Nedel 2004; Ohshima 1993;
CMU SPHINX trainer 2008; Young 1994; Hiyassat
et al. 2005; Al-Zabibi 1990). These corpuses are used
for other Latin languages such as French or Italian due
to similarities between these languages and the Eng-
lish language from phoneme point of view.
Unfortunately, there are great differences between
English language and Arabic language from phoneme
point of view due to the existence of some special
phoneme such as Dhad , Dh , Tah, , aeen enough, due to the high cost of the resources (Mimer
, ghaeen , haa , ssad , KHaa and et al. 2004).
141
Qaaf . Although some researchers have already
used English corpus for Arabic Speech Recognition
purposes but most of these approaches did not offer
good performance results (according to Katrin Kirch-
hoff et al. (2002) WER obtained is 59.9% for Roman-
ized Arabic which is not comparable to English ASR
WER) (Kirchhoff et al. 2002). To this effect, we de-
cided to build pure formal Arabic corpus that will be
used in testing our algorithm. This corpus will may be
used as benchmark for future researches.
In building a corpus for any language certain do-
main should be selected, a domain dependent tran-
scription should be obtained. Recording this transcrip-
tion is done using different speakers in a sound iso-
lated booth and sampled using deferent sampling rates
(Rosti 2004; Huang et al. 2003; Raj 2000; Alghamdi
et al. 2004; Killer et al. 2003).
Of course, such tasks are exhaustive in both time
and cost and beyond individuals capabilities. Usually
such tasks are done through bodies such as Defense
Advanced Research Projects Agency (DARPA), Jon
Hopkins University (JHU), Carnegie Mellon Univer-
sity (CMU) or Harvard Tool Kit (HTK) and Network
for Euro-Mediterranean Language Resources (NEM-
LAR) (CMU SPHINX Open Source Speech Recogni-
tion Engines 2007; Huang et al. 2003; Fukada et al.
1999; Mimer et al. 2004; Black et al. 1998).
As mentioned earlier The Arabic alphabet only
contains letters for long vowels and consonants. Short
vowels and other pronunciation phenomena, such as
consonant doubling, can be indicated by diacritics
(short strokes placed above or below the preceding
consonant). However, Arabic texts are almost never
fully diacritized and are thus potentially unsuitable for
recognizer training except The Holly Quraan and few
other school text books. The Holly Quraan is consid-
ered as the most important reference for Arabic lan-
guage.
5.2 Corpuses design criteria
Developing speech corpus is not a trivial task and need
resources to be allocated, unfortunately such resources
does not exists for the purpose of this research, in some
researches a figure of hundreds of thousand of dollars
is considered limited budget. Some researchers con-
sider that the size of 40 hours of broadcast News is
142
5.3 The Holly Quraan Corpus HQC-1
The development process of the HQC-1 is started by
collecting the recording for different reciters. The au-
dio file collected, are of different format, some of it
are MP3 format while the others are of wav format.
All of the collected audio files were converted using
this process using Sound Forge software which is open
source sound processing software (Black et al. 1998).
Audio files have the following characteristics:
Format mono .wav files
Sampling frequency 16000 Hz; 16 bit
5.3.1 Filenames conventions
Each file have been give unique file name, filenames of
the resources bear three ordered types of information:
a. Reciter name.
b. Sora (chapter) name.
c. Serial number identifying audio file.
Example of the file name is: Huthaifi-Isra-001.wav,
the first part of the file name, is stand for the name of
reciter. In this case al-Huthiafi (one of the famous re-
citers for Holly Quraan), the second part is sora name,
in this example the Isra sora (the sora number seven-
teen of the Holly Quraan) and the last part is the serial
number of the audio file with respect to both the reciter
and the sora number. The file extension (.wav) stands
for file format.
5.3.2 Directory structure
Directory of the audio file was divided into subdirecto-
ries, each reciter in separate directory, the second level
of sub directories are sora of the Holly Quraan. Ex-
ample: HQC/Huthaifi/Isra/filename.wav.
5.4 Languages and character sets encodings
Language used in the transcription files is the dia-
critized Arabic, the encoding used is Unicode (UTF-8)
and end line with line feed only format.
Int J Speech Technol (2006) 9: 133150
Fig. 1 Example of control file content and format
5.4.1 Tools/software used for marking silences
After choosing good quality recording, it is splitted
into sub recordings of about 10 to 30 seconds length
each. Sound forge software is used to automatically
detect the silence in the original recording by using
the Auto Cue feature. A single cue is added in the cen-
ter of each detected silent region. The Threshold value
sets the volume level for the silence. In most cases,
the value should be 40 dB or higher otherwise any
background hiss, pops, or clicks will be treated as non-
silence and no silence would be marked at all. If no cue
points appear, we try increasing this value to 30 dB
or higher.
5.4.2 Silence length
The Silence length value determines how much silence
is required before it is marked. Some recordings con-
tain brief silences that you usually do not want to be
marked. This value helps to avoid marking any brief
pauses within a wave file. Values between 1.0 to 1.5
seconds are used to ignore these brief silences and only
mark longer silences between recordings.
5.4.3 Splitting recordings
Recordings were split into small recordings of 10 to 30
seconds length each, according to cue points location.
Each of these files is given unique file identification
name. Control file containing list of file name and path
for all recordings are created, the file extension name
for this file is .fileids example an4_train.fileids as
shown in Fig. 1. File extension here is not provided,
its defined in the configuration file, since SPHINX-IV
accept different file format as wave, raw or NIST for-
mat.
Int J Speech Technol (2006) 9: 133150 143

5.4.4 Feature extraction
For every recording in the training corpus, a set of fea-
tures files are computed from the audio training data.
Each recording can be transformed into a sequence
of feature vectors using the front-end executable pro-
vided with the SPHINX-IV training package, as ex-
plained earlier.
The process start with Pre-emphasis, this is ap-
plied first, in order to remove noise by applying hi-
pass filter. Then hamming window is applied, in or-
der to slices up Data objects into a number of overlap-
ping windows, (usually referred to as frames in the
speech world). After applying the hamming window
the (FFT) is applied to compute the Discrete (FT) of
the input sequence. This is done by analyzing a signal
into its frequency components. Filters bank (MFFB) is
applied to the output of the FFT, the output is an array
of filtered values, typically called Mel-spectrum, each
corresponding to the result of filtering the input spec-
trum through an individual filter. Therefore, the length
of the output array is equal to the number of filters cre-
ated. In order to obtain the MFCC the (DCT) is applied
and its mean is computed for the MFCC to obtain the
(CMN)), this is done in order to reduce the distortion
caused by the transmission channel. After computing
the (CMN), the first and the second derivative of it are
computed in order to model the speech signal dynam-
ics (all as explained earlier).
5.4.5 Transcription file
to 16 kHz and divided into small utterancesand then
transcribed, resulting in a total of 59428 words and
25740 unique words with about 18.35 of recording
hours. It took a total of about 732 working hours to
build this corpus.
6 Pronunciation Dictionary creation
In order to create the PD, the APDT described earlier
should be invoked; APDT needs a transcription file to
produce PD based on this transcription file. Once the
APDT is invoked, two files are created; one is the (PD)
and the other is a file containing the transcription with
pronunciation alignment, so that each word in the tran-
scription is mapped to its pronunciation in the PD file.
The PD file has all acoustic events and words in the
transcripts mapped onto the acoustic units we want to
train. Redundancy in the form of extra words is permit-
ted. The dictionary must have all alternate pronuncia-
tions marked with parenthesized serial numbers start-
ing from (2) for the second pronunciation. The marker
(1) is omitted. Each word in the dictionary is followed
by its pronunciation as shown in Fig. 3.
6.1 Filler dictionary
Filler dictionary usually lists the non-speech events as
words and maps them to user-defined phones. This
dictionary must at least have the entries (Fig. 4).

Each recording is accurately transcribed; any error in

the transcription will mislead the training process later.
The transcription process done manually, that is, we
listen to the recording then we match exactly what we
hear into text even the silence or the noise should be
represented in the transcription. The transcription then
is followed by the file name without the path as shown
in Fig. 2 this is to map the transcription to its corre-
sponding recording.
The HQC-1 corpus consists of about 7742 record-
ings, these recordings were processeddown sampled

Fig. 2 Transcription file contents Fig. 3 Sample of pronunciation dictionary

144
Fig. 4 Filler dictionary
minimum content
Format Mono .wav files
Sampling frequency 16000 Hz; 16 bit
The entries stand for:
<s> : beginning-utterance silence
<sil> : within-utterance silence
</s> : end-utterance silence
Note that the words <s>, </s> and <sil> are treated
as special words and are required to be present in the
filler dictionary. At least one of these must be mapped
on to a phone called SIL. The phone SIL is treated
in a special manner and is required to be present. The
SPHINX-IV expects us to name the acoustic events
corresponding to our general background condition as
SIL. For clean speech these events may actually be
silences, but for noisy speech these may be the most
general kind of background noise that prevails in the
database. Other noises can then be modeled by phones
defined by us.
6.2 Phone list
Phone list is a list of all acoustic units that we want
to train models for. The SPHINX-IV does not permit
us to have units other than those in our dictionaries.
All units in the two dictionaries must be listed here.
In other words, phone list must have exactly the same
units used in your dictionaries, no more and no less.
Each phone must be listed on a separate line in the
file, beginning from the left, with no extra spaces after
the phone. an example is shown in Fig. 5.
By creating the transcription file, the PD, the filler
dictionary, control file and the features of the audio
files, the system now can be trained in order to test the
PD accuracy.
6.3 Command and Control Corpus (CAC-1)
The development process of the CAC-1 is started by
collecting the recording for different speakers. The au-
dio file recorded format is
6.3.1 Filenames conventions
Filenames of the resources contain the following in-
formation:
Int J Speech Technol (2006) 9: 133150
Fig. 5 Phones list used in training
a. Speaker name.
b. Serial number identifying audio file.
Example of the file name: Mohamad-001.wav, the
first part stand for the speaker name the second part is
the serial number of the audio file. The file extension
(.wav) stand for file format.
6.3.2 Directory structure
Directory of the audio files is divided into sub di-
rectories for each speaker. Example: CAC/Mohamad/
filename.wav.
6.3.3 Languages and character sets encodings
Language used in the transcription files is the dia-
critized Arabic, the encoding used is Unicode (UTF-8)
and end line with line feed only format.
6.3.4 Recording and processing
CAC-1 corpus is totally recorded for this research.
CAC-1 consists of two disjoint sets of utterances: 5628
training utterances collected from 103 male and 74
female speakers, and 372 testing utterances from 15
male and 8 female speakers details are shown in Ta-
ble 5. The total length of the training utterances is
about 4248 seconds. Different speakers are used for
the training and testing data. Both the training and
the testing utterances in the CAC-1 database were
recorded simultaneously using mono single unidirec-
tional microphone placed on the desktop nearby. It
is originally recorded at 16000 kHz with manually
transcribed and annotated acoustic environment for
speakers dialect and other conditions.
Int J Speech Technol (2006) 9: 133150 145

The CAC-1 corpus is considered as a small vocabu- Dictionariespronunciation and fillerare devel-
lary set (approximately 30 words in lexicon), the utter- oped in a similar way as for HQC-1. Since the CAC-
ances consist of command and control word as shown 1 corpus is a planned one, transcription were done
in Table 6. easilyeach speaker is asked to say exactly the same
Baseline system is trained using about 2 hours of word in the same orderonly mapping of the record-
speech in CAC-1, including all conditions together. ing to the control file is needed. Of course making sure
About 10 minutes of evaluation data are used for the that the recording reflects exactly the same transcrip-
test in this research. tion is essential; otherwise we fool the system in the
training phase.
7 Arabic Digits Corpus (ADC)
7.1 Training and evaluation
The third corpus is the ADC corpus is totally devel-
oped in this research too. This corpus is built for devel- Once the model definition file is ready, training is
oping Arabic digit model recognition for digits zero, started by initializing the model parameter and run-
one . . . nine ... , . ning the Baum-Welch algorithm described in next sec-
The ADC corpus is developed using the recording tions.
of 142 speakers, Table 7 shows the details of those
speakers. This corpus was developed exactly in the 7.2 Effect of number of Gaussians on the ADC
same manner and the same environment of the CAC-1
corpus. The ADC Consists of two disjoint sets of ut- Table 8 through Table 14 show the different perfor-
terances: 1213 training utterances collected from 73 mance measure of the ADC, from these tables it is
male and 49 female speakers, and 143 testing utter-
ances from 12 male and 8 female speakers details are Table 8 Number of Number Accuracy
shown in Table 7. The total length of the training ut- Gaussians versus word of Gaus-
accuracy versus
terances is about 0.67 hr. sians
Baseline system is trained using about 35 minutes
of speech, including all conditions together. About 7 1 80.159
minutes of evaluation data are used for the test in this 2 84.127
research. 4 88.889
8 89.683
Table 5 Details of Gender Total 16 88.889
speakers participated in
32 77.778
CAC-1 corpus
Male 118 64 69.048
Female 82 128 65.079
256 65.079
Table 6 Words used in
CAC-1 corpus
Table 9 Number of Gaussians versus number of errors
Number of Gaussians Errors
Sub Ins Del

1 24 0 1
2 18 0 2
4 12 0 2
8 9 0 4
16 9 0 5
Table 7 Details of Gender Total 32 9 0 19
speakers participated in 64 9 0 30
ADC corpus
Male 85 128 9 0 35
Female 57 256 9 0 35
146 Int J Speech Technol (2006) 9: 133150
Table 10 Number of Number of WER Table 13 Number of Number of Speed X
Gaussians versus WER Gaussians Versus speed as
Gaussians ratio of real time audio Gaussians real time

1 19.841 1 0.06
2 15.873 2 0.06
4 11.111 4 0.05
8 10.317 8 0.07
16 11.111 16 0.08
32 22.222 32 0.09
64 30.952 64 0.11
128 34.921 128 0.13
256 34.921 256 0.15

Table 11 Number of Number of Words Table 14 Number of

Gaussians versus words Number of Average
Gaussians matches Gaussians versus memory
matches usage Gaussians memory used

1 101 1 7.4
2 106 2 7.53
4 112 4 8.44
8 113 8 9.4
16 112 16 11.39
32 98 32 16.08
64 87 64 25.17
128 82 128 43.13
256 82 256 79.02

Table 12 Number of Gaussians versus sentences matches

Number of Gaussians Sentences matches Sentence accuracy

1 101 80.159
2 106 84.127
4 112 88.889
8 113 89.683
16 112 88.889
32 98 77.778
64 87 69.048
128 82 65.079
256 82 65.079

easily noticeable that best performance obtained when

Fig. 6 Overall Likelihood versus number of Gaussians Densi-
the distributions splitted into 8 Gaussians. ties with five states per HMM

7.3 HQC-1 overall likelihood of training

frame likelihood and dividing it by number of frames.
SPHINX-IV provides a tool to calculate per frame It is found that as the number of Gaussians densi-
training likelihood and overall training likelihood, the ties increases the Overall Likelihood increases too as
over all is obtained simply by summing up the per shown on Fig. 6.
Int J Speech Technol (2006) 9: 133150
8 Summary and conclusions
In this research we have used SPHINX-IV for Arabic
speech recognition, build speech recognition resources
for Arabic, build new tools that do not originally ex-
ist in SPHINX-IV that are suitable for Arabic Recog-
nition such as APDT and linguistic questions and we
also investigated fine-tuning SPHINX-IV parameters
for this purpose too. In this section, we present a sum-
mary of this research and the relevant observations
that we have drawn from our investigation in train-
ing, fine-tuning and testing using the three acoustic
models (HQC-1, CAC-1 and ADC) based on our pro-
posed dictionary. Some comments on future research
directions and unresolved questions are presented too.
The section is closed with final summary and conclu-
sions. What is mostly unique about this research is
the (APDT) algorithm we developed and tested. Three
Arabic corpuses, namely HQC-1, CAC-1 and ADC
were created for providing acceptable level of train-
ing and testing of our system. The recognition per-
formance obtained using these corpuses and the dic-
tionary obtained using our (APDT) for Arabic is very
successful. To the best of our knowledge, neither this
tool nor the HQC-1 corpus were exist prior to this re-
search. SPHINX-IV parameters were tuned using a
global search algorithm, training data was extended
with help of neural network, and a features-based sys-
tem (not Romanized) for Arabic speech recognition
based on SPHINX-IV technology is finally found. Our
system could be the basis for any future open source
research on Arabic Speech Recognition and we intend
to keep it open for the research community. Automatic
tool kit for generating (PD) is fully developed and
tested in this work. This tool kit is a Rule based pro-
nunciation tool. (PD) (HUSDICT60) is produced for
Fig. 7 Test summaries for Holly Quraan corpus
the HQC-1
[java] Accuracy: 70.813% Errors: 750(Sub: 467 Ins: 276
Del: 7)
[java] Words: 1624 Matches: 1150 WER: 46.182%
[java] Sentences: 273 Matches: 57 SentenceAcc: 20.879%
[java] This Time Audio: 7.62s Proc: 5.68s Speed: 0.75 X
real time
[java] Total Time Audio: 2205.66s Proc: 1638.93s Speed:
0.74 X real time
147
both formal and Holly Quran. HUSDICT60 contain
59424 words, this dictionary will be freely available
on Arabic. Three corpuses are entirely developed by
the author of this work. In the Holly Quraan Corpus
(HQC-1) about 7,742 recordings were processed and
then transcribed, which results in total of 59,428 words
and 25,740 unique words, and about 18.35 of record-
ing hours, this process consumes about 432 working
hours. Note that one research at Carnegie Mellon Uni-
versity (CMU) is done on about 1,400 hours of speech
for training one system (CMU SPHINX Open Source
Speech Recognition Engines 2007). Results are shown
in Fig. 7 and Fig. 8
The CAC-1 corpus consists of two disjoint sets
of utterances: 5628 training utterances collected from
103 male and 74 female Arabic native speakers, and
372 testing utterances from 15 male and 8 female
speakers. The CAC-1 corpus is considered as a small
vocabulary set (approximately 30 words in lexicon),
final results for this corpus is shown in Fig. 9.
ADC corpus is developed using the recording of
142 Arabic native speakers. This corpus is concerned
in developing Arabic digit model recognition for dig-
its zero, one . . . nine ... , . The ADC
Consists of two disjoint sets of utterances: 1213 train-
ing utterances collected from 73 male and 49 female
speakers, and 143 testing utterances from 12 male and
8 female speakers. The total length of the training ut-
terances is about 2431 seconds (Fig. 10).
From the results of research throughout this work,
many suggestions for future work are recommended
as shown in coming subsections. One major weakness
of conventional HMMs is that they do not provide an
adequate representation of the temporal structure of
speech. This is because the probability of state occu-
148 Int J Speech Technol (2006) 9: 133150

Fig. 8 Output of the performance test for HQC-1

Command and Control Corpus

Accuracy: 98.182% Errors: 1 (Sub: 1 Ins: 0 Del: 0)

Words: 55 Matches: 54 WER: 1.818%
Sentences: 55 Matches: 54 Sentence Acc: 98.182%
Total Time Audio: 53.29s Proc: 13.24s Speed: 0.25 X
real time
Mem Total: 126.62 Mb Free: 101.30 Mb
Used: This: 25.33 Mb Avg: 20.14 Mb Max: 25.49 Mb

Fig. 9 Test summaries for the CAC-1

Int J Speech Technol (2006) 9: 133150
Performance test of the digits corpus
Accuracy: 99.213% Errors: 1 (Sub: 0 Ins: 0 Del: 1)
Words: 127 Matches: 126 WER: 0.787%
Sentences: 127 Matches: 126 SentenceAcc: 99.213%
This Time Audio: 1.39s Proc: 0.09s Speed: 0.07 X
real time
Total Time Audio: 143.24s Proc: 9.91s Speed: 0.07 X
real time
Mem Total: 126.62 Mb Free: 114.17 Mb
Used: This: 12.46 Mb Avg: 12.76 Mb Max: 18.44 Mb
Fig. 10 Test summaries for the ADC
pancy decreases exponentially with time. This issue
is a promising area to investigate and many issues on
the HMM modeling and temporal structuring can be
studied. Another issue is the training of the HMM, al-
though Ant Colony Optimization is a stochastic and
discrete optimization algorithm, we believe that this
could be a promising algorithm if adapted for training
speech recognition models or at least can be used in
optimizing the training process. Final words, it should
be noticed that most Arabic texts are almost never
fully diacritical, and are thus potentially unsuitable for
recognizer training except the Holly Quraan and few
other Text Books and some religion old books. In addi-
tion to that, the existence of electronic versions of such
text is not always available. There should be an Ara-
bic effort to create diacritical corpus for both speech
recognition and text to speech research. During this re-
search, about 200,000 unique diacritic words are col-
lected and are now available on our free website cor-
pus as mentioned earlier.
References
Al-Zabibi, M. (1990). An acoustic-phonetic approach in auto-
matic Arabic speech recognition. The British Library in
Association with UMI.
Alghamdi, M. (2001). Arabic phonetics. Riyadh: Altawbah
Printing.
Alghamdi, M., Al-Muhtaseb, H., & Elshafei, M. (2004). Ara-
bic phonological rules. Journal of King Saud University:
Computer Sciences and Information, 16, 125 (in Arabic).
Andersen, O., & Kuhn, R., et al. (1996). Comparison of two
tree-structured approaches for grapheme-to-phoneme con-
version. In ICSLP 96 (Vol. 3, pp. 17001703) Oct. 1996.
Baugh, A. C., & Cable, T. (1978). A history of the English lan-
guage. Oxon: Redwood Burn Ltd.
149
Billa, J., et al. (2002a). Arabic speech and text in Tides On Tap.
In Proceedings of HLT, 2002.
Billa, J., et al. (2002b). Audio indexing of broadcast news. In
Proceedings of ICASSP, 2002.
Black, A., Lenzo, K., & Pagel, V. (1998). Issues in building gen-
eral letter to sound rules. In Proceedings of the ESCA work-
shop on speech synthesis, Australia (p. 7780) 1998.
Christensen, H. (1996). Speaker adaptation of hidden Markov
models using maximum likelihood linear regression. Ph.D.
Thesis, Institute of Electronic Systems Department of
Communication Technology, Aalborg University.
CMU SPHINX Open Source Speech Recognition Engines.
URL:http://www.speech.cs.cmu.edu/ (2007).
CMU SPHINX trainer Open Source Speech Recognition En-
gines, URL: http//:www.cmusphinx.org/trainer (2008).
Doh, S.-J. (2000). Enhancements to transformation-based
speaker adaptation: principal component and inter-class
maximum likelihood linear regression. Ph.D. Thesis,
Department of Electrical and Computer Engineering,
Carnegie Mellon University.
El Choubassi, M. M., El Khoury, H. E., Jabra Alagha, C. E.,
Skaf, J. A., & Al-Alaoui, M. A. (2003). Arabic speech
recognition using recurrent neural networks. Electrical and
Computer Engineering Department, Faculty of Engineer-
ing and ArchitectureAmerican University of Beirut.
Essa, O. (1998). Using prosody in automatic segmentation of
speech. In Proceedings of the ACM 36th annual south east
conference (pp. 4449). Apr. 1998.
Fukada, T., Yoshimura, T., & Sagisaka, Y. (1999). Automatic
generation of multiple pronunciations based on neural net-
works. Speech Communication, 27, 6373.
Ganapathiraju, A., Hamaker, J., & Picone, J. (2000). Hybrid
SVM/HMM architectures for speech recognition. In Pro-
ceedings of the international conference on spoken lan-
guage processing (Vol. 4, pp. 504507). November 2000.
Gouva, E. B. (1996). Acoustic-feature-based frequency warp-
ing for speaker normalization. Ph.D. Thesis, Department
of Electrical and Computer Engineering, Carnegie Mellon
University.
Hadj-Salah, A. (1983). A description of the characteristics of
the Arabic language. In Applied Arabic linguistics, signal
& information processing, Rabat, Morocco, 26 September
5 October 1983.
150
Hain, T., et al. (2003). Automatic transcription of conversational
telephone speechdevelopment of the CU-HTK 2002 sys-
tem. (Technical Report CUED/F-INFENG/TR. 465). Cam-
bridge University Engineering Department. Available at
http://mi.eng.cam.ac.uk/reports/.
Hermansky, H. (1990). Perceptual linear predictive (PLP) analy-
sis of speech. Journal of the Acoustic Society of America,
87, 17381752.
Hiyassat, H., Nedhal, Y., & Asem, E. Automatic speech recog-
nition system requirement using Z notation. In Proceedings
of Amse 05, Roan, France, 2005.
Huang, X., Alleva, F., Wuen, H., Hwang, M.-Y., & Rosen-
feld, R. (2003). The SPHINX-II speech recognition sys-
tem: an overview . In School of Computer Science Carnegie
Mellon University, Pittsburgh, 15213, 2003.
Huerta, J. M. (2000). Robust speech recognition in GSM codec
environments. Ph.D. Thesis, Department of Electrical and
Computer Engineering, Carnegie Mellon University.
Killer, M., Stker, S., & Schultz, T. (2003). Grapheme based
speech recognition. Eurospeech, Geneva, Switzerland,
September 2003.
Killer, M., Stker, S., & Schultz, T. (2004). A grapheme based
speech recognition system for Russian. In SPECOM2004:
9th conference, speech and computer, St. Petersburg, Rus-
sia, September 2022.
Kirchhoff, K., Bilmes, J., Das, S., Duta, N., Egan, M., Ji, G.,
He, F., Henderson, J., Liu, D., Noamany, M., Schone, P.,
Schwartz, R., & Vergyri, D. (2007). Novel approaches to
Arabic speech recognition. The 2002 Johns-Hopkins sum-
mer workshop, 2002.
Lee, K., Hon, H., & Reddy, R. (1990). An overview of
the SPHINX speech recognition. IEEE Transactions on
Acoustics, Speech, and Signal Processing, ASSP-28(1),
3545.
Lee, T., Ching, P. C., & Chan, L. W. (1998). Isolated word
recognition using modular recurrent neural networks. Pat-
tern Recognition, 31(6), 751760.
Liu, F.-H. (1994). Environmental adaptation for robust speech
recognition. Ph.D. Thesis, Department of Electrical and
Computer Engineering, Carnegie Mellon University, Pitts-
burgh, PA.
Mimer, B., Stuker, S., & Schultz, T. (2004). Flexible decision
trees for grapheme based speech recognition. In Proceed-
ings of the 15th conference elektronische sprach signal ve-
rarbeitung (ESSV), Cottbus, Germany, 2004.
Nedel, J. P. (2004). Duration normalization for robust recog-
nition of spontaneous speech via missing feature methods.
Int J Speech Technol (2006) 9: 133150
Ph.D. Thesis, Department of Electrical and Computer En-
gineering, Carnegie Mellon University.
Ohshima, Y. (1993). Environmental robustness in speech recog-
nition using physiologically-motivated signal processing.
Ph.D. Thesis, Department of Electrical and Computer En-
gineering, Carnegie Mellon University.
Pallet, D. S., et al. (1999). 1998 Broadcast news bench-
mark test results. In Proceedings of the DARPA broadcast
news workshop, Herndon, Virginia, February 28March 3,
1999.
Rabiner, L. R., & Juang, B.-H. (1993). Fundamentals of speech
recognition. Englewood Cliffs: Prentice-Hall.
Raj, B. (2000). Reconstruction of incomplete spectrograms for
robust speech recognition. Ph.D. Thesis, Department of
Electrical and Computer Engineering, Carnegie Mellon
University.
Rosti, A.-V.I. (2004). Linear Gaussian models for speech recog-
nition. Ph.D. Thesis, Wolfson College, University of Cam-
bridge.
Rozzi, W. A. (1991). Speaker adaptation in continuous speech
recognition via estilsiation of correlated mean vectors.
Ph.D. Thesis, Department of Electrical and Computer En-
gineering, Carnegie Mellon University.
Russell, S., Binder, J., Koller, D., & Kanazawa, K. (1995). Local
learning in probabilistic networks with hidden variables.
Computer Science, IJCAI.
Schultz, T. (2002). Globalphone: a multilingual speech and text
database developed at Karlsruhe University. In Proceed-
ings of the ICSLP, Denver, CO, 2002.
Schultz, T., Alexander, D., Black, A., Peterson, K., Suebvisai,
S., & Waibel, A. (2004). A Thai speech translation system
for medical dialogs. In Proceedings of the human language
technologies (HLT), Boston, MA, May 2004.
Seltzer, M. L. (2000). Automatic detection of corrupt spectro-
graphic features for robust speech recognition. Master de-
gree Thesis, Department of Electrical and Computer Engi-
neering, Carnegie Mellon University.
Siegler, M. A. (1999). Integration of continuous speech recogni-
tion and information retrieval for mutually optimal perfor-
mance. Ph.D. Thesis, Department of Electrical and Com-
puter Engineering, Carnegie Mellon University.
Young, S. J. (1994). The HTK hidden Markov model toolkit:
design and philosophy (CUED/F-INFENG/TR.152). Engi-
neering Department, University of Cambridge.
Zavagliakos, G., et al. (1998). The BNN Byblos 1997 large vo-
cabulary conversational speech recognition system. In Pro-
ceedings of ICASSP, 1998.