SAMARAWEERA ET AL: ENGLISH SINHALA MACHINE TRANSLATION – PSRS; VOL. 3, DEC 2009

37

Application of English – Sinhala – English Speech Machine Translation S.A.M.D. Samaraweera, W.A.H.N. Weerasinghe, H.R. Bopitiya, D.A. Ekanayake, K.A.W. Kumuduni, and K. Pulasinghe Department of Information Technology, Sri Lanka Institute of Information Technology, Malabe This paper brings together the development of speech machine translation system which is capable of translating a meaningful English spoken sentence in to its corresponding Sinhala spoken sentence and vise versa. System consists of speech recognizer, translator and speech synthesizer and Pages coming so forth describe the latest technologies that have been applied in the system to make it more precise and unambiguous translation approach. In addition to the speech translator this system comes with features such as an inbuilt dictionary with brief descriptions, a word adding tool, a grammar checking tool, and a test hypothesis tool for the words recognitions. As the majority of the worldwide population carries out their day-to-day work in English, it has become a necessary and important language that people should learn. Therefore, this system is highly useful for careers of the people who are unfamiliar in English and Sinhala languages especially with spellings and grammar. In addressing this need, the challenge of building an English– Sinhala–English speech translator was taken up. Index Terms - English, Sinhala language, Speech machine translation, Translator.

I.

INTRODUCTION

T

here is a huge language barrier between English and Sinhala languages as most of the students in Sri Lanka finish their secondary education in Sinhala medium. It is because the language English is considered the second language. In universities most of the subjects like Engineering, medicine, Information Technology with regard are taught in English and all the references are also in English language. Due to that most of the students have found that it is difficult to get rid of with language problem. As both the primary and secondary education are done using Sinhala, students who leave the school might find it difficult to find a good occupation in the private sector. Lack of knowledge in the English language has created difficulties for many Sri Lankans especially in job interviews. The language barrier would always embarrass people in various situations. In addition, foreigners also face difficulties when trying to understand and communicate in a society where majority speak Sinhala when they visit Sri Lanka. There is no existing software system which is capable of translating given English speech to its corresponding Sinhala speech and vice versa. Hence our goal is to remove that barrier by developing the System Digital Language Translator (DLT). DLT consists of Speech Recognizer, language Translators and Speech Synthesizers which are going to be further discussed in the Methodology. There are some existing systems which content the above mentioned features individually. However, according to the methodologies they had used, some drawbacks can be seen. •

Less speed in the translation

Even though there are number of software systems to do English to Sinhala and Sinhala to English language translation the time taken to process the results they are time consuming. But this research is focused on the Example–based approach for the language translation process. By using this translation

strategy, it is possible to obtain high quality translations, with accuracy [1]. •

Less accuracy of the Language recognizer

Most of English speech recognizers have used Hidden Markov Model technology (HMM) to do the recognizing. But the Microsoft Software Development Kit (SDK) is capable of maximizing the accuracy of recognizing the speech. In this system’s English speech recognizer was created using SDK and the Sinhala speech recognizer using HMM Tool Kit (HTK). It contains some parameters. These HTK parameters can be changed to maximize the accuracy of the recognizer. This research mainly focuses on spoken Sinhala and spoken English. For this purpose in the translation process, all the three “persons” in both languages and all the passive voice sentences were not considered but all the tenses in the English and Sinhala Language were considered when developing the system. Translation is done using Example based technology to process the output in a minimal time. The research methodologies, future research directions and improvements are discussed in below sections. II. METHODOLOGY This system DLT contains six major sections. 1. 2. 3. 4. 5. 6.

English Speech Recognizer Sinhala Speech Recognizer English to Sinhala Translator Sinhala to English Translator English Speech Synthesizer Sinhala Speech Synthesizer

The overall design process is illustrated in the Fig. 1. This is a graphical representation of, how a human speech signal will convert to a translated speech signal successfully.

SAMARAWEERA ET AL: ENGLISH SINHALA MACHINE TRANSLATION – PSRS; VOL. 3, DEC 2009

Human Speech

Microphone

Sound Waveform

Output Transcription

Software Interfacing

Text File

Text File

Wave Format

Output Speech

Translator

Translation Engine

this, a pronunciation dictionary has to be created for training. This will be used to evaluate the accuracy of the recognized words at a latter phase of the process. This dictionary will consist of a set of words which will be used in the training session. Also the dictionary will be stored in a separate file and will be created according to the HTK standards, using HTK tools [2]. 2.

GUI Front-end

Recognizer

Recognizing Engine

Parameterize

Fig.1.- System Overview

38

Modeling and Training

After the data is prepared, the system has to be modeled to identify the language primitives. The modeling is done by using a set of HMMs. In this purpose HTK will be used to develop HMMs. The formed HMMs can be stored as separate files and will be used at the run time. All the HMMs will be created under the conventions and constraints of this topology. This research needs continuous speech recognition. So the initial step would be creating a well-trained set of singleGaussian monophonic HMMs to map each and every Sinhala phoneme. Then monophonic HMMs will be used to create tied-state tri phone HMMs once it’s completed.

A. English Speech Recognizer The English speech recognizer transforms the speech to text conversion using Microsoft Speech DLL. It is developed using Microsoft Speech Lib DLL. It gives many functions that can be used to develop Speech Recognizing tasks. The DLL does the recognition of the spoken word and also is capable of giving the hypothesis for the word. In DLT both recognized word and also its hypothesis are also used to fulfill the language translation process, because sometimes the correct word for the given speech is not recognized but can be available under hypothesis of the word. Therefore if user sees that the correct word is not available and if it is in the hypothesis list, he can simply select it and replace the incorrect word. In this application it is used to convert the voice signal (input to the system) to text. Practically the recognizing is limited. The words that are to be identified must be given to the application first. In the application the developers can add new words that should be identified. That facility is given to improve the performance and enhance the recognition part of the application. B.

Sinhala Speech Recognizer

The Sinhala speech recognizer transforms the speech to text conversion using HMM. This will perform the role interpreter of converting speech to the desirable text format. Basically there are three stages to be covered in order to make a Sinhala speech recognizer [2]. The three phases are; 1.

Data Preparation

Data preparation signifies initial preparation of preliminary input data, which will be used when modeling and training the recognizer. The necessary input to the recognizer is a .wav file which is captured, encoded and parameterized with the required input transcription. All input files have to be converted to the parametric form as they read-in. In addition to

Training Speech

Input Transcript

Training

Intermediate HMMs

Recognizer

Output Transcript

Unknown Speech

Fig.2.- Train and Recognize

The recognizing will be done with use of several in-built probability functions and the engine will automatically map the phoneme with the HMM and select the matching value which has the highest level of probability. The training will require a well formatted phonetically balanced Sinhala paragraphs. This has to be converted to .wav files along with the transcripts. It is illustrated in Fig. 2. The training will result the initial mapping of each and every phoneme to a monophonic HMM. There are several integrated tools that are in HTK for training HMMs. By using these tools the training process can be initiated, reset or smoothened [3]. 3. Testing and Analyzing Once the recognizer is modeled and implemented the performance and the accuracy has to be evaluated. At this level, there will be several dictionaries and several sets of test data to be recorded. The recognizer will be tested for several times to appraise the level of accuracy. After each and every test the HMMs will be reset and tuned until it achieves a considerable throughput. Like in previous processes there are in-built utilities to achieve this task. The test results can be

SAMARAWEERA ET AL: ENGLISH SINHALA MACHINE TRANSLATION – PSRS; VOL. 3, DEC 2009

directly printed on the screen by using a test console and it will show the percentage of the accuracy level. This will be done at the testing and experiment phase of the project. C. Translator The translator uses a vocabulary database that contains Sinhala definitions for English words after the recognizer has converted the English speech to English text. Digital Language Translator

Fig.3.- Main Interface of the English to Sinhala Translator

The process of translation is further divided in to two sections. To do an exact translation first the source language sentence pattern should be well analyzed and should put in to two groups in a way that normal sentence is in one group and an interrogative sentence is in another. Then in the next process it should be mapped in to the target language. To do the translation part successfully the example based translation mechanism is used. The reason of using Example based translation among the rest of the others is the project DLT is involving translating English speech to Sinhala speech and Sinhala speech to English speech. But the translation part is not going to depth considering all the grammar rules in both languages. And that is a tree like structure where the examples can be put and matched with the words to get a meaningful sentence in Sinhala. This translated Sinhala sentence is shown in a Sinhala Font and if the user cannot read Sinhala he can get it read as a Sinhala speech using text to speech. 1.

English to Sinhala translation

As the first step in the translation process, the translation engine takes in the input text from the recognizer that must be a sentence. Each sentence’s tense should be grammatically correct. This is essential for the translation process to produce accurate sentences. The translation engine then analyses the inserted text from the recognizer. First, the system checks the first word of the sentence. Then it checks the next word. Once the system notices a word from Table I, it recognizes the sentence as an Interrogative sentence. And all the other sentences are recognized as normal simple sentences.

39

TABLE I MATCHING WORDS AND TENCE OF THE INTERROGATIVE SENTENCES

Words am / is / are was / were do / does did what / where / who when / why / which how / who can could shall should may might

Tense Question : Present Continuous Question : Past Continuous Question : Present Question : Past Question : Present / Past Question : Present Question : Past Question : Present Question : Past Question : Present Question : Past

Each word of that sentence is added to a sentence object, which is then added to an Array List. Then system checks the sentence word by word with the database. If there is a space, “a”, “an”, “the”, “to” then, the system does not check those words with the database. Those words are not included in the Database. Table II illustrates the steps carried out by the translation engine when translating the sentence “I want to eat rice” from English to Sinhalese [4]. TABLE II MATCHING WORDS WITH THE DATABASE

Word in the Database I want eat rice

Meaning in the Database mm uvmn` @v k bW

The system checks the first word and special words (“to”, “from” etc.) in the sentence and identifies the pattern of the sentence and then fills the postfixes of the Sinhala words. t - m+t nv` - uvmn` @v + nv` Nant - k + Nant If the sentence is “I eat rice” then, Eat - k nv` - k + nv` Fig. 4 illustrates the process of the English to Sinhala Translator.

SAMARAWEERA ET AL: ENGLISH SINHALA MACHINE TRANSLATION – PSRS; VOL. 3, DEC 2009

Get the input sentence from the recognizer

The translation engine then makes the string pattern for the Sinhalese sentence, which it will then use to make the English string pattern of the sentence [5].

English Sentence

Split the sentence in to words and Add to an Array

Get the (next) word from the array

English word

Stem until found in the Database

Find the word information home Get the relevant Sinhala word +t Append the word to given postfixes

End of Array

D. Text To Speech synthesizer 1.

Sinhala speech Synthesizer

This system uses Mbrola Di-phone synthesizing technology to convert Sinhala text to Sinhala speech. Because Mbrola synthesizes the exact Sinhala speech output from a written Sinhala words using limited Sinhala phonemes [6]. So the out come of the language translator is a notepad, containing the Sinhala meanings with using Sinhala font, and those Sinhala characters compare with Sinhala Unicode map one by one and pass the exact Unicode into the Mbrola sound player, and then real-time Mbrola synthesizes the Sinhala characters using Indian Di-phone database in order to given Sinhala sentence or word [6].

home

DB

40

No

Yes Get the meanings for the word information of full sentence

Validate the sentence according to the Sinhala pattern

2.

English speech Synthesizer

Sinhala speech to English Speech translator is used to obtain the translated English sentence in English speech. This research uses Microsoft Speech Lib DLL to synthesize the English Text to English Speech. E. Data Adding Tool Add New Nouns

English Noun

Sinhala Meaning Add

Displays the Sinhala meaning of the English sentence

Present Verbs

English Present Verb

Sinhala Meaning

Past Verbs

English Past Verb

Sinhala Meaning

Clear

Meaning’s Last Letter

Fig.4.- Diagram of English to Sinhala Translation Engine

2.

Sinhala to English translation

Fig.5.- Data adding Tool for Sinhala Dictionary

Same techniques and steps were applied to develop the other end of the translation engine that convert the Sinhala sentence to English sentence. The system checks the first word and realizes the pattern of the sentence and then checks the postfixes of the words. Table III illustrates different English words that are used when translating the above postfixes of Sinhala words. TABLE III SINHALA POSTFIXES WITH ENGLISH WORDS

Sinhalese K

English a / an

eN

from a / from an

N

from

@gN

from the

k

in a

t

to

@G

of / of the

vl

in

Sometimes there may be situations where even the system is not recognized words. Or may be not translated a sentence properly. Because the words in the sentence that user was inserted into the system is not in the database. So user can use this data adding Tool to insert data. F. Grammar Checking Tool The Grammar checking tool is used to check the grammar of the inserted sentence. Then user can check the word if it is a noun or a verb. G. English and Sinhala Dictionary Tool The system can use as English and Sinhala dictionary. The user can get the original meaning of the inserted word and also user can get a small definition about the word. H. Embedded System The research project Digital Language Translator which consists of the above mentioned six sections gives the final outcome working altogether thus can be considered as an embedded system.

SAMARAWEERA ET AL: ENGLISH SINHALA MACHINE TRANSLATION – PSRS; VOL. 3, DEC 2009

III. RESULTS AND DISCUSSIONS Sinhala speech recognizer, language translator and the integration of the separated systems were the main challenges and successfully completed after adopting them with new technologies. A speech recognizer will never attain the overall quality of human speech recognition because a machine does not have the human dimension required for dealing accurately with the subtle nuances of pronunciation. However, a recognition test for 100 sample sentences was conducted using the system, and an acceptable recognition result with more than 80 percent was achieved. The testing of the English to Sinhala translation was based on 12 test cases. These test cases were designed depending on the 12 tenses in Table IV under active voice available in the English Language. The subject is not an important part because the system is not focused on the grammar. Otherwise, 84 test cases have to be designed with the personal pronouns (I, We, You, They, He, She and It) for each tense. So, only one test case is sufficient for these pronouns. TABLE IV TWELVE TENSES IN ENGLISH LANGUAGE

Present Tense

Past Tense

Future Tense

Simple Present

Simple Past

Simple Future

The next sets of sentences are examples of Sinhala sentences that were correctly translated by the Sinhala English translation engine. TABLE V TEST RESULTS FOR CORRECTLY TRANSLATED SENTENCE

English to Sinhala 1.

I need to go home to get the lunch. mt Øv` a`hry gNnt @gqr yNnt uvmn` @vnv`.

2.

What do you want to eat? @m`nvq obt kNnt av|&y?

3.

Can I reserve a room? mt k`mryK @vNkrgNn ÕUvNq?

4.

She did not do the homework yesterday. a#y I@Y @gqr v#d k@R n#h#.

5.

How did you play football? @k`hmq ob p`pNÚ æd` krN@n?

Sinhala to English 6.

a#y aq pAñyt °@Y n#h#.

She did not go to classroom today. 7.

aMmû mmû m@g w`Wwv blNn @r~hlt °y`.

Mother and I went to hospital to see my father.

Present Continuous

Past Continuous

Future Continuous

Present Perfect

Past Perfect

Future Perfect

Present Perfect Continuous

Past Perfect Continuous

Future Perfect Continuous

Testing of the Sinhala to English translation handles eight out of the twenty regulations for the active voice in Fig. 6. And here also the system is not covered all the thirty-two grammatical rules of Sinhala Language. Thirty two regulations

Twenty rules (Active Voice)

41

Twelve rules (Passive Voice)

8.

@m`nvq obt @b`Nnt av|&y?

What do you want to drink? 9.

oÒ iw`mWm nrk ]m@yK.

He is a very bad boy. 10.

mm @ht p`slt yN@n n#h#.

I will not go to school tomorrow. A. Future Research Directions The presented system can be improved in following ways. • This system can be improved in a way which gives the portability. • Can get modified and used to do online chats using Sinhala and English. • Can use to help disables to learn languages by adding new suitable features.

Fig.6.- Grammatical regulations in Sinhala Language

Table V contains test results and some example sentence patterns that were correctly translated by the system. In the first sentence contains two verbs with the word “to”. Next four sentences are examples of an interrogative sentence (W/H question) that is in present tense, an interrogative sentence (yes/no question) that is in present tense, a sentence that is in negative form, and an interrogative sentence (W/H question) that is in past tense, respectively.

The system cannot get accurate translations for sentences with verbs such as “a#w” and “âè”, wishes proverbs Interrogative sentences orders. These are the areas that we hope to look into in the future. We have already begun work in some of them. In addition, more focus has to be placed on the passive voice (fourth sentence in Table V), as the current translator is mainly focused on the active voice sentences in the Sinhala language. By increasing the number of words in the dictionary, users will have the benefit of being able to get more meanings and example sentences in Sinhala and English. The speed of the

SAMARAWEERA ET AL: ENGLISH SINHALA MACHINE TRANSLATION – PSRS; VOL. 3, DEC 2009

grammar checker tool can also be optimized. The grammar tool can be further developed in a way that it would enable a foreigner to study Sinhala grammar in the English medium. B. Difficulties There are some difficulties in the recognition and the translation process. The system cannot recognize words “bird’s” and “birds” separately. And also it cannot recognize “red” and “read” (past tense) separately (1 and 2 in Table V). Same phonemes are used to pronounce these words. In third sentence of Table V, the system cannot recognize “,” and it is a major difficulty to translate the subject with more than one noun. Fifth sample sentence in Table V is grammatically correct. The word “°@yË” is not translated by the system. TABLE V SOME EXAMPLES FOR INACCURATE TRANSLATION

Inaccurately recognized sentences / words 1. bird’s / birds red / read

3.

Father, you and I go to town.

Inaccurately translated sentences / words 4. Fish is eaten by dog. 5.

Translator) unique is that it combines six major sections in the communication process which mentioned earlier in the methodology. This system will help users to convert English and Sinhala words or sentences to Sinhala and English languages easily and to facilitate their learning. ACKNOWLEDGMENT The authors wish to acknowledge the parents, friends and co-workers who provided us support in various ways throughout the preparation of this research paper and also the authors wish to thank the anonymous reviewers for their valuable suggestions. REFERENCES [1]

[2]

[3] [4]

2.

aÓ p`sL °@yË.

[5]

[6]

C. Suggestions The system is designed to work in an indoor environment with an average amount of noise. The presence of background noise to a large extent might disturb either the Training or the recognition process. Further the use of a microphone which is capable of capturing sounds within a limited area will help to enhance the speech recognizer. If it does not capture the background noise, there won't be a hindrance to the phases. But the background noise should be minimal in order for the system to work properly. Since the HTK is used as the foundation of the voice recognition engine, design is restricted to the architecture of the HTK. But the design is done in a manner in which it will be compliant with future versions of HTK. The system cannot work for all available phonemes because it will create a large number of utterances for users to utter and therefore is going to be a tedious task. Since the numbers of phonemes are limited it is difficult to generate meaningful sample utterances for the users. Researches those who are going to do researches based on the DLT can use SDK instead of using HTK for the English speech recognizer as SDK gives higher accuracy compared to HTK. IV. CONCLUSION This system proposes a way of how technology based learning can enhance the quality of the current learning system for students who seek to understand English and Sinhala languages. What makes the DLT (English – Sinhala – English

42

[7]

[8]

H. W. John and H. L. Somers. "An Introduction to Machine Translation”, London: Academic Press.1992,ISBN 0-12-362830-X. Available: http://www.hutchinsweb.me.uk/IntroMT-TOC.htm. J. Holmes and W. Holmes. “Speech Synthesis and Recognition”, 2nd Edition, CRC: 2001.ISBN 0748408568. The HMM-based Speech Synthesis System. Available: http://hts.sp.nitech.ac.jp. S. Young, et al., “The HTK Book”, HTK Version 3.2.1. R. Murphy. “English grammar book Intermediate English grammar”, published by arrangement with Cambridge University press. D.I. De Silva, P.K. Alahakoon, P.V.I. Udayangani, D. Kolonnage, M.H.P. Perera, and S. Thelijjagoda. “Application of Transfer Based Machine Translation from Sinhala to English”. In Proceedings of the 4th SLIIT Research Symposium, 2008, vol. 2, pp. 33–36. T. Dutoit, “An Introduction to Text-to-Speech Synthesis”. Kluwer Academic Publishers, Dordrecht, Hardbound, ISBN 0-7923-4498-7, April 1997, vol. 3, 312 pp. The MBROLA PROJECT Synthesizing, Technology. Available: http://tcts.fpms.ac.be/synthesis/mbrola.html, [accessed: March. 17, 2009]. M. Chitnis, P. Tiwari, and L. Ananthamurthy. “Developer”. Internet: http://www.developer.com/design/article.php/3080941/, [Accessed: March. 20, 2009]. J. Allen, M. S. Hunnicutt, and D. Klatt, “From Text to Speech: The MITalk system”. Cambridge University Press: 1987. ISBN 0521306418.