Audio files on a UIP

This is a simple overview about one aspect of the UIP. The purpose is to transfer some knowledge inside L2 team in order to better handle support cases.

UIP documentation and L3 support will help to get a deeper knowledge.

Audio files on a UIP

This overview is about : UIP data Referential data & synchronisation Configuration file on UIP Configuration file on referential server
Former USMS Referential audio files mapper.cfg

sync

sync

sync

UIP1
audio files links uip.cfg

UIP2
audio files links uip.cfg

UIP3
audio files links uip.cfg

Audio files & config. on UIP

On the UIP itself Two subdirectories on the UIP containing
real audio files copied from a referential
( $MESSAGES/APP_8/UIP_audio)

Unix links related to each audio file

( $MESSAGES/APP_8/GM_00 )

A configuration file uip.cfg containing the information about the referential

Configuration file uip.cfg

[prompt_reference] sync_on_start=yes host=du-p-newsms-gui directory=/global/IN/UIP/ local_cmd=/opt/FERMA/sbin/uip_generate_prompts remote_cmd=null protocol=ssh

Configuration file uip.cfg

[prompt_reference]
Section defining the vocal prompts reference repository

sync_on_start=yes
If "yes", the prompts repository of the UIP will be synchronised when starting UIP (uip start). If "no", only the command "uip sync" will synchronise the prompts repository

Configuration file uip.cfg

host=du-p-newsms-gui
The host name or IP address of the reference platform which hosts the vocal prompts reference repository

directory=/global/IN/UIP/
The path to the prompts repository on the reference platform

Configuration file uip.cfg

local_cmd=/opt/FERMA/sbin/uip_generate_prompts
The name of a command to be run on that machine if, at least, one file was transferred/modified/deleted.

remote_cmd=null
The name of a command to be run on the remote machine if, at least, one file was transferred/modified/deleted.

protocol=ssh
The protocol to be used to transfer the files. Possible values are : ssh or rsh.

Audio files repository

On the referential Formerly hosted on a USMS, could be on the same UIP or on another one A directory containing the whole list of audio files, managed by the customer A configuration file mapper.cfg defining the structure of the referential

Media base & clips

Some understanding of the media base structure is needed to use the service, to fill or update the mapper.cfg file. This database is made of clips. A clip is an elementary vocal element materialized by an audio file (typically a WAV file).

Media base & clips

Each clip is identified by
a type, a language a message number.

The UIP media base distinguishes 2 types of clips Synthesis clips and Service clips.

Media base & clips

Synthesis clips Used by the synthesis engine to build and play the variable part of announcements.
Example : if the SCP asks the UIP to synthesize the number 1001 in English, the synthesis engine will play the message containing the number one in English, then the message containing thousand in English and finally again the message containing one. The identifiers of synthesis clip are determined by the synthesis engine.
NB: the Id of a message has no relation with their content. The message playing one in English can have the Id 1001, the message playing one in French can have the Id 3001etc

Media base & clips

Service clips
Represent the specific messages played for the service. Example : UIP is asked to play the prompt #2003. UIP determines that this prompt is made of 2 clips : Sorry but you have exceeded the maximum number of selections allowed and Please try again. Thank you. so the caller will hear the message Sorry but you have exceeded the maximum number of selections allowed. Please try again. Thank you.

Configuration file mapper.cfg

The announcements and clips descriptions are stored in a text file named mapper.cfg. This file provides :
The name of the audio file and the description of each clip The list of clips contained in each prompt.

Three types in mapper.cfg

Each line corresponds to a clip or an announcement (see 3rd column) :
2003 : 0 : announce : ar-AE-sorry.wav , ar-AE-du-tryagain.wav ; 3 : 0 : service : ar-AE-sorry.wav, ?"ar-Sorry but you have exceeded the maximum number of selections allowed"; 4 : 0 : service : ar-AE-du-try-again.wav, ?"ar-Please try again. Thank you."; 1001 : 0 : synthesis : ar-AE-1001.wav , ?"ar-one" ;

Type announcement
When the line defines an announcement ( 3rd column = announce), the line format is :
announcement_id : language_id : announce : file1 [, file2, file 3,, file 10] ; announcement_id is the announcement number language_id is the language number fileX is either a service clip file name or the star character * meaning that a variable part is expected but also a VXML URL

Type service clip

When the line defines a service clip ( 3rd column = service), the line format is :
clip_id : language_id : service : file, ? content ; clip_id is a UIP internal, integer identifier for the clip. This is a unique integer whose range is [1 .. 3999]. language_id is the language number. file is the service clip file name. content is the clip content to be displayed in traces.

Type synthesis clip

When the line defines a synthesis clip (3rd column = synthesis), the line format is :
clip_id : language_id : synthesis : file, ? content ; clip_id is a UIP internal, integer identifier for the clip. This depends on the language to be synthetised. language_id is the language number. file is the synthesis clip file name. content is the clip content to be displayed in traces.

Synchronization
The purpose is to update audio files on the UIP with the audio files on the referential. The synchronization or media base deployment - is started from the UIP via uip sync while the application is running. To minimize the deployment, the UIP uses a cache to prevent from deploying an already deployed clip.

Full synchronization
To force the deployment of the whole content, a full synchronization can be done
Purge the cache file: >/opt/ESG/resources/AudioTrt.idx Purge the deployed files: /bin/rm -rf $MESSAGES/APP_8
NB: When using MNVO capabilities, you have to remove all directories $MESSAGES/APP_X where X is an operator number.

Force a redeployment: /opt/ESG/sbin/uip_generate_prompts

Location of files on the UIP

Connect on a UIP and go under directory $MESSAGES.
Usually : /export/home/lsav/serveur/CHANSONS

You will find subdirectory APP_8. For (rare) customers using MVNO, you will find several subdirectories APP_*

Files & links on the UIP

Under $MESSAGES/APP_8, you will find the two subdirectories: UIP_audio it contains all the real audio files, the same than on the referential GM_00 it contains a hierarchical structure of directories hosting links to the audio files

Files & links on the UIP

The relation between the name of the Unix link and the audio file is made by an algorithm described in UIP-IP Media Base 4.0 Technical Guide (chapter Understanding Media Base Deployment) . It uses the entry in mapper.cfg. Basically,
for a service the algorithm is:
(Language Id) x 10000 + 6000 + (Prompt Id) then a convertion in hexa

for an announce or a composed service the algorithm is:

(Language Id) x 10000 + (Prompt Id) then a convertion in hexa

There is an example in KA UIP : understand the mapper.cfg and the related trace when a prompt is played

Files & links : a live example

Lets try to locate an audio file called ar-AE-du6091.* This is only an information about the name of the audio file.

We will find the file itself in the repository, find its Id, then will check it exists on the UIP, on UIP_audio (the same audio file) and on GM_00 (the link pointing to the audio file).

Files & links : a live example

In the directory (on referential server)
Check the existence of the file, its size & date. lsav@du-p-sms06> ls -l ar-AE-du-6091.* ... 35414 Feb 5 2010 ar-AE-du-6091.wav

Files & links : a live example

In the mapper.cfg (on referential server)
See below that this file is used as a service clip, several times, for the same language (Id 0). One service is composed of two audio files.
lsav@du-p-sms06> fgrep 'ar-AE-du-6091' mapper.cfg 2075 : 0 : service : ar-AE-du-6091.wav, ar-AE-du-8074.wav; 2156 : 0 : service : ar-AE-du-6091.wav; 2155 : 0 : service : ar-AE-du-6091.wav; 91 : 0 : service : ar-AE-du-6091.wav , ? "ar- Sorry. the card number you have entered is incorrect. ;

Files & links : a live example

In the subdirectory UIP_audio (on the UIP)
The extension and the size can change (.wav on the referential, .MSG on the UIP, size is a little bit different to a convertion from .wav to .MSG).
lsav@uip1> cd $MESSAGES/APP_8/UIP_audio lsav@uip1> ls -l ar-AE-du-6091* -rwxr-xr-- 1 lsav calapp 33306 Feb 14 2010 ar-AE-du6091.MSG

Files & links : a live example

In the subdirectory GM_00 (on the UIP)
We can see the three links pointing to our audio file corresponding to the three simple service. The complex service composed of two prompts is not associated our single audio file.
lsav@uip1> cd $MESSAGES/APP_8/GM_00 lsav@uip1> ls -l */* |fgrep 'ar-AE-du-6091' 68 Oct 22 01:15 GM_17/A8M0017CB.MSG -> /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/ar-AE-du-6091.MSG 68 Oct 22 01:19 GM_1F/A8M001FDB.MSG -> /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/ar-AE-du-6091.MSG 68 Oct 22 01:19 GM_1F/A8M001FDC.MSG -> /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/ar-AE-du-6091.MSG

Files & links : a live example

In the subdirectory GM_00 (on the UIP)
We could have retrieved the hexadecimal part used for the name of the links by applying the algorithm. For a simple service, we should calculate (Language Id) x 10000 + 6000 + (Prompt Id) and convert it in hexadecimal. Prompt Id 2155 -> 0 + 6000 + 2155 = 8155 -> 1FDB in hexadecimal Prompt Id 2156 -> 0 + 6000 + 2156 = 8156 -> 1FDC in hexadecimal Prompt Id 91 -> 0 + 6000 + 91 = 8155 -> 17CB in hexadecimal

For a complex service, we should calculate (Language Id) x 10000 + (Prompt Id) and convert it in hexadecimal. Prompt Id 2075 -> 0 + 2075 = 2075 -> 081B in hexadecimal

Files & links : a live example

In the subdirectory GM_00 (on the UIP)
This way, with the hexadecimal codes (1FDB, 1FDC, 17CB, 081B) we can see the four links (even the complex one).
lsav@uip1> ls -l */* |egrep '1FDB|1FDC|17CB|081B' Oct 22 01:19 GM_08/A8M00081B.MSG -> /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/ar-AE-du-2075.MSG Oct 22 01:15 GM_17/A8M0017CB.MSG -> /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/ar-AE-du-6091.MSG Oct 22 01:19 GM_1F/A8M001FDB.MSG -> /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/ar-AE-du-6091.MSG Oct 22 01:19 GM_1F/A8M001FDC.MSG -> /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/ar-AE-du-6091.MSG

We can see that when the complex service is asked, it is the audio file ar-AE-du-2075 which is played

Files & links : SVI_maptools.log

As we are not always found of hexadecimal conversion, there is another way to know the audio file pointed by each link.
See SVI_maptools.log ( in standard logs directory) generated during the synchronization. For each prompt Id, it shows the associated link.

Files & links : SVI_maptools.log

Example for prompt Id 2075 and prompt Id 2155:

lsav@uip1> cd /var/opt/FERMA/logs lsav@uip1> fgrep 2075 SVI_maptools.log MsgId[2075] Lg[0][synthesis] link /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/ar-AE-du-2075.MSG-> /export/home/lsav/serveur/CHANSONS/APP_8//GM_00/GM_08/A8M00081B.MSG lsav@uip1> fgrep 2155 SVI_maptools.log MsgId[2155] Lg[2][service] link /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/hi-AE-du-26091.MSG-> /export/home/lsav/serveur/CHANSONS/APP_8//GM_00/GM_6D/A8M006DFB.MSG MsgId[2155] Lg[1][service] link /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/en-AE-du-16091.MSG-> /export/home/lsav/serveur/CHANSONS/APP_8//GM_00/GM_46/A8M0046EB.MSG MsgId[2155] Lg[0][service] link /export/home/lsav/serveur/CHANSONS/APP_8/UIP_audio/ar-AE-du-6091.MSG-> /export/home/lsav/serveur/CHANSONS/APP_8//GM_00/GM_1F/A8M001FDB.MSG lsav@uip1>

Audio files format

The recording studios produce stereo, 48kHz, 16bits clips, not suitable for telecom network. To convert the audio files (UIPv4):
If the encoding law is a, then perform: /opt/ESG/bin/MEDIA_audio2wav -f alaw -i <input file> -o <output file> If the encoding law is u, then perform: /opt/ESG/bin/MEDIA_audio2wav -f ulaw -i <input file> -o <output file>

Audio files format

To convert the audio files (UIPv2):
SVI_audio -i $name -o $name -f ALAW
To get info: SVI_audio -i $name -w