WWW.ANITE.

COM/NEMO

NEMO VOICE QUALITY GUIDE

DOCUMENT NUMBER
NNNNNNNNN ISSUE N

© Anite 2016
 NEMO VOICE QUALITY GUIDE

Copyright
The information contained in this document is confidential and no part of it may be copied or
reproduced in any form without the written consent of Anite Network Testing. Additionally, this
document is not to be passed to or discussed with third parties without the prior written permission
of Anite Network Testing.

Anite’s products are subject to continual development and specifications may change. Users of
Anite’s products and documentation should exercise their own independent judgement to evaluate
the suitability of Anite’s products and documentation for their particular use. Anite does not accept
any liability arising from the application or use of the product or this documentation.

All reasonable care has been made to ensure that this document is accurate. If you have any
comments on this document, or would like details of any Anite products, services or equipment,
please contact us through the Anite website.

Page 2
 NEMO VOICE QUALITY GUIDE

CONTENTS

1 NEMO VOICE QUALITY GUIDE .............................................................................. 4

2 VOICE QUALITY SYSTEM DESCRIPTION ................................................................ 5

2.1 Mobile-to-fixed line..................................................................................... 5
2.2 Mobile-to-mobile ........................................................................................ 5
2.3 Nemo Server remote configuration ............................................................... 5
2.3.1 Nemo Server settings ......................................................................... 6
2.4 Mute call detection ..................................................................................... 8

3 NEMO OUTDOOR VOICE QUALITY – NEMO MEDIA ROUTER .................................... 11

3.1 Setting up the system ............................................................................... 11

4 MAKING VOICE QUALITY MEASUREMENTS .......................................................... 15

4.1 Simultaneous PESQ and POLQA measurements ............................................ 16
4.2 Mobile-to-mobile VQ measurements ........................................................... 17
4.3 Mobile-to-fixed-to-mobile VQ measurements ............................................... 22
4.4 End-to-end call instructions ....................................................................... 25
4.4.1 Nemo Outdoor script settings............................................................. 25
4.4.2 Nemo Server configuration ................................................................ 28
4.4.3 Starting a measurement and running a script ...................................... 28
4.4.4 Configuring manual end-to-end voice calls .......................................... 28

Page 3
 NEMO VOICE QUALITY GUIDE

1 NEMO VOICE QUALITY GUIDE

Nemo Voice Quality is an option for the Nemo Outdoor measurement system. Nemo Outdoor
supports mobile-to-mobile and mobile-to-fixed-to-mobile measurements. The measurements enable
the assessment of the quality of voice during voice calls as perceived by end-users. The
measurement is based on the ITU-T recommendations P.862.1, P862.2, and P863 and it produces a
voice quality score that is mapped on an EMOS (Estimated Mean Opinion Score) scale. Also
measurements based on the WB-AMR codec are supported. Nemo Outdoor supports voice quality
measurements with Nemo Media Router running on Android-based smartphones.

The Nemo Voice Quality system with the Nemo Media Router consists of Nemo Outdoor, 1-6
Android-based smartphones, and optionally Nemo Outdoor Multi. It is supported with the Nemo
Outdoor platform, and it supports the USB 2.0 audio system, four channels, and the GSM, WCDMA,
CDMA2000, and TD-SCDMA technologies. For a list of supported terminals, please see the Voice
Quality data sheet.

It is possible to perform two types of voice quality measurements: mobile-to-mobile and fixed-to-
mobile measurements.

In mobile-to-mobile measurements, one terminal is calling another and the mobiles send a test
sample. These measurements present the combination of the uplink quality of the transmitting
terminal and the downlink quality of the receiving terminal.

In mobile-to-fixed-to-mobile measurements, a terminal calls a fixed number (PSTN) that is

connected to a Nemo Server with the Audiotest option. These measurements present the downlink
quality of the terminal at the Nemo Outdoor end, the uplink quality of the terminal at the server
end, and the downlink quality of the receiving terminal.

Page 4
 NEMO VOICE QUALITY GUIDE

2 VOICE QUALITY SYSTEM

DESCRIPTION

2.1 Mobile-to-fixed line

When Nemo Outdoor calls a fixed line that is connected to Nemo Server, the server answers the call
and Nemo Outdoor starts sending the configured test sample. The server is already configured to
use the same sample. After the first sample is received, the server sends in turn the reference
sample to Nemo Outdoor that records it and calculates the voice quality score (MOS DL). Server
side calculates scores for received samples in parallel to the testing procedure, and opens a
measurement file and stores the result there.

Along with the test samples, short synchronization tones are sent. They are used for
synchronization purposes and do not affect the score. The synchronization tone is always sent from
the side which sends the audio sample. For example, with mobile-to-fixed-line testing
synchronization tone is sent first from the mobile to the server.

After receiving the test sample, Nemo Outdoor sends again the reference sample to the server, the
server records it and calculates the score (MOS UL), and so on until the call is disconnected.

The measurement file on server side is closed when the call is disconnected and FilecloseTimeout
seconds is elapsed from disconnection (default 200s). The test goes on alternating until a call
drops, or until Nemo Outdoor or Nemo Server ends the call. If the server does not detect the call
disconnection, for example, it cannot detect the busy tone because the country code is wrong, the
server does not hang up the line. In this case, use NSM to stop the VQD service and change the
country code. If it does not help, consult the Nemo Server manual for further instructions.

2.2 Mobile-to-mobile
The same logic is used also in mobile-to-mobile testing as with mobile-to-fixed line.

♦ PSTN lines of Nemo Server must be calibrated before use.

♦ Busy call detection wizard of Nemo Server must be done before use.

2.3 Nemo Server remote configuration

Nemo Server remote control allows field units to issue commands to Nemo Server and to configure
and query a subset of Nemo Server settings before and during measurements. Real-time
commands are needed to match the call event count of the server and the field in abnormal
situations, such as, when a call does not connect or the call goes to voicemail, provided that data
connection is successful in the first place.

Communication between a field unit and Nemo Server is done over a packet data connection using
the Nemo Server API interface (HTTP API). Internet access is required for Nemo Server. Servers
without internet access use the existing method (DTMF) to order voice call callback. As opposed to

Page 5
 NEMO VOICE QUALITY GUIDE

DTMF, a packet data connection does not require a preliminary call to the server to send the
predefined call sequence information before an actual measurement session. Enabling the Nemo
Server connection via a packet connection also makes it possible to start a measurement session
with an MTC call where the Nemo Server initiates a call which is answered by the field unit. All voice
quality call related settings from Nemo Outdoor are synchronized with Nemo Server. Packet
connection is the preferred method for call synchronization between the field unit and Nemo Server.
Automatic callback option under the channel definitions on Nemo Server should be off.

2.3.1 Nemo Server settings

The following Nemo Server related settings can be configured via Nemo Outdoor user interface:

Number defines the server phone number (line number/ B-party phone number)

Own number defines the A-party phone number.

Page 6
 NEMO VOICE QUALITY GUIDE

Call reception timeout defines the max. time to receive the incoming call. If call is not received
within the timeout, call attempt failure with the cause “Incoming call not received” is written to the
log file.

Enable Nemo Server connection opens the HTTPS Options dialog where you can define
connection settings.

Global options like CountryCode and FileCloseTimeout cannot be controlled at all as they are
adjusted once when the Nemo Server is installed and there is no reason to touch them afterwards
unless server is moved or lines changed or so, that require new installation anyway by local user.

Only one channel can be controlled and/or configured with single request as there is only one
RemoteNumber parameter. Supplied channel parameters are permanent until changed by new
request.

♦ Nemo Server version 4.40 or later is required.

The following example script file can be used to start voice quality testing between field unit and
fixed end initiated by fixed end (Nemo Server).

Script must be started by adding the Receive call command specifying call duration, call reception
timeout value and end to end call control settings including own number and server line number
what will be used to execute the callback to field unit. Check that callback setting for particular
channel on Nemo server is set to off. This must be done via Nemo Server user interface. Continue
to build the script by adding the make voice call command and specify all required settings for voice
quality call and also end to end call control settings.

Page 7
 NEMO VOICE QUALITY GUIDE

Figure 1. Voice quality call script initiated with MTC.

Figure 2. Nemo Server channel definitions. Callback.

2.4 Mute call detection

Sometimes end users are experiencing random audio muting during phone calls. When a call is
answered, only silence can be heard.

When audio is lost during a phone call, usually it remains silent until the end of the call. These calls
can be seen also by observing the audio quality MOS score as it drops significantly during silent
calls. The mute call detection functionality in Nemo Outdoor makes it easy to analyze phone calls
where low MOS score is caused by a silent or muted call.

A parameter called “Audio reception state” is displayed real-time and it indicates if audio is present
or not. During RX mode the voice quality measurement system is monitoring the energy of the
received signal continuously for each received audio frame and write audio quality signal status to
the measurement file. If the energy of the received signal drops below specified threshold for
specified amount of time, the audio reception state is set to “Not detected”. If the energy of the
received signal goes above specified threshold for specified amount of time, the audio reception
state is set to “Detected”. Threshold values are fixed.

Audio reception state can be seen in different views e.g. with line graph. Detection starts when
phone audio mode is in reception (RX) mode. Synchronization tone can be seen at the beginning of
the RX mode followed by the actual received audio sample file. Depending on the reference sample

Page 8
 NEMO VOICE QUALITY GUIDE

file it is normal that during the reception of the received sample file, audio is not necessarily
detected all the time. E.g. the case sample file includes a pause in audio longer than threshold
value used for the detection.

Figure 3. Audio RX mode. Normal situation.

Audio lost

Figure 4. Silent call. Audio not detected.

Audio lost.
Muted call

Figure 5. Muted call. Audio lost during call. DL MOS scores seen as low as 1.49MOS.

Page 9
 NEMO VOICE QUALITY GUIDE

Figure 6. Audio gap. Lost audio frames during RX mode.

Page 10
 NEMO VOICE QUALITY GUIDE

3 NEMO OUTDOOR VOICE QUALITY –

NEMO MEDIA ROUTER

This chapter will describe the steps for setting up and configuring the voice quality system using
Nemo Media Router. Handsets ordered with Nemo Media Route come preinstalled from Anite
Network Testing. If you wish to use your existing Nemo Outdoor test terminals, please install the
Nemo Media Router first.

3.1 Setting up the system

Requirements:

• Nemo Handy firmware

1. Next install the Nemo Media Router on the mobile (if not preinstalled by Anite Finland).
Connect the device to your laptop with a USB cable. Copy the NemoMediaRouter.apk file on
your device in the Nemo folder.

2. Select Apps | My files on your device and browse to the Nemo folder. Tap the
NemoMediaRouter.apk file.

Page 11
 NEMO VOICE QUALITY GUIDE

3. Select Install. When the installation is finished, select Open.

4. After the Nemo Media Router application has started, restart the device.

5. When the device has restarted, connect the device with Nemo Outdoor/Nemo Invex as
instructed in the Device Configuration Guide included in the Nemo Outdoor product
package.

6. When the device is connected with Nemo Outdoor/Invex, the Nemo Media Router
application is started automatically. If the Nemo Media Router does not appear, start the
application manually, reboot the device, and try again adding the device to Nemo Outdoor.

Page 12
 NEMO VOICE QUALITY GUIDE

7. The Outdoor option should be selected as the USB configuration. The Mass memory and
Mass memory ADB options are not needed with Nemo Outdoor.

Page 13
 NEMO VOICE QUALITY GUIDE

8. In the Nemo Outdoor Device Configuration – Voice/video quality page, select the Nemo
Media Router option and define which voice quality algorithm you wish you use. Click OK.
The device is now ready.

Page 14
 NEMO VOICE QUALITY GUIDE

4 MAKING VOICE QUALITY

MEASUREMENTS

After software installations and hardware setup you are ready to start the measurements. You will
receive some default test samples along with Nemo Outdoor but if you wish to use your own
samples, please read the requirements below.

Test sample requirements:

• 8kHz, 16kHz or 48kHz sampling rate (narrowband NB, wideband WB and super-wideband
SWB)

• 16-bit linear PCM

• Signal scaled to 90% of full scale

• PESQ-NB: Band-pass filtered with corner frequencies at 250Hz and 3500Hz

• POLQA-NB: Band-pass filtered with corner frequencies at 50Hz and 3800Hz

• PESQ-WB: Band-pass filtered with corner frequencies at 50Hz and 7400Hz

• POLQA-SWB: Band-pass filtered with corner frequencies at 50Hz and 14000Hz

• Minimum/maximum durations for sample: 4s / 10s.

• Speech activity between 60-80%

• Leading and trailing silences: 0.5s

The following samples are provided in the Nemo Outdoor setup package. They can be found in
folder \Nemo Tools\Samples.

• 4s_m.wav

• 8s.wav

• 10s.wav

• AmEnglish_NB_m1s1_f2s2_6s.wav

• AmEnglish_SWB_m1s1_f2s2_6s.wav

• BrEnglish_NB_f1s4_m1s3_6s.wav

• BrEnglish_SWB_f1s4_m1s3_6s.wav

• German_NB_m2s1_f1s1_6s.wav

• German_SWB_m2s1_f1s1_6s.wav

• Italian_NB_f1s2_m1s2_6s.wav

• Italian_SWB_f1s2_m1s2_6s.wav

• Japanese_NB_f1s1_m1s1_6s.wav

Page 15
 NEMO VOICE QUALITY GUIDE

• Japanese_SWB_f1s1_m1s1_6s.wav

• Russian_NB_f2s6_m1s3_6s.wav

• Russian_SWB_f2s6_m1s3_6s.wav

NB-samples can be used in PESQ and POLQA testing. Note that samples that do not have NB or
SWB in the file name are NB samples.

SWB-samples can be used in mobile-to-mobile WB-AMR testing.

4.1 Simultaneous PESQ and POLQA measurements

When analyzing PESQ/POLQA results it is important to know which version, narrowband or
wideband, of the algorithm was used. Although both versions will generate MOS scores on the same
five point scale, it is strictly forbidden to mix wideband (WB) and narrowband (NB) results. If
wideband networks have to be compared to narrowband networks, then the wideband version of
PESQ/POLQA algorithm has to be used in both cases.

In general, the scores achieved in good conditions with POLQA are generally higher than PESQ
scores for the same samples. In addition, the POLQA-WB maximum score has been increased to
4.75MOS (it is still 4.5MOS for NB). As a result POLQA may occasionally score 4.5MOS in good
conditions that will never happen with PESQ. Both algorithms are commonly used at the moment
but it is estimated that when WB, SWB codecs, and VoLTE are widely in use, POLQA will become
commonly adopted by operators. Therefore, side by side live comparison of these two algorithms is
needed during this transition phase. Comparison can be done with Nemo Outdoor where both PESQ
and POLQA MOS scores are calculated simultaneously.

To activate both PESQ and POLQA measurements, go to Device Configuration – Voice/video

quality page and select both options. Note that both PESQ and POLQA voice quality options are
required.

Page 16
 NEMO VOICE QUALITY GUIDE

4.2 Mobile-to-mobile VQ measurements

♦ Note that the mobile must NOT be in silent mode during measurements, as it may impair
sample flow from the terminal.

♦ Remember to switch off the alert and warning tones. Beeping during a measurement will
lower the score.

1. Define the voice quality settings for the first mobile, i.e., the mobile that you are calling
from.

2. Open the Measurement Properties, Script dialog for the first mobile (Measurement |
<device> | Script Properties). Click on the Script Editor button to create a new script.

3. In the Script Editor, click Add and select Voice Call from the menu.

Page 17
 NEMO VOICE QUALITY GUIDE

4. In the Voice Call Properties dialog:

• Define the phone number of the second mobile, i.e., the mobile that you are calling.

• Select Request voice call callback if you want the application to send callback command to
Nemo Server using DTMF tones.

• Select the Enable voice quality option.

• Select one of the default test samples in the Reference sample filename field. Click OK.

Page 18
 NEMO VOICE QUALITY GUIDE

5. In the Script Editor dialog, double-click on the End a voice call item to define whether end
command type is normal or forced. Define how many times you would like to repeat the
script by editing the Repeat Script parameter.

6. Click Save and type a file name for the script. Finally click Close.

7. Define the voice quality settings for the second mobile, i.e., the mobile that you are calling
to.

8. Open the Measurement Properties dialog for the second mobile (Measurement |
<device> | Measurement Properties).

Page 19
 NEMO VOICE QUALITY GUIDE

9. In the Measurement Properties – Measurement page, double-click the Voice Call

item.

Page 20
 NEMO VOICE QUALITY GUIDE

10. In the Voice Call Properties dialog:

• Define the phone number of the first mobile, i.e., the mobile that you are receiving calls
from.

• Select the Enable voice quality option

• Select Request voice call callback if you want the application to send callback command
to Nemo Server using DTMF tones.

• Define a Prefix for the received files. The prefix will be added at the beginning of the file
name of all received files.

• Select one of the default test samples in the Reference sample filename field. Select the
same file as for the other mobile. Click OK.

11. Click the Start Recording button. Click the Start Scripts button to start the
measurement. Nemo Outdoor will run through the script and make the voice quality
measurements.

Page 21
 NEMO VOICE QUALITY GUIDE

4.3 Mobile-to-fixed-to-mobile VQ measurements

♦ Please complete the Nemo Server installation before continuing. See Nemo Server
manual for detailed instructions.

1. Connect the test mobiles to the Nemo Outdoor measurement system.

♦ Note that the mobile must NOT be in silent mode during measurements, as it may impair
sample flow from the terminal.

2. Define the voice quality settings in the Voice Call Properties dialog:

• Select one of the default test samples. Make sure that the server has the same samples in
its reference sample folder (see Nemo Server manual for details).

3. If you are using scripts, click the Start Scripts button to start the measurement. Nemo
Outdoor will run through the script and make the voice quality measurements.

4. If you are performing measurements manually, select Start Voice Call from the
Measurement menu or by clicking the Device Commands button. With each voice call, the
voice quality measurement is run.

5. Define the voice quality settings for the mobile.

• Open the Measurement Properties, Script dialog for the mobile (Measurement | <device> |
Script Properties).

• Click the Script Editor button to create a new script.

6. In the Script Editor, click Add and select Voice Call from the menu.

Page 22
 NEMO VOICE QUALITY GUIDE

7. In the Voice Call Properties dialog:

• Define the phone number of the fixed end, i.e., the server that is answering the calls and
sending back the samples.

• Select the Voice Quality option.

• Select one of the default test samples in the Reference sample filename field. Please
make sure that the same reference file exists also on the server. Click OK.

Page 23
 NEMO VOICE QUALITY GUIDE

8. In the Script Editor dialog, define how many times you would like to repeat the script by
editing the Repeat Script parameter.

9. Click Save and type a file name for the script. Finally click Close.

10. Click the Start Recording button. Click the Start Scripts button to start the
measurement. Nemo Outdoor will run through the script and make the voice quality
measurements.

Page 24
 NEMO VOICE QUALITY GUIDE

4.4 End-to-end call instructions

In the User Interface Properties – Measurement dialog, set option Use ETSI compatible call
connection trigger to Yes. With this option, the call duration will be calculated from Alerting to
Call disconnected. It corresponds better to the call duration seen by the server.

Set option Enable end to end calls when server connection fails to Yes. With this option Nemo
Outdoor will perform the MO call, even if HTTP commands to the server would fail.

4.4.1 Nemo Outdoor script settings

1. When end-to-end call synchronization is used, all the devices must have the same group ID,
which can be set in device script settings as the Script group.
Page 25
 NEMO VOICE QUALITY GUIDE

2. Devices in the same script group must use the same script files. Only the line number and
the own number can be different between scripts on different devices. Other call settings
must be the same. Otherwise, the server will not be able to predict the behavior of the
different terminals in the group correctly.

♦ It is mandatory to set a Wait for script synchronization command before the MO or MT

calls. Always use wait commands between call commands. The minimum wait time
should be 10 sec.

Figure 7. Mobile originated call settings (make a voice call)

Page 26
 NEMO VOICE QUALITY GUIDE

Figure 8. Incoming call setting (receive call)

3. Own number (subscriber telephone number) must be defined in both Make a voice call
and Receive call commands.

4. Server line (Nemo Server phone number) must be defined in the Receive call command.

5. Only end-to-end call settings should be defined. For example, the Request voice call
callback in Voice Call Configuration must be set to No.

Figure 9. Nemo e2e server settings

Page 27
 NEMO VOICE QUALITY GUIDE

6. Nemo Server settings are global settings that are written in the PC registry. This means that
these settings are not transferred within a script file. If the script files are copied from a
Control PC to another user, these settings must be written again.

4.4.2 Nemo Server configuration

1. Channel callback settings must be set to off.

2. Configure all server channels with correct RemoteNumbers and LineNumbers before making
any end-to-end commands.

3. Restart the VQD service after channel configuration is done and each time the configuration
is changed.

4. If measurements are stopped and restarted, wait for 200 seconds between the changes so
that the possibly running server side timers are finished before the new test begins.

5. If grouping is changed so that, for example, one handset is removed from the script group,
change the group name. Otherwise the grouping stays the same on the server side also for
channels that were part of the last group. Alternatively, configure the removed handset with
an empty group and let it command the server once to clear the group from that channel.

6. Make a configuration command (e.g. one test call cycle) on good RF conditions.

♦ NOTE! Nemo Server side grouping stays valid until automatic service restart happens at
3:00 am. A new configuration command executed by the field unit is required on daily
basis. Also when the VQD service is stopped or started or when the Nemo Server is shut
down or rebooted.

4.4.3 Starting a measurement and running a script

It is important, that after starting a measurement, the HTTP commands go through from all the
terminals to the server at least once. This enables the server to synchronize and detect all the
devices belonging to the same group.

♦ When changing from synchronized measurements to non-synchronized or individual

tests, the field user has to wait at least until the server-side parameter FileCloseTimeout
(seconds) has been elapsed from the end of the last call. This is because the server
purges the channel grouping only after the measurement file is closed for all group
channels. By default the FileCloseTimeout is 200 seconds.

4.4.4 Configuring manual end-to-end voice calls

With Nemo Outdoor it is possible to make end-to-end voice call testing between two devices
connected to the same Nemo Outdoor system.

1. Add two phones to Nemo Outdoor: Device A and Device B

2. Open the Voice Call Properties for Device A by double-clicking the Voice Call item in the
Devices view.

Page 28
 NEMO VOICE QUALITY GUIDE

3. Enter Device B phone number in the Number field.

4. Select the Enable end-to-end call option.

5. Select the Local device option and select Device B from the list. Click OK.

11. You can now start making voice calls from Device A to Device B.

Page 29