8 Voice Service Performance Evaluation

8.1 QoS Requirements

Section 6.1.7 in 3GPP TS 23.203 of Release 10 provides the QoS requirements of services
with standardized QCIs. Table 8-1 lists the QoS requirements of QCI 1 services.
Table 8-1 QoS requirements of QCI 1 services
Resource Type Priority Packet Delay
Budget (ms)
Packet Error
Loss Rate
Typical Service
GBR 2 100 10-2 Voice service
l The preceding Packet Delay Budget indicates the threshold delay between UE and PDN
gateway (P-GW). The corresponding user satisfaction rate is 98%.
l The preceding Packet Error Loss Rate indicates the threshold proportion of SDUs that
are not successfully sent from the data link layer of the transmission end to the
corresponding upper layer of the receive end.
The packet delay budget, uplink and downlink Uu interface packet loss rate, and downlink
PDCP packet loss rates of QCI 1 services can be measured using counters. For details about
related counters, see 9.2.4.2 Voice QoS.

8.2 Quality Evaluation

The mean opinion score (MOS) is an important indicator for evaluating voice quality. MOSbased
evaluation involves subjective evaluation, objective evaluation, and measurement-based
evaluation.
8.2.1 Subjective Evaluation
MOS is a subjective measurement where listeners perceive raw speech materials and
processed degraded speech materials and then score speech quality.
Table 8-2 lists the MOS standards defined in ITU-T G.107.
eRAN
VoLTE Feature Parameter Description 8 Voice Service Performance Evaluation
Issue 03 (2015-06-30) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
64
Table 8-2 MOS standards
MO
S
Quality Level Auditory Distortion Extent Required Auditory
Effort
5 Excellent Distortion not noticed Listening in a relaxed
manner
4 Good Distortion noticed, but not
unpleasant
Listening with a little effort
3 Fair Distortion noticed, and
annoying
Listening in an attentive
manner
2 Poor Unpleasant but not annoying Listen with much effort
1 Bad Unpleasant and annoying Cannot understand the
contents
8.2.2 Objective Evaluation
Perceptual Evaluation of Speech Quality (PESQ), defined in ITU-T P862, is a mainstream
method of objectively evaluating AMR-NB speech quality. Perceptual Objective Listening
Quality Assessment (POLQA), defined in ITU-T P863, is the evolution of PESQ; POLQA
supports wider sampling scope and provides more accurate evaluation of AMR-WB or AMRNB
speech quality. PESQ and POLQA are based on special tools, for example, Digital Speech
Level Analyzer (DSLA).
Objective evaluation is commonly used by operators to evaluate speech quality. However, it
requires the collaboration of drive tests. As it cannot monitor speech quality in real time, it is
generally used for tests in labs, network entry tests, and third-party evaluation of network
quality.
8.2.3 Measurement-based Evaluation
Overview
Using the third-party's drive test tools to evaluate speech quality is time and cost consuming
and cannot monitor speech quality in real time. Therefore, it is important for vendors to
develop their own measurement-based evaluation methods. Huawei uses a Voice Quality
Monitoring (VQM) algorithm to calculate the MOS scores of uplink and downlink voice
services on the Uu interface.
VQM is mainly used for network monitoring, network optimization, VIP guarantee, and user
complaint handling. VQM reduces the necessity of drive tests required for obtaining voice
quality.
VQM applies only to the AMR speech codec scenario. VQM is controlled by the
EnodebAlgoSwitch.VQMAlgoSwitch parameter, which is turned off by default. After this
parameter is turned on, the eNodeB automatically identifies whether the eNodeB supports
AMR speech codec. The eNodeB calculates only the MOS of voice services that use the
AMR speech codec scheme.
eRAN
VoLTE Feature Parameter Description 8 Voice Service Performance Evaluation
Issue 03 (2015-06-30) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
65
VQM Principles
Figure 8-1 shows the VQM implementation process.
Figure 8-1 VQM implementation process
1. The eNodeB monitors counters for voice packets of QCI of 1, including the frame error
rate (FER) on the Uu interface, long frame error rate (LFER), and handover state. When
detecting that the delay variation for voice packets of QCI of 1 exceeds the
VQMALGO.ULDelayJitter parameter value, the eNodeB determines that a packet loss
occurs.
2. Then, the eNodeB inputs the monitoring results to the Huawei proprietary voice quality
indicator (VQI) model and estimates the MOS scores of uplink and downlink voice
services on the Uu interface at intervals of 2.5 seconds. The VQI model is based on the
algorithms specified in ITU-T P.863 and simulates an MOS score using mathematical
formulas.
– The eNodeB separately simulates the MOS scores of uplink and downlink voice
packets.
– The eNodeB separately simulates the MOS scores of voice packets coded at
different rates.
3. The MOS scores are saved in call history records (CHRs) and are used to collect the
statistics of cell-level voice quality counters and monitor user-level performance.
NOTE
The voice quality monitoring results including statistics about cell-level voice quality counters and
user-level performance monitoring results as well as CHRs do not include user privacy
information.
eRAN
VoLTE Feature Parameter Description 8 Voice Service Performance Evaluation
Issue 03 (2015-06-30) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
66
Voice Quality Evaluation
The eNodeB evaluates voice quality based on the MOS provided by the VQM and voice
quality thresholds. In addition, the eNodeB collects the statistics of counters related to voice
quality, as listed in Table 8-3.
Table 8-3 Voice quality evaluation rules and related counters
MOS Scores Provided by the VQI Model Voice Quality
Level
Related Counters
MOS > VQMAlgo.VqiExcellentThd Excellent For details, see Table
9-8.
VQMAlgo.VqiGoodThd < MOS ≤
VQMAlgo.VqiExcellentThd
Good For details, see Table
9-9.
VQMAlgo.VqiPoorThd < MOS ≤
VQMAlgo.VqiGoodThd
Fair For details, see Table
9-10.
VQMAlgo.VqiBadThd < MOS ≤
VQMAlgo.VqiPoorThd
Poor For details, see Table
9-11.
MOS ≤ VQMAlgo.VqiBadThd Bad For details, see Table
9-12.
Application Limitations
AMR coding rate is an important input for the VQI model. Currently, the VQM algorithm
cannot monitor the AMR coding rate in real time. If the AMR coding rate changed, the output
of the VQI model would be inaccurate. Up to now, it has not been found that the AMR coding
rate changes in any scenario.
In the following scenarios, the eNodeB cannot determine the AMR coding rate and therefore
does not use the VQI model.
l One RTP packet carries multiple AMR speech frames.
l One RTP packet carries multiple speech frames that are coded at different rates.
The IP headers need to be removed when the eNodeB identifies the coding rate. The IP
header length is different after the encryption. The IP header of a voice packet may
include an Authentication Header (AH) or (Encapsulation Security Payload (ESP) header
for encryption. The length of AHs is not fixed, and the latter part of ESP headers is
encrypted, and the latter part of ESP headers is encrypted.

8.3 Capacity Evaluation

The voice service capacity can be evaluated based on the number of users performing QCI 1
services. The related counters are listed below and described in 9.2.4.4 Voice Capacity.
l L.Traffic.ActiveUser.UL.QCI.1
l L.Traffic.ActiveUser.DL.QCI.1
eRAN
VoLTE Feature Parameter Description 8 Voice Service Performance Evaluation
Issue 03 (2015-06-30) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
67

8.4 Performance Evaluation

The performance of voice services can be evaluated using the following KPIs, which are
described in 9.2.4.1 Voice KPIs
l E-RAB Setup Success Rate (VoIP)
l Call Drop Rate (VoIP)
eRAN
VoLTE Feature Parameter Description 8 Voice Service Performance Evaluation
Issue 03 (2015-06-30) Huawei Proprietary and Confidential
Copyright © Huawei Technologies Co., Ltd.
68