Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
KPI
GSM-to-UMTS Training Series_V1.0
Change History
Date
Revision Version
Description
Author
2008-12-13
1.0
Draft completed.
Zang Liang
2009-01-08
1.1
Guo Hao
Page 2
Objectives
statistics analysis
Simple signaling analysis
Page 3
GSM KPI
Coverage rate
Coverage rate
SHO rate
None
Delay
Delay
Page 4
Contents
Introduction to KPI
Test Method of KPI
RAN KPI Counters
Page 5
Introduction to KPI
Definition of KPI
Key Performance Indicator
Management of KPI
RNC-level, cell-level, and customized level
Function of KPI
Define the network performance counters
Monitor the network performance
Locate and rectify the network faults
Analyze the network trend
Purpose of KPI Test
The verification and enhancement of the network optimization.
That is, evaluate the network performance through the network
KPI test, solve the network problem, continue to optimize the
network to achieve the objective of building a refined network.
Page 6
Introduction
to KPI
End-to-End Network Performance Evaluation System
Other PLMN
Core Network
Service Network
Radio/Transmissi
User
Service
Core Network
on
Equipmen
Network
Performance
Network
t
Performance
Performance
Performance
End-to-End Service Performance
PSTN
Internet
External
Network
Performance
Page 7
Introduction to
KPI
Quality of Experience
(QoE)
Availability
Key Quality
Indicator (KQI)
Key
Performance
Indicator
(KPI)
Accuracy
Speed
Retain
ability
Integrit
y
Mobilit
y
Page 8
Coverage
Availabilit
y
Introduction to KPI
Customer:
As a customer, I do not care about
KPIs, only in the hope of obtaining
good and rich services.
Operator:
Our ultimate purpose is to meet increasing
requirements of network subscribers and
provide wireless networks with perfect user
experience.
As a channel between subscribers and the network, subscribers experience the network
Page 9
Introduction
to KPI
Huawei RAN KPIs
Coverage AccessabilityRetainability Service Integrity Mobility
Coverage
rate
RRC connection
setup success rate
SHO rate
RAB connection
setup success rate
Call setup
success rate
Pilot
pollution
ratio
PDP activation
success rate
Call completion
rate of the calling
party (voice, video,
and SMS)
Call completion
rate of the called
party (voice, video,
and SMS)
Call drop
rate (voice,
video, and
PS services)
Mean opinion
score (MOS)
Soft handover
success rate
Video call
(synchronization
performance of
voice and image
Intra-frequency
hard handover
success rate
Uplink/downlink
BLER (Voice,
Video Phone)
Average
throughput of PS
services
(64K/128K/384
K/HSPA)
Inter-frequency
hard handover
success rate
Inter-RAT
handover
success rate
Delay
Availability
PDP activation
delay
Admission
rejection rate
Point-to-point
user plane delay
Paging
congestion rate
Ping delay
Downlink
transmission
resource usage
rate
Service
disruption delay
Flow delay
Page 10
Congested cell
rate
Contents
Introduction to KPI
Test Method of KPI
RAN KPI Counters
Page 11
Test Method
Prerequisite
Page 12
DTDrive Test
Classification of
KPI Test Methods
STStatic/Stationary Test
Stat.Statistics
Page 13
Test Method of
KPI
Con
Page 14
Another prerequisite of the KPI of the high-quality network is that all the
KPIs of DT/ST/Stat must be covered and all kinds of counters must be
complementary to each other.
Test Method of
KPI
Network lifecycle
Contract
Requirements
for KPIs
Planning
Implementation
Optimization
Cluster
Optimization by
Field Test
Field Test KPIs KPIs (Drive
Test/
(Drive Test/
Stationary Test) Stationary
Test)
for PAC
Statistic KPIs
for FAC
Page 15
Ongoing
Optimization
Commercial
Launch
PAC
ST:
Call Setup Delay
PDP Context
Activation Delay
Ping Delay
...
PAC
DT:
CDR
CSSR
MOS
...
Network
Expansion
FAC1
Stat.:
RRC/RAB SSR
CDR
SHO SSR
...
Ongoing Optimization
by
FAC2
Stat.:
RRC/RAB SSR
CDR
SHO SSR
...
Contents
Introduction to KPI
Test Method of KPI
RAN KPI Counters
Page 16
KPI Name
Coverage rate
......
......
......
Flow delay
......
......
......
Availability
......
Page 17
Delay
Retainability
Mobility
SHO overhead
Accessibility
KPI Name
SHO rate
Coverage
KPI Type
Remarks:
1. All the KPIs have two modes: Stat. mode and DT/ST mode.
2. According to the services, some KPIs have sub-item.
3. According to the calling part or the called party, some KPIs have
sub-item.
Page 18
Coverage Analysis
KPI (DT/CQT)
Coverage Rate
At the early phase of the network deployment, the
coverage rate is an important factor in the network
planning and optimization.
KPI (stat.)
SHO Rate and SHO Overhead
The impact of the overlapped coverage on the GSM
network is not obvious. For the WCDMA network, however,
it may decrease the network capacity and deteriorate the
coverage quality (pilot pollution).
Page 19
Coverage
Coverage rate
Page 20
Test Method: DT
Coverage rate A1/A2*100%
A1: length of valid test route (km) (CPICH Ec/Io-12dB & RSCP95dBm)
A2: total length of test route
The coverage rate is calculated after the distance binning of 5 m is
performed.
Main Influencing Factors
NodeB CPICH transmit power
NodeB distribution condition
Environmental factor
Total interference level
Test Conditions
Perform the test on the customer-specified route.
The area without coverage should not be included in the customerspecified route.
Reference value of class A network: coverage rate >=96
Coverage
SHO rate
SHO rate = (Number of samples that 2 radio links in active set + Number of
samples that 3 radio links in active set)/Total number of samples x 100%
SHO_FACTOR_UE = (A2 + A3) / (A1 + A2 + A3)
Page 21
Traffic load of the sector (mainly specifies the load of the Um interface, and
is not related to the actual capacity of the network)
ACTIVESET_3_2SOFTER_UE_MEAN: average number of UEs configured with three radio links that two links are combined.
SHO_Ratio = 100*{[VS.SHO.AS.1 +
(VS.SHO.AS.2Softer + VS.SHO.AS.2Soft)*2+
(VS.SHO.AS.3Softer+
VS.SHO.AS.3Soft+VS.SHO.AS.3Soft2Softer)*3)]/(
VS.SHO.AS.1 + VS.SHO.AS.2Softer +
VS.SHO.AS.2Soft + VS.SHO.AS.3Softer +
VS.SHO.AS.3Soft + VS.SHO.AS.3Soft2Softer )-1}
SHO overhead
Page 22
Page 23
KPIs (stat.):
Accessibilit
y Analysis
RRC connection setup success rate and RAB connection setup success rate
These counters are mapped onto the success rate of immediate
assignments and success rate of assignments in the GSM network. Similar
to the analysis of the GSM accessibility, you can focus on these counters in
the daily KPI monitor and network optimization. There is no KPI about the
congestion in the WCDMA network. Therefore, you need to analyze the
network congestion through these counters.
KPIs (DT/CQT)
Call completion rate of the calling/called party and PDP activation success
rate
The test requirements of KPIs in the WCDMA network are basically the
same as the test requirements of KPIs in the GSM network. The service
type in the WCDMA test is increased, so the classification of the tested
KPIs is also increased. Like the KPIs of GPRS/EDGE services, the KPIs of PS
services are classified according to different rates. For the CS services in
the WCDMA network, the VP service test is added.
Page 24
RRC Connection
Setup Success Rate
Stat. Number of
Stat. Number of
Stat. Number of
Figure1
Stat. Number of
UE
RNC
RRC CONNECTION REQUEST
A1
UE
RNC
A2
RRC CONNECTION REQUEST
B
RRC CONNECTION REJECT
RRC CONNECTION SETUP COMPLETE
Page 25
Setup Success
Rate
RRC Setup KPI (reference value of class A network: RRC connection setup success rate
>=98.5%)
RRC Connection Setup Success Rate = RRC connection setup success time / RRC connection
request time received by RNC 100
Note here the RRC connection setup request time is the request time of initiating the service
RRC Reject Power Congestion CE Congestion NodeB RL Fail ALCAP AAL2 Fail FP
Fail Code Congestion Other
Page 26
Exclude the success rate of the WCDMA networks in covering the edge cell of the network.
The failed RRC connection setup due to the network congestion is excluded.
RAB Connection
Setup Success Rate
UE
RNC
CN
UE
RNC
CN
RAB ASSIGNMENT
REQUEST
RAB ASSIGNMENT
REQUEST
A
A
RADIO BEARER
SETUP
RADIO BEARER
SETUP
RADIO BEARER
SETUP COMPLETE
RADIO BEARER
SETUP FAILURE
RAB ASSIGNMENT
RESPONSE
B
RAB ASSIGNMENT
RESPONSE
Page 27
Page 28
Page 29
Service
Voice/VP
Test conditions
Test methods
Page 30
The
Thenumber
numberofofcall
callconnections
connectionsofofthe
thecalled
calledparty
partyisisthe
thenumber
numberofofAlerting
Alertingmessages
messages
Reference
value
of classUE.
A network
received
by
the
calling
received
by
the
calling
UE.
Voice >= 98%, VP >= 98%
Service
Voice/VP
Test conditions
Test methods
Collect statistics on the voice call and VP call separately. Adopt WCDMA ONLY of SCANNER + Voice/VP to consecutively
make a call (short-time call).
The calling party uses a fixed-line phone or the UE with strong signals. The calling party cannot uses a fixed-line phone during a
VP call.
The calling party makes a call. Then, the number of call requests of the called party increases by 1. If the called party hears the
Page 31
ring back tone, the number of call completions of the called party increases by 1. Otherwise, the call fails to be set up.
Test conditions
Carry out the test in the area of Ec/Io > -11dB and RSCP > -90dBm.
Carry out the test in the case of low traffic, that is, the uplink load and the downlink load cannot exceed the specified load. Exclude the access
failure caused by the admission failure.
Exclude the access failure caused by problems not related to the access networks or by the UE.
The activation success rate of the service varies slightly with the rate. The service with a higher rate has a relatively lower activation rate.
Test methods
If the PS service is established on the DCH, adopt WCDMA ONLY of SCANNER + PS128K to consecutively perform the PDP activation
test, that is, make short-time calls. If the PS service is established on the HSDPA, adopt WCDMA ONLY of Scanner + HSDPA to
consecutively perform the PDP activation test.
Carry out the test or collect statistics based on the typical subscription rate.
Carry out the PDP activation test.
If the PDP activation is successful, deactivate the PDP after 10 seconds.
If the PDP activation fails, activate the PDP again after 10 seconds.
Calculate the number of successful PDP activations and the number of PDP activation requests.
Page 32
RAN KPI
Counters
Coverage
Accessibility
Retainability
Delay
Mobility
Page 33
Analysis of Call
Drop Rate
Call drop rate is a KPI that telecom operators are
required to be tested, regardless of the WCDMA network
or the GSM network. It is an important counter for the
analysis of the traffic measurement and DT/CQT.
On the basis of the GSM network, the video call service
is added to the WCDMA network. Compared with the
analysis of the call drop rate in the GSM network, the
analysis of the call drop rate in the WCDMA network is
very difficult. Therefore, you should make more efforts to
analyze it during the network optimization.
Page 34
Test conditions
Test methods
Adopt WCDMA ONLY of SCANNER + Voice/VP to consecutively make a call (long-time call).
During a voice/VP call, the called party is in the strong-signal area of RSCP > -80dBm and Ec/Io > -8dB
Set the call duration of the UE to 90 seconds and the waiting time after the call is terminated to 15 seconds.
If the call drop occurs, initiate a call after 30 seconds.
Calculate the number of call connections and the number of call drops.
At least, carry out the test 200 times.
Page 35
Test conditions
Test methods
For the PS call drop rate of R99, adopt WCDMA ONLY period of SCANNER + PS full-coverage rate to download the test (short-time call). For
HSDPA PS call drop rate, adopt WCDMA ONLY period of SCANNER + HSDPA to download the test (short-time call).
Carry out the test or collect statistics according to the full-coverage rate.
Perform the PDP activation. If the PDP activation is successful, the number of successful PDP activations is increased by 1.
Start the FTP server to download. After 90s, perform the PDP deactivation. If the call drop occurs during the download, the number of PS call
drops is increased by 1.
At least, carry out the test 200 times.
Page 36
RNC
CN
RNC
CN
IU RELEASE REQUEST
IU RELEASE COMMAND
IU RELEASE COMPLETE
A
A
Page 37
Page 38
Page 39
Page 40
Overall counter
CS call drop
CS call drop rate
PS call drop
PS call drop rate
Traffic class based counter
Conversational
Streaming
Interactive
Background
Typical bearer based counter
AMR speech
CS 64 kbit/s
PS 64 kbit/s
PS 128 kbit/s
PS 384 kbit/s
Main Influencing Factors
RF loss
RLC reset
AAL2 failure
Other cause
RAN KPI
Counters
Coverage
Accessibility
Retainability
Delay
Mobility
Page 41
Delay
Delay is mainly tested by the call quality test (CQT).
There is no great difference in the GSM network and
the WCDMA network. For the CS services, the VP
service is added to the WCDMA network. For the PS
services, more channel occupation modes and
multiple rates are applied to the WCDMA network.
Delay is applicable to the delivery or when
customers perform the network CQT survey. Daily
KPI monitor and network optimization are mainly
performed on the basis of the exported data of the
KPIs.
Page 42
AMR/VP
Voice Delay
Test conditions
Connection delay: Refers to the interval from the time when the UE
sends the RRC Connection Request for the first time to the time
when the UE receives an Alerting message.
1. UEs of the calling party and called party are in the strong-signal
area of RSCP > -90dBm and Ec/Io > -11dB.
2. Both the calling party and called party are in the local office.
3. When the static test is performed, the location of UE remains
unchanged.
Test methods
1. If you use a stopwatch to calculate the delay, the timing begins when
you press the dial key and ends when a ring back tone is heard.
2. If the ring back tone from the peer end is not received, the call is
defined to be unsuccessful and this delay is not recorded.
3. Record call-related messages (the interval from the time when the UE
sends the RRC Connection Request to the time when the UE
receives an Alerting message).
4. At least, carry out the test 20 times. Then, calculates the average
delay.
5. Change a cell to perform the test at five locations.
Page 43
VP
Image
Delay
Test conditions
1. UEs of the calling party and called party are in the strong-signal area
of RSCP > -90dBm and Ec/Io > -11dB.
2. Both the calling party and called party are in the local office.
3. When the static test is performed, the location of UE remains
unchanged.
Test methods
1. Record the time when you see the first frame image after you press
the talk key.
2. If the Alerting message from the peer end is not received, the call is
defined to be unsuccessful and this delay is not recorded.
3. At least, carry out the test 20 times. Then, calculates the average
delay.
4. Change a cell to perform the test at five locations.
5. Exclude the excessive access delay caused by non-network reasons.
Page 44
AMR/VP Call
Delay
UE->UE
Means 4.5s
95 <5.5s
UE->PSTN
Means 3.3s
95 <3.54s
UE->UE
Means 4.5s
95 <5.5s
Voice
VP-Alerting
VP-display
UE->UE
Means 9s
(The VP delay difference is
great between different UEs)
Page 45
Ping Delay
Test conditions
When the uplink or downlink TCH is mapped onto the DCH, the DCCC algorithm
must be disabled to perform the Ping delay test. If the DCCC algorithm is
enabled, the speed may decrease during the Ping test. In this case, the delay
may increase.
The server adopts the Windows operating system, and the Ping server is
directly connected to the Gi interface.
Test methods
1. According to the requirements, the size of the Ping packets should be 0 bytes and
1460 bytes.
2. Exclude the excessive delay caused by the non-network reason.
3. If the timeout occurs, the average value of the delay should be calculated
manually.
Do not use the average value of the delay displayed in the CMD command.
4. At least, carry out 100 times. The average value is used as the value of the delay.
5. Change a cell to perform the test at five locations.
6. Exclude the excessive access delay caused by non-network reasons.
Page 46
Related to Ping
Delay
Counter Name
Definition
Service
Reference Value of
Delay
DCH Ping
UL 128/DL 384
0 bytes: 170 ms
1460 bytes: 380 ms
FACH Ping
RACH/FACH
0 bytes: 200 ms
HSDPA Ping
UL 128k/DL HSDPA
0 bytes: 90ms
1460 bytes: 150ms
1300 ms
FACH to HSDPA
Ping
Ping 1460Bytes
1000 ms
2800ms
2650 ms
Page 47
RAN KPI
Counters
Coverage
Accessibility
Retainability
Delay
Mobility
Page 48
Analysis of
Handover Success
Rate
Like the call drop rate, the data of the handover success rate needs to
be also exported from the test software and the drive test (DT). Call
drop rate and handover success rate are important KPIs for
engineering delivery.
According to the attributes of the target cell, the KPIs of the WCDMA
network are categorized into the following: soft handover success
rate, intra-frequency/inter-frequency hard handover, inter-RAT
handover. During the analysis process, the soft handover success rate
(similar to the G-BSC internal handover) is mainly analyzed. The intrafrequency handover occurs in the edge of the Radio Access Network
(RAN) (refer to the GSM outgoing BSC handover). The inter-frequency
handover occurs in the multi-band network (refer to the GSM dualband handover). The inter-RAT handover indicates the WCDMA-to-GSM
handover.
Compared with the GSM network, the handover counters related to the
PS services is added to the WCDMA network. The procedure for the PS
service handover in the WCDMA network can refer to the procedure for
the CS handover in the GSM network.
Page 49
Note:
Note:The
Thenumber
numberofofsoft
softhandover
handoverattempts
attemptsisisthe
thenumber
numberofofActive
ActiveSet
SetUpdate
Update
Reference
value
of
class
A
network
messages
sent
by
the
RNC
messages
sent
by
the
RNC
>= 99.5%
The
Thenumber
numberofofsuccessful
successfulsoft
softhandovers
handoversisisthe
thenumber
numberofofActive
ActiveSet
Set
Service
Update
CS & PS
UpdateComplete
Completemessages
messagesreceived
receivedby
bythe
theRNC.
RNC.
Test conditions
Test methods
Perform a long-conversation test by adopting different services, and place the UE of the called party in the strong-signal
area.
The RNC starts the single user signaling tracing of the tested UE.
The UE records the RRC signaling process.
Count the number of activity sets to be updated sent by the RNC and the number of updated activity sets received by the
RNC.
If the conversation is interrupted during the test, re-establish the conversation.
Page 50
Soft Handover
Success Rate
Main
Influencing Factors
Handover
Delay
RNC
MEASUREMENT REPORT
factor
Movement
speed
Interference
Signal
UE
parameter
UE
RNC
factor
change
UE
RNC
MEASUREMENT REPORT
MEASUREMENT REPORT
A
A
ACTIVE SET UPDATE
B
WAITING ACTIVE SET UPDATE
COMPLETE TIME OUT
Page 51
Service
CS & PS
Test conditions
Perform the test in the area of Ec/Io > -11dB and RSCP > -90dBm.
Carry out the test in the case of low traffic, that is, the uplink load and the downlink load cannot exceed the specified load. Exclude the
failure to add handover links caused by network congestion.
Exclude the hard handover failure caused by the failure to establish the user plane.
Exclude the hard handover failure caused by the UE.
Test methods
Page 52
UE
NodeB
RNC
MEASUREMENT REPORT
UE
RNC
UE
RNC
PHYSICAL CHANNEL
RECONFIGURATION
PHYSICAL CHANNEL
RECONFIGURATION
PHYSICAL CHANNEL
RECONFIGURATION FAILURE
UE
SRNC
CN
RELOCATION
REQUIRED
A
RELOCATION
COMMAND
HANDOVER FROM
UTRAN COMMAND
B
C
IU RELEASE
COMMAND
D
Successful inter-RAT HO outgoing procedure
Page 54
Test conditions
The speed of the WCDMA network signal decreases by less than 30dB within 3 seconds.
Exclude the handover scenario in the elevator.
When the handover occurs, RxLex of the GSM network signal must be greater than -85dBm.
Exclude handover failures caused by GSM network congestion.
Exclude handover failures caused by GSM networks.
Exclude handover failures caused by the UE.
It is not a failure when the UE returns to the WCDMA network during the handover.
Test methods
Determine three to five scenarios in which the UE is handed over from the WCDMA network to the GSM network. For example, the GSM
network is equipped with the indoor distribution system, and the WCDMA network does not have the indoor distribution system.
Make a long-time Voice call for a test and move the UE from the WCDMA coverage area to the GSM coverage area.
Check whether the UE is handed over from the WCDMA network to the GSM network, that is, check whether the icon on the screen changes
from WCDMA to GSM. If the conversation is not interrupted, the number of successful handovers is increased by 1. If the conversation is
interrupted, the number of failed handovers is increased by 1. Return to the WCDMA area. Perform the cell reselection forcibly for the UE to
camp on a WCDMA cell. Record signaling messages on the UE side and the RNC side.
At least, perform the test over 20 times in each handover scenario.
Formula for calculating the handover success rate: Number of successful handovers/(Number of successful handovers + Number of failed
handovers)
Success rate of inter-RAT handovers (outgoing) in the CS domain = 100 x (VS.IRATHO.SuccCSOut) / (VS.IRATHO.AttCSOut.RNC)
In the Nastar software, the counter of the success rate of inter-RAT handovers (outgoing) in the CS domain is
VS.IRATHO.SuccCSOut.RNC.Rate.
Page 55
PS Inter-RAT Handover
Success Rate (Outgoing)
PS inter-RAT handover success rate (outgoing) = Number of successful PS inter-RAT handovers / Number of PS inter-RAT handover attempts x 100%
PS inter-RAT handover success rate (outgoing) = Number of successful PS inter-RAT handovers / Number of PS inter-RAT handover attempts x 100%
Note: The number of successful PS inter-RAT handovers is the number of Routing Area Update Complete messages sent by the UE.
Note: The number of successful PS inter-RAT handovers is the number of Routing Area Update Complete messages sent by the UE.
The number of PS inter-RAT handover attempts is the number of CELL CHANGE ORDER FROM UTRAN messages received by the UE.
The number of PS inter-RAT handover attempts is the number of CELL CHANGE ORDER FROM UTRAN messages received by the UE.
In the flow chart of the PS inter-RAT handover,
SRNC
CN
MEASUREMENT
REPORT
CELL CHANGE
ORDER FROM
UTRAN
B
IU RELEASE
COMMAND
IU RELEASE
COMPLETE
Page 56
Adopt PS UL 64K/DL 64K to test Ping packets by using the command Ping
The
number
ofofsuccessful
PS
inter-RAT
handovers
isisthe
number
ofofIU
The
number
successful
PS
inter-RAT
handovers
the
number
IU
Reference value of class A network
RELEASE
RELEASECOMMAND
COMMANDmessages
messagesreceived
receivedby
bythe
theRNC
RNCininwhich
whichthe
thecause
causevalue
value
PS is
inter-RAT
handover
success
rate:
Successful
Relocation
ororNormal
Release.
is
Successful
Relocation
Normal
Release.
>= 90%
Formula for calculating the PS inter-RAT
handover success rate
PS inter-RAT handover success rate =
100 x
(VS.IRATHO.SuccPSOutUTRAN.RNC)
/ (VS.IRATHO.AttPSOutUTRAN.RNC)
In the Nastar software
Counter of the PS inter-RAT handover
Page 58
Conclusion
Page 59
Thank You
www.huawei.com