Analysis of Transport Network QoS and QoE of IPTV Services

Tran Trong Son, Bong-Kyun Lee, Young-Tak Kim

Dept. of Information and Communication Engineering,
Graduate School, Yeungnam University
214-1, Dae-Dong, Kyeong-San, Kyeongbuk, 712-749, KOREA
trongson@ynu.ac.kr, zakare@ynu.ac.kr, ytkim@yu.ac.kr

Abstract

This paper introduces an approach for measuring the Quality of Experience (QoE) and
Quality of Service (QoS) for Internet Protocol Television (IPTV) services employing MPEG-2
codec and MPEG-2 transport stream. A testbed which emulates the Internet and xDSL access
networks is used for analyzing the QoE and QoS. We figure out the relationships between QoE
and QoS parameters. Our results show that the jitter and loss take most effect on the QoE
while the behavior of delay is likely not predictable.

I. Introduction
The advances in broadband internet and scalable
video coding techniques such as MPEG-1, 2, and
MPEG4/H264, etc., have made the Internet Protocol
Television (IPTV) possible [1]. IPTV describes a system
capable of sending, receiving, and displaying a video
stream as a series of IP packets.
IPTV can be viewed in several ways. A computer may
be used to reassemble protocol packets and convert them
into a television signal that can be recognized by standard
television set. Another option is the set top box, commonly
used by cable companies. The set-top box would be used
in place of a computer and would do the same job a
computer does.
The coming together of voice, data, and multimedia
(IPTV) services brings to customers convenience but from
the service provider side, there are much tasks needed to
be done for the real triple-play service. There are some
limitations to the IPTV. IPTV is based on the Internet
Protocol, and then it is very sensitive to packet loss, delay
and jitter. A windows Media-encoded stream, for instance,
takes up 1.0 to 1.5Mbps for Standard Definition TV (SDTV).
Ten channels could be sent at one without any effect on the
voice or data traffic. The situation is explicitly changed
when High Definition TV (HDTV) joins the market. The
bandwidth necessary for HDTV is about 20 to 25 Mbps. In
this situation, we need QoE/QoS-supported mechanisms.
There are many well-known organizations involved in
studying general architecture capable of supporting
QoE/QoS for IPTV services. The most active one is ITU-T
(International Telecommunication Union) [2]. The
recommendation Y.1541 [9] and G.1010 [10] of ITU-T could
be a good reference for supporting QoS and network
performance, and for supporting QoS at application layer,
respectively. The IPTV Focus Group of ITU-T is currently
working on the network architecture and QoS model for
IPTV system. The National Telecommunications and
Information Administration (NTIA), United States of America,
also has actively activities on introducing a QoE (Quality of
Experience) metric, VQM (Video Quality Metric), for
evaluating the quality of video stream (IPTV stream, etc.).
Other objective metrics of service quality such as the
duration of periods degraded service (e.g. Degraded
Seconds, Errored Seconds, Unavailable Seconds) provide
less information but may be easier to measure.
There are many studies on supporting QoS at network
and application layer, and on testing the quality of picture
(such as VQM, PSNR (Peak Signal to Noise Ratio), etc.).
But we do not see much study on the relation between QoE
and QoS. In this paper, we discuss about our study on this
relation.
The rest of this paper is organized as follow. In section
II, we explain about the IPTV architecture, and the
QoE/QoS requirements for IPTV services. We present our
approaches to fully comprehend the relation between QoS
and QoE in section III. We show our experimental results in
section IV. In section V, we conclude our paper and indicate
directions for future work.

II. Overview of IPTV services, architecture,
and QoE/QoS considerations
2.1. Overview of IPTV services and
architecture
Recently, the ITU-T FG IPTV reached a consensus on
the following definition of IPTV [4]: IPTV is defined as
multimedia services such as
television/video/audio/text/graphics/data delivered over IP
based networks managed to provide the required level of
QoS/QoE, security, interactivity and reliability.
Figure 1 depicts the meaning and scope of IPTV
definition incorporating all discussions and results.
Fig. 1 Meaning and Scope of IPTV

Generally, the basic reference model of IPTV service is
shown in Figure 2. This architecture is studied in ITU-T with
the analysis of relevant roles that involve in the provision of
an IPTV service. As describer in the Figure 2, there are four
roles [4]:
Content Provider: The entity that owns or is licensed
to sell content or content assets.
Service Provider: The entity that provides the IPTV
services to Customer. The Service Provider relieve
on the QoS in the networks provided by Network
Provider to guarantee QoE of IPTV services.
Network Provider: The entity that provides network
services for both Customer and Service Provider.
The responsibility of this entity is very high. And it
may need to have contracts with not only Service
Provider but also Customer about the network
parameters (e.g. bandwidth, availability of service,

etc.).
Customer: The entity that consumes and pays for
the IPTV services.
The basic model looks quite simple, but this clear
definition let us distinguish IPTV services further by
considering each role. For example, we can classify the
IPTV services in terms of Content (e.g. live content,
managed services, unmanaged services) [1].

Fig. 2 IPTV Reference Model

Looking at the basic IPTV Reference, there are many
factors that impact to the Quality of the IPTV services.
Typically, the area that covered by Network Provider
contains of wide area network (e.g. Internet or BcN, etc.)
and access network (e.g. xDSL, FTTH, wireless access
networks). These networks influence the quality much. The
delay in a small sector of the Internet typically excesses
one hundred milliseconds on the normal load. During the
high traffic time, the delay, jitter and loss is very high. This
is one of the biggest challenges in guaranteeing QoS of the
network and so QoE. We also need to consider the
Customer network. Nowadays, the Customer network is
considered as an infrastructure with many kinds of end-
point device and Layer 2 technologies, such as Ethernet,
IEEE1394, Wireless, and so on. Study on QoS support in
this network is also needed for support the overall QoS [3].
To fully support QoE/QoS we also need to consider the
Content and Service Provider roles such as video/audio
encoding/compression technologies (e.g. MPEG-2, MPEG-
4/H.264, etc,), and also transport methods.
In this paper, we analyze the impact of network
parameters on the Quality of Video service (e.g. QoE of
IPTV service).

2.2. QoE/QoS Considerations
The DSL Forums technical report TR-126 [5] defines
Quality of Service (QoS) is a measure of performance at
the packet level from the network perspective. Quality of
Service (QoS) also refers to a set of technologies (QoS
mechanisms) that enable the network operator to manage
the effects of congestion on application performance as well
as providing differentiated service to selected network traffic
flows or to selected users.
Quality of Experience (QoE) is the overall performance
of a system from the point of view of the users. QoE is a
measure of end-to-end performance at the services level
from the user perspective and an indication of how well the
system meets the users needs.
QoE and QoS are two different type of metrics used for
evaluating the quality of service. But there is the major
difference between them. This difference is the point-of-
view. The QoS is considered at network side while the
QoE is considered by customers. Customers do not know
about the QoS, the only thing they concern is the quality of
video (e.g. picture frames in video stream) and the quality
of audio. Therefore, the QoE is a measure of end-to-end
performance, while the QoS can be measured at different
points inside the network.
The Video QoE can be measured in three ways [5]:
Subjectively using a controlled viewing experiment
and participants who grade the quality using rating
scales such as Mean Opinion Score (MOS).
Objectively at the service layer by using electronic
test equipment to measure various aspects of the
overall quality of the video signal.
Indirect using measurements of network
impairments (loss, delay, jitter, duration of the
defect) to estimate the impact on video quality,
where there is an established relationship between
QoE and QoS.
The first way is dependent on participants, and the
results may be changed widely. There are many studies on
the send method, such as PSNR, VQM, etc.
VQM accurately tracks subjective quality judgments of
video scenes and in addition to providing technology-
independent perception-based estimates of subjective
quality, the VQM has low computational complexity and can
be used for continuous real-time in-service quality
monitoring applications. The VQM consists of a linear
combination of four parameters that have been optimized
for the standard viewing distance of six times picture height.
Three parameters are extracted from spatial gradients of
the luminance component (Y) of ITU-R BT.601 [7] input and
output video streams while one parameter is extracted from
the vector formed by the chrominance components (CB, CR).
The sampled input and output video streams are assumed
to have been calibrated before the processes described
herein are performed. This calibration includes
compensation for system gain and level offset, as well as
spatial and temporal registration of the images [8].
PSNR is a classic metric to assess quality of
compressed video. PSNR is defined according to the
following formulae:
2
10
2
1
255
10log (1)
1
( ( , , ) ( , , ))
( 2 1 1)( 2 1 1)( 2 1 1)
n
n N
PSNR
MSE
MSE d p mn o p mn
P P M M N N
=

=


=
+ + +
2 2 2
1 1
(2)
p P m M N
p P m M
= = =
= =


Where d(p,m,n) and o(p,m,n) represent respectively
degraded and original pixel value at frame p, row m and
column n.
In spite of this metric does not generally conform to
human perception of image distortion, it is a common
medium to estimate video codecs quality for the last
decades.
Currently, there are several implementations of Video
quality measurement based on VQM, PSNR, etc, metrics
such as Tektronix [11] and Elecard StreamEye Tools [12].
The recommendations Y.1541 [9] and G.1010 [10] of
ITU-T provide guidance for supporting QoS at IP layer and
application layer. These recommendations, which are
shown in Table 1 and Figure 3, respectively, could be good
references for Network and Service Providers for
guaranteeing QoE to customers.

Table 1 ITU-T Y.1541 IP network QoS class definitions
and network performance
QoS classes Network
performance
parameter
0 1 2 3 4 5
IPTD
100ms 400ms 100ms 400ms 1s U
IPDV
50ms 50ms U U U U
IPLR 10
-3
10
-3
10
-3
10
-3
10
-3
U
IPER 10
-4
U

The ITU-T recommendation Y.1541 focuses on the IP
layer performance with five different QoS classes. It also
provides guidance for IP QoS classes, but no mention

about IPTV. The ITU-T recommendation G.1010 provides
delay guidance for several kinds of service. There is a
comment for streaming audio and video (timely, delay ~10s,
see Figure 3), it is likely not suitable for IPTV because in
IPTV service scenario, the content may come from some
live sources. This content may come to customers in a
different way such as traditional TV. In this case, if the delay
guaranteed for IPTV services is about 10s there is a big
gap in displaying systems. Moreover, IPTV is provided
through IP network where the fluctuation in delay and jitter
is very high. If we guarantee the delay about 10s it can
result in the high jitter.


Conversational
voice and video
Command/control
(eg Telnet,
interactive
games)
Transactions
(eg E-commerce,
WWW browsing,
email access)
Messaging,
Downloads
(eg FTP, still
image)

Background
(egUsenet)

Fax


Voice/video
messaging


Streaming audio
and video

Interactive
(delay<<1sec)
Interactive
(delay ~2sec)
Timely
(delay ~10sec)
Non-critical
(delay >>10sec)
Error
tolerant
Error
intolerant

Fig. 3 ITU-T G.1010 Application Delay Classifications

Other consideration about QoE and QoS is the relation
between QoE and QoS. Typically there will be multiple
service level performance (QoS) metrics that impact overall
QoE. The relation between QoE and service performance
(QoS) metrics is typically derived empirically. Having
identified the QoE/QoS relationship, it can be used in two
ways [5]:
Given a QoS measurement, one could predict the
expected QoE for a user
Given a target QoE for a user, one could deduce the
network required service layer performance.
Having this relation is very useful in system design and
traffic engineering process. When Service Providers start
providing IPTV services they need to ensure the level of
QoE guaranteed for customers and by translating the QoE
into QoS and vice versa they can construct a good network
design, maintenance, and planning in future.
In the rest of this paper, we will present our study on
investigating how the network parameter (e.g. loss, jitter,
and delay) affects to the QoE of IPTV service.

III. Testbed and Approaches
In this study, we configure a testbed for the
performance analysis of the end-to-end IPTV services. The
IPTV client utilizes the IPTV services from Service Provider
through IP networks. Figure 4 shows our testbed
implementation.
NMS
Streaming Server
Client
Performance Monitoring
Reports
Internet
Access
network
xDSL
Access
network
xDSL
RTP/UDP

Fig. 4 Testbed for QoE/QoS relation discovery

The Streaming Server acts as IPTV server which
delivers IPTV services to the Client. The Client utilizes the
IPTV services and also acts as Performance Monitoring
point. We choose to measure the network performance at
the RTP layer because RTP is the highest layer under the
application layer. In order to measure the performance of
IPTV services in an environment which is most similar to
the real one, we configure the backbone network as the
Internet with the Wide Area Network Emulator (WANEM)
[13]. The access networks are configured as xDSL ones.
Using WANEM, we can simulate the Internet characteristics
like network delay, jitter, packet loss, etc.
We use Elecard StreamEye Tools [12] as the main tool
to analyze the VQM of video streams.
In our scenario, we study the relation between QoE
and QoS by investigating SDTV service:
MPEG-2 codec
MPEG-2 transport stream
Seven 188-byte packets per RTP/UDP/IP packet.
No loss concealment
Network Metrics are for RTP flows containing video
stream only.
Figure 5 represents our IPTV model implementation for
measuring the QoE and QoS.

Fig. 5 Our IPTV model for QoE/QoS measurement

IV. Experimental Results
VQM is used to investigate the effects of jitter and
delay on the QoE of IPTV services. Normally, the set-top
box (STB) buffers every incoming RTP packets, reorders,
and after that forwards them to the Decoder. The RTP
reordering time in the STB could affect much to the quality
of real-time video streams because in some cases, the jitter
and delay in the network are very high, which may result in
the misorder of RTP packets, the STB still keeps old RTP
packets of one video frame in the buffer and waits for the
complete income of others. In this case, other video frames
are delayed to the video output even though they are ready.
VQM vs Delay
0
1
2
3
4
5
6
7
8
50 100 200 300 400 500 600
Del ay (ms)
V
Q
M
averJ itter=30ms
averJ itter=50ms
averJ itter=60ms
averJ itter=80ms
averJ itter=100ms

Fig. 6 Relationship between Delay and VQM

To analyze the effects of delay and jitter on real-time
characteristics of IPTV services, we implement a specific
buffer in the STB. The STB receives, buffers RTP packets,
and waits for 50 milliseconds delay between two packets. It
means that once one RTP packet comes into the buffer if
the next RTP packet does not come after this moment

50ms, it will be considered to be lost. By using this model,
we could analyze the effects of jitter and delay on the real-
time IPTV services.
The effects of delay and jitter on QoE are shown in
Figures 6 and 7, respectively.
By setting the jitter in the network to relatively stable
values, we can measure the impact of delay on VQM with
specific jitter values.
As shown in Figure 6, while the delay increases the
VQM increases (the quality is decreased). In the case
where the average jitter is set to 30ms, the delay does not
affect to the VQM. It is because of the RTP reordering time
is set to 50ms.
VQM vs Jitter
-1
0
1
2
3
4
5
6
7
8
0 20 40 60 80 100 120
Ji tter (ms)
V
Q
M
averDelay=50ms
averDelay=100ms
averDelay=200ms
averDelay=300ms
averDelay=400ms
averDelay=500ms
averDelay=600ms

Fig. 7 Relationship between J itter and VQM

Similarly, for measuring impact of J itter on VQM, we
set the packet delay in the network to relatively stable
values. Figure 7 represents the trend of impact of jitter on
VQM. While jitter is less than 80ms, the VQM increases
exponentially as the jitter increases. But if the jitter is higher
(about 80 to 100ms), the VQM increases slowly.
From Figure 6 and 7, good VQM values (less than
about 1.5) can be archived if the jitter is less than 50ms and
delay is less than 300ms.
VQM vs Loss
-1
0
1
2
3
4
5
6
7
0 0.0001 0.0002 0.0003 0.0004 0.0005 0.0006
Loss rati o (%), short vi deo
V
Q
M
averVQM
maxVQM
minVQM

Fig. 8 Effect of loss on VQM

Figure 8 represents effect of loss on VQM. Obviously,
loss takes much effect on the VQM. The quality of video
stream is guaranteed if the loss ratio is less than about
0.00015 percent (one packet loss in about 6666 packets).

V. Conclusion
In this paper, we analyzed the QoE and QoS
requirements for IPTV services as well as the relationship
between these parameters.
We configured a testbed for investigating the
relationship between QoE and QoS by using VQM metric.
The results showed that the effects jitter and loss is very
critical, while the behavior of delay is likely not predictable.
The results for VQM and PSNR metrics are quite similar.
In this paper, we introduced our study on the QoE/QoS
of typical SDTV services using MPEG-2 transport stream.
Our future work is to study on other kinds of IPTV services
such as HDTV and other type of Video Transport
techniques. Currently, we are studying on the QoE/QoS
provisioning for IPTV services in the home/office
environment.

VI. References
[1] J ames She, Fen Hou, Pin-Han Ho, and Liang-Liang
Xie, IPTV over WiMAX: Key Success Factors,
Challenges, and Solutions, IEEE Communications
Magazine August 2007, page(s) 87 - 93.
[2] http://www.itu.int/net/home/index.aspx.
[3] Min Ho Park, Yeonjoon Chung, Wan Ki Park and Eui
Hyun Paik, A Novel QoS Guaranteed Mechanism
for Multicast Traffic Controls in the Home Network,
ISCE '06. 2006 IEEE Tenth International Symposium
on Consumer Electronics, 2006., page(s) 1 6.
[4] Chae-Sub Lee, IPTV over Next Generation
Networks in ITU-T, Broadband Convergence
Networks, 2007. BcN '07. 2nd IEEE/IFIP
International Workshop, May 2007.
[5] Architecture and Transport Working Group, DSL
Forum, Technical Report TR-126, Triple-play
Services Quality of Experience (QoE)
Requirements, 13 December 2006.
[6] http://www.ntia.doc.gov/.
[7] ITU-R BT.601-5 (1995), Studio encoding
parameters of digital television for standard 4:3 and
wide-screen 16:9 aspect ratios.
[8] ITU-T J .144 (2001), Objective perceptual video
quality measurement techniques for digital cable
television in the presence of a full reference.
[9] ITU-T Y.1541 (2002), End-user multimedia QoS
categories.
[10] ITU-T G.1010 (2001), Objective perceptual video
quality measurement techniques for digital cable
television in the presence of a full reference.
[11] http://www.iptv-industry.com/sp/tkx/tkxhome.htm.
[12] http://www.elecard.com/.
[13] http://wanem.sourceforge.net/.