human factors engineering

&
QoE driven network design

jose.de.francisco@alcatel-lucent.com

this study provides references to a variety of sources and third parties for illustration and
discussion purposes only, which does not imply any kind of endorsement
The illuminated crowd

http://www.flickr.com/photos/albany_tim/3536902765/
 As an important measure of the end-to-end performance at the
services level from the user's perspective, QoE is an important
metric for the design of systems and engineering processes.

This is particularly relevant for video services because bad network

performance may highly affect the user's experience, mainly
because these services are compressed and have low entropy.

So, when designing systems the expected output, i.e. the expected
QoE, is often taken into account also as a system output metric.

http://en.wikipedia.org/wiki/Quality_of_experience
 QoE is an emerging but very difficult concept and has
baffled large corporations.

So, a number of companies are capitalizing on the

possibilities of measuring QoE.

http://www.springerlink.com/content/l1446505n3834055/fulltext.pdf
 The ultimate measure of a network and the services it offers is
how subscribers perceive the performance. Quality of
Experience (QoE) is the term used to describe this perception
and how usable the subscribers think the services are.

http://www.nokia.com/NOKIA_COM_1/About_Nokia/Press/White_Papers/pdf_files/whitepaper_qoe_net.pdf
 What happens when QoE is poor?
A survey by Accenture suggests that around 82% of customer
defections are due to frustration over the product or service and
the inability of the provider/ operator to deal with this effectively.

Moreover, this leads to a chain reaction as, on average, one

frustrated customer will tell 13 other people about their bad
experiences.

About 90% of customers will not complain before defecting – they

will simply leave once they become unsatisfied.

QoS is the ability of the network to provide a service with an assured service level.

QoE is how a user perceives the usability of a service when in use.

 1. Speed
First and foremost, we believe that speed is more than a feature.

Speed is the most important feature.

carsonified.com/blog/business/fred-wilsons-10-golden-principles-of-successful-web-apps/
 Steve Souders

400 ms slower
= 5-9% drop in full-page traffic

strong negative impact

roughly linear changes with increasing delay
time to click changed by roughly the delay

increase in abandonment heuristic = less satisfaction

active users are more sensitive

shaved 2.2 seconds off the average page load time

and increased download conversions by 15.4%!

http://www.slideshare.net/souders/souders-wpo-web-20-expo
 “we are not going to launch
something unless it is fast enough”

“we will talk about latency issues

when loading the APIs”

“it takes 13 seconds more on

the iPhone and Android
platforms, we think that’s pretty Marcelo Camelo Pamela Fox
unacceptable”

http://www.youtube.com/watch?v=zI8at1EmJjA
Eric Klinker
 Chetan Sharma

http://www.chetansharma.com/Managing_Growth_and_Profits_in_the_Yottabyte_Era.pdf
X15= ?
Moray Rumney
policy management &
 The difficulties at measuring something as "irrational" as happiness, forced economists to
reconsider utility as a behavioral measure. So a behavior became rational when it
maximizes utility - what ever that might be.

In contrast to the utility function, quality of service is very clearly defined, including in
several international standards.

Given a network that combines several technologies,

including IP-routed networks, QoS will give us a measure of the network's capability to
provide the best possible service given a selected network traffic.

The QoS function must take into account factors such as bandwidth, latency and loss
characteristics.

QoS, particularly when measured by the Mean Opinion Score (MOS) has been used as a
proxy for QoE, but it is limited because it requires a highly controlled environment.

http://www.springerlink.com/content/l1446505n3834055/fulltext.pdf
http://www.qoesystems.com/docs/QMaster2Datasheet.doc
 Although it is tied to the concept of Quality of Service, which attempts to objectively measure
the service delivered, QoE also takes in account the needs and the desires of the subscribers
when using mobile services.

For example an operator may provide reliable data services, and have a high QoS, but the
users may still be very unhappy with the content causing a low QoE.

• The operator's radio network - coverage of the operator's network, handovers that affect
network performance, radio capacity, etc.
• The devices or terminals - speed of the device, the radio accessibility
• The user interface - more accessible, easy to navigate, and aesthetically pleasing.
• The content - relevancy and quality of the content
• The operator's application servers - long delays times during peak usage times
• Customer service
• Provisioning for WAP, MMS - streaming settings and set up errors
• Network security - virus proliferation with MMS as well as hacking attempts
• Price & billing - for the operator's service
• The core network - IP equipment, SGSN, firewall

http://www.thetanetworks.com/resources/quality_of_experience.html
http://www.hpl.hp.com/techreports/2001/HPL-2001-179.pdf
http://www.hpl.hp.com/techreports/2001/HPL-2001-179.pdf
http://www.hpl.hp.com/techreports/2001/HPL-2001-179.pdf
The illuminated crowd

http://www.flickr.com/photos/albany_tim/3536902765/
The illuminated crowd

“A crowd has gathered, facing a light, an illumination brought about by a

fire, an event, an ideology - or an ideal.”

The strong light casts shadows, and as the light moves toward the back
and diminishes, the mood degenerates; rowdiness, disorder and
violence occur, showing the fragile nature of man.”

“Illumination, hope, involvement, hilarity, irritation, fear, illness,

violence, murder and death - the flow of man's emotion through space.”

Sculpture and words by Raymond Mason, 1985

http://www.flickr.com/photos/albany_tim/3536902765/
a XXI century approach to quality management:

http://consultaglobal.wordpress.com/2011/01/28/user-centric-engineering-the-3q-model/
 leading indicators lagging indicators
Usually change before the Usually change after the economy as
economy as a whole changes. a whole does.

They are therefore useful as short- Improved customer satisfaction is the

term predictors of the economy. result of initiatives taken in the past

http://en.wikipedia.org/wiki/Economic_indicator
 leading indicators lagging indicator

QoS
Engagement
QoExp
metrics
coverage
connectivity

bandwidth
raw peak/edge
opinion score
speed recommendations
Jitter reputation
Engagement loyalty
bit rate metrics predictability
packet loss network effects

session duration
number of sessions revenue per user
periodicity • direct
impressions • indirect
depth (immersion)
personalization / creation
social graph / conversations
abandonment rate
http://radywire.files.wordpress.com/2007/08/organizational-behavior.jpg
 QoL
While Quality of Life (QOL) has long been an explicit or implicit policy
goal, adequate definition and measurement have been elusive.

Diverse "objective" and "subjective" indicators across a range of

disciplines and scales, and recent work on subjective well-being
(SWB) surveys and the psychology of happiness have spurred
renewed interest.

Robert Costanza
University of Vermon

http://en.wikipedia.org/wiki/Quality_of_life
 transgenerational design?

public safety?

participatory democracy?
QoL – quality of life – how is your solution contributing to individuals’ well-being, our
communities and culture, and… what about growth and wealth generation?

QoE – quality of experience(s) – how are users leveraging this solution? what’s their
engagement level? what are the network effects?

QoS – quality of service – what are the specific service levels? and the supporting
technologies and solutions that enable the above two?