Queuing Delay and

Queuing Analysis

1
 RECALL: Delays in Packet
Switched (e.g. IP) Networks
 End-to-end delay (simplified) =
– (dprop + dtrans + dqueue + dproc) … on each link

A B
 Where:
 Propagation delay (dprop) = d/s (dependent on path)
 Transmission delay (dtrans) = L/R (dependent on path)
 Queuing delay (dqueue) = (dependent on load)
 Processing delay (dproc) = (minimal-insignificant/node)
 Number of links (Q) = (dependent on path)
2
Introduction
Projected vs. Actual Response
Time

Why??

3
Queuing Analysis
Queueing delay (revisited)
 R: link bandwidth
(bps)

queueing delay
average
 L: packet length
(bits)
 a: average packet
arrival rate traffic intensity
= La/R

La/R ~ 0
 La/R ~ 0: avg. queueing delay small
 La/R -> 1: avg. queueing delay large
 La/R > 1: more “work” arriving
than can be serviced, average delay infinite!
La/R -> 1
4
Queuing Analysis
 Introduction- Motivation
 Address how to analyze changes in
network workloads (i.e., a helpful tool
to use)
 Analysis of system (network) load and
performance characteristics
– response time
– throughput
 Performance tradeoffs are often not
intuitive
 Queuing theory, although
mathematically complex, often makes
analysis very straightforward 5
Queuing Analysis
 Important Note
 Queuing theory is heavily dependent
on basic probability theory (a pre-
requisite for our graduate program)
 If you need to refresh your
knowledge in this area, please review
the Stallings textbook, Chapter 7:
Overview of Probability and
Stochastice Processes.
 I will not test you specifically on
probability theory, but will reference
it in coverage of the queuing topics
addressed in this module.
6
Queuing Analysis
Single-Server Queuing System

Queuing
Items Arriving
Items Departing
(rate: )
(rate: R)
System
(Delay Box)
(message, packet, cell)

Items Lost

7
Queuing Analysis
Router output port functions

datagram
link
switch buffer(s) layer line
fabric protocol termination
queueing (send)

Queue Queue server

 buffering/queuing required when datagrams arrive

from fabric faster than the transmission rate
 scheduling discipline chooses among queued
datagrams for transmission
 sending discipline (servicing the queue) on the
output link as determined by link protocol 8
Introduction
The Fundamental Task of
Queuing Analysis

Given: Determine:
• Arrival rate,  • Items waiting, w
• Service time, Ts • Waiting time, Tw
• Number of servers, N • Items queued, r
Queuing Analysis
• Residence time, Tr 9
Parameters for Single-Server
Queuing System

Comments, assuming queue has infinite capacity:

1. At  = 1, server is working 100% of the time (saturated), so items are queued
(delayed) until they can be served. Departures remain constant (for same L).
2. Traffic intensity, u = L/R. Note that Ts = L/R, so:
max = 1 / Ts = 1 / (L/R) is the theoretical maximum arrival rate, and that
Lmax/R = u = 1 at the theoretical maximum arrival rate
10
Queuing Analysis
Queuing Process Example: Single-
Server FIFO Queue
Depth of General Expression:
the Queue
(r)
TRn+1 = TSn+1 + MAX[0, Dn – An+1]

11
Queuing Analysis
General Characteristics of
Network Queuing Models
 Item population
– generally assumed to be infinite therefore,
arrival rate is persistent through time
 Queue size
– infinite, therefore no loss
– finite, more practical, but often immaterial
 Dispatching discipline
– FIFO, typical
– LIFO (when is this practical?)
– Relative/Preferential, based on QoS
12
Queuing Analysis
Multiserver Queuing System

Comments:
1. Assuming N identical servers, and  is the utilization of each server.
2. Then, N is the utilization of the entire system, and the maximum utilization is
N x 100%.
3. Therefore, the maximum supportable arrival rate that the system can handle is:
max = N / Ts = NR/L
13
Queuing Analysis
Multiple Single-Server Queuing
Systems

14
Chapter 8 Overview of Queuing Analysis
Basic Queuing Relationships

Single
General Multiserver
Server
r = Tr Little’s Formula  = Ts  = Ts
N
w = Tw Little’s Formula r=w+ u = Ts = N

Tr = Tw + Ts r = w + N

15
Queuing Analysis
Kendall’s notation
 Notation is X/Y/N, where:
X is distribution of interarrival times
Y is distribution of service times
N is the number of servers
 Common distributions
G = general distribution if interarrival times or
service times
 GI = general distribution of interarrival time with
the restriction that they are independent
 M = exponential distribution of inter-arrival times
(Poisson arrivals – p. 167) and service times
 D = deterministic arrivals or fixed length service

Queuing Analysis
M/M/1? M/D/1? 16
Important Formulas for Single-
Server Queuing Systems

Note Coefficient of variation:

if Ts = Ts => exponential
if Ts = 0 => constant
17
Queuing Analysis
Important Formulas for Single-
Server Queuing Systems

18
Queuing Analysis
Mean Number of Items in System
(r)- Single-Server Queuing
Ts/Ts = Coefficient of variation

M/M/1

M/D/1

19
Queuing Analysis
Mean Residence Time – (Tr)
Single-Server Queuing

M/M/1

M/D/1

20
Queuing Analysis
Network Queue Performance:
Key Fact

The higher the variability in arrival rate

at the router, relative to the service
time on the output link(s), i.e., Ts/Ts
(coefficient of variation) the poorer the
performance of the router, especially at
high rates of utilization.

21
Queuing Analysis
Multiple Server Queuing Systems

Multiserver
Queuing
System

Multiple Single-
Server Queuing
System

22
Queuing Analysis
Important Formulas for Multiserver
Queuing

Note:
Useful only in
M/M/N case,
with equal
service times
at all N
servers.

23
Queuing Analysis
Multiple Server Queuing Example
(p. 203)

Single server
M/M/1 (2nd Floor)

Multiserver
M/M/? (2nd Floor)

Multiple
Single server

M/M/1 (1st Floor)

M/M/1 (2nd Floor)

24
Queuing Analysis
MultiServer vs. Multiple Single-
Server Queuing System Comparison
(from example problem, pp. 203-204)

Single server case (M/M/1):

Single server utilization:  = 10 engineers x 0.5 hours each / 8 hour work day
= 5/8 = .625
Average time waiting: Tw = Ts / 1 -  = 0.625 x 30 / .375 = 50 minutes
Arrival rate:  = 10 engineers per 8 hours = 10/480 = 0.021 engineers/minute
90th percentile waiting time: mTw(90) = Tw/ x ln(10) = 146.6 minutes
Average number of engineers waiting: w = Tw = 0.021 x 50 = 1.0416 engineers

25
Queuing Analysis
Example: Router Queuing
Internet  = 5 packets/sec
…
9600 L = 144 octets
bps

From data provided:

• Ts = L/R = (144x8)/9600 = .12sec
•  = Ts = 5 packets/sec x .12sec = .6
Determine:
1. Tr= Ts / (1-) = .12sec/.4 = .3 sec
2. r =  / (1-) = .6/.4 = 1.5 packets
ln(1-.90)
3. mr(90) = - 1 = 3.5 packets For 3 & 4, use:
ln (.6)
ln(1-.95) ln(1 – y/100)
4. mr(95) = - 1 = 4.8 packets mr(y) = -1
ln (.6) ln 
26
Queuing Analysis
 Priorities in Queues – Two
priority classes

27
Queuing Analysis
 Priorities in Queues – Example
Tr
        
 64Kbps

Router queue services two packet sizes: Find the average Queuing Delay (Tr)
• Long = 800 octets through the router:
• Short = 80 octets 1 Ts 1 + 2 Ts 2
• Lengths exponentially distributed Tr1 = Ts1 +
1 - 1
• Arrival rates are equal, 8packets/sec
• Link transmission rate is 64Kbps .08 x .01 + .8 x .1
= .01 + = 0.098 sec
• Short packets are priority 1, 1-.08
• Longer packets are priority 2
Tr 1 - Ts 1
Tr2 = Ts2 +
From data above, calculate: 1-
Ts 1 = Lshort/R = (80 x 8) / 64000 = .01 sec .098 - .01
= .1 + = 0. 833 sec
Ts 2 = Llong/R = (800 x 8) / 64000 = .1 sec 1 - .88
1 =  Ts 1 = 8 x 0.01 = 0.08
1 2
2 =  Ts 2 = 8 x 0.1 = 0.8 Tr = T +
 r1  Tr2
 = 1 + 2 = 0.88
= .5 x .098 + .5 x .833 = 0.4655 sec
28
Chapter 8 Overview of Queuing Analysis
Network of Queues

29
Queuing Analysis
Elements of Queuing Networks

30
Queuing Analysis
Queuing Networks

31
Queuing Analysis
Jackson’s Theorem and Queuing
Networks
 Assumptions:
– the queuing network has m nodes, each providing
exponential service
– items arriving from outside the system at any node
arrive with a Poisson rate
– once served at a node, an item moves immediately to
another with a fixed probability, or leaves the network
 Jackson’s Theorem states:
– each node is an independent queuing system with Poisson
inputs determined by partitioning, merging and tandem
queuing principles
– each node can be analyzed separately using the M/M/1
or M/M/N models
– mean delays at each node can be added to determine
mean system (network) delays
32
Queuing Analysis
Jackson’s Theorem - Application
in Packet Switched Networks Internal load:
L
 =  i
i=1
Packet Switched where:
Network
 = total on all links in network
i = load on link i
L = total number of links

Note:
• Internal > offered load
External load, offered to network:
• Average length for all paths:
N N
E[number of links in path] = /
 =   jk
j=1 k=2 • Average number of item waiting
where: and being served in link i: ri = i Tri
 = total workload in packets/sec • Average delay of packets sent
jk = workload between source j through the network is:
1 L Mi
and destination k T=   (See p. 210)
i=1 Ri - Mi
N = total number of (external)
sources and destinations where: M is average packet length and 33
Queuing Analysis Ri is the data rate on link i
Estimating Model Parameters
 To enable queuing analysis using
these models, we must estimate
certain parameters for the network:
– Mean and standard deviation of arrival
rate
– Mean and standard deviation of service
time (or, packet size)
 Typically, these estimates use
sample measurements taken from an
existing system

34
Queuing Analysis
Sample Means for Underlying
Exponential Distribution
Sampling:
• The mean is generally
the most important
quantity to estimate:


N
( ) = 1 Xi
N i=1
• Sample mean is itself a
random variable
• Central Limit Theorem:
the probability
distribution tends to
normal as sample size,
N, increases for
virtually all underlying
distributions
• The mean and variance
of X can be calculated
as:
E[ ]= E[X] = 
Var[ ]= 2x/N

35
Queuing Analysis