QOS Review:

This is a review of NCARs current campus network and the potential FRGP QOS implementation. Since
NCARs initial QOS deployment, new hardware and IOS versions have been deployed in the network,
CatOS has been converted to IOS, and the QOS toolset has changed. The intent of this document is to
validate the existing QOS configuration and/or propose new QOS configurations using QOS best
practices outlined in Ciscos Enterprise QoS Solution Reference Network Design (Enterprise QoS SRND)
as a guideline.
The intent of NCARs QoS deployment is to ensure VOIP traffic has priority over other traffic in the face
of network congestion. QoS is not a one size fits all configuration it is highly dependent on company
policy decisions (i.e. prioritize application X) and on the overall composition of network traffic. NCARs
QoS implementation should be reviewed on a regular and as-needed basis to ensure QoS is functioning
as designed when new hardware or changes in traffic profiles occur.
Here is a summary of the recommendations that have been reviewed and approved by NETS engineers:

Ciscos Enterprise QoS and Medianet Campus Qos SRNDs will be used as the basis for QoS
configurations
Input queuing and policing will not be used
Only voice traffic will be prioritized
Transmit queue design will use the default settings applied by the mls qos command except in
a few cases where the default deviates from the SRND
Cut and paste QoS files will be created for each queue type and be used as the default
configuration for new cards
Packet drops per queue and threshold will be monitored to ensure QoS policy is working
properly
Netflow will be enabled on internal NCAR traffic for additional QoS monitoring and statistics

This document is organized in the following manner:

1.
2.
3.
4.
5.
6.
7.
8.
9.

QoS Overview Review of QoS concepts and config examples

NCARs Existing QoS configuration - Review of NCARs current QoS configurations
NCARs queue types
NCARs traffic profile VOIP traffic vs Other traffic
QoS Configuration Recommendations
Monitoring QoS
Telepresence and QoS at NCAR
Telepresence and QoS at the FRGP
References

1. QoS Overview

There are many different ways to implement QoS but the SRND recommends using DSCP markings to
ensure consistent PHB (per hop behavior) of packets. A DSCP based QoS deployment consists of the
following components:
1.
2.
3.
4.

Input Queuing
Classification and Marking
Policing
Transmit Queuing

The example commands presented below are for a 6500 with 1p3q8t queue. To complicate matters, the
QoS implementation varies across the different hardware platforms and IOS versions. Commands may
vary slightly depending on switch and/or queue type, however the basic procedure remains the same.

Input Queuing
Ingress queues (RX) are extremely difficult to congest. Ingress congestion implies that the combined
ingress rates of traffic exceed the switch fabric channel speed, and thus would need to be queued simply
to gain access to the switching fabric. On newer platforms, such as the Catalyst 6500 Sup720, this means
that a combined ingress rate of up to 40 Gbps per slot would be required to create such an event*. The
6500 line cards do have ingress queues in the event that congestion does occur, but since this is highly
unlikely, only the egress (TX) queues will be addressed in this document.
* End-to-end qos network design, By Tim Szigeti, Christina Hattingh

Classification and Marking

The Enterprise QoS SRND follows standards-based DSCP PHB markings to ensure interoperability and
future expansion.
A general rule of thumb is to classify traffic as close to the source as possible and to establish trust
boundaries. At the edge of our network, the Cisco IP phones appropriately mark their traffic as voice
bearer (COS 5, DSCP 46|EF) and voice signaling traffic (COS 3, DSCP 24|CS3), so well trust the DSCP
markings from the phone. A PC attached to the phones PC port will not be trusted - should the PC try
to send packets marked other than DSCP = 0, the switch will remark the packets to DSCP = 0. By default,
the 6500s, with QoS enabled, will mark packets with DSCP 0 from any untrusted source.
The trust boundary can be established on a per port or per vlan basis. Cisco recommends per vlan when
possible.
Once classification is performed at the edge for inbound traffic, the router/switch may need to map COS
to DSCP markings (and vice-versa) before transmitting the packet. Use the IOS commands below to
configure the mapping (global config):

mls qos map cos-dscp

mls qos map dscp-cos
Here is the CatOS command (global):
set qos cos-dscp-map
set qos dscp-cos-map
The DSCP/COS values are then trusted on the uplinks from Access to Distribution, and Distribution to
Core.

Policing
Policing can be used to markdown excess traffic in a class or drop offending traffic. Policing will not be
used in NCARs campus network at this time.

Transmit Queuing
Transmit queue types are hardware dependant. IOS command show interface capabilities -> QoS
scheduling displays the type:
ml-16c-c1-gs#show interfaces capabilities mod 4
GigabitEthernet4/1
Model:
WS-X6148A-GE-45AF
Type:
10/100/1000BaseT
Speed:
10,100,1000,auto
Duplex:
half,full
Trunk encap. type:
802.1Q,ISL
Trunk mode:
on,off,desirable,nonegotiate
Channel:
yes
Broadcast suppression: none
Flowcontrol:
rx-(off,on,desired),tx-(off,on,desired)
Membership:
static
Fast Start:
yes
QOS scheduling:
rx-(1q2t), tx-(1p3q8t)

The CatOS command is

show qos info config 1p3q8t tx

In this example, the transmit queue is 1p3q8t.

1p = one priority queue. Traffic in the priority queue is serviced until the queue is empty before
the other queues are serviced.
3q = there are three queues
8t = each queue has 8 drop thresholds that can be configured (does not apply to the priority
queue)
For each queue type, the following needs to be configured:

Map traffic to queue

Set queue buffer size

Set queue WRR weights
Set queue WRED thresholds

Map traffic to queue

There are a number of RFCs (2474, 2597, 3246, 4595) which describe the PHB for the different DSCP
markings. Proper queue design (buffer size, WRR weights, and WRED thresholds) determines whether
the queue behaves as Best Effort or CS3 or some other type. DSCP marked traffic must be mapped to a
queue designed to handle that type of traffic. Cisco has taken all these factors into account and
provided queuing models in the SRND for all the different types of queues found in their hardware:
http://www.cisco.com/en/US/docs/solutions/Enterprise/WAN_and_MAN/QoS_SRND/QoS-SRNDBook.html
Here are the IOS commands (per interface):
wrr-queue cos-map <queue> [<thresh>] <cos> <cos>
priority-queue cos-map <queue> <cos> <cos>
Here is the CatOS command (global)
set qos map 1p2q2t tx 3 1 cos 5
Set queue buffer size
The size of the buffers must be set. Typically the lower weighted queues will have larger buffers since
they are not serviced as often. Here is the IOS command to set the buffers (per interface):
wrr-queue queue-limit
priority-queue queue-limit
Here is the CatOS command (global):
set qos txq-ratio 1p2q2t 10 90
Set queue WRR weights
With WRR, the switch uses a configured weight value for each egress queue. This weight value
determines the implied bandwidth of each queue. The higher the weight value, the higher the priority
that the switch applies to the egress queue. Priority queues do not have assigned weights and are
serviced before other queues, until empty.
For example, consider the case of a Catalyst 3550 switch configured for QoS and WRR. The Catalyst 3550
uses four egress queues. If queues 1 through 4 are configured with weights 50, 10, 25, and 15,
respectively, queue 1 can utilize 50 percent of the bandwidth when there is congestion. Here is the IOS
command to set the queue weights (per interface):

wrr-queue bandwidth
Here is the CatOS command (global):
set qos wrr 1p2q2t 30 70
Set queue WRED thresholds
There is a minimum and maximum WRED (weighed random early detection) threshold configured. The
minimum indicates the point at which packets will be randomly dropped for that queue/threshold and
the MAX threshold determines when tail drop (i.e. all pkts in the queue) will begin. Here is a link to how
WRED works:
http://www.cisco.com/en/US/docs/ios/qos/configuration/guide/congestion_avoidance.html#wp100095
9

IOS example (per interface):

! Sets Min WRED Threshold for Q1T1 to 80% and all others to 100%
wrr-queue random-detect min-threshold 1 80 100 100 100 100 100
100 100
! Sets Max WRED Threshold for Q1T1 to 100% and all others to 100%
wrr-queue random-detect max-threshold 1 100 100 100 100 100 100
100 100
CatOS example (global):
set qos wred 1p3q8t tx queue 3 <[thr1Lo:]thr1Hi>
<[thr8Lo:]thr8Hi>

2. NCARs Existing QoS configurations

The existing QOS policy is, where possible, to use the default configuration when mls qos global
command is applied. Exceptions to the default configurations are COS to DSCP mappings, and
queue/threshold mappings for COS 3 and COS 5 marked traffic.
Here is a typical configuration from one of our CATOS switches.
set
set
set
set
set
set
set
set

qos
qos
qos
qos
qos
qos
qos
qos

enable
map 2q2t tx 2 1 cos 3
map 2q2t tx 2 2 cos 5
wrr 2q2t 100 255
map 1p2q2t tx 2 1 cos 3
wrr 1p2q2t 100 255
wred 1p2q2t tx queue 1 40:80 70:100
wred 1p2q2t tx queue 2 40:80 70:100

set qos map 1p2q1t tx 2 1 cos 3

set qos map 1p3q8t tx 3 3 cos 3
set qos cos-dscp-map 0 8 16 24 32 46 48 56
set qos ipprec-dscp-map 0 8 16 24 32 46 48 56
clear qos acl all
#ACL-IP-PHONES
set qos acl ip ACL-IP-PHONES trust-cos ip any any
#ACL-TRUST-DSCP
set qos acl ip ACL-TRUST-DSCP trust-dscp ip any any
#ACL-TRUST-IPPREC
set qos acl ip ACL-TRUST-IPPREC trust-ipprec ip any any
#
commit qos acl all
set qos acl map ACL-IP-PHONES 700
A survey of all the CATOS switches showed that queue mapping is performed for all the queues, but
buffer sizes, weights, and thresholds are not consistently applied to the different queues. A survey of
IOS switches shows that qos is enabled globally, but are currently using the defaults for all other
settings.
In CATOS, show qos info config 1p3q8t tx shows the current queue settings (for IOS its show queueing
interface <interface name>):
QoS setting in NVRAM for 1p3q8t transmit:
QoS is enabled
Queue and Threshold Mapping for 1p3q8t (tx):
Queue Threshold CoS
----- --------- --------------1
1
0
2
1
1 2
3
3
3
3
5
4
3
7
6 7
4
1
5
Tx drop thresholds:
Tx drop-thresholds feature is not supported for this port type.
Tx WRED thresholds:
Queue # Thresholds - percentage
------- -----------------------------------------1
70%:100% 100%:100% 100%:100% 100%:100% 100%:100%
100%:100% 100%:100% 100%:100%
2
70%:100% 100%:100% 100%:100% 100%:100% 100%:100%
100%:100% 100%:100% 100%:100%
3
40%:70% 40%:70% 50%:80% 50%:80% 60%:90% 60%:90% 70%:100%
70%:100%
Tx queue size ratio:
Queue # Sizes - percentage
------- ------------------------------------1
65%

2
15%
3
15%
4
5%
Tx WRR Configuration of ports with 1p3q8t:
Queue # Ratios
------- ------------------------------------1
20
2
100
3
200

3. NCAR Queue Types

Model
VS-S720-10G
WS-X6K-S2-MSFC2
WS-X6K-S2U-MSFC2
WS-X6K-SUP2-2GE
WS-SUP32-10GE-3B
WS-SUP720-3B
WS-SUP720-3BXL
WS-SUP720-BASE
WS-X6148-RJ45V
WS-X6348-RJ-45
WS-X6148A-GE-45AF
WS-X6408A-GBIC
WS-X6416-GBIC
WS-X6516A-GBIC
WS-X6704-10GE
WS-X6716-10GE
WS-X6748-GE-TX
WS-X6748-SFP
3560E
IE3000
3750 - NME-XD-48ES-2S-P

Description
Supervisor Mod 720 10G (enabled/disabled G ports)
2 port 1000BaseX Supervisor Mod 2 (GBIC)
2 port 1000BaseX Supervisor Mod 2 (GBIC)
2 port 1000BaseX Supervisor Mod 2 (GBIC)
2 port 10GBaseX Supervisor module
Supervisor Mod 720 base board
Supervisor Mod 720 base board
Supervisor Mod 720 base board
48 port 10/100BaseTX (RJ-45)
48 port 10/100BaseTX (RJ-45)
48 port 10/100/1000BaseTX (RJ-45)
8 port 1000BaseX (GBIC),Enhanced QoS
16 port 1000BaseX (GBIC)
16 port 1000BaseX (GBIC)
4 port 10 GE
16 port 10 GE
48 port 10/100/1000 (RJ-45)
48 port 1000Base FX (SFP GBIC)
Industrial Ethernet Switch
NME-XD-48ES-2S-P: EtherSwitch SM 48 10/100T PoE + 2 SFP

Queue Type
Scheduler
1p3q4t/1p7q4t DWRR or SRR
1p2q2t
1p2q2t
1p2q2t
1p3q8t
DWRR or SRR
1p2q2t
WRR
1p2q2t
WRR
1p2q2t
WRR
2q2t
WRR
2q2t
WRR
1p3q8t
WRR
1p2q2t
WRR
1p2q2t
WRR
1p2q2t
WRR
1p7q8t
DWRR
1p7q4t
DWRR or SRR
1p3q8t
DWRR
1p3q8t
DWRR
1p3q3t
SRR
1p3q3t
SRR
1p3q3t
SRR

Table 1: NCAR Queue Types

4. NCAR Traffic Profile

To get a rough sense of the percentage of VoIP traffic on the network vs. other traffic, in/out bps was
pulled from all the VLANs at ML, FL, and CG for a 31 day period. For voice vlans, the max values were
pulled along with average values to evaluate the worst case scenario. The table below shows percent of
VoIP (all traffic on voice vlans) traffic per campus, which was typically less than 1%.

% CGRA VOIP traffic

% CGRA MAX VOIP traffic

Inbound
(%)
0.085
0.421

Outbound
(%)
0.055
0.275

% FLRA VOIP traffic

% FLRA MAX VOIP Traffic

0.065
0.926

0.032
0.183

% MLRA VOIP Traffic

% MLRA MAX VOIP Traffic

0.043
0.246

0.042
0.655

5. QoS Configuration Recommendations

AutoQoS vs. Manual configuration AutoQoS will automatically apply QoS configurations on a switch.
Lab tests show that AutoQoS selected different buffer sizes, weights and thresholds than what is
recommended in the SRND. Manual configuration is recommended.
Input Queuing and Policing Input queuing and Policing will not be used at this time per the reasons
discussed above.
Classification and Marking: The SRND classifications recommendations assume that all traffic will be
classified and have configured transmit queues accordingly. To keep things simple, only voice bearer
and call signaling traffic will be explicitly prioritized on NCARs campus. Queue configurations will
therefore need to be adjusted accordingly.
The proposed classification of NCARs campus traffic is as follows:
1. Prioritize voice bearer traffic as COS 5, DSCP 46|EF
2. Prioritize call signaling traffic as COS 3, DSCP 24|CS3.
3. Router/switch control traffic (i.e. routing protocols, HSRP, etc) are marked as CS6 by the
router/switch.
4. All other traffic is marked with the default COS 0, DSCP 0.
All of NCARs campus traffic will be placed into a 4 level classification policy. Additional classification
levels can easily be added should we need to prioritize traffic other than voice.
Application
Voice
Network
Control
Call Signaling
All Other

L3 Classification
EF, DSCP 46
CS6, DSCP 48
CS3, DSCP 24
DSCP 0

Trust boundaries will be extended to the IP phones, all other *access* ports will be untrusted. All
uplinks between access and distribution/core switches will be trusted. Here is a sample IOS
configuration that extends the trust boundary to the IP phone:
Per Interface Configs:
interface gig <mod/port>
mls qos vlan-based
switchport voice vlan 700

<- Applied on

interface Vlan700
no ip address
service-policy input ACL-IP-PHONES-policy

Global Configs:
class-map match-all ACL-IP-PHONES-1-class
match access-group name ACL-IP-PHONES-1
!
policy-map ACL-IP-PHONES-policy
class ACL-IP-PHONES-1-class
trust dscp

Here is a sample IOS configuration to enable trust on an uplink (per interface):

mls qos trust cos

COS to DSCP and DSCP to COS mappings will remain unchanged.

Transmit Queuing, Generic Proposal:

Transmit queuing is partially determined by the hardware (1p3q8t, 2q2t, etc) and IOS version being used
on the switch/router. While different types of queuing and thresholds (WRR, SRR, WRED, WTD, etc) are
implemented on the different platforms, the user has the ability to control buffer size, weights, and
thresholds regardless of the type. The queues must behave in an appropriate manner as determined by
the type of traffic (DSCP/COS) assigned to the queue. Based on the proposed classification of traffic,
here are a couple of generic queue design proposals.
Option 1:
Use the SRND as a guide for designing queues, making adjustments as necessary based on our
classification policy.
1. Mapping of DSCP/COS to queues will be performed per the SRND even though NCAR will only
use 4 classification levels. In the future, should we decide to prioritize traffic other than voice,
the mappings will already be in the standard configuration. I dont see any real downside to
doing this.

2. Buffer space will only be allocated to queues with DSCP/COS values for which traffic has been
explicitly mapped. For example, 1p7q8t has a total of 8 queues available 1 priority and 7 other
queues. The proposed classification calls for at most 4 different queues, so 4 of the 1p7q8t
queues will have a buffer size of 0 assigned since no traffic would ever be assigned to these
queues. The SRND will be used as a guide to determine buffer size. It will be necessary to
deviate from the SRND because we will be marking all traffic except voice as DSCP/COS 0 and a
larger buffer will be needed to accommodate this traffic.
3. Bandwidth servicing weights will only be allocated to queues that have buffers assigned. The
SRND will be used as a guide to determine bandwidth servicing weights. For the most part,
these should follow pretty closely with the SRND except where buffers have been set to 0.
4. Buffer thresholds will be explicitly defined to queues that have buffers assigned, for all DSCPs
assigned to the queue regardless of whether or not any traffic will be marked with that DSCP
value. The SRND will be used as a guide. For example, the SRND for 1p3q8t queues assigns CS7
to q3t7. The proposed classification does not include CS7, however, the q3t7 threshold will still
be set. This will have no affect on traffic that is using the queue.
PROs:

Follows Ciscos recommendation for queue design which is standards based.

Queue design is known and less likely to change due to IOS version changes.

Ciscos queue design is based on marking/classifying ALL traffic. NCAR is only

marking/classifying VOIP traffic, so adjustments will need to be made for buffers,
weights, and thresholds.
Queue design results in 14+ lines of config per port depending on queue type.

CONs:

Option 2 (RECOMMENDED):
As a sanity check, we ran option1 past David and Option 2 came from that review. Option 2 primarily
relies on the default values that result from enabling the mls qos global command (Currently
implemented in NCARs campus network CATOS devices).
1. Use the default mapping of DSCP/COS to queues that result from enabling the mls qos global
command except where it differs for DSCP 24 (call signaling) and DSCP 46 (voice traffic). In
those cases, follow the SRND.
2. Use the default buffers, bandwidth servicing weights, and thresholds that result from enabling
the mls qos global command. For the most part, these settings are in line with SRND and
where they deviate, its hard to quantify how it might affect QoS.

PROs:

Easier admin, fewer config lines per port

Still maps COS to queues per SRND

Queue design could change, if defaults change due to IOS upgrade

Depending on queue type, some resources could go unused i.e. buffers, but its hard to
quantify how this would affect QoS (see 1p3q8t example below).

CONs:

Note that once a Tx queuing option has been agreed upon, it will need to be applied to all queue
types described in Table 1.

Transmit Queuing 1p2q2t:

Below are some examples of 1p2q2t (older 1G uplink queue type) queue design based on option 1 and
option 2. Option 1 compares three queue designs: SRND queue design if the SRND is exactly followed,
Default queue design created by enabling mls qos global command (same for IOS and CATOS), and
Proposed the proposed design as described above. Option 2 compares two queue designs: Default
queue design created by enabling mls qos global command, and Proposed the proposed design as
described above.
The grayed out COS/DSCP values represent unused COS/DSCP values. For example, no traffic will be
marked as AF21, CS2, COS2. The blue and orange cell colors are used to visually separate the queues
and have no other meaning.

OPTION 1
SRND

Default

1p2q2t - Buffer Size% &

BW Servicing Weight COS to Threshhold Map
n/a
EF, Cos5

70

Threshold
Value
n/a

Priority Queue - 30%

CS7; Cos7
q2t2 80:100
CS6; Cos6
AF31,CS3; Cos3
q2t1 40:80
AF21,CS2; Cos2
AF41,CS4; Cos4

1p2q2t - Buffer Size% & Threshold

BW Servicing Weight COS to Threshhold Map Value
n/a
EF, Cos5
n/a

255

100 (5 catos)

30

Queue 2 - 40%
DF; Cos0
q1t2 80:100
CS1; Cos1
q1t1 40:80

Queue 1 - 30%

Priority Queue - 15%

CS7; Cos7
q2t2 70:100
CS6; Cos6
q2t1 40:70
AF41,CS4; Cos4

Queue 2 - 15%
AF21,CS2; Cos2
q1t2 70:100
AF31,CS3; Cos3
CS1; Cos1
q1t1 40:70
DF; Cos0

Queue 1 - 70%

Proposed (based on SRND)

1p2q2t - Buffer Size% & Threshold
BW Servicing Weight COS to Threshhold Map Value
n/a
EF, Cos5
n/a

70

30

Priority Queue - 15%

CS7; Cos7
q2t2 80:100
CS6; Cos6
AF31,CS3; Cos3
q2t1 40:80
AF21,CS2; Cos2
AF41,CS4; Cos4
Queue 2 - 15%
DF; Cos0
q1t2 80:100
CS1; Cos1
q1t1 40:80

Queue 1 - 70%

Option 1: Proposed IOS Queue Configuration Commands (per interface):

wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue

cos-map 2 2 6 7
cos-map 2 1 2 3 4
cos-map 1 2 0
cos-map 1 1 1
bandwidth 30 70
random-detect 1
random-detect 2
random-detect min-threshold 1 40 80
random-detect max-threshold 1 80 100

Option 1: Proposed CATOS Queue Configuration Commands (per queue type):

set qos map 1p2q2t tx 2 1 cos 2 3 4
set qos map 1p2q2t tx 2 2 cos 6 7
set qos map 1p2q2t tx 1 1 cos 1
set qos map 1p2q2t tx 1 2 cos 0
set qos wred 1p2q2t tx queue 2 40:80 80:100
set qos wred 1p2q2t tx queue 1 40:80 80:100
set qos wrr 1p2q2t 30 70

Option 2
Default
1p2q2t - Buffer Size% & Threshold
BW Servicing Weight COS to Threshhold Map Value
n/a
EF, Cos5
n/a

255

100 (5 catos)

Priority Queue - 15%

CS7; Cos7
q2t2 70:100
CS6; Cos6
q2t1 40:70
AF41,CS4; Cos4

Queue 2 - 15%
AF21,CS2; Cos2
q1t2 70:100
AF31,CS3; Cos3
CS1; Cos1
q1t1 40:70
DF; Cos0

Proposed (based on default)

1p2q2t - Buffer Size% & Threshold
BW Servicing Weight COS to Threshhold Map Value
n/a
EF, Cos5
n/a

255

100

Priority Queue - 15%

CS7; Cos7
q2t2 70:100
CS6; Cos6
q2t1 40:70
AF41,CS4; Cos4
AF31,CS3; Cos3
Queue 2 - 15%
AF21,CS2; Cos2
q1t2 70:100
CS1; Cos1
q1t1 40:80
DF; Cos0

Queue 1 - 70%

Queue 1 - 70%

Option 2: Proposed IOS Queue Configuration Commands (per interface):

wrr-queue cos-map 2 1 3
Option 2: Proposed CATOS Queue Configuration Commands (per queue type):
set qos map 1p2q2t tx 2 1 cos 3
set qos wrr 1p2q2t 100 255
Transmit Queuing 1p3q8t:
Below are some examples of 1p3q8t queue design based on option 1 and option 2. Option 1 compares three queue
designs: SRND queue design if the SRND is exactly followed, Default queue design created by enabling mls qos
global command (different defaults for IOS and CATOS), and Proposed the proposed design as described above.
Option 2 compares two queue designs: Default queue design created by enabling mls qos global command (different
defaults for IOS and CATOS), and Proposed the proposed design as described above.
The grayed out COS/DSCP values represent unused COS/DSCP values. For example, no traffic will be marked as
AF21,CS2,COS2

OPTION 1

SRND
BW Servicing 1p3q8t - Buffer Size% & Threshold
Weight
COS to Threshhold Map Value
EF, Cos5
n/a

Default CatOS
BW Servicing 1p3q8t - Buffer Size% & Threshold
Weight
COS to Threshhold Map Value
EF, Cos5
n/a
n/a
Priority Queue - 5%
CS7; Cos7
q3t7 70:100
CS6; Cos6
AF41,CS4; Cos4
q3t5 60:90
200
AF31,CS3; Cos3
q3t3 50:80
Queue 3 - 15%
CS1; Cos1
q2t1 70:100
AF21,CS2; Cos2
100
Queue 2 - 15%
DF; Cos0
q1t1

70:100

n/a
20

70

Priority Queue - 30%

CS7; Cos7
q3t5 90:100
CS6; Cos6
q3t4 80:90
AF31,CS3; Cos3
q3t3 70:80
AF21,CS2; Cos2
q3t2 60:70
AF41,CS4; Cos4
q3t1 50:60

Queue 3 - 40%
DF; Cos0
q2t1 80:100

Queue 1 - 65%

Default IOS
BW Servicing 1p3q8t - Buffer Size% & Threshold
COS to Threshhold Map Value
Weight
n/a
EF, Cos5
n/a

40

25

Queue 2 - 25%
CS1; Cos1
q1t1 80:100
Queue 1 - 5%

Proposed (based on SRND)

BW Servicing 1p3q8t - Buffer Size% & Threshold
Weight
COS to Threshhold Map Value
EF, Cos5
n/a
n/a
Priority Queue - 10%
CS7; Cos7
q3t5 100:100
CS6; Cos6
q3t4 90:100
AF31,CS3; Cos3
q3t3 80:90
60
AF21,CS2; Cos2
q3t2 70:80
AF41,CS4; Cos4
q3t1 70:80
Queue 3 - 15%
DF; Cos0
q2t1 80:100

200

150

Priority Queue - 15%

CS7; Cos7
q3t1 70:100
CS6; Cos6
Queue 3 - 15%
AF31,CS3; Cos3
q2t2 70:100
AF41,CS4; Cos4
CS2:Cos2
q2t1 40:70
40:70
Queue 2 - 20%
CS1; Cos1
q1t2 70:100
DF; Cos0
q1t1 40:70

100

Queue 1 - 50%

Queue 2 - 75%
CS1; Cos1
q1t1 80:100
Queue 1 - 0%

Option 1: Proposed IOS Queue Configuration Commands (per interface):

wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue
wrr-queue

cos-map 3 5 7
cos-map 3 4 6
cos-map 3 3 3
cos-map 3 2 2
cos-map 3 1 4
cos-map 2 1 0
queue-limit 0 75 15
bandwidth 0 40 60
random-detect 1
random-detect 2
random-detect 3
random-detect min-threshold
random-detect max-threshold
random-detect min-threshold
random-detect max-threshold
random-detect min-threshold
random-detect max-threshold

1
1
2
2
3
3

80 70 70 70 70 70 70 70
100 100 100 100 100 100 100 100
80 70 70 70 70 70 70 70
100 100 100 100 100 100 100 100
70 70 80 90 70 70 70 70
80 80 90 100 100 100 100 100

Option 1: Proposed CATOS Queue Configuration Commands (per queue type):

set qos
set qos
set qos
set qos
set qos
set qos
set qos
set qos
set qos
set qos
100:100
set qos
100:100
set qos
100:100

map 1p3q8t tx 1 1 cos 1

map 1p3q8t tx 2 1 cos 0
map 1p3q8t tx 3 1 cos 4
map 1p3q8t tx 3 2 cos 2
map 1p3q8t tx 3 3 cos 3
map 1p3q8t tx 3 4 cos 6
map 1p3q8t tx 3 5 cos 7
txq-ratio 1p3q8t 0 75 15
wrr 1p3q8t 0 40 60
wred 1p3q8t tx queue 3 70:80 40:70 70:100 100:100 100:100 100:100 100:100
wred 1p3q8t tx queue 2 80:100 100:100 100:100 100:100 100:100 100:100
100:100
wred 1p3q8t tx queue 1 80:100 100:100 100:100 100:100 100:100 100:100
100:100

Option 2
Default CatOS
BW Servicing 1p3q8t - Buffer Size% & Threshold
COS to Threshhold Map Value
Weight
EF, Cos5
n/a
n/a
Priority Queue - 5%
CS7; Cos7
q3t7 70:100
CS6; Cos6
AF41,CS4; Cos4
q3t5 60:90
200
AF31,CS3; Cos3
q3t3 50:80
Queue 3 - 15%
CS1; Cos1
q2t1 70:100
AF21,CS2; Cos2
100
Queue 2 - 15%
DF; Cos0
q1t1

70:100

Proposed (based on Current/Default)

BW Servicing 1p3q8t - Buffer Size% & Threshold
Weight
COS to Threshhold Map Value
n/a
EF, Cos5
n/a

20

200
Queue 1 - 65%

Default IOS
BW Servicing 1p3q8t - Buffer Size% & Threshold
Weight
COS to Threshhold Map Value
n/a
EF, Cos5
n/a

200

150

Priority Queue - 15%

CS7; Cos7
q3t1 70:100
CS6; Cos6
Queue 3 - 15%
AF31,CS3; Cos3
q2t2 70:100
AF41,CS4; Cos4
CS2:Cos2
q2t1 40:70
40:70
Queue 2 - 20%
CS1; Cos1
q1t2 70:100
DF; Cos0
q1t1 40:70

100

Queue 1 - 50%

Priority Queue - 15%

AF31,CS3; Cos3
q3t3 70:100
CS7; Cos7
q3t1 70:100
CS6; Cos6
Queue 3 - 15%
AF41,CS4; Cos4
q2t2 70:100
CS2:Cos2
q2t1 40:70

150
Queue 2 - 20%
CS1; Cos1
q1t2
DF; Cos0
q1t1
100

Queue 1 - 50%

70:100
40:70

Option 2: Proposed IOS Queue Configuration Commands (per interface):

wrr-queue cos-map 3 3 3
wrr-queue cos-map 2 2 4
Option 2: Proposed CATOS Queue Configuration Commands (per queue type):
set qos map 1p3q8t tx 1 2 cos 1
set qos map 1p3q8t tx 2 1 cos 2
set qos map 1p3q8t tx 2 2 cos 4
set qos map 1p3q8t tx 3 1 cos 6 7
set qos map 1p3q8t tx 3 3 cos 3
set qos txq-ratio 1p3q8t 50 20 15
set qos wrr 1p3q8t 100 150 200
set qos wred 1p3q8t tx queue 3 70:100 40:70 70:100 50:80 60:90 60:90
70:100 70:100
set qos wred 1p3q8t tx queue 2 40:70 70:100 100:100 100:100 100:100
100:100 100:100 100:100
set qos wred 1p3q8t tx queue 1 40:70 70:100 100:100 100:100 100:100
100:100 100:100 100:100
****Recommend using Option 2****

Transmit Queuing, 3560E/3750/IE3000:

The 3560E uses SRR (shaped round-robin) queues with WTD (weighted tail drop). Weighted Tail Drop
is a simpler congestion avoidance mechanism than WRED. The weight sets the % of the queue that,
when full, will begin tail drop. The queues can be configured as 4qt3 or 1p3q3t. Well configure it to
use 1p3q3t.
We will add this configuration information based upon the decisions for the other module types.
Transmit Queuing, Voice Gateways:
As of 12.2.11T and later, the voice gateways, by default will mark voice traffic with DSCP 46 | EF and call
signaling with DSCP 24 | CS3.
Yet another queuing configuration format is used for voice gateways - HQF (Hierarchical queuing
Framework). Mar-26a-c1-gw uses HQF on its T1 link.
The SRNDs for Enterprise QoS and Unified Communications dont explicitly cover this. From a BW
perspective, the links are lightly used and even if all PRIs were in use, the max load on the uplinks would
be : 3 pri * 24 calls/pri * 80kbps/call = 7.68 Mbps. For Gig uplinks, that would consume less than 1%
of available BW.
Recommendation: No changes.

Applying QoS Configurations:

Smart port macros are available, and we could create one macro per queue configuration type. The
other option is to follow our current setup of configuring the modules as we insert them. The settings
are determined by the modules, so this would be a good way to keep us in sync as we change modules.
http://www.cisco.com/en/US/docs/switches/lan/catalyst6500/ios/12.2SX/configuration/guide/smrtport.html

When the macro is applied to the interface, the configuration shows all the commands and which macro
was applied. Here is an example of a 1p3q8t port configured using a macro:
interface GigabitEthernet3/1
no ip address
shutdown
wrr-queue bandwidth 0 40 60
wrr-queue queue-limit 0 65 20 wrr-queue random-detect minthreshold 2 80 100 100 100 100 100 100 100
wrr-queue random-detect min-threshold 3 100 100 80 90 100 100
100 100
wrr-queue random-detect max-threshold 2 100 100 100 100 100 100
100 100
wrr-queue random-detect max-threshold 3 100 100 90 100 100 100
100 100
wrr-queue cos-map 1 1 1
wrr-queue cos-map 2 1 0
wrr-queue cos-map 3 1 4
wrr-queue cos-map 3 2 2
wrr-queue cos-map 3 3 3
wrr-queue cos-map 3 4 6
wrr-queue cos-map 3 5 7
macro description pd-test
end
Note that MAX thresholds should be configured before MIN thresholds. This will avoid errors where the
new MIN threshold might be greater than the existing MAX threshold.

6. Monitoring QoS
QoS configurations need to be validated to ensure packets are properly marked and monitored to
ensure they are operating as designed.
To validate configurations, grab a packet trace (or use netflow) at the end points and check that in/out
bound DSCP markings are set correctly (SCCP = CS3; RTP = EF routing protocols = CS6, and all other set
to 0).

Locations to check
o
o
o

Switch ports that connect to CMs and voicemail servers

Switch ports that connect to gateways
PC port on phone (random sampling).

Config checkers should be used to validate that configurations are applied properly.
Queue drops can be monitored per interface, per queue and per threshold. Excessively high drops in a
particular queue could indicate the need to adjust queuing parameters. The CLI command 'show
queuing interface <blah> detailed' provides per interface, per queue and threshold drops, however, the
"detailed" portion of this command is only available in IOS ver x.x.x SXI.
These are the OIDs:
IOS - 1.3.6.1.4.1.9.9.580.1.5.1.1.4 - CISCO-SWITCH-QOS-MIB,

csqIfStatsDropPkts

CATOS - .1.3.6.1.4.1.9.9.179.1.4.5.1.4 - CISCO-CATOS-ACL-QOS-MIB

Here is an IOS example:
SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4.191.2.1.1 = Counter64: 9124503
SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4.191.2.1.2 = Counter64: 0
SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4.191.2.1.3 = Counter64: 0
SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4.191.2.1.4 = Counter64: 0
SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4.191.2.1.5 = Counter64: 0
SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4.191.2.1.6 = Counter64: 0
SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4.191.2.1.7 = Counter64: 0
SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4.191.2.1.8 = Counter64: 0
SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4.191.2.2.1 = Counter64: 0
SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4.191.2.2.2 = Counter64: 142487

SNMPv2-SMI::enterprises.9.9.580.1.5.1.1.4 = csqIfStatsDropPkts. The next number is the

ifIndex of the interface, followed by the direction (1=inbound, 2=outbound),
then by the queue, and finally the threshold.

We currently lack the ability to determine amount of traffic per COS/DSCP which could be useful for
planning and troubleshooting purposes. This would be remedied if we ran netflow internally.

7. Telepresence (TP) and QoS at NCAR

Here is a brief review of some key characteristics of Telepresence (TP):

TP reserves 192.168.0.0/24 thru 192.168.3.0/24 for internal use but does not have a default
gateway and therefore cannot route beyond the TP codec.
TP appears as two SIP endpoints in CM,
1) the TP codec
2) 7975 IP phone
Bandwidth requirements for TP: 1.628 Mbps to 12.756 Mbps per stream depending on
configuration
End to End Jitter target is < 10ms. Jitter greater than 20ms peak to peak will downgrade call
quality. Jitter 40ms or greater call will terminate.
Target packet loss is 0.5%
Marks video traffic as CS4, voice as CS5, and signaling as CS3. CS4 marked traffic is to be
assigned to the priority queue
The SRND recommends a separate cluster for TP, unless there are no other video devices on the
existing cluster

There are many different ways to deploy TP, i.e. point-to-point, point-to-multipoint, intra-enterprise,
inter-enterprise, etc. If TP were to be deployed on NCARs campus, qos configurations would also have
to be adjusted. Using the QoS configuration recommendations from above as a foundation, here is an
overview of the changes that would be needed assuming an Intra-NCAR TP deployment:

Input Queuing: No changes

Classification and Marking: Extend trust boundary to TP equipment; configure CM to mark
video traffic as CS4
Policing: No changes
Transmit Queuing: Re-map CS4 marked traffic to the priority queue; possibly re-allocate buffers

8. Telepresence (TP) and QoS at the FRGP

The FRGP uses DSCP markings to facilitate commodity policers at the FRGP. Inbound commodity traffic,
destined for an FRGP/UPOP member not directly connected to the same router as the commodity
provider, is marked as DSCP 1. To implement the policer strategy, QoS was globally enabled on the
6500s at the FRGP. QoS is NOT implemented at the FRGP to provide preference to specific traffic types
i.e. voice, video, etc except for VOIP traffic destined to/from RAF. In this case, we trust the QoS
markings along the path.
Telepresence (or other video conferencing) capabilities appear to be on the horizon for FRGP member
institutions either via NLR, between FRGP members, or a single FRGP member with multiple connections

to the FRGP. Given the strict latency, jitter, and packet loss requirements for TP, its possible that
member institutions will request preferential treatment of TP traffic transiting the FRGP routers.
A single FRGP member requesting preferential treatment for video will impact all FRGP members in all
but the simplest case (ports are on the same router) because some QoS policy would need to be applied
on ports connecting l3-gw-1 <-> frgp-gw-2 <-> frgp-gw-3. At that point, one members traffic is being
preferred over all of the other members traffic.
In all the use cases mentioned above, the FRGP is the middleman transiting traffic from participant A to
participant B. A generic path may look like this:
particpantAs internal network <-> Tail circuit provider network<->FRGP<-> Tail Circuit provider
network<->participantBs internal network. In some cases, there may be a direct connection to a
participants network, eliminating the tail circuit provider network.
Telepresence marks video CS4, voice CS5, and signaling as CS3 (this is configurable at the TP end). Both
CS4 and CS5 are recommended to be placed in the priority queue. Ultimately we dont really care how
Telepresence is deployed in the participants networks, but we will need to know the COS/DSCP types
and bandwidth requirements for each COS/DSCP marking.
Should it be deemed necessary to implement QoS at the FRGP, a QoS policy will need to be created that
clearly defines the number and types of queues and COS/DSCP mappings to those queues. When
deploying QoS, coordination with participants and tail circuit vendors will be necessary to determine a
mapping between the different queue structures. If they dont match, then traffic will need to be
remarked either inbound and/or outbound to ensure similar treatment across different administrative
boundaries. Participants will be responsible for re-marking.
Using the QoS configuration recommendations from above as a foundation, here is an overview of the
changes that would be needed to create an FRGP QoS policy recommendation:

Input Queuing: No changes

Classification and Marking:
o Inbound Option 1: Extend trust boundaries to participant/tail circuit vendor. Any DSCP
1 marked traffic from a participant/tail circuit vendor will need to be remarked to DSCP
0 to avoid conflict with the commodity policer.
Pros: easier to config and maintain, participants responsible for correctly
marking traffic
Cons: potential for abuse
o Inbound Option 2: explicitly classify and mark (example: a policy statement that
identifies SIP traffic and marks it as CS3)
Pros: clearly identifies what traffic will be marked and at what priority
Cons: more complex policy statements, FRGP engineers responsible for
correctly marking traffic

Recommend Inbound Option 1 Note that for either option, DSCP values might have to be remarked on a per interface basis depending on how a participants/tail circuit providers queue
strategy aligns with the FRGPs. Participants should be responsible for remarking where
possible.

Outbound: Any packet marked DSCP 1 by the commodity policer will need to be
remarked DSCP 0. If not there is a possibility that a participant/tail circuit vendor would
deferentially treat this traffic as it is typically assigned to the Scavenger Class. .

Policing:
o Inbound Option 1: No policing of COS/DSCP marked traffic
Pro: easier to admin
Con: Potential for abuse i.e. mark all traffic as priority
o Inbound Option 2: Place strict limits on COS/DSCP marked traffic
Pro: Prevents potential abuse
Con: more complicated policers
Recommend Inbound Option 2

Outbound: No change

Transmit Queuing:
o Option 1: Come up with FRGP QoS templates for services and how they will be mapped.
Similar to QMOE QoS templates.
Pro: limited set of configurations to maintain, easier to admin
Con: less flexible
o Option 2: Configure queuing on a per participant/tail circuit vendor basis.
Pro: Very flexible
Con: More configs to maintain, but not that many members
o Its possible that neither options is viable. The QoS configuration on some cards is
applied per ASIC (8 ports per ASIC on a 6148a) and not per port. More investigation is
needed.

Open questions:

Charge members $$ for priority traffic??

9. References
1) Enterprise QoS SRND
http://www.cisco.com/en/US/docs/solutions/Enterprise/WAN_and_MAN/QoS_SRND/QoS-SRND-Book.html
2) Medianet Campus Qos Design 4.0
http://www.cisco.com/en/US/docs/solutions/Enterprise/WAN_and_MAN/QoS_SRND_40/QoSCampus_40.html

3) Cisco Unified Communications SRND Based on Cisco Unified Communications Manager 6.x
http://www.cisco.com/en/US/docs/voice_ip_comm/cucm/srnd/6x/uc6_0.html

4) Comparison of the Cisco Catalyst and Cisco IOS Operating Systems for the Cisco Catalyst 6500
Series Switch
http://www.cisco.com/en/US/customer/prod/collateral/switches/ps5718/ps708/prod_white_paper09186a00800c
8441.html

5) Implementing Quality of Service Policies with DSCP

http://www.cisco.com/en/US/tech/tk543/tk757/technologies_tech_note09186a00800949f2.shtml

6) Quality of Service - The Differentiated Services Model

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6558/ps6610/product_data_sheet0900aecd803
1b36d.html

7) Understanding How Routing Updates and Layer 2 Control Packets Are Queued on an Interface
with a QoS Service Policy
http://www.cisco.com/en/US/customer/tech/tk543/tk544/technologies_tech_note09186a0080094612.shtml

8) Understanding Quality of Service on Catalyst 6000 Family Switches

http://www.cisco.com/en/US/customer/tech/tk543/tk762/technologies_white_paper09186a00800b0828.shtml#e
igth

9) Cisco AutoQoS Data Sheet

http://www.cisco.com/en/US/customer/tech/tk543/tk759/tech_digest09186a00801348a6.html

10) Monitoring Voice over IP Quality of Service

http://www.cisco.com/en/US/customer/tech/tk543/tk759/technologies_tech_note09186a0080094bc3.shtml

11) Cisco TelePresence Network Systems 2.0 Design Guide

http://www.cisco.com/en/US/docs/solutions/Enterprise/Video/TP-Book.html