Sei sulla pagina 1di 111

CIS 188 CCNP TSHOOT (Troubleshooting)

Ch. 5 Maintaining and Troubleshooting


Routing Solutions
Rick Graziani
Cabrillo College
graziani@cabrillo.edu
Fall 2014

Materials
Book:
Troubleshooting and Maintaining
Cisco IP Networks (TSHOOT)
Foundation Learning Guide:
Foundation learning for the CCNP
TSHOOT 642-832
By Amir Ranjbar
Book
ISBN-10: 1-58705-876-6
ISBN-13: 978-1-58705-876-9
eBook
ISBN-10: 1-58714-170-1
ISBN-13: 978-1-58714-170-6

Troubleshooting Routing

Network Layer Connectivity


EIGRP
OSPF
Route Redistribution
BGP

Note: You always have the option of reviewing my CCNP ROUTE


PowerPoints or attending the lectures for a review.
At the time of this presentation we will be discussing
Manipulating Routing Updates with route maps and distribution
lists in my CIS 185 ROUTE class on Tuesday.

Network Connectivity

Just like we did when we looked at Layer 2 connectivity


To troubleshoot Layer 3 connectivity, you need to have a good
understanding of the processes that are involved in routing a packet from a
host through multiple routers to the final destination.
Lack of network layer connectivity indicates that the problem is at or below
network layer.

Which decisions does Host A make to successfully send a packet destined


for Host B to the first hop Router C?
What information does it need?
Addressing
ARP Cache
Which actions does it perform?
DHCP Request
Host or GW Address
DNS Query
ARP Request
Layer 2 encapsulation

Which decisions does Router C make to successfully send the packet from
Host A destined for Host B to the next hop Router D?
TTL ICMP Time Exceeded
CEF or layer 3
Longest prefix match
Layer 2 Next-hop address
layer 2 encapsulation
Fragmentation
Which decisions does Router D and E make to successfully send the packet
from Host A destined for Host B to the next hop Router E? (Same questions)
Are there any differences in the processes and information required in order
to successfully transmit return packets from Host B back to Host A?

Alex Zinins Routing Table Principles


I know about my remote
networks but it is not my
responsibility if R2 and R3
know about their remote
networks.

Principle 1: Every router makes its decision alone, based on the


information it has in its own routing table.
R1 makes forwarding decisions based solely on the information in the
routing table.
R1 does not consult the routing tables in any other routers.
Making each router aware of remote networks is the responsibility of the
network administrator.

Alex Zinins Routing Table Principles


Just because I know how to
get to R3s LAN,
192.168.2.0/24 and I send
that packet to R2, doesnt
mean R2 knows how to get
there.

???

Principle 2: The fact that one router has certain information in its
routing table does not mean that other routers have the same
information.
8

Alex Zinins Routing Table Principles


And if the packet for R3s
LAN reaches 192.168.2.0/24,
I dont know if R3 has a route
back to 172.16.3.0/24 for any
return traffic.
???

Principle 3: Routing information about a path from one network


to another does not provide routing information about the
reverse, or return, path.

To forward packets, a router combines information from various control


plane data structures.
The most important of these data structures is the routing table.
Other structures include:
LSDB or topology table
Neighbor table
ARP or Frame Relay Map table
Unlike switches, which flood unknown frames, routers drop any packet for
which they cannot find a matching entry in the routing table.
Searches for longest possible prefix match of the destination IP address.
Associated with this entry is an egress interface - most cases, a next-hop IP
address.

10

The mapping between the next hop-IP address and the Layer 2 address or
identifier is stored in a data structure that is specific for that Layer 2
protocol.
Ethernet: ARP cache
Frame Relay: Frame Relay map table (except for point-to-point)
A routing table lookup may need to be followed up by a lookup in a Layer 3
to Layer 2 mapping table to construct a frame, encapsulate the packet, and
transmit it.

11

Routing table and layer 2 mapping to construct a frame every time a packet
needs to be routed is an inefficient approach to forwarding IP packets.
Cisco Express Forwarding (CEF) improves this process.
Can be used on all routers and layer 3 switches.
On most platforms, the CEF switching method is enabled by default.

12

CEF combines the information from the routing table and other data structures
such as Layer 3 to Layer 2 mapping tables, into two data plane data
structures:
Forwarding Information Base (FIB)
Mostly reflects the routing table with all the recursive lookups resolved.
A lookup in the FIB results in a pointer to an adjacency entry in the
CEF adjacency table.
CEF adjacency table
point-to-point interface: egress interface only
multipoint interface: egress interface and next-hop IP address

13

Troubleshooting EIGRP

14

At a high level, each routing protocol consists of the following elements and
processes: (RIP is a notable exception to some of these.)
Reception of routing information from neighbors:
Neighbor relationships
Routing protocol data structures:
Neighbor table, topology table
Route injection or redistribution:
Directly connected, dynamic routes, static routes
Route selection and installation:
Best path, equal cost load balancing, unequal cost load balancing
Transmission of routing information to neighbors:
Summarization, Broadcast, Multicast, LSAs, Route updates, ACKs

15

To diagnose and resolve problems related to EIGRP you must be able to:
Apply your knowledge of EIGRP data structures
Neighbor table, Topology Table
Apply your knowledge of EIGRP processes
DUAL, adjacency process, summarization, load balancing
Use Cisco IOS commands to gather information from the EIGRP data
structures and track the flow of EIGRP routing information
Note: As we will see these procedures can be applied to other routing
protocols as well.

16

R3# show ip eigrp interfaces


IP-EIGRP interfaces for process 1

Interface
Se0/1
Se0/0
Se0/2
Se0/3
R3#

Peers
1
1
1
1

Xmit Queue
Un/Reliable
0/0
0/0
0/0
0/0

Mean
SRTT
60
607
29
24

Pacing Time
Un/Reliable
0/15
0/15
0/15
0/17

Multicast
Flow Timer
299
3031
143
50

Pending
Routes
0
0
0
0

EIGRP stores its operational data, configured parameters, and statistics in


three main data structures:
Interface table:
Lists all interfaces that have been enabled for the processing of EIGRP
packets
Passive interfaces are not listed in this table.

17

R3# show ip eigrp neighbors


IP-EIGRP neighbors for process 1
H
Address
Interface
Type
3
2
1
0
R3#

10.0.0.18
10.0.0.14
10.0.0.5
10.0.0.9

Se0/3
Se0/2
Se0/0
Se0/1

Hold Uptime

SRTT

(sec)
(ms)
13 00:17:37
24
14 00:17:50
29
14 00:23:35 607
12 00:24:01
60

RTO

Seq

200
200
3642
360

Cnt
0
0
0
0

Num
5
4
13
21

Neighbor table:
Keeps track of all active EIGRP neighbors.
Neighbors are added to this table on the reception of hello packets
Neighbors are removed when:
Hold-time expires
Interface goes down or is removed from the interface table

18

R3# show ip eigrp topology


<output omitted>
P 10.0.0.0/30, 2 successors, FD is 2681856
via 10.0.0.9 (2681856/2169856), Serial0/1
via 10.0.0.5 (2681856/2169856), Serial0/0
<output omitted>

Feasible distance

P 172.16.0.0/16, 1 successors, FD is 2172416


via 10.0.0.14 (2172416/28160), Serial0/2
via 10.0.0.18 (2312192/28160), Serial0/3

FS if Reported
Distance is less
than Feasible
distance
successor
feasible successor

Feasible distance: if this router was the successor.


Topology table:
Holds all the routes that were received from neighboring routers, locally
injected, or redistributed into EIGRP.
EIGRP will select the best path from among the available possible paths
EIGRPs best path selection is based on the Diffusing Update Algorithm
(DUAL).

19

Initial Route Discovery


A

B
Updated

Updated
EIGRP
Neighbor
Table

Hello, I am Router A. Is anyone there?

EIGRP
Neighbor
Table

Hello, I am Router B.

Updated
EIGRP
Topology
Table

Here is all my routing information.


Im using split horizon.

Thanks for the information!


That is very nice of you.

Successor

Updated
IP
Routing
Table

Updated
EIGRP
Topology
Table
Successor

Here is all my routing information.


Im also using split horizon.
Thanks for the information!
Weve reached convergence.

Updated
IP
Routing
Table

20

After the initial updates have been exchanged, routing updates will only be
exchanged as a result of changes on the networks.
Changes can be caused by changes in connectivity, such as:
Loss of a link or neighbor
Configuration events

21

R2# debug ip routing


IP routing debugging is on
R2# conf t
R2(config)# int fa0/0
R2(config-if)# ip address 172.16.1.1 255.255.255.0
R2(config-if)# no shutdown
%LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
%LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0,
changed state to up
RT: add 172.16.1.0/24 via 0.0.0.0, connected metric [0/0]
RT: interface FastEthernet0/0 added to routing table

debug ip routing:
Not specific to EIGRP
Displays any changes that are made to the routing table, such as
installation or removal of routes.
22

debug eigrp packets:


Displays the transmission and reception of EIGRP packets.
Note: Cisco Press Implementing Cisco IP Routing (ROUTE) by Diane
Teare provides detailed explanation of the output.
23

debug ip eigrp:
Displays EIGRP routing events, such as updates, queries, and replies
sent to or received from neighbors.
Focuses on the routing information contained in the packets and the
actions that EIGRP takes as a result of the information received.
Note: Cisco Press Implementing Cisco IP Routing (ROUTE) by Diane
Teare provides detailed explanation of the output.

24

Troubleshooting Example: Routing


Problem in an EIGRP Network

25

Traffic path unexpectedly goes


through BRO2
ping from BRO1 to CRO1
succeeds, you can conclude that
the WAN link is operational at
Layers 3 and below
BRO1# traceroute 10.1.220.1
Type escape sequence to abort.
Tracing the route to cro1.mgmt.tshoot.local (10.1.220.1)
1 10.1.163.130 0 msec 0 msec 0 msec
2 10.1.194.5 12 msec 12 msec *
BRO1# ping 10.1.194.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.1.194.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 28/29/32 ms

26

Two likely reasons why the traffic is routed through router BRO2 instead of
directly across the WAN to router CRO1:
BRO1 has not learned about the direct route to router CRO1.
BRO1 selects the route through router BRO2 as the best route.
Lets look at the EIGRP topology table

27

EIGRP Troubleshooting
Example Cont.

EIGRP show commands indicate that


there is only one BR01 topology entry for
CR01 Lo0 and that BR01
BRO1# show ip eigrp topology 10.1.220.1 255.255.255.255
IP-EIGRP (AS 1): Topology entry for 10.1.220.1/32
State is Passive, Query origin flag is 1, 1 Successor(s), FD is 40642560
Routing Descriptor Blocks:
10.1.163.130 (FastEthernet0/1.30), from 10.1.163.130, Send flag is 0x0
Composite metric is (40642560/40640000), Route is Internal
Vector metric:
Minimum bandwidth is 64 Kbit
Total delay is 25100 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 2

The topology table lists all routes that were received, so you can see if the
direct route to CRO1 is missing.
Note: This command displays all entries in the topology table for this
destination, not just feasible successors similar to all-links option.
No direct route!
Need to determine whether the route was not learned because:
A neighbor relationship with CRO1 was never established
The relationship was established but the specific route was not
exchanged.
Good next step is to display the neighbor table

29

EIGRP Troubleshooting
Example Cont.

BRO1# show ip eigrp neighbors


IP-EIGRP neighbors for process 1
H
Address
Interface
0

10.1.163.130

Fa0/1.30

Hold Uptime
SRTT
(sec)
(ms)
12 00:09:56
4

RTO

Q Seq
Cnt Num
200 0 585

CRO1 is not listed as a neighbor on router BRO1.


Could be one or the other routers is not:
Sending Hello packets
Ignoring Hello packets
Lets look at the EIGRP interfaces to make sure the serial interface is
configured for EIGRP.

EIGRP Troubleshooting
Example
Cont. interface is
Only the BR01Fa0/1.30
participating in EIGRP.
BRO1# show ip eigrp interfaces
IP-EIGRP interfaces for process 1

Interface
Fa0/1.30

Peers
1

Xmit Queue
Un/Reliable
0/0

Mean
SRTT
4

Pacing Time
Un/Reliable
0/1

Multicast
Flow Timer
50

Pending
Routes
0

Does not list the serial interface of router BRO1.


So, even if hello packets are received on the serial interface, router BRO1
does not process them.
Two conditions need to be met for an interface to be added to the EIGRP
interface table:
The interface has to be up and its IP address must match one of the
configured network statements.
The interface should not be configured as a passive interface.
Lets look at the running-config

EIGRP Troubleshooting
Example Cont.
The show run command reveals
that network statement for
10.1.194.1 is the problem.
BRO1# show running-config | section router eigrp
router eigrp 1
network 10.1.163.129 0.0.0.0
network 10.1.194.1 0.0.0.0
no auto-summary

Problem with one of the network statements.


The statement network 10.1.194.1 0.0.0.0 matches IP address 10.1.194.1
the IP address of router CRO1
Does not match the serial interface of BRO1
Replaced it with the statement network 10.1.194.2 0.0.0.0 or some other
network statement

EIGRP Troubleshooting
Example
EIGRP
Cont.
After
correcting the
network

statement, both BR01 interfaces are


participating in EIGRP and BR02 and CR01
are BR01 neighbors.

BRO1# show ip eigrp interfaces


IP-EIGRP interfaces for process 1

Interface
Fa0/1.30
Se0/0/0.111

Peers
1
1

Xmit Queue
Un/Reliable
0/0
0/0

Mean
SRTT
1
707

BRO1# show ip eigrp neighbors


IP-EIGRP neighbors for process 1
H
Address
Interface
1
0

10.1.194.1
10.1.163.130

Se0/0/0.111
Fa0/1.30

Pacing Time
Un/Reliable
0/1
10/380

Multicast
Flow Timer
50
4592

Hold Uptime
SRTT
(sec)
(ms)
14 00:10:10 707
12 01:34:49
1

RTO

Pending
Routes
0
0

Q
Cnt
4242 0
200 0

Seq
Num
783
587

The interface table now lists subinterface Serial 0/0/0.111 in addition to subinterface
FastEthernet 0/1.30.
This means that EIGRP packets are now processed on interface Serial 0/0/0.111.
IP address of router CRO1 (10.1.194.2) is now listed in the neighbor table on the
WAN interface.

Successor

The EIGRP topology table now lists two entries for network 10.1.220.1/32.
Note: BR02 is listed but is not a Feasible Successor (RD is not less than
FD)

34

The EIGRP topology table now


lists two entries for network
10.1.220.1/32.
Note: BR02 is listed but is not a
Feasible Successor (RD is not
less than FD)

BRO1# show ip eigrp topology 10.1.220.1 255.255.255.255


IP-EIGRP (AS 1): Topology entry for 10.1.220.1/32
State is Passive, Query origin flag is 1, 1 Successor(s), FD is 40640000
Routing Descriptor Blocks:
10.1.194.1 (Serial0/0/0.111), from 10.1.194.1, Send flag is 0x0
Composite metric is (40640000/128256), Route is Internal
Vector metric:
Minimum bandwidth is 64 Kbit
Total delay is 25000 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 1
10.1.163.130 (FastEthernet0/1.30), from 10.1.163.130, Send flag is 0x0
Composite metric is (40642560/40640000), Route is Internal
Vector metric:
Minimum bandwidth is 64 Kbit
Total delay is 25100 microseconds
Reliability is 255/255
Load is 1/255
Minimum MTU is 1500
Hop count is 2

show ip route command with network 10.1.220.1 confirms that the path
through router CRO1 has been installed in the routing table
traceroute command confirms that this path is now used to forward packets
to the loopback of CRO1.

36

show ip route command with network


10.1.220.1 confirms that the path
through router CRO1 has been
installed in the routing table
traceroute command confirms that
this path is now used to forward
packets to the loopback of CRO1.
BRO1# show ip route 10.1.220.1 255.255.255.255
Routing entry for 10.1.220.1/32
Known via "eigrp 1", distance 90, metric 40640000, type internal
Redistributing via eigrp 1
Last update from 10.1.194.1 on Serial0/0/0.111, 00:20:55 ago
Routing Descriptor Blocks:
* 10.1.194.1, from 10.1.194.1, 00:20:55 ago, via Serial0/0/0.111
Route metric is 40640000, traffic share count is 1
Total delay is 25000 microseconds, minimum bandwidth is 64 Kbit
Reliability 255/255, minimum MTU 1500 bytes
Loading 1/255, Hops 1
BRO1# traceroute 10.1.220.1
Type escape sequence to abort.
Tracing the route to cro1.mgmt.tshoot.local (10.1.220.1)
1 10.1.194.1 16 msec 12 msec *

Troubleshooting OSPF

38

To diagnose and resolve problems related to OSPF you must be able to:
Apply your knowledge of OSPF data structures
Apply your knowledge of OSPF processes within an area
Apply your knowledge of OSPF processes between areas
Use Cisco IOS commands to gather information from the OSPF data
structures and track the flow of OSPF routing information
39

At a high level, each routing protocol consists of the following elements and
processes: (RIP is a notable exception to some of these.)
Reception of routing information from neighbors:
Neighbor relationships and LSAs
Routing protocol data structures:
LSDB and SPF
Route injection or redistribution:
Directly connected, dynamic routes, static routes
Route selection and installation:
Best path, equal cost load balancing
Intra-area, Inter-area and external routes, area types
Transmission of routing information to neighbors:
Forward to other neighbors
Internal routers, ABRs, ASBRs

40

R1# show ip ospf interface serial 0/0/0


Serial0/0/0 is up, line protocol is up
Internet Address 192.168.10.1/30, Area 0
Process ID 1, Router ID 10.1.1.1, Network Type POINT_TO_POINT, Cost: 64
Transmit Delay is 1 sec, State POINT_TO_POINT,
Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
<output omitted>

OSPF stores its operational data, configured parameters, and statistics in


four main data structures:
Interface table:
Lists all interfaces that have been enabled for OSPF.
When an interface is configured as a passive interface, it is still listed in
the OSPF interface table, but no neighbor relationships are established
on this interface.
41

R1# show ip ospf neighbor


Neighbor ID
10.3.3.3
10.2.2.2

Pri
1
1

State
FULL/ FULL/ -

Dead Time
00:00:30
00:00:33

Address
192.168.10.6
192.168.10.2

Interface
Serial0/0/1
Serial0/0/0

Neighbor table:
Keeps track of all active OSPF neighbors.
OSPF goes through a number of states while establishing a neighbor
relationship (also known as adjacency)
Lists the current state for each individual neighbor

42

Link-state database:
This is the main data structure that OSPF uses to store all its network
topology information.
Contains a wealth of network topology information
One of the most important data structures to gather information from
when troubleshooting OSPF problems.

43

R33# show ip route


33.0.0.0/32 is subnetted, 1 subnets
C
33.33.33.33 is directly connected, Loopback0
172.16.0.0/24 is subnetted, 2 subnets
C
172.16.1.0 is directly connected, FastEthernet0/0
O
172.16.2.0 [110/2] via 172.16.1.1, 00:02:13, FastEthernet0/0
172.30.0.0/24 is subnetted, 2 subnets
C
172.30.1.0 is directly connected, FastEthernet0/1
O
172.30.2.0 [110/2] via 172.16.1.3, 00:02:23, FastEthernet0/0
O*IA 0.0.0.0/0 [110/2] via 172.16.1.1, 00:02:13, FastEthernet0/0

Routing information base:


After executing the SPF algorithm, the results of this calculation are
stored in the RIB (Routing Information Base).
Routes might be added to or deleted from the RIB without the need for a
SPF recalculation.
From the RIB, OSPF offers its routes to the IP routing table.
Note: The collection of best paths is known as the OSPF RIB.
There is not a separate physical data structure called the OSPF RIB.
Many writings refer to the IP routing table as the RIB to allow it to be
distinguished from the FIB that CEF creates.

44

LSA 1s Router LSAs

LSA 1s

LSA 1s
LSA 1s

show ip ospf database Router Link States (LSA 1s)


Should display all the RouterIDs of routers in that area,
including its own.
show ip route O routes
Routes within that area

45

LSA 2s Network LSAs

LSA 2s
LSA 2s

show ip ospf database Net Link States (LSA 2s)


Net Link States (LSA2s) should display the RouterIDs of the
DRs on all multi-access networks in the area and their IP
addresses.
show ip route O routes
Routes within that area

46

LSA 3 Summary LSAs

LSA 3s

LSA 3s
LSA 1s

show ip ospf database Summary Net Link States (LSA 3s)


Link ID = IP network addresses of networks in other areas
ADV Router = ABR Router ID sending the LSA-3
show ip route IA (Inter-Area Routes)
Routes in other areas

47

LSA 4 ASBR Summary Link States

LSA 4

LSA 1s
ebit
LSA 4

show ip ospf database Summary Net Link States (LSA 3s)


Link ID = IP network addresses of networks in other areas
ADV Router = ABR Router ID sending the LSA-3
show ip route IA (Inter-Area Routes)
Routes in other areas

48

LSA 5 External Link States


R2 (ASBR)
router ospf 1
redistribute static
ip route 57.0.0.0 255.0.0.0 ser 0/3

LSA 5

LSA 5s
LSA 5

Redistribute command creates an ASBR router.


Originated by the ASBR.
Describes destination networks external to the OSPF Routing Domain
Flooded throughout the OSPF AS except to stub and totally stubby areas
49

Stub Area

LSA 1s still
sent within
each area.

LSA 3
LSA 4
LSA 5

Stub Area
LSA 3
LSA 4 Blocked
LSA 5 Blocked

Default
route to
ABR
injected

We only see routes in


our area, other areas,
and a default route.
No external routes.

Sent by ABR: LSA 3s (Inter-Area routes)


Blocked:
LSA 4s (reachability to ASBR)
LSA 5s (External routes)
The ABR injects a default route into the stub area, pointing to the ABR.
This does not mean the ABR has a default route of its own.
Changes in External routes no longer affect Stub Area routing tables.

50

Totally Stubby Area


Totally Stubby
Area
Blocked LSA 3
Blocked LSA 4
Blocked LSA 5

Default
route to
ABR
injected

We only see routes in our area and


a default route.
No inter-area or external routes.

LSA 1s still
sent within
each area.

Stub Area
LSA 3
LSA 4 Blocked
LSA 5 Blocked

Default
route to
ABR
injected

We only see routes in


our area, other areas,
and a default route.
No external routes.

Blocked:
LSA 3s (Inter-Area routes)
LSA 4s (reachability to ASBR)
LSA 5s (External routes)
The ABR injects a default route into the stub area, pointing to the ABR.
This does not mean the ABR has a default route of its own.
51
Changes in other areas and external routes no longer affect Stub Area routing tables.

For both areas

For both areas

If the network is a transit network (more than one router is connected to it)
the Designated Router generates a type-2 LSA that describes the link state
for that link including its subnet and connected routers.
52

R1# show ip ospf neighbor


Neighbor ID
10.3.3.3
10.2.2.2

Pri
0
0

State
FULL/ FULL/ -

Dead Time
00:00:35
00:00:36

Address
192.168.10.6
192.168.10.2

Interface
Serial0/0/1
Serial0/0/0

Two routers will become neighbors only if the following parameters match in
the Hello packets:
Hello and dead timers:
Use the same Hello and dead time.
Broadcast and point-to-point type networks are 10-second Hello and 40second dead time.
OSPF area number:
Both consider that link to be in the same area.
OSPF area type:
Only if they both consider the area to be the same type of area (normal,
stub, or not-so-stubby area [NSSA]).
IP subnet and subnet mask:
If they are on the same subnet.
The exception to this rule is on a point-to-point link, where the subnet
mask is not verified.

53

Steps to OSPF Operation with States


Hello 10.6.0.1 10.5.0.1
Hello 10.6.0.1
Down
Init
2-way

Down
Init
2-way

Hello 10.5.0.1
Hello 10.5.0.1 10.6.0.1
Down State - Init State Two Way State
Down State - OSPF routers send Hello packets at regular intervals (10 sec.) to establish
neighbors.
When a router (sends or) receives its first Hello packet, it enters the init state.
Hello packet contains a list of known neighbors.
When the router sends a Hello packet (unicast reply) to the neighbor with its RouterID and
the neighbor sends a Hello packet packet back with that Router ID, the routers interface will
transition to the two-way state.
Now, the router is ready to take the relationship to the next level.
54

Steps to OSPF Operation with States (cont)

Explanations in Notes Section

55

debug ip ospf packet:


Displays the transmission and reception of OSPF packets.
Only the packet headers are displayed, not the content of the packets.
This command is useful to verify if Hellos are sent and received as expected.
debug ip ospf events:
Displays OSPF events.
This includes reception and transmission of Hellos, but also the establishment of
neighbor relationships and the reception or transmission of LSAs.
This command can also provide clues (mismatched parameters such as timers,
area number, etc.) as to why neighbor Hellos might be ignored.
debug ip ospf adj:
Displays events that are related to the adjacency building
See a neighbor relationship transition from one state to the next.
Observe possibly the state the relation getting stuck at.
debug ip ospf monitor:
Monitors when the SPF algorithm is scheduled to run and displays the triggering
LSA and a summary of the results after the SPF algorithm has completed.
Allows you to discover which LSA was received and triggered an SPF
computation.
Can easily discover a flapping link.
56

Troubleshooting Example: Routing


Problem in an OSPF Network

57

CRO1# show ip route 10.1.152.0 255.255.255.0


Routing entry for 10.1.152.0/24
Known via "ospf 100", distance 110, metric 2, type inter area
Last update from 10.1.192.1 on FastEthernet0/0, 00:00:11 ago
Routing Descriptor Blocks:
* 10.1.192.1, from 10.1.220.252, 00:00:11 ago, via FastEthernet0/0
Route metric is 2, traffic share count is 1

CR01#showiproute10.1.152.0
Examine the routing table on router CRO1, you only find a single entry, the
path through router CSW1.
Should be two equal cost paths

This result is unexpected because there are two equal cost paths are available
to CRO1:
via CSW1
via CSW2
Ping CSW2
Because this ping succeeds, conclude that the Fast Ethernet link between
router CRO1 and router CSW2 is operational at Layers 3 and below.
Need to find out why the second, equal cost path through router CSW2 is not
installed in the routing table
There are two main reasons why this could be happening.
CSW2 is not advertising subnet 10.1.152.0/24 to area 0
Cost to reach subnet 10.1.152.0/24 through router CSW2 from router CRO1
59
is considered to be worse than the cost through router CSW1.

It is expected that both routers CSW1 and CSW2 advertise a type-3


summary LSA for subnet 10.1.152.0/24.
CRO1 shows two entries both a cost of 1.
Router ID 10.1.220.252 (CSW1)
Router ID 10.1.220.253 (CSW2)

60

Therefore, the preference for the path to 10.1.152.0/24 via CSW1 must be
based on the topology within area 0.
Given that router CRO1 has a direct connection in area 0 to both router CSW1
and CSW2, there are only two plausible explanations for the fact that router
CRO1 is not using the path via router CSW2.
Either the direct path to router CSW2 is not used because routers CSW2
and CRO1 have not become neighbors
The path is not used because the cost for interface FastEthernet 0/1 is
higher than the cost for interface FastEthernet 0/0

61

Note: Unlike the show ip eigrp interfaces command,


the show ip ospf interface command will
display interfaces that are enabled for OSPF,
but configured as passive interfaces.

To verify if router CRO1 has established, a proper neighbor relationship with


router CSW1, the show ip ospf interface
There could be several reasons why router CSW2 is not listed as a
neighbor of CRO1:
CSW2 is not sending Hellos
Hellos are received, but ignored due to mismatched Hello parameters.
Hellos are sent, but not received, because interface FastEthernet 0/1
has not been activated for OSPF and therefore does not listen to the
OSPF multicast group 224.0.0.5

62

The is a problem with one of the network statements.


The statement network 10.1.192.9 0.0.0.0 area 0 matches IP address
10.1.192.9, which is not one of router CRO1s IP addresses, but an IP
address of router CSW2.
network statement needs to be replaced with the statement network
10.1.192.10 0.0.0.0 area 0 or some other network statement

63

After change, verify interface and neighbor relationship


64

Verify route

65

Troubleshooting the book

Through no fault of the author the diagram in the book is incorrect.


Why is this diagram incorrect given this output?
CR01 cannot receive LSA 3s from CSW2 unless:
Establishes a neighbor adjacency with CSW2
CSW2 forwards the LSA3 to CSW1 within area 0 (CR01s area)

66

Troubleshooting Route
Redistribution

67

Ideally, no more than one interior (intra-AS) routing protocol is used within an
organization.
Reasons for running multiple routing protocols:
Company mergers and different IGPs are used
Company has different divisions with the network under separate control
for business or political reasons
Company has connections between business partners
To allow multivendor interoperability (OSPF on non-Cisco, EIGRP on
Cisco, for instance)
Improper route redistribution can lead to:
Suboptimal routing
Routing feedback (routing loops)

68

Router(config-router)# network network-address [protocol options]

Router(config-router)# redistribute protocol [protocol options]

There are two ways for routes to be injected in a routing protocol:


Directly connected:
Enabling the routing protocol on an interface.
These routes are considered internal by the routing protocol.
External:
Networks from a different source that are present in the routing table
Because these routes were not originated by the routing protocol,
they are considered external.
69

There are two important conditions that must be met for a prefix learned
from one protocol (using redistribution) to be successfully advertised
through another protocol:
The route needs to be installed in the routing table doing the
redistribution.
A proper seed metric is assigned to the redistributed route (except
for OSPF which defaults to 20)

70

Example
1000 33 255 1

BW=20

BW=20

R2 summary:
router eigrp 1
network 172.30.0.0
auto-summary
redistribute ospf 1

OSPF learned networks are distributed


into the EIGRP domain

default-metric 1000 33 255 1 1500 Use the metrics for BW DLY RLY Load
redistribute connected metric 1000 33 255 1 1500

Distribute any directly connected networks


and use these metrics for BW DLY RLY Load

!
router ospf 1

network
10.0.0.0 0.0.0.255 area 0
So far
network 172.16.0.0 0.0.0.3 area 0
EIGRP learned networks are distributed
into the OSPF domain, default metric of 20
redistribute connected Distribute any directly connected networks
and use default metric of 20

redistribute eigrp 1 subnets

Using Route Maps to Avoid Route Feedback


RIPv2
192.168.1.0/24

OSPF
E

0 E2 192.168.1.0/24

Multi-point boundary routers may cause suboptimal routing or routing loops.


RIPv2 on Router C advertises network 192.168.1.0.
Routers A and B redistribute the network into OSPF.
OSPF then advertises the route to its neighbor OSPF routers as an OSPF
external route.
The route passes through the OSPF AS and eventually makes its way back
to the other edge router.
Router B (or A) then redistributes 192.168.1.0 from OSPF back into the
original RIPv2 network; this is a routing feedback loop.
Depending upon the metrics this may result in suboptimal routing or even a
routing loop!

72

router ospf 10
redistribute rip subnets
router rip
redistribute ospf 10 route-map OSPF_into_RIP
route-map OSPF_into_RIP deny 10
match ip address 1

192.168.1.0/24 network will not be redistributed into RIP

route-map OSPF_into_RIP permit 20

All other routes will be redistributed into RIP

access-list 1 permit 192.168.1.0 0.0.0.255

To prevent the routing feedback loop, a route map called OSPF_into_RIP has
been applied to Routers A and B when redistributing OSPF routes into RIP.
The decision to filter a route or allow the route through is based on the deny
or permit in the route-map command, and not the deny or permit in the ACL
or prefix list.

73

Using Route Maps With Tags (Reminder from Rick )


router eigrp 100
redistribute rip metric 1000 100 255 1 1500 route-map into-eigrp
route-map into-eigrp deny 10
match tag 40
route-map into-eigrp permit 20
set tag 20

route tag A unitless 32-bit integer that most routing protocols can assign
to any given route.
Tag follows the route advertisement, even through the redistribution
process.
Another router may use an IOS tool such as route-maps to match routes
with a given route tag to make a decision.
Can be used to help to solve the domain loop problem.
74

RIPv2

Tag 40

EIGRP

R1

Tag 20

172.16.0.0/16

Tag 40

R2

10.0.0.0/8

Tag 20

router eigrp 100


redistribute rip metric 1000 100 255 1 1500 route-map into-eigrp
router rip
redistribute eigrp 100 metric 3 route-map into-rip
route-map into-eigrp permit 20
set tag 20
route-map into-rip permit 20
set tag 40

Routes redistributed into EIGRP are tagged with the value 20


Routes redistributed into RIP are tagged with the value 40
BUT BEFORE we tag and allow a route into the domain we want to deny any routes
that the other router already tagged and allowed in from the other domain.

75

RIPv2

R1

Tag 40

172.16.0.0/16

EIGRP

Tag 40

Tag 40

R2

Tag 40

10.0.0.0/8

router eigrp 100


redistribute rip metric 1000 100 255 1 1500 route-map into-eigrp
router rip
redistribute eigrp 100 metric 3 route-map into-rip
route-map into-eigrp deny 10
match tag 40

Updates with the tag 40 are not allowed to go back into EIGRP

route-map into-eigrp permit 20


set tag 20
All other updates get a tag 20 are allowed into EIGRP
route-map into-rip deny 10
match tag 20
route-map into-rip permit 20
set tag 40

76

RIPv2

Tag 20

EIGRP

R1

Tag 20

172.16.0.0/16
11.0.0.0/8

Tag 20

R2

10.0.0.0/8

Tag 20

router eigrp 100


redistribute rip metric 1000 100 255 1 1500 route-map into-eigrp
router rip
redistribute eigrp 100 metric 3 route-map into-rip
route-map into-eigrp deny 10
match tag 40
route-map into-eigrp permit 20
set tag 20
route-map into-rip deny 10
match tag 20

Updates with the tag 20 are not allowed to go back into RIP

route-map into-rip permit 20


set tag 40

All other updates get a tag 40 are allowed into RIP

77

R3
11.0.0.0.0/8

Notice that the route tags do not appear on the routes within the R1 and R2
routers, because these routers learn about all routes from both RIP and
EIGRP directly.

78

20
20

11.0.0.0.0/8

R3

However, show ip route output from router R3, an internal router in the
EIGRP network.
Notice that router R3 does see network 11.0.0.0 with a tag of 20
This tag is carried with the route as R3 advertises it to other routers in the
EIGRP network, including R1 and R2.
When routers R1 and R2 see the tag of 20, they do not redistribute the
11.0.0.0 route back into RIP.
79

Troubleshooting BGP

80

BGP is classified as an EGP or an inter-autonomous-system (inter-AS)


routing protocol.
BGP does not use metrics such as bandwidth, delay or path cost to find the
best paths within an AS
It is used to:
Exchange routing information with external networks (other autonomous
systems)
Selection of guaranteed loop-free path using policies (attributes)
81

BGP
Characteristics

BGP updates are carried using TCP on port 179.


In contrast:
RIP updates use UDP port 520
EIGRP uses EIGRPs RTP
OSPF does not use a Layer 4 protocol but uses OSPF mechanisms
for reliability (OSPF ACKs)
Because BGP requires TCP:
IP connectivity must exist between BGP peers.
TCP connections must also be negotiated between them before updates
can be exchanged.
BGP inherits those reliable, connection-oriented properties from TCP.
Flow control properties as well (sliding windows)
BGP assumes that its communication is reliable and therefore, BGP does not
have to implement any retransmission or error-recovery mechanisms, like
EIGRP or OSPF does.

82

BGP Neighbor Relationships

Neighbors or peers - Two routers that establish a TCP-enabled BGP


connection between each other.
BGP speaker - Each router running BGP.
A BGP speaker has a limited number of BGP neighbors with which it peers
and forms a TCP-based relationship.
BGP peers can be either:
Internal to the AS
External to the AS

83

External BGP Neighbors


EBGP: BGP is running between routers in different autonomous systems.
Routers running EBGP are usually directly connected to each other
EBGP multi-hop allows EBGP neighbors not to be directly connected
(later)

84

There are several requirements for EBGP neighborship:


Different AS number:
EBGP neighbors must reside in different autonomous systems to be
able to form an EBGP relationship.
Define neighbors:
A TCP session must be established prior to starting BGP routing update
exchanges.
Reachability:
The IP addresses used in the neighbor command must be reachable
EBGP neighbors are usually directly connected.

85

EBGP
RTA(config)#router bgp 100
RTA(config-router)#
RTB(config)#router bgp 200
RTB(config-router)#
Configuring EBGP neighbors (more later)
To begin configuring a BGP process, issue the following familiar command:
Router(config)#router bgp AS-number
BGP configuration commands appear similar to familiar IGP but it is different!
Note: Cisco IOS permits only one BGP process to run at a time, thus, a router cannot
belong to more than one AS.
Because the two AS numbers are different, BGP will start an EBGP connection with
RTA.

86

EBGP
RTA(config)#router bgp 100
RTA(config-router)#neighbor 10.1.1.1 remote-as 200
RTB(config)#router bgp 200
RTB(config-router)#neighbor 10.1.1.2 remote-as 100
Configuring EBGP neighbors (more later)
Neighbor command - Used to establish a neighbor relationship with another BGP
router.
Router(config-router)#neighbor ip-address remote-as AS-number
Identifies a peer router with which the local router will establish a session.
The AS-number argument determines whether the neighbor router is an EBGP or an
IBGP neighbor
Different AS numbers mean EBGP peers
Same AS numbers mean IBG peers

87

Internal BGP Neighbors


IBGP: When BGP is running between routers within the same AS
IBGP is run within an AS to exchange BGP information so that:
All internal BGP routers have the same BGP routing information about
outside autonomous systems
This information can be passed to other autonomous systems.
Typically full-mesh on all routers in the transit path between ASs
88

There are several requirements for IBGP neighborship:


Same AS number:
IBGP neighbors must reside in the same AS to be able to form an IBGP
relationship.
Define neighbors:
A TCP session must be established between neighbors prior to start
exchanging BGP routing updates.
Reachability:
IBGP neighbors must be reachable; an IGP typically runs inside the AS.
Do not have to be directly connected.

89

RTB(config)#router bgp 200


RTB(config-router)#neighbor 172.16.1.2 remote-as 200

RTC(config)#router bgp 200


RTC(config-router)#neighbor 172.16.1.1 remote-as 200

The remote-as value (200) is the same routers will attempt to establish an
IBGP session.
Note: AS 200 is not a remote AS , for simplicity, the keyword remote-as is
used.

90

ISP1#show ip bgp
Network
128.107.0.0/19

*
*>
* 200.0.0.0
*>

Next Hop
10.1.1.5
10.1.1.1
10.1.1.5
10.1.1.1

Metric LocPrf Weight Path


0
0 100 i
0
0 100 I
0 100 2 i
0 100 2 i

# show ip bgp neighbors


BGP neighbor is 129.213.1.1, remote AS 200, external link
BGP version 4, remote router ID 175.220.12.1
BGP state = Established, table version = 3, up for 0:10:59
Last read 0:00:29, hold time is 180, keepalive interval is 60 seconds
Minimum time between advertisement runs is 30 seconds
Received 2828 messages, 0 notifications, 0 in queue
Sent 2826 messages, 0 notifications, 0 in queue
Connections established 11; dropped 10

Specifics of BGP
Reception of routing information from neighbors:
Neighbors need not be directly connected.
Neighbors are manually configured, not automatically discovered through
a hello protocol.
Routing protocol data structures:
BGP has two main data structures.
Neighbor table to keep track of the state of configured neighbors.
BGP table, which BGP uses to store all the prefixes, including those
received from the neighbors.

91

Route injection or redistribution:


BGP does not automatically inject any routes into the BGP table
Learned via:
Redistribute into IGP (ex: OSPF or EIGRP)
Configure iBGP: subject to BGP synchronization rule
BGP synchronization rule states that a BGP router E1 should not
advertise to external neighbors (ISP1) destinations (200.0.0.0) learned from
inside BGP neighbors (E2) unless those destinations are also known via an
IGP (OSPF).
See my BGP presentations for more information.

92

Route selection and installation: BGP has a complex decision algorithm


to compare paths received from different neighbors and select the best one
for each prefix.

93

BGP FSM

BGP FSM includes six states:


1. Idle
2. Connect
3. Active
4. OpenSent
5. Open Confirm
6. Established
Note: Curved arrows should show
pointing back to the same state.
94

Transmission of routing information to neighbors:


Paths that are selected as best in the BGP table can be advertised to
other BGP routers.
Several rules, such as the one commonly referred to as IBGP splithorizon rule
BGP Split Horizon Rule - Routes learned via IBGP peers are not
propagated to other IBGP peers.
95

BGP Synchronization

I learned about 172.16.0.0 via IBGP from


Router B. I will not advertise 172.16.0.0 to
Router E via EBGP unless I see this network
in my routing table leaned via an IGP (OSPF).

Note: There is not a physical


link b/t A and B

OSPF

?
IBGP

AS 65000 networks
Into OSPF

BGP synchronization rule states: A BGP router should not use or advertise
to an external BGP neighbor a route learned by IBGP, unless that route is
directly connected or learned from the IGP.
In the past this use to be the default.
If there were small enough number of BGP routes they could be
redistributed into the IGP (by Router A and Router B).
Routers C and D would then know about 172.16.0.0 and all AS 65000
networks via redistribution by Router B.
Then IBGP would not have to run on all routers in the transit path.
96

BGP Synchronization

OSPF

I learned about 172.16.0.0 via IBGP from


Router B. I will not advertise 172.16.0.0 to
Router E via EBGP unless I see this network
in my routing table leaned via an IGP (OSPF).

?
IBGP

AS 65000 networks
Into OSPF

It is important that Router C and Router D learn about the networks from AS
650000 (172.16.0.0).
Otherwise, when Router A forwards a packet to Router C destined for
172.16.0.0, Router C would drop the packet because that network is not in
its routing table.
This is why synchronization was the default on BGP routers.
However, in the modern Internet it not practical to redistribute so my
networks into the IGP this is no longer the best practice!
97

I learned about 172.16.0.0 via IBGP from


Router B. I can
will not
advertise
advertise
172.16.0.0
172.16.0.0
to to
Router E via EBGP even
unlessif Ithis
seenetwork
this network
in
in myinrouting
NOT
my routing
table table
leaned
leaned
via anvia
IGP
an(OSPF).
IGP
(OSPF).

No BGP
Synchronization
OSPF

?
IBGP

AS 65000 networks
Into OSPF

Best practice is to no longer redistribute BGP networks into the IGP.


Instead, all routers in the transit path should be fully meshed IBGP.
Synchronization is disabled by default in Cisco IOS 12.2(8)T and later.
This means that a BGP router can advertise to an external BGP neighbor a
route learned by IBGP regardless whether or not that network is in the
routing table via IGP.
All routers in the in the transit path must be running IBPG and be fully
meshed.
Each transit router in the AS must have a neighbor relationship with all
other transit routers.

98

No BGP
Synchronization

I learned about 172.16.0.0 via IBGP from


Router B. I can advertise 172.16.0.0 to
Router E via EBGP even if this network in
NOT in my routing table leaned via an IGP
(OSPF).

OSPF

The BGP synchronization rule does ensure consistency of information


throughout the AS and avoids black holes
For example, advertising a destination to an external neighbor when not all the
routers within the AS can reach the destination) within the autonomous
system.
So, when synchronization is disabled you must make sure that all routers in
the transit path are fully meshed via IBGP.
Otherwise, when Router A forwards a packet to Router C destined for 172.16.0.0,
Router C would drop the packet because that network is not in its routing table.
This allows the routers to carry fewer routes in IGP and allows BGP to converge
more quickly because it can advertise the routes as they are learned.
99

Troubleshooting Example: Routing Problem


in a BGP Network

100

When a traceroute command is executed on router IRO1 to determine the


path that is used to reach IP address 192.168.224.1, which belongs to an IP
address block that is owned by ISP1
Traffic path goes through router IRO2

101

The successful ping to ISP1s access router verifies that the link between
router IRO1 and ISP1 access router is operational at Layers 3 and below.
Need to find out why the direct path to Internet service provider 1 is not
being used to route traffic to IP address 192.168.224.1.
There are two likely reasons:
IRO1 has not learned about the prefix directly from ISP1 at all
IRO1 router has learned about the route, but it incorrectly prefers the
path through router IRO2
Lets look at the routing table

102

Output shows that the route to network 192.168.224.0/19 is:


The best match for destination IP address 192.168.224.1 and that this
route is an internal BGP path and its source is the router with IP address
10.1.220.4, which is IRO2.
However, the IP routing table does not show if the path through IRO2 is
used because:
No other path is available, or
The path through IRO2 was selected as the best path.
Need to look at the BGP table

103

Output shows that:


The only the path through router IRO2 is present in the BGP table and (no
other BGP learned paths are available)
The path through IRO2 is the best (only) path
ISP1 is not present in the BGP table can have several different causes:
ISP1 might not advertise the route
ISP1 advertises it, but router IRO1 rejects or ignores the advertisement
IRO1 and ISP1 have not successfully established a peering relationship
and no routes have been exchanged at all

104

Output reveals:
IRO1 peering to IP address 10.1.220.4 (IRO2) has been established and
6 prefixes have been received from the neighbor
But peering to IP address 192.168.224.244 is in the Active state.
This means that this router is trying to establish a TCP session to neighbor
192.168.224.244, but has not succeeded yet.
In the Active state, the BGP process is trying to initiate a TCP connection
with the neighbor.

105

IRO1#show run
...
router bgp 64568
neighbor 192.168.224.244 remote-as 65525
...

Ping to the IP address 192.168.224.244 which is the configured neighbor


address for ISP1 fails.
Two possible causes ping not succeeding:
The IP address 192.168.224.244 is not the correct IP address and/or
IP address 192.168.224.254 should have been used for the peering to
ISP1 instead
Note: It is possible that we could ping ISP1 but still not able to establish a
BGP peering because of something like a wrong AS in the neighbor
statement.

106

Change neighbor command to correct IP Address


Neighbor is now sending Prefixes to IRO1
New best path via ISP1

107

Output shows that router IRO2 has received a path to 192.168.224.0/19


from router IRO1 and installed it in its BGP table.
IRO2 selects the path through (learned from) IRO1 as the best path to prefix
192.168.22.0/19 in AS 65525.

108

Output confirms that the path through neighbor 192.168.224.254 (ISP1) has
been installed in the routing table
The result of the traceroute command confirms this.

109

To summarize, each routing protocol consists of the following elements and


processes: (RIP is a notable exception to some of these.)
Reception of routing information from neighbors:
Neighbor relationships
Routing protocol data structures:
Neighbor table, topology table
Route injection or redistribution:
Directly connected, dynamic routes, static routes
Route selection and installation:
Best path, equal cost load balancing, unequal cost load balancing
Transmission of routing information to neighbors:
Summarization, Broadcast, Multicast, LSAs, Route updates, ACKs

110

CIS 188 CCNP TSHOOT (Troubleshooting)

Ch. 5 Maintaining and Troubleshooting


Routing Solutions
Rick Graziani
Cabrillo College
graziani@cabrillo.edu