Chapter 10 IP Services Troubleshooting

I. NAT Troubleshooting
Types of NAT Type of NAT Static NAT Dynamic NAT Description A one-to-one mapping of private internal IP addresses to public external IP addresses A dynamic mapping of private internal IP addresses to a pool of public external IP addresses

NAT Overloading Allows multiple private internal IP addresses to use a single public external IP addresses by keeping track of Layer 4 port numbers, which make each session unique (that is, Port Address Translation [PAT]) Overlapping NAT Used when private internal IP addresses at one location overlap destination private internal IP addresses at another location. Names of NAT IP addresses Advantage Definition Inside Local Inside Global Outside Local Outside Global A private IP address referencing an inside device A public IP address referencing an inside device A private IP address referencing an outside device A public IP address referencing an outside device

Dynamic NAT Sample Configuration R1# show run ...OUTPUT OMITTED... ! interface FastEthernet1/0 ip address 10.1.1.100 255.255.255.0 ip nat inside ! interface Serial 0/0 ip address 172.16.1.100 255.255.255.0 ip nat outside ! ip nat pool OUTSIDE_POOL 172.16.1.1 172.16.1.10 netmask 255.255.255.0 ip nat inside source list 1 pool OUTSIDE_POOL ! access-list 1 permit 10.0.0.0 0.0.0.255 ! ...OUTPUT OMITTED...

In the example, ACL 1 identifies the inside addresses (the 10.1.1.0/24 network in this example) to be translated. A pool of addresses named OUTSIDE_POOL is defined as IP addresses in the range 172.16.1.1 to 172.16.1.10. The ip nat inside source list 1 pool OUTSIDE_POOL command associates the internal range of addresses defined by ACL 1 with the range of outside addresses defined by the OUTSIDE_POOL pool. Finally, you need to indicate what router interface is acting as the inside interface and what interface is acting as the outside interface. Note that you can have multiple interfaces acting as inside or outside interfaces. The ip nat inside command is issued for interface Fast Ethernet 1/0, and the ip nat outside command is issued for Serial 0/0. 1. Potential NAT Troubleshooting Issues 1. Using NAT over a VPN Some VPN protocols check the checksum of a packet to verify its integrity. The checksum calculated for a packet before NAT is different from a checksum calculated for that same packet after NAT (because performing NAT on a packet changes IP address information). Therefore, a VPN protocol (for example, IPSec) might reject such a packet because it appears to have been altered. Workarounds are available, including NAT Traversal, NAT Transparency, and IPSec over TCP/UDP. 2. NAT hiding true IP address information Because NAT translates an inside IP address to an outside IP address, tracing a data flow from end to end for troubleshooting purposes can be challenging. You can start troubleshooting by using the show ip nat translation command to verify whether the translation does exist in the translation table. 3. Applications that are not NAT compatible When some applications initialize, they randomly determine what ports are going to be used for communication, which might be incompatible with how NAT handles incoming traffic. Some Voice over IP (VoIP) protocols face such an issue, as they select the User Datagram Protocol (UDP) port numbers to be used for their Real-time Transport Protocol (RTP) media streams. Also, when setting up communication with a remote device, an application might include IP address information in the payload of a packet. If the remote device attempted to return traffic to the IP address embedded in that payload, that IP address might be unreachable because of the NAT translation. Therefore, you should avoid NAT for some applications; use NAT-aware applications, or configure NAT to work with NAT-unaware applications. 4. Delays experienced due to NAT's processing Because NAT manipulates Layer 3 information of packets, the packets are subject to a bit more delay than they would otherwise experience. This delay might become more evident on routers performing numerous NAT translations.

2. Order of Operations for an Interface 1. Following is a listing of the order of interface operations for traffic flowing from the inside network into the outside network 1. Decryption of IPSec traffic 2. Input ACL applied 3. Input policing applied 4. Input accounting applied 5. Policy-based routing (PBR) 6. Redirecting traffic to a web cache 7. NAT translating local to global addresses 8. Crypto map application 9. Output ACL applied 10. Cisco IOS Firewall inspection performed 11. TCP intercept feature applied 12. Encryption performed 2. Following is a listing of the order of interface operations for traffic flowing from the outside network into the inside network. Notice, for example, that an putout ACL might need to reference a translated IP address for a packet as opposed to the original IP address of the packet. 1. Decryption of IPSec traffic 2. Input ACL applied 3. Input policing applied 4. Input accounting applied 5. NAT translating global to local addresses 6. Policy Based Routing (PBR) 7. Redirecting traffic to a web cache 8. Crypto map application 9. Output ACL applied 10. Cisco IOS Firewall inspection performed 11. TCP intercept feature applied 12. Encryption performed 3. Most common causes for a NAT issue: 1. An ACL referenced by a NAT configuration is incorrect 2. Inside and outside interfaces are not correctly assigned 3. Incorrect IP addresses (or address ranges) are referenced by a NAT configuration 4. Applications are not NAT aware 5. A routing loop occurs as a result of a NAT address translation

3. NAT Troubleshooting Syntax NAT Troubleshooting Commands Command Description Clear ip nat translation * Show ip nat translations Show ip nat statistics Removes all dynamic entries from a router's NAT translation table Used to see all entries in a router's NAT translation table Used to display NAT configuration and statistical information on a router, such as inside and outside interfaces, total translations, number of expired translation, inside address ACL, and outside address pool information Provides real-time information about NAT translations as they occur, including the IP address being translated and the IP identification number that can be used to match packets in the output with packets captured with a protocol analyzer

Debug ip nat

ip nat pool pool-name start-ip Global configuration mode command that defines a pool of inside end-ip {netmask subnet-mask global addresses into which inside local addresses can be translated | prefix-length prefix-length} ip nat inside source list access-list pool pool-name [overload] ip nat translation maxentries number ip nat {inside | outside} Global configuration mode command that associates an ACL defining an inside local address space with the specified pool of inside global addresses (Note: The overload keyword enables PAT, which allows multiple inside addresses to share a common outside address.) Global configuration mode command that specifies the maximum number of entries permitted in a router's NAT table Interface configuration mode command that identifies an interface as an inside or outside NAT interface.

II. DHCP Troubleshooting

1. Basic DHCP Operation 1. Steps of a client obtaining a DHCP address 1. When a DHCP client initially boots, it has no IP address, default gateway, or other such configuration information. Therefore, the way a DHCP client initially communicates is by sending a broadcast message (that is, a DHCPDISCOVER message) to a destination address of 255.255.255.255 in an attempt to discover a DHCP server. 2. When a DHCP server receives a DHCPDISCOVER message, it can respond with a DHCPOFFER message. Because the DHCPDISCOVER message is sent as a broadcast, more than one DHCP server might respond to this discover request. However, the client typically selects the server that sent the first DHCPOFFER response it received. 3. The DHCP client communicates with this selected server by sending a DHCPREQUEST message asking the DHCP server to provide IP configuration parameters. 4. Finally, the DHCP server responds to the client with a DHCPACK message. This DHCPACK message contains a collection of IP configuration parameters.

2. DHCP Configurations 1. DHCP Relay 1. The service dhcp command is used to enable DHCP service on a router. Without it you cannot configure DHCP relay. Most routers already have this configured but you should look whether this is enabled or not when troubleshooting 2. The ip helper-address address command is used to configure DHCP relay, specifically where to send DHCP messages to: A DHCP server. 3. You can also configure a directed broadcast, but is not recommended by Cisco as the DHCP messages are sent to the entire subnet as a broadcast. IE: ip helperaddress 10.1.1.255 (a 24 bit subnet broadcast address). 2. Some other protocols that are forwarded by a DHCP relay agent 1. TFTP 2. Domain Name System (DNS) 3. Internet Time Service (ITS) 4. NetBIOS name server 5. NetBIOS datagram server 6. BootP 7. TACACS DHCP Message Types DHCP Message DHCPDISCOVER

Description A client sends this message in an attempt to locate a DHCP server. This message is sent to a broadcast IP address of 255.255.255.255 using UDP port 67. A DHCP server sends this message in response to a DHCPDISCOVER message using UDP port 68 This message is a request for IP configuration parameters sent from a client to a specific DHCP server. This message is sent from a client to a DHCP server to inform the server that an IP address is already in use on the network. A DHCP server sends this message to a client and includes IP configuration parameters. A DHCP Server sends this message to a client and informs the client that the DHCP server declines to provide the client with the requested IP configuration information A client sends this message to a DHCP server and informs the DHCP server that the client has released its DHCP lease, thus allowing the DHCP server to reassign the client IP address to another client. This message is sent from a client to a DHCP server and requests IP configuration parameters. Such a message might be sent from an access server requesting IP configuration information for a remote client attaching to the access server.

DHCPOFFER DHCPREQUEST DHCPDECLINE DHCPACK DHCPNAK

DHCPRELEASE

DHCPINFORM

3. Example of configuring a router interface to obtain a DHCP address R1#conf term R1(config)# int fa 0/0 R1(config-if)# ip address dhcp 4. Example of a router configured to BE a DHCP server R1# show run ...OUTPUT OMITTED... ip dhcp excluded-address 10.8.8.1 ! ip dhcp pool POOL-A network 10.8.8.0 255.255.255.0 default-router 10.8.8.1 dns-server 192.168.1.1 netbios-name-server 192.168.1.2 ...OUTPUT OMITTED... 3. Potential DHCP Troubleshooting Issues 1. A router not forwarding broadcasts By default;, a router does not forward broadcasts, including DHCPDISCOVER broadcast messages. Therefore, a router needs to be explicitly configured to act as a DHCP relay agent if the DHCP client and DHCP server are on different subnets. 2. DHCP pool out of IP addresses A DHCP pool contains a finite number of addresses. Once a DHCP pool becomes depleted, new DHCP requests are rejected. 3. Misconfiguration The configuration of a DHCP server might be incorrect. For example, the range of network addresses to be given out by a particular pool might be incorrect, or the exclusion of addresses statically assigned to routers or DNS servers might be incorrect. 4. Duplicate IP addresses A DHCP server might hand out an IP address to a client that is already statically assigned to another host on the network. These duplicate IP addresses can cause connectivity issues for both the DHCP client and the host that had been statically configured for the IP address. 5. Redundant services not communicating Some DHCP servers can coexist with other DHCP servers for redundancy. For this redundancy to function, these DHCP servers need to communicate with one another. If this interserver communication fails, the DHCP servers can hand out overlapping IP addresses to their clients. 6. The pull nature of DHCP When a DHCP client wants an IP address, it can request an IP address from a DHCP server. However, the DHCP server has no ability to initiate a change in the client IP address after the client obtains an IP address. In other words, the DHCP client pulls information from the DHCP server, but the DHCP server cannot push information to the DHCP client.

2. Useful info to gather before troubleshooting DHCP issues 1. The configuration of the DHCP server For example, confirm that the pools are correctly defined with appropriate network addresses, default gateways, and other relevant IP address information 2. The configuration of the DHCP relay agent For example, determine if the target addresses a unicast IP address or a directed broadcast address 3. Determine the size of a DHCP pool Because a pool in a DHCP server accommodates only a limited number of IP addresses, determine how many IP addresses (if any) are still available from a given DHCP pool 4. DHCP troubleshooting Syntax DHCP Troubleshooting Commands Command Description Show ip dhcp conflict Identifies any IP address conflicts a router identifies, along with the method the router used to identify the conflicts (this is, via ping or gratuitous ARP) Displays IP addresses that an IOS DHCP server assigns, their corresponding MAC addresses, and lease expirations Releases all current DHCP leases Clears all currently identified DHCP conflicts

Show ip dhcp binding Clear ip dhcp binding * Clear ip dhcp conflict *

Debug ip dhcp server events Provides real-time information about DHCP address assignments and database updates Debug ip dhcp server packet Displays real-time decodes of DHCP packets ip helper-address ip-address Interface configuration mode command that causes an interface to forward specific received UDP broadcasts to the destination IP address, which can be either a specific IP address or a directed broadcast address. ip dhcp excluded-address Specifies a range of IP addresses not to be assigned to DHCP clients beginning-ip-address [endingip-address] ip dhcp pool pool-name Network network-address subnet-mask Default-router ip-address dns-server ip-address Netbios-name-server ipaddress Lease {days hours minutes | infinite} Creates a DHCP pool Identifies a subnet to be used by a DHCP pool Specifies the IP address of a default gateway to be given to a DHCP client Configures the IP address of a DNS server to be given to a DHCP client Defines the IP address of a WINS server to be given to a DHCP client Determines the duration of a DHCP lease given to a DHCP client.