Wednesday, December 23, 2009

“Broken MPLS? Troubleshoot it we should, mmmmm.” … yoda

Cheers from London! I learned this week that a “Christmas Cracker” is not a food item, OR a person.  ;-)     There is so much to know.  I am grateful for students willing to show me the ropes here in the UK.   Thank you all.    Now on to the topic at hand.
MPLS is an important part of the RS Bootcamp, including troubleshooting MPLS.
Here is an MPLS troubleshooting scenario, that has 1 (one,одну,un,uno) configuration issue.  Can you spot it?  Lets get to it!  Here is the diagram.
mpls-ine-blog
Problem: Clients on the 5.5.5.0 network are not able to ping the server, or any other devices, on the 1.1.1.0 network. Your challenge, based on the provided IOS show commands only, is to identify the 1 configuration problem that is causing the network failure.
For this scenario, the PC and Server configurations are correct, as well as all the layer 2 switching infrastructure on the Catalyst switches. R1 and R5 are CE routers, R2 and R4 are PE routers, and R3 is a P router.
We will work from the diagram, using the show commands from right to left, starting with R5.    (I have grouped the show commands together, then the individual commands and their output.   This way, the post may be used as a study/reference tool.)
R5
show ip int brief | ex una
show ip route
show ip protocols
ping 136.1.45.255
R5#show ip int brief | ex una
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet0/0            5.5.5.5         YES manual up                    up
FastEthernet0/1            136.1.45.5      YES NVRAM  up                    up
Loopback0                  150.1.5.5       YES NVRAM  up                    up
R5#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     136.1.0.0/24 is subnetted, 2 subnets
R       136.1.12.0 [120/1] via 136.1.45.4, 00:00:08, FastEthernet0/1
C       136.1.45.0 is directly connected, FastEthernet0/1
     1.0.0.0/24 is subnetted, 1 subnets
R       1.1.1.0 [120/1] via 136.1.45.4, 00:00:08, FastEthernet0/1
     5.0.0.0/24 is subnetted, 1 subnets
C       5.5.5.0 is directly connected, FastEthernet0/0
     150.1.0.0/24 is subnetted, 1 subnets
C       150.1.5.0 is directly connected, Loopback0
R5#show ip protocols
Routing Protocol is "rip"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Sending updates every 30 seconds, next due in 11 seconds
  Invalid after 180 seconds, hold down 180, flushed after 240
  Redistributing: rip
  Default version control: send version 2, receive version 2
    Interface             Send  Recv  Triggered RIP  Key-chain
    FastEthernet0/0       2     2
    FastEthernet0/1       2     2
  Automatic network summarization is not in effect
  Maximum path: 4
  Routing for Networks:
    5.0.0.0
    136.1.0.0
  Routing Information Sources:
    Gateway         Distance      Last Update
    136.1.45.4           120      00:00:08
  Distance: (default is 120)

R5#ping 136.1.45.255

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 136.1.45.255, timeout is 2 seconds:

Reply to request 0 from 136.1.45.4, 8 ms
Reply to request 1 from 136.1.45.4, 32 ms
Reply to request 2 from 136.1.45.4, 16 ms
Reply to request 3 from 136.1.45.4, 4 ms
Reply to request 4 from 136.1.45.4, 24 ms
R5#
R4
show ip route
show ip route vrf Vrf1
show ip protocols
show mpls interface
show mpls ldp neighbor
show mpls forwarding-table
show ip bgp summary
show ip bgp vpnv4 all
ping vrf Vrf1 136.1.45.255
ping 136.1.34.255
R4#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     136.1.0.0/24 is subnetted, 2 subnets
O       136.1.23.0 [110/65] via 136.1.34.3, 00:04:28, FastEthernet0/0
C       136.1.34.0 is directly connected, FastEthernet0/0
     150.1.0.0/32 is subnetted, 2 subnets
C       150.1.4.4 is directly connected, Loopback0
O       150.1.2.2 [110/66] via 136.1.34.3, 00:04:28, FastEthernet0/0
R4#show ip route vrf Vrf1

Routing Table: Vrf1
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     136.1.0.0/24 is subnetted, 2 subnets
B       136.1.12.0 [200/0] via 150.1.2.2, 00:04:23
C       136.1.45.0 is directly connected, FastEthernet0/1
     1.0.0.0/24 is subnetted, 1 subnets
B       1.1.1.0 [200/2] via 150.1.2.2, 00:04:23
     5.0.0.0/24 is subnetted, 1 subnets
R       5.5.5.0 [120/1] via 136.1.45.5, 00:00:01, FastEthernet0/1
R4#show ip protocols
Routing Protocol is "ospf 234"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 150.1.4.4
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
  Routing on Interfaces Configured Explicitly (Area 0):
    FastEthernet0/0
    Loopback0
 Reference bandwidth unit is 100 mbps
  Routing Information Sources:
    Gateway         Distance      Last Update
    150.1.2.2            110      00:04:28
    150.1.3.3            110      00:04:28
  Distance: (default is 110)

Routing Protocol is "rip"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Sending updates every 30 seconds, next due in 0 seconds
  Invalid after 180 seconds, hold down 180, flushed after 240
  Redistributing: rip
  Default version control: send version 1, receive any version
  Automatic network summarization is in effect
  Maximum path: 4
  Routing for Networks:
  Routing Information Sources:
    Gateway         Distance      Last Update
  Distance: (default is 120)

Routing Protocol is "bgp 24"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  IGP synchronization is disabled
  Automatic route summarization is disabled
  Neighbor(s):
    Address          FiltIn FiltOut DistIn DistOut Weight RouteMap
    150.1.2.2
  Maximum path: 1
  Routing Information Sources:
    Gateway         Distance      Last Update
  Distance: external 20 internal 200 local 200

R4#show mpls interface
Interface              IP            Tunnel   Operational
FastEthernet0/0        Yes (ldp)     No       Yes
R4#show mpls ldp neighbor

R4#show mpls forwarding-table
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
tag    tag or VC   or Tunnel Id      switched   interface
17     Untagged    150.1.2.2/32      0          Fa0/0      136.1.34.3
18     Untagged    136.1.23.0/24     0          Fa0/0      136.1.34.3
19     Untagged    5.5.5.0/24[V]     19950      Fa0/1      136.1.45.5
20     Aggregate   136.1.45.0/24[V]  1684
R4#show ip bgp summary
BGP router identifier 150.1.4.4, local AS number 24
BGP table version is 1, main routing table version 1

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
150.1.2.2       4    24     148     149        1    0    0 02:08:57        0
R4#show ip bgp vpnv4 all
BGP table version is 22, local router ID is 150.1.4.4
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf Vrf1)
*>i1.1.1.0/24       150.1.2.2                2    100      0 ?
*> 5.5.5.0/24       136.1.45.5               1         32768 ?
*>i136.1.12.0/24    150.1.2.2                0    100      0 ?
*> 136.1.45.0/24    0.0.0.0                  0         32768 ?
R4#ping vrf Vrf1 136.1.45.255

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 136.1.45.255, timeout is 2 seconds:

Reply to request 0 from 136.1.45.5, 4 ms
Reply to request 1 from 136.1.45.5, 28 ms
Reply to request 2 from 136.1.45.5, 16 ms
Reply to request 3 from 136.1.45.5, 32 ms
Reply to request 4 from 136.1.45.5, 8 ms
R4#ping 136.1.34.255

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 136.1.34.255, timeout is 2 seconds:

Reply to request 0 from 136.1.34.3, 8 ms
Reply to request 1 from 136.1.34.3, 4 ms
Reply to request 2 from 136.1.34.3, 16 ms
Reply to request 3 from 136.1.34.3, 8 ms
Reply to request 4 from 136.1.34.3, 4 ms
R4#
R3
show ip route
show ip protocols
show mpls interface
show mpls ldp neighbor
show mpls forwarding-table
R3#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     136.1.0.0/24 is subnetted, 2 subnets
C       136.1.23.0 is directly connected, Serial0/0.23
C       136.1.34.0 is directly connected, FastEthernet0/0
     150.1.0.0/32 is subnetted, 3 subnets
O       150.1.4.4 [110/2] via 136.1.34.4, 00:05:16, FastEthernet0/0
C       150.1.3.3 is directly connected, Loopback0
O       150.1.2.2 [110/65] via 136.1.23.2, 00:05:26, Serial0/0.23
R3#show ip protocols
Routing Protocol is "ospf 234"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 150.1.3.3
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
  Routing on Interfaces Configured Explicitly (Area 0):
    FastEthernet0/0
    Serial0/0.23
 Reference bandwidth unit is 100 mbps
  Routing Information Sources:
    Gateway         Distance      Last Update
    150.1.4.4            110      00:05:16
    150.1.2.2            110      00:05:26
  Distance: (default is 110)

R3#show mpls interface
Interface              IP            Tunnel   Operational
FastEthernet0/0        Yes (ldp)     No       Yes
Serial0/0.23           Yes (ldp)     No       Yes
R3#show mpls ldp neighbor

R3#show mpls forwarding-table
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
tag    tag or VC   or Tunnel Id      switched   interface
16     Untagged    150.1.2.2/32      0          Se0/0.23   point2point
17     Untagged    150.1.4.4/32      0          Fa0/0      136.1.34.4
R2
show ip route
show ip route vrf Vrf1
show ip protocols
show mpls interface
show mpls ldp neighbor
show mpls forwarding-table
show ip bgp summary
show ip bgp vpnv4 all
ping 136.1.23.255
ping vrf Vrf1 136.1.12.255
R2#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     136.1.0.0/24 is subnetted, 2 subnets
C       136.1.23.0 is directly connected, Serial0/0.23
O       136.1.34.0 [110/65] via 136.1.23.3, 00:05:41, Serial0/0.23
     150.1.0.0/32 is subnetted, 2 subnets
O       150.1.4.4 [110/66] via 136.1.23.3, 00:05:41, Serial0/0.23
C       150.1.2.2 is directly connected, Loopback0
R2#show ip route vrf Vrf1

Routing Table: Vrf1
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     136.1.0.0/24 is subnetted, 2 subnets
C       136.1.12.0 is directly connected, FastEthernet0/0
B       136.1.45.0 [200/0] via 150.1.4.4, 00:05:18
     1.0.0.0/24 is subnetted, 1 subnets
O       1.1.1.0 [110/2] via 136.1.12.1, 02:18:47, FastEthernet0/0
     5.0.0.0/24 is subnetted, 1 subnets
B       5.5.5.0 [200/1] via 150.1.4.4, 00:05:18
R2#show ip protocols
Routing Protocol is "ospf 234"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 150.1.2.2
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
  Routing on Interfaces Configured Explicitly (Area 0):
    Serial0/0.23
    Loopback0
 Reference bandwidth unit is 100 mbps
  Routing Information Sources:
    Gateway         Distance      Last Update
    150.1.4.4            110      00:05:41
    150.1.3.3            110      00:05:51
  Distance: (default is 110)

Routing Protocol is "bgp 24"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  IGP synchronization is disabled
  Automatic route summarization is disabled
  Neighbor(s):
    Address          FiltIn FiltOut DistIn DistOut Weight RouteMap
    150.1.4.4
  Maximum path: 1
  Routing Information Sources:
    Gateway         Distance      Last Update
  Distance: external 20 internal 200 local 200

R2#show mpls interface
Interface              IP            Tunnel   Operational
Serial0/0.23           Yes (ldp)     No       Yes
R2#show mpls ldp neighbor

R2#show mpls forwarding-table
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop
tag    tag or VC   or Tunnel Id      switched   interface
17     Untagged    136.1.34.0/24     0          Se0/0.23   point2point
18     Untagged    150.1.4.4/32      0          Se0/0.23   point2point
21     Aggregate   136.1.12.0/24[V]  1040
22     Untagged    1.1.1.0/24[V]     18240      Fa0/0      136.1.12.1
R2#show ip bgp summary
BGP router identifier 150.1.2.2, local AS number 24
BGP table version is 1, main routing table version 1

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
150.1.4.4       4    24     151     150        1    0    0 02:10:15        0
R2#show ip bgp vpnv4 all
BGP table version is 13, local router ID is 150.1.2.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf Vrf1)
*> 1.1.1.0/24       136.1.12.1               2         32768 ?
*>i5.5.5.0/24       150.1.4.4                1    100      0 ?
*> 136.1.12.0/24    0.0.0.0                  0         32768 ?
*>i136.1.45.0/24    150.1.4.4                0    100      0 ?
R2#ping 136.1.23.255

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 136.1.23.255, timeout is 2 seconds:

Reply to request 0 from 136.1.23.3, 12 ms
Reply to request 1 from 136.1.23.3, 16 ms
Reply to request 2 from 136.1.23.3, 32 ms
Reply to request 3 from 136.1.23.3, 28 ms
Reply to request 4 from 136.1.23.3, 28 ms
R2#ping vrf Vrf1 136.1.12.255

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 136.1.12.255, timeout is 2 seconds:

Reply to request 0 from 136.1.12.1, 4 ms
Reply to request 1 from 136.1.12.1, 28 ms
Reply to request 2 from 136.1.12.1, 16 ms
Reply to request 3 from 136.1.12.1, 28 ms
Reply to request 4 from 136.1.12.1, 12 ms
R2#
R1
show ip int brief | ex una
show ip route
show ip protocols
ping 136.1.12.255
R1#show ip int brief | ex una
Interface                  IP-Address      OK? Method Status                Protocol
FastEthernet0/0            136.1.12.1      YES NVRAM  up                    up
FastEthernet0/1            1.1.1.1         YES manual up                    up
Loopback0                  150.1.1.1       YES NVRAM  up                    up
R1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     136.1.0.0/24 is subnetted, 2 subnets
C       136.1.12.0 is directly connected, FastEthernet0/0
O E2    136.1.45.0 [110/1] via 136.1.12.2, 00:06:20, FastEthernet0/0
     1.0.0.0/24 is subnetted, 1 subnets
C       1.1.1.0 is directly connected, FastEthernet0/1
     5.0.0.0/24 is subnetted, 1 subnets
O E2    5.5.5.0 [110/1] via 136.1.12.2, 00:06:20, FastEthernet0/0
     150.1.0.0/24 is subnetted, 1 subnets
C       150.1.1.0 is directly connected, Loopback0
R1#show ip protocols
Routing Protocol is "ospf 12"
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 150.1.1.1
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    1.1.1.0 0.0.0.255 area 0
    136.1.12.1 0.0.0.0 area 0
 Reference bandwidth unit is 100 mbps
  Routing Information Sources:
    Gateway         Distance      Last Update
    136.1.12.2           110      00:06:20
  Distance: (default is 110)

R1#ping 136.1.12.255 

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 136.1.12.255, timeout is 2 seconds:

Reply to request 0 from 136.1.12.2, 8 ms
Reply to request 1 from 136.1.12.2, 20 ms
Reply to request 2 from 136.1.12.2, 12 ms
Reply to request 3 from 136.1.12.2, 24 ms
Reply to request 4 from 136.1.12.2, 8 ms
R1#
What do you see as the problem, and what may be done to correct it?
Good luck!

By Keith Barker, CCIE #6783

No comments:

Post a Comment