OSPF in a VRF requires a box-unique router ID

It's obvious why two routers in the same OSPF domain cannot have the same router ID. But requiring unique router IDs on OSPF processes running in different VRFs is probably too harsh, although it does prevent confusion if two VRFs ever get connected through a customer site. Anyhow, if you have overlapping IP addresses on loopback interfaces in different VRFs, OSPF process might not start.

Here's a short example: two VRFs have loopback interfaces with the same IP address. Perfectly legal setup from the MPLS/VPN perspective.

c7200#show ip vrf interfaces
Interface    IP-Address   VRF            Protocol
Lo1003       10.0.0.1     EIGRP_OSPF     up
Lo1001       10.0.0.1     OSPF_1         up

However, when you try to configure the second OSPF process, it fails to start as it cannot get a box-unique router ID. You have to enter a different router ID manually.

c7200#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
c7200(config)#router ospf 1 vrf OSPF_1
c7200(config-router)#network 0.0.0.0 255.255.255.255 area 0
c7200(config-router)#exit
c7200(config)#router ospf 3 vrf EIGRP_OSPF
%OSPF-4-NORTRID: OSPF process 3 cannot pick a router-id.
  Please configure manually or bring up an interface with an ip address.
c7200(config-router)#router-id 10.0.0.2

Multihomed EIGRP sites in MPLS VPN network

Deploying EIGRP as the PE-CE routing protocol in MPLS VPN networks is easy if all sites have a single PE-CE link and there are no backdoor links between the sites. Real life is never as simple as that; you have to cope with various (sometimes undocumented) network topologies. Even that would be manageable if the customer networks would have a clean addressing scheme that would allow good summarization (that happens once in a blue moon) or if the MPLS VPN core could announce the default route into the EIGRP sites (wishful thinking; the customer probably has one or more Internet exit points).

In the end, you’re left with two-way route redistribution between core MP-BGP and edge EIGRP, resulting in nightmarish scenarios (probably a good half of the blog posts of the CCIE candidates talk about redistribution horrors). Fortunately, Cisco implemented two extra features supporting EIGRP-to-MP-BGP redistribution: BGP cost community and BGP Site-of-Origin. The Multihomed MPLS VPN sites running EIGRP article I wrote in CT³ wiki describes how you can use both features to get a reliable network even when doing two-way redistribution.

Fragments | 2008-06-28

This week in Fragments, NIL's corporate blog:

• Klemen Štular, our CEO, shares his thoughts on Cisco's partner strategy.
• Boštjan Šuštar, one of our MPLS gurus, describes the hidden dangers of VPLS.
• Marjan Bradeško, the e-learning evangelist, found an interesting reason why you might prefer training in SecondLife-like environment: the classroom simulation might focus your mind on the learning experience (something, which is sorely missing in most e-learning offerings).
• Mitja Robas, CCIE focused on storage networks, gives you a few hints on securing SAN networks.
• Jan Bervar, security guru, thinks that Cisco should embrace VMsafe environment.
• And lastly, I had a few things to say about IOS stability.

Use the RSS feed to subscribe to Fragments

Simple EIGRP in MPLS VPN networks

A few days ago I've been involved in an interesting EIGRP-within-MPLS/VPN troubleshooting session. I wanted to reproduce the behavior in the lab, so I set up a simple EIGRP lab … and realized that it's been a while since we wrote the MPLS and VPN Architectures, Volume II, which covers EIGRP used as PE-CE routing protocol. To make things more interesting, a few details have changed in the meantime; you have to configure the following features to get EIGRP running within MPLS/VPN environment:

• The autonomous-system command within the VRF address family is mandatory, even if the VRF AS number matches the EIGRP process number.
• The default BGP-to-EIGRP redistribution metric has to be configured, otherwise remote EIGRP routes will not be redistributed even though they have EIGRP metric encoded in extended BGP communities.
• Things work best if you disable auto-summary on PE-routers.

You can find more details and complete configuration examples in the EIGRP in MPLS VPN networks article I wrote in the CT³ wiki.

BGP Templates on MPLS VPN PE-routers

BGP session and policy templates are a fantastic feature if you want to design scalable, easy to maintain BGP configurations. The IOS documentation doesn't even mention how they could be used in designs using multiple address families, for example in MPLS VPN networks; the article I wrote in the CT³ wiki should resolve any ambiguities.

Use the explicit "address-family ipv4" in BGP configurations

If you use multiprotocol BGP (MP-BGP) in your network to support MPLS VPN, IPv6 or IP Multicast over BGP, it's best if you go all the way and configure an explicit ipv4 address family; the resulting BGP configuration is significantly easier to read and understand as the session-specific parameters are clearly separated from the routing-specific parameters and the IPv4 settings are nicely grouped in an explicit section.

To change the format of the BGP configuration, configure the IPv4 address family with the address-family ipv4 unicast router configuration command (the neighbor statements and other configuration settings pertinent to IPv4 configuration are automatically moved into the new address family) or manually activate a BGP neighbor for IPv4 route exchange with the neighbor activate router configuration command.To illustrate the differences between the traditional BGP configuration and the per-address-family configuration, consider a simple MPLS VPN+Internet setup. First the traditional approach:

router bgp 65000
 template peer-policy Internal
  send-community both
 exit-peer-policy
 !
 template peer-session Internal
  remote-as 65000
  update-source Loopback0
 exit-peer-session
 !
 no synchronization
 bgp log-neighbor-changes
 network 10.0.1.1 mask 255.255.255.255
 neighbor 10.0.1.5 inherit peer-session Internal
 neighbor 10.0.1.5 description PE-C(RR)
 neighbor 10.0.1.5 inherit peer-policy Internal
 no auto-summary
 !
 address-family vpnv4
  neighbor 10.0.1.5 activate
  neighbor 10.0.1.5 send-community extended
 exit-address-family

... and the changed configuration after the address-family ipv4 command has been entered:

router bgp 65000
 template peer-policy Internal
  send-community both
 exit-peer-policy
 !
 template peer-policy InternalV6
  send-label
  inherit peer-policy Internal 1
 exit-peer-policy
 !
 template peer-session Internal
  remote-as 65000
  update-source Loopback0
 exit-peer-session
 !
 bgp log-neighbor-changes
 neighbor 10.0.1.5 inherit peer-session Internal
 neighbor 10.0.1.5 description PE-C(RR)
 !
 address-family ipv4
  no synchronization
  network 10.0.1.1 mask 255.255.255.255
  neighbor 10.0.1.5 activate
  neighbor 10.0.1.5 inherit peer-policy Internal
  no auto-summary
 exit-address-family
 !
 address-family vpnv4
  neighbor 10.0.1.5 activate
  neighbor 10.0.1.5 send-community extended
 exit-address-family

The “fallback global” VRF option does not exist in Cisco IOS

Cheng sent me an interesting question:

I'm reading your book MPLS and VPN Architectures and I've found the ip vrf forwarding name fallback global command in the “Additional Lookup in the Global Routing Table” section. I can only find this command in Junos, but not in IOS.

… and he was right. When we were writing the book, we described several features that were still in development as it looked like they would be in the production code by the time the book was published. Many of them made it into the public IOS releases (for example, the Carrier's Carrier architecture), but some of them (like this command) simply vanished from the surface.

However, it looks like the engineers that switched from Cisco to Juniper took the concept with them and implemented it in JunOS, so JunOS has this feature but IOS doesn't.

This article is part of You've asked for it series.

The “IP fragmentation” quick learning module

In the IP Corner article The Never-Ending Story of IP Fragmentation, I've described the various challenges posed by IP fragmentation in modern IP networks. If you want a more in-depth look at that topic, check the new IP fragmentation e-lesson. It includes a recorded presentation and a remote lab exercise where you can test various fragmentation scenarios, including MPLS VPN backbone, GRE tunnels and GRE+IPSec combination.

E-lessons are subscription-based; you can repeat each module in the lesson (including the lab) as many times as needed.

Building Core Networks with BGP, OSPF and MPLS

Do you need advanced knowledge and skills needed in designing and implementing core MPLS networks? We have developed exactly the course you need.

The Building Core Networks with BGP, OSPF and MPLS (NIL_BCMPL) course focuses on MPLS applications such as MPLS VPN (with special attention to Internet access from a VPN), Any Transport over MPLS (AToM), Carrier Supporting Carrier (CsC) and MPLS Traffic Engineering (MPLS TE). As a prerequisite for MPLS deployment the routing protocols are explained as well - the Open Shortest Path First (OSPF) and Border Gateway Protocol (BGP). The scalability issues of the protocols as well as multiprotocol BGP are addressed as well. Several practice labs enable you to gain the necessary experience in deploying the MPLS-based core networks.

Layer-2 or Layer-3 VPN services?

Occasionally someone would try to persuade me that the layer-2 VPN services are like aspirin (you know, totally harmless plus it could get rid of all your headaches). OK, that might be true if you take the layer-2 VPN offering as a pure transport solution and plug in an extra router (sometimes also called a layer-3 switch by marketing people) between the Service Provider's Ethernet (or whatever they give you) and your LAN. But there are people who don't know the details and plug the SP Ethernet straight into their L2 switch … and things might even work for a while … until the whole network collapses.

In my opinion, we need both L2 and L3 VPN services, but it's important that they are positioned and deployed correctly. You can read more about my views on this topic in the SearchTelecom article Making the case for Layer 2 and Layer 3 VPNs.

The tale of the three MTUs

An IOS device configured for IP+MPLS routing uses three different Maximum Transmission Unit (MTU) values:

• The hardware MTU configured with the mtu interface configuration command
• The IP MTU configured with the ip mtu interface configuration command
• The MPLS MTU configured with the mpls mtu interface configuration command

The hardware MTU specifies the maximum packet length the interface can support … or at least that's the theory behind it. In reality, longer packets can be sent (assuming the hardware interface chipset doesn't complain); therefore you can configure MPLS MTU to be larger than the interface MTU and still have a working network. Oversized packets might not be received correctly if the interface uses fixed-length buffers; platforms with scatter/gather architecture (also called particle buffers) usually survive incoming oversized packets.

IP MTU is used to determine whether a non-labeled IP packet forwarded through an interface has to be fragmented (the IP MTU has no impact on labeled IP packets). It has to be lower or equal to hardware MTU (and this limitation is enforced). If it equals the HW MTU, its value does not appear in the running configuration and it tracks the changes in HW MTU. For example, if you configure ip mtu 1300 on a Serial interface, it will appear in the running configuration as long as the hardware MTU is not equal to 1300 (and will not change as the HW MTU changes). However, as soon as the mtu 1300 is configured, the ip mtu 1300 command disappears from the configuration and the IP MTU yet again tracks the HW MTU.

The MPLS MTU determines the maximum size of a labeled IP packet (MPLS shim header + IP payload size). If the overall length of the labeled packet (including the shim header) is greater than the MPLS MTU, the packet is fragmented. The MPLS MTU can be greater than the HW MTU assuming the hardware architecture and interface chipset support that (and the router will warn you that you might be getting into trouble). Similar to the ip mtu command, the mpls mtu command will only appear in the running configuration if the MPLS MTU is different from the HW MTU. However, contrary to the behavior of the IP MTU, any change in HW MTU with the mtu configuration command also resets the MPLS MTU to HW MTU.

The behavior as described above was tested on a 3725 router running IOS release 12.4(15)T1. Although the MPLS MTU Command Changes document claims that you cannot set MPLS MTU larger than then interface MTU from IOS release 12.4(11)T, I was still able to do it in 12.4(15)T1.

mturoute: A utility that measures hop-by-hop path MTU

I wanted to get in-depth details on how various MTU parameters interact in GRE/IPSec/MPLS environment. Before going into router configuration details, I wanted to have a tool that would reliably measure actual path MTU between the endpoints. After a while, Google gave me a usable link: supposedly the tracepath program on Linux does what I needed. As I'm a purely Windows user (for me, PCs are just a tool), I needed a Windows equivalent … and found mturoute, the utility that does exactly what I was looking for.Unfortunately, the original mturoute had an interesting bug: ICMP unreachable generated due to DF bit on oversized packet was accepted as a successful ping. The second run of the program always reported the correct MTU size (as Windows caches the maximum MTU per destination host), but I wanted to be more precise. Half a day later, after installing Windows SDK and Visual Studio Express on my PC, rediscovering my C programming skills and reading a lot of Winsock documentation, I've managed to fix the bugs and even add a “retry on ping timeout” feature. You can download the fixed version (source + exe) from the Articles area of my web site. It was compiled and tested on Windows XP, if you can test it on other platforms (2000, Vista), please let me know the results.

Here is a sample run of the program (the reduced path MTU is due to an MPLS-enabled GRE tunnel in the path):

$ mturoute -t 10.0.3.3
mturoute to 10.0.3.3, 30 hops max, variable sized packets
* ICMP Fragmentation is not permitted. *
* Maximum payload is 10000 bytes. *
 1  --+---+++-+++-++  host: 10.0.0.1  max: 1500 bytes
 2  --+---++---+++  host: 10.2.0.2  max: 1476 bytes
 3  --+---+-+++++++  host: 10.0.3.3  max: 1472 bytes

Stop Inter-VRF static route leaking

The MPLS VPN implementation on Cisco IOS has always allowed you to create VRF static routes that pointed to interfaces belonging to other VRFs. The feature can be used to implement interesting overlapping VPN (or common services VPN) designs, some of which are explained in the MPLS and VPN Architectures books.

However, quite often the ability to create inter-VRF static routes is considered a major security problem, as an operator configuration error could establish undesired inter-VPN connectivity. In these cases, use the no ip route static inter-vrf configuration command to prevent such routes from being installed in the VRF routing table.

You might also want to read a good explanation of MPLS VPN route leaking from Cisco systems

Increased number of OSPF processes in MPLS VPN environments

When we were writing the MPLS and VPN architectures books, there was a limit on the number of OSPF processes you could configure per PE-router. The limit was based on the fact that IOS supports up to 32 sources of routing information; two of them are static and connected plus you need an IGP and BGP in the MPLS VPN backbone, resulting in 28 OSPF processes that could be configured on a single PE-router. This "feature" severely limited OSPF-based MPLS VPN deployments until IOS release 12.3(4)T when the limitation was removed, resulting in availability of up to 30 routing processes per VRF.

RIP, BGP and EIGRP never experienced the same limitations as you configure VRF-specific routing instances within address families of a single routing protocol

The details of the actual change that took place are not obvious from the IOS documentation; they simply changed the global allocation of routing process identifiers to per-routing-table identifiers. For example, in the following show ip protocol summary printouts, the two OSPF processes (one in global routing table, one in VRF test) share the same index.

router#show ip protocols summary
Index Process Name
0 connected
1 static
2 ospf 112
router#show ip protocols vrf Test summary
Index Process Name
0 connected
1 static
2 ospf 111

Introduction to Traffic Enginnering

In the TechTarget article Traffic engineering's role in next-generation networks I'm describing why one would need traffic engineering in a Service Provider network, what options were available using the “legacy” WAN technologies and what you can do to deploy traffic engineering in a modern end-to-end IP network.

Update: Preparing for the MPLS CCIP exam

Following my post about the relationship between the MPLS and VPN architectures books and CCIP MPLS exam, Peter Dob had an excellent idea: combine the MPLS and VPN architectures (Volume I, CCIP edition would be even better) with the MPLS fundamentals from Luc de Ghein. By reading Luc's book, you'll also get exposure to other MPLS-related topics (for example, AToM) on top of MPLS TE overview that you need for the exam.

This article is part of You've asked for it series.

Using MPLS VPN books to study for the CCIP exam

Every now and then I'm getting questions from my readers regarding the suitability of my MPLS books for the CCIP exam, for example:

“I'm pursuing my CCIP and have a hard time finding the right MPLS study guide. I know you have the CCIP edition that was written in 2002, but I think the exam topics have changed. Can you recommend what book or books are best for the CCIP MPLS exam?”

“Are MPLS VPN Architectures Volume 1 & 2 two completely separate books or is Volume 2 a newer release. I was thinking of going for the CCIP and wanted to know if I should get both books or just the more recent one.”

Here is the full story: MPLS and VPN Architectures Volume I and II are completely separate books with only slight overlap. Volume I was written when MPLS and MPLS VPNs were an emerging technology, thus the coverage of some solutions (like Carrier's Carrier architecture) was scarce (as they were mostly on the drawing board at that time). We've later released CCIP edition of Volume I, which includes a few bug fixes and two chapters on troubleshooting to match the requirements of the early version of Cisco's MPLS course.

The Volume II covers advanced MPLS topics, including remote access, inter-AS MPLS VPN, Carrier's carrier architecture, IP Multicast in MPLS VPN etc. Reading Volume II without having sound foundations from Volume I does not make sense.

The current MPLS course that's part of the CCIP curriculum has been significantly redesigned from the original one (primarily shifting the focus from baseline MPLS + MPLS VPN to “a bit about everything MPLS has become”), but at the moment Cisco Press has no plans to release another CCIP edition book to cover the changes. The new course (Implementing Cisco MPLS 2.2) has dropped any ATM-specific information (finally) and includes a chapter on MPLS TE. While Cisco's web site claims MPLS TE is included, it's not listed in their Course Outline section. The information on our web is more accurate, as we build the course outline from student materials, not from supplementary documents.

I would definitely recommend CCIP edition of Volume I (you can still get it as an on-line book) as the basis of your learning efforts, with a few topics from Volume II (EIGRP as PE-CE routing protocol, more in-depth troubleshooting information) also being applicable. MPLS TE is not covered in any of my books.

Test drive Carrier's Carrier MPLS VPN service

Carrier's Carrier MPLS VPN service is one of the more confusing aspects of MPLS VPN technology; there are simply so many different bits and pieces that have to fit together just right to make it work (although we did a pretty good job describing it in the Cisco Press book MPLS and VPN architectures, Volume II). If you would like to set it up in a test environment, here's what you can do: if you have partner-level Cisco Connection Online access, you can do it free of charge:

• Start Partner e-learning connection.
• After landing on the main PEC page, click here (or paste http://cisco.partnerelearning.com/peclms/lang-en/management/LMS_TrainLedInfo.asp?UserMode=0&LEDefId=17619 in the browser address bar)

If you're not working for a Cisco partner, you can buy the whole set of advanced MPLS remote labs from NIL Data Communications.

Practice EIGRP configuration in MPLS VPN environment

If you would like to test how EIGRP works within an MPLS VPN, you can do that in our remote labs. If you have partner-level Cisco Connection Online access, you can do it free of charge:

• Start Partner e-learning connection.
• After landing on the main PEC page, click here (or paste http://cisco.partnerelearning.com/peclms/lang-en/management/LMS_TrainLedInfo.asp?UserMode=0&LEDefId=20318 in the browser address bar)

If you're note working for a Cisco partner, you can buy the whole set of MPLS remote labs from NIL Data Communications.

MPLS VPN half-duplex VRF works only on virtual template interface

IOS release 12.3(11)T introduced Half-duplex VRF, a great feature for those of us who have to implement hub-and-spoke VPN (the VPN where all traffic has to pass through the central site), but hate the configuration hassle associated with it. Unfortunately, the way this feature is implemented, you can only configure it on virtual access/template interface, making it useless in most access networks. Too bad ...