IS-IS+MPLS TE+Native IPv6=FAIL

The information in this article applies only to single-topology IS-IS setup. Multi-topology IS-IS works correctly.

It’s a shame but it looks like IPv6 and MPLS TE don’t work together in current IOS releases. The standards are there, but Cisco didn’t find the motivation to implement them yet. The situation is particularly grave if you happen to:

• run IS-IS as your routing protocol (many large Service Provider networks do);
• use IS-IS for both IPv4 and IPv6 (makes perfect sense) in single-topology mode (not the best idea);
• use MPLS TE for traffic engineering and/or fast reroute (not uncommon);
• use autoroute on MPLS TE tunnels (most people do);
• run IPv6 natively (without end-to-end MPLS LSP and 6PE).

In this scenario, IS-IS will install autoroute-enabled MPLS TE tunnels in the SPF tree (see autoroute basics for more details) but as IPv6 and MPLS TE tunnels don’t mix, the IPv6 destinations behind the MPLS TE tunnels will not be reachable at all.

MPLS TE issues in IPv6 networks are covered in Building IPv6 Service Provider core webinar (register here).

MPLS TE autoroute basics

A few days ago I had an interesting discussion about the need for bidirectional MPLS TE tunnels when using MPLS TE forwarding adjacency. Although I’d always thought I knew how MPLS TE FA works, it took me a while to figure out all the details, so I decided to write an in-depth description of this interesting feature ... and realized after a few botched attempts it would be better to start with the fundamentals.

If you want to understand the benefits of MPLS TE FA, you need to know how MPLS TE autoroute works (and what its limitations are), so we’ll start with the MPLS TE autoroute basics.

Upgrade, Reboot, Repeat

Disclaimer: No, this post has absolutely nothing to do with any software I regularly blog about.

The FTP Butterfly Effect

Anyone dealing with FTP and firewalls has to ask himself “what were those guys smokingthinking?” As we all know, FTP is seriously broken interestingly-designed:

• Command and data streams use separate sessions.
• Layer-3 addresses and layer-4 port numbers are carried in layer-7 messages.
• FTP server opens a reverse session to a dynamic port assigned by the FTP client.

Once upon a time, there was a very good reason for this weird behavior. As Marcus Ranum explained in his Internet nails talk @ TEDx (the title is based on the For Want of a Nail rhyme), the original FTP program had to use two sessions because the sessions in the original (pre-TCP) Arpanet network were unidirectional. When TCP was introduced and two sessions were no longer needed (or, at least, they could be opened in the same direction), the programmer responsible for the FTP code was simply too lazy to fix it.

Innovative coincidences

In another close-to-perfect series of events, Scott Berkun has just published his latest speech on innovation delivered at The Economists’ Ideas Economy event. I loved this part (you might have noticed I’m following the Schneier Blogging Template) ...

You can put the word innovation on the back of a box, or in an advertisement, or even in the name of your company, but that does not make it so. Words like radical, game-changing, breakthrough, and disruptive are similarly used to suggest something in lieu of actually being it. You can say innovative as many times as you want, but it won’t make you an innovator, nor make inventions, patents or profits magically appear in your hands.

… but you should really take the time to read the whole article; it's a gem.

Any similarity to the recent Innovation is Everywhere event is obviously pure coincidence. If you don’t believe me, read some more statistics-based debunking from the resident skeptic Michael Shermer.

Cisco TelePresence Calculator

It’s pretty hard to sell something to people who haven’t realized they need it. My workshops and Cisco TelePresence are perfect examples. While I’m struggling to find the right approach, infinitely more resourceful Cisco’s marketing created an ideal tool: the Cisco TelePresence Calculator.

IPv6 Deployment Scenarios recording now on YouTube

One of the interesting suggestions I got when asking you to help me fix my webinars was to publish samples of the recorded presentations (this would allow the potential buyers to see what they look like). I’ve posted the first one on YouTube this weekend; in this short segment from the Market Trends in Service Provider Networks webinar I’m discussing viability of Service Provider IPv6 deployment in various customer environments (enterprise/hosting/residential).

IP Multicast is like Banyan Vines

Every now and then I stumble upon an elegy lamenting the need to study IP Multicast to pass one or the other certification exam. The history obviously repeats itself; we’ve been dealing with similar problems in the past and one of my favorite examples is Banyan VINES.

If you’ve been working with Cisco routers for more than 15 years, you might still have fond memories of Router Software Configuration (RSC) course, at its time one of the best networking courses. In those prehistoric days, the networks were multi-protocol, running all sorts of things in parallel with IPv4. The week-long RSC course thus covered (at least) the following protocols: IPv4, AppleTalk, Novell IPX, DecNET, XNS, Banyan VINES, CLNP and SNA (I probably forgot one or two). By the third day, everyone (including the instructor) was sick-and-tired of the endless stream of lookalike protocols and ready to skip a section or two.

Poll results: Internet traffic transport options

I was somewhat surprised by the results of my recent “Do you use MPLS to transport Internet traffic?” poll. I had assumed that most Service Provider networks today use MPLS-based BGP-free core, but almost half of the respondents transport Internet traffic natively.

Off-topic: Sounding like the Tin Man

One of the things I wanted to do in the last week was to publish samples of my webinars on YouTube. Sounds simple: you take the Webex recording, convert it to another file format, add an opening and closing slide and you’re done. Like always, the devil is in the details.

Webex has a standalone conversion utility that runs on Linux. The audio retrieval part reliably crashes on my Fedora, so I end up having the advancing slides video with no audio. The conversion process takes as long as the original recording; each try takes quite a long time. No wonder I gave up.

Borderless Networks, Take Two

Another cloudy product launch happened on Wednesday: the next step in the Borderless Networks saga with the tagline Innovation is Everywhere (what a revelation; we were not aware of that before the event).

Must read: why is cloud computing a bad metaphor

I wanted to entertain you with some juicy opinions about the webcast, but that will have to wait; I’m going rock climbing in a few minutes. In the meantime, you can satisfy your inner Dilbert with a comprehensive technical (what a relief!) summary of the products and technologies launched on Wednesday published by Jennifer McAdams in the Cisco’s Innovation blog. Thank you, Jennifer! Great job; exactly what the engineers need.

CLNS and CLNP

Yap Chin Hoong has been looking at the OSI protocol stack I’ve published in the CT3 wiki and asked an interesting question: “where is CLNS in that protocol stack?”

The OSI protocol stack has a major advantage over the TCP/IP stack: it defines both the protocols and the APIs between the layers. CLNS (Connection-less network Service) is the API (the function calls that allow transport layers to exchange datagrams across the network) while CLNP (Connection-less network Protocol) is the layer-3 protocol that implements CLNS. In my diagram, CLNS would be a thin line above CLNP between L3 and L4 boxes.

IOS developers did not escape the confusion between CLNS and CLNP. The clns routing command does not make sense; you cannot route an API. The command should have been called clnp routing.

Do you use MPLS to transport Internet traffic?

One of the interesting questions I’ve got after announcing the Building IPv6 Service Provider core webinar was “Why would you push IPv6 into the core?”

Obviously there are two mechanisms to transport Internet traffic across your (next-generation ;) Service Provider core network: forward it natively or transport it between BGP next-hops across MPLS LSP. It would be really interesting to know how many people use one or the other mechanism, so if you’re working in a Service Provider environment, please respond to the poll (use this link if you’ve disabled JavaScript).

Both transport mechanisms are described in details in my Building IPv6 Service Provider core webinar (don’t forget to register). Conversely, if you’re still not sure why you should have an IP/MPLS backbone, the Market trends in Service Provider networks webinar will definitely persuade you (or your boss).

More details: Seven IPv6 myths

Two weeks ago I listed six IPv6 myths, asking you to add your own favorites. Obviously the MythBusters are not reading my blog and everyone else decided to focus on a single provocative sentence (got you!) and expressed strong feelings about NAT being (or not being) a security feature.

I've described the myths (including the mobility myth to get their number up to the nearest magic number) in more details in the Seven IPv6 networking myths that don't match reality article published by SearchTelecom.

If you’re working in the Service Provider environment and plan to roll out IPv6 services, don’t forget to register to my Building IPv6 Service Provider core webinar.

SNMP over XML over HTTP?

The snmp-server host command in Cisco IOS has an interesting option: you can specify an URL as the destination host, for example: snmp-server host http://1.2.3.4/xxx traps snmpv2c public. However, I was not able to make it work; the router would accept the configuration command, but the outbound HTTP session never starts. Has anyone managed to get this to work? Any ideas what else is required?

Building IPv6 Service Provider Core: Follow-up questions

I’ve received several follow-up questions after announcing the Building IPv6 Service Provider core webinar. Two were extremely easy:

Q: Can I register after April 5th?
A: Yes, you can. The registration deadline for all my webinars is 1 day before the event.

Q: Is this a Webex event (not a presentation on a location)?
A: Yes, a webinar is a web-delivered event (and I’m in fact using Webex). You can be anywhere in the world, as long as you have a decent Internet connection (we’re using VoIP).

The third one was a bit trickier:

Q: We are looking at moving to IPv6 within our Enterprise. Would this training give us clearer insight into that?

A: This webinar assumes you already know the IPv6 basics (addressing, routing). If your network is big enough that you use BGP or MPLS internally, you'd definitely benefit from this training, but it covers primarily the backbone part and a few select access issues. If your network is smaller (so you don’t use BGP internally) or you don’t know enough about IPv6, I would recommend standard Cisco’s IPv6 training courses.

And don’t forget: the webinar is less than a month away. Register now.

The cult of busy

I love reading Scott Berkun’s blog. For years I’ve been doing (and preaching) most of the things he writes about, but sometimes he manages to describe them so eloquently that the reading of familiar thoughts becomes pure pleasure. You simply must read the Cult of Busy post; I’ve seen too many people working 12+ hours a day and achieving nothing or pointy-haired bosses who judged the productivity of their team solely by the time they left the office (and consequently managed to end with a heap of useless individuals).

Secure BGP

One of the decades-long grudges most people have with BGP is that it’s so easy to insert bogus routing information into the Internet if your upstream ISP happens to be a careless idiot (as Google discovered when Pakistan decided to use blackhole routing for Youtube and leaked the routes). There are two potential solutions that use X.509 certificates to authenticate BGP information: Secure BGP (which uses optional transitive attributes) authenticates the originator as well as the whole AS-path (using AS-by-AS certificates), while the significantly simpler Secure Origin BGP (which uses new BGP messages) authenticates only the originator of the routing information.

New webinar: Building IPv6 Service Provider Core

When building an IPv6-enabled Service Provider (or large enterprise) core, you have three design options: dual-stack deployment, running IPv6 over MPLS (6PE) or running IPv6 inside a VPN (6VPE). My new webinar, Building IPv6 Service Provider core, describes the principles of all three design options and their comparative benefits and drawbacks. It’s targeted at CCNP/CCIE-level engineers with basic IPv6 knowledge and includes detailed configuration examples and router printouts. After attending the webinar, you’ll get complete router configurations which you can use in your own lab to test the scenarios discussed in the webinar.

If you want to attend this webinar on April 14^th, register now (and don’t forget – I’m experimenting with reduced webinar pricing; it would be great if we could prove that it works).

CRS-3: The marketing flop of the year

When I received the first invitations to Cisco’s product announcement that will “forever change the Internet”, I knew it would be another case of overpromising and underdelivering. But even being prepared for the let down, I was totally disappointed when the “magic” product was another high-end router. No doubt it’s an important product, no doubt it will give the Tier-1 service providers a tenfold improvement of the total network throughput, no doubt it’s a wonderful piece of engineering (quoting the Cisco’s press release: it unifies the combined power of six chips to work as one ... you see how banal and degrading the engineering efforts look when described by marketing?), but it will “forever change the Internet” in the same way that AGS+, Cisco 7000, Cisco 7500, Cisco 12000 and CRS-1 did ... by providing ever-higher core network throughput.

New webinar pricing

The responses to the “Can you help me fix the webinar marketing?” question were quite explicit: it’s too expensive for most people. Several readers indicated that they feel the optimal price would be around $50 (and some off-line respondents told me the original $90 price tag was too hard to expense). As Julian pointed out, “the market is always right”, so I’ve decided to reduce the price for the next webinars to $49.99, including the “Choose the optimal VPN service” on May 11^th and the “Market trends in Service Provider networks” on June 2^nd.

Off-topic: The survey bias

Bad designer, one of my favorite devil’s advocates asked an interesting question about post-course survey results:

Once in a course or similar and you get to know someone it becomes v difficult to give bad results, in particular, if it is life effecting in some way such as bonuses or future work etc. In fact one can argue that you should get high results just for high effort with integrity.

The bias toward higher scores is definitely present and is in fact so strong that 4.0 usually represents a barely acceptable result; sometimes the minimum acceptable average score for an instructor is set to 4.3 – 4.5. It’s also very important to understand how the questions are phrased and what the results actually mean.

Anyone attending Cisco Expo 2010 in Slovenia?

The local Cisco office sent me such a nice invitation to the Cisco Expo Slovenia event that I simply had to register. So, if you’ll be in Portorož during the event and would like to join me for a cup of coffee or a beer (hopefully on a terrace overlooking the sea), get in touch ... or look for the guy asking nasty questions from the back row ;)

This is how you design a useful protocol

A post in the My CCIE Training Guide pointed me to the GoogleTechTalk given by Yakov Rekhter (one of the fathers of BGP) in 2007. You should watch the whole video (it helps you understand numerous BGP implementation choices), but its most important message is undoubtedly the Design by Pragmatism approach:

• They had a simple, manageable problem (get from a spanning-tree Internet topology to a mesh topology).
• They did not want to solve all potential future problems; they left that marvelous task to IDRP (which still got nowhere the last time I've looked).
• They started with simple specifications (three napkins), had two interoperable implementations in a few months, and wrote the RFC after BGP was already in production use.
• They rolled it out, learnt from its shortcomings and fixed it.
• They gradually made it easily extensible: TLV encoding, optional attributes, capabilities negotiations. This approach made it possible to carry additional address families in BGP and use it for applications like MPLS VPN and VPLS.

One could only hope that the IPv6 architects had used the same approach ... but as Yakov said in his talk, that’s “water under the bridge”.

Can you help me fix the webinar marketing?

The Market trends in Service Provider networks webinar was obviously well received by the attendees ... the “only” problem was that there were so few of them. The conversion ratios were murderous:

• From over a hundred thousand visitors who have seen the webinar announcement, approximately 1% clicked on the registration link. This is normal and expected; most people are banner-blind and many visitors are not interested in the particular topic or don’t want to attend a webinar.
• Over a thousand visitors decided that the registration page is worth looking at, but only around 1% actually registered for the webinar. This ratio needs some serious fixing; increasing it by a few percentage points would make the whole idea viable.

If you were among those that were interested enough in the webinar to look at the registration page but did not proceed, please tell me what stopped you from registering. And, obviously, if you’ve spotted a glaring stupidity I made, please share it with me.

Lies, damned lies and independent competitive test reports

When the friendly sales guy from your favorite vendor honors you with an “independent test lab” report on the newest wonderful gadget he’s trying to sell you, there’s one thing you can be sure of: the box behaves as described in the report. The “independent” labs are earning too much money verifying the test results to participate in outright lies. Whether the results correlate with your needs is a different story, but we’ll skip this discussion.

However, when you’re faced with a competitive report from an independent test lab “sponsored” (read: paid) by one of the vendors, rest assured it’s as twisted as it can be (you should also suspect the sponsoring vendor has some significant issues he’s trying to cover). The report will dutifully list the test configurations and the test results ... without mentioning that the configurations and the tests were cherry-picked by the sponsoring vendor. You don’t believe me? Put on your most cynical glasses and read the About us statement from the premier independent test lab.

You still want me to prove my point: look at the latest HP-versus-Cisco blade server test results (paid by HP). They took an oversubscribed UCS chassis (it had 4 10GB uplinks and up to 8 servers) and compared it to an HP chassis with 8 servers and 8 10GB uplinks. Furthermore, Kevin Tolly himself admitted in the comments that they’ve really tested the bandwidth between the servers within the chassis (absolute kudos for being so frank), which you might suspect could be somewhat irrelevant in a typical deployment scenario.

Market trends in SP networks: the survey results

The Market trends in Service Provider networks webinar happened just a few days before my winter vacations, so it took me a while to process the post-session survey results (actually, my paperwork day pushed me to do it). Here are the results:

Overall, I was satisfied with this web session	4,8
This session was a worthwhile investment	4,8
Based on this experience, I would recommend this session to others	Yes (100%)
Based on this experience, I would you attend another similar session	Yes (100%)
Considering other educational experiences, I would rate this session	4,4
Instructor score (3 questions)	4,93
Course materials (5 questions)	4,76

The Eventbrite registration system got a high-five (not surprisingly, it’s an excellent system) and Webex got 4.0 due to not so good voice quality experienced by some participants.

Beta version of Cisco VPN client for Windows 7

Update (2010-05-03): Cisco VPN Client 5.0.07 has been released.

Regardless of the supposed focus of my blog, its most popular post still remains the Cisco VPN client in 64-bit Windows 7. This requirement is obviously so common that Cisco decided to implement the 64-bit client natively; it’s currently in beta testing. Philip Remaker left the following comment at my original post:

QPPB in MPLS VPN

Quick link for the attention-challenged: QPPB works in MPLS VPNs (with a few limitations).

And now for the long story: A while ago I’ve noticed that my LinkedIn friend Joe Cozzupoli changed his status to something like “trying to get QPPB to work in MPLS VPN environment”. I immediately got in touch with him and he was kind enough to send me working configurations; not just for the basic setup, but also for Inter-AS Option A, B and C labs.