Followup: zone-based firewall performance

The Zone-based firewall performance post has generated a few interesting comments. William Chu and an anonymous reader posted links to a Cisco ZBFW performance document. The document claims that the performance of TCP session inspection was significantly increased in 12.4(4)T (which would apply to CBAC as well, since zone-based firewalls were introduced in 12.4(6)T), but the maximum HTTP throughput numbers for ZBFW are way lower than the Cisco IOS Firewall Performance (table 3 of the Cisco Integrated Firewall Solutions document). One could only guess that the discrepancy does not indicate that CBAC is twice as fast as ZBFW but illustrates the gap between the real-life test scenarios and marketing figures.

David has also mailed me an interesting observation: CBAC inspects all traffic exiting (or entering) an interface; ZBFW inspects only inter-zone traffic. This distinction does not matter in common scenarios where there is not much traffic between external interfaces, but it could become important if you use IOS firewall to filter traffic between two IP networks and have multiple transit interfaces in each network.

Zone-based firewall performance

David asked me an interesting question:

Can you comment on the performance differences between zone-based firewalls and the classic Content-Based Access Control (CBAC) IOS firewall? I’m running into issues where the router is running VoIP and CBAC, and call quality issues are appearing during heavy data usage.

I never did performance tests with one or the other, but I wouldn’t expect the zone-based firewall (ZFW) performance to exceed CBAC. They use the same (or at least very similar) code, ZFW is primarily a different method of configuring the same functionality.

Does anyone have different experience? It looks like Colin McNamara disagrees with me, but the document with performance data I found at Cisco’s web site does not list different figures for CBAC and ZFW (and they would surely make them public if the ZFW would be way better than CBAC).

This article is part of You've asked for it series.

Running OSPF across a PIX/ASA firewall: TTL details

Sharath Samanth has recently asked an interesting question:

I have seen the post on running OSPF across a PIX firewall. Since I did not have a PIX, I tested the solution by replacing PIX with a router.

I had configured the neighbor statements on both routers, but the OSPF was failing to come up. The debug indicated that the router emulating PIX was sending time exceeded ICMP to both OSPF-speaking routers.

The OSPF hello by default has a TTL of 1 which I think is an issue with this scenario. Is there anything special thats done on PIX to get OSPF working?

The answer is quite simple: PIX is not behaving like a router, but rather like a bridge with additional IP features (NAT and traffic filters). It does not decrement the TTL of a transit packet (which could lead to interesting loops if you badly mess up a redundant topology) … and I have to congratulate Sharath for an excellent diagnosis of the problem.

This article is part of You've asked for it series.

Cisco 851 and 871 bridge between LAN and WAN interfaces during boot process

Euphrates Greene sent me a report of a very annoying “functionality” of Cisco 851/871: they're bridging between the inside (LAN) ethernet and outside (WAN) ethernet interfaces while they're running the ROMMON code (from the reload/power-up throughout the software decompression process until the control is transferred to the Cisco IOS). It's worth mentioning that these routers are commonly used as SOHO firewalls and that the internal LAN is exposed while the router is in the bridging mode.

Our security experts have replicated the behavior and reported it to Cisco PSIRT. Fortunately it's a known vulnerability, documented as CSCsd60259 (release note is available on CCO to registered users) and fixed with a ROMMON upgrade.

New routers are shipped with new ROMMON version, so you shouldn't be seeing this behavior on brand new boxes … but one cannot help but wonder why such a nasty behavior was not documented as a field notice/security advisory.

RFC 3514 implemented by the ASR series of routers

The information on the IOS XE software used by the recently launched ASR 1000 router is pretty scarce (there is still no link to the documentation available on CCO), but obviously some backdoor links already exist, as I was able to find some IOS XE-related documents with Google. One of the most amazing features I've found is the support for the security-oriented RFC 3514 which allows you to mark the security level of an IP packet.

The RFC 3514 requires the end host to participate in the process, but as most operating system vendors still don't have a trusted computing platform, a transparent proxy has to be implemented on the network edges to properly tag the ingress packets. ASR 1000 has the first high-speed implementation of the RFC 3514 proxy thanks to its non-deterministic parallel QuantumFlow processors.

The configuration of the RFC 3514 proxy is extremely simple: all you need to do is to configure auto-secure mark on the ingress interfaces of the ASR 1000. Once the security bit has been set, you can use the match ip security-bit 0|1 command in a class-map or a route-map on any router running IOS release 12.4(11)T or later (the command is still hidden).

SNPA labs available on Partner Education Connection

Our new remote lab exercises covering the Securing Networks with PIX and ASA v5.0 course have been made available free of charge to Cisco partners on
Partner Education Connection. To start them, just click this link, log in and select the desired exercise.

If you're not a Cisco partner, you can buy the same exercises on our web site.

Running OSPF across a firewall

When I was reading Joe Harris' post detailing how you can run OSPF across a PIX/ASA firewall (not with it, but between the adjacent routers), I wondered why anyone would want to do it. Our security gurus immediately gave me an answer: suppose you have two separate DMZs similar to the following setup:

You definitely don't want to attract traffic to a DMZ if the Internet uplink is down. One of the ways to handle it is to conditionally source the OSPF default route on the outside routers (based on availability of BGP prefixes, for example) and pass it to the inside routers that propagate it to the rest of the network.

The question that remains to be answered is “why wouldn't you run OSPF with the firewall?” If nothing else, if someone breaks into the outside router, he could manipulate the IP routing on the firewall, which is definitely not a good idea (you can't filter routing updates in OSPF like you can in RIP, EIGRP or BGP).

Practice the PIX and ASA configuration in a remote lab

If you're studying for your CCSP exam or have to test some of the new features available on PIX and ASA, the remote lab exercises supporting the Securing Networks with PIX and ASA course from Cisco might be just the right thing for you. You'll be able to configure firewall and VPN features of PIX/ASA, as well as test its integration in a network, for example, usage of AAA server and deployment of WebVPN. The lab exercises also cover interesting improvements like transparent firewall, virtual firewall and active/active failover.

More information is available here.

Inspect router-generated traffic

A while ago a reader has asked me whether you could modify an IP access-list when the interface IP address changes. While that's definitely doable with Tcl and Embedded Event Manager, it's not a trivial task, so I've tried to understand why he would need such a functionality.

The answer was quite interesting: he's running NTP on his firewall router and thus needs to accept incoming NTP responses from an external NTP server. While that could be easily achieved with the following configuration (only the relevant bits-and-pieces are shown), he didn't want to make the access-list too generic (allowing NTP from the external server to any IP address).

ip inspect name DEFAULT100 tcp
ip inspect name DEFAULT100 udp
!
interface Dialer0
 description $FW_OUTSIDE$
 ip access-group 102 in
 ip inspect DEFAULT100 out
!
access-list 102 remark #### Dialer0 incoming ####
access-list 102 remark #### non-relevant lines deleted
access-list 102 permit udp host 1.2.3.4 eq ntp any eq ntp

This problem nicely illustrates a broader issues: the router does not inspect it's own traffic and thus does not prepare conduits for the return packets; you have to specify all the return traffic you're expecting in the incoming access list. This drawback has been fixed in IOS release 12.3(14)T with the introduction of the Inspection of Router-Generated Traffic feature. In our scenario you only need to change the inspect rules:

ip inspect name DEFAULT100 tcp router-traffic
ip inspect name DEFAULT100 udp router-traffic

... and the router synchronizes to an external NTP server:

sp#show ip inspect sessions
Established Sessions
 Session 474032B4 (192.168.1.3:123)=>(10.0.0.1:123) udp SIS_OPEN
sp#
01:04:34: %NTP-5-PEERSYNC: NTP synced to peer 10.0.0.1
01:04:34: %NTP-6-PEERREACH: Peer 10.0.0.1 is reachable

Note: This article is part of You've asked for it series.

The self zone in zone-based firewall configuration

One of my readers made an interesting observation when faced with configuring zone-based firewall on Cisco IOS: „My main issue is a confusion between when to use self and when to use in/outside.“

The rules are simple:

• Whenever you filter traffic transiting the router, you control it with a zone-pair specifying an inside and an ouside zone.
• The self zone controls traffic sent to the router itself or originated by the router.
• Unless you specify a zone-pair combining self zone with another zone, all traffic from that zone sent to the router itself is allowed (the router is not protected)
• To control traffic that the router can send into a zone use a zone-pair from self to another zone. Use inspect in the service-policy to allow the return traffic.
• To filter the traffic that the router can accept, use a zone-pair from another zone to self. Only the packets accepted by this zone-pair's service-policy will be accepted by the router.

More information about the self zone (as well as other aspects of zone-based firewall configuration) can be found in my digital book Deploying Zone-Based Firewalls published by Cisco Press.

Sinkholes and blackholes

In his latest Q&A post, Scott Morris mentioned an excellent Cisco article that describes routing tricks needed to implement sinkholes and remote blackholes in great details. Highly recommended reading.

Improve your hands-on PIX and ASA configuration skills

If you're not deploying Cisco firewalls on a regular basis, you'd definitely benefit from going through Securing Networks with PIX and ASA (SNPA) remote lab exercises practicing common configuration tasks, from simple to pretty complex ones ... not to mention that SNPA is part of the Cisco Certified Security Professional (CCSP) certification.

If you have partner-level Cisco Connection Online access, you can get these labs free of charge:

• Start Partner e-learning connection.
• Select Lab central.
• In the Cisco Career Certification Labs table, select CCSP.
• Enter SNPA into keywords field, and perform search

If you're note working for a Cisco partner, you can buy these remote labs from NIL Data Communications.

Where did the CBAC go?

I've got an interesting question a while ago: Do new Cisco routers still use CBAC?

Of course they do, it's just been renamed. The marketing department has decided that Context Based Access Control (CBAC) does not sound nearly so nice as the Cisco IOS Firewall. Even the command structure hasn't changed, you still use the ip inspect commands to configure it, unless, of course, you have IOS release 12.4(6)T or newer, where you can use zone-based policy firewall configuration.

This article is part of You've asked for it series.

Default action in firewall policy maps

Marko asked a very interesting question: What is the default class policy in a firewall policy-map (policy-map type inspect)? Or, using his original wording, "is it mandatory to use class class-default/drop" at the end of every policy map?

As it turns out, the default action for any class (unless you specify otherwise) is drop. By default, packets not matched by any traffic class are therefore dropped (unless you specify a different action in the class-default), similar to well-known ip access-list behavior.I've tested this feature using a reverse setup: web traffic mapped into a class with no action and all other traffic being inspected in the class-default:

class-map type inspect match-all web
 match protocol http
!
policy-map type inspect InToOut
 class type inspect web
 class class-default
  inspect
!
zone security Inside
zone security Outside
zone-pair security InToOut source Inside destination Outside
 service-policy type inspect InToOut

As expected, HTTP traffic (port 80) was the only traffic blocked by the firewall.

Use access-lists to filter IP packets with IP options

In the security advisory detailing the Crafted IP Option Vulnerability (a nasty bug that allows an intruder to reset your router with a ping packet), the authors forgot to mention a great tool available in IOS release 12.3(4)T (and integrated in 12.4): the ability to filter packets with IP options in an access-list.I've used this proof-of-concept access list that logs all ping packets with IP options:

ip access-list extended LogOptions
 permit icmp any any echo option any-options log
 permit icmp any any echo-reply option any-options log
 permit ip any any

After an extended ping from my Windows XP workstation, the router (as expected) logged the offending packets:

%SEC-6-IPACCESSLOGDP: list LogOptions permitted icmp 10.0.0.10 -> 172.16.0.12 (8/0), 1 packet
%SEC-6-IPACCESSLOGDP: list LogOptions permitted icmp 10.0.0.10 -> 172.16.0.12 (8/0), 3 packets

Note: you can easily generate ICMP packets with IP options with the -r, -s, -j or -k options of the Windows XP ping program (and I am positive Linux ping has equivalent options).

Reading the Cisco advisory, the IP access-list that would drop all dangerous packets would be similar to this one (not tested, use at your own discretion):

ip access-list extended ProtectRouter
 deny icmp any any echo option any-options
 deny icmp any any information-request option any-options
 deny icmp any any timestamp-request option any-options
 deny icmp any any mask-request option any-options
 deny pim any any option any-options
 deny 113 any any option any-options
 deny tcp any any eq 465 option any-options
 permit ip any any

Firewalls kill TCP performance when faced with out-of-order packets

In my discussion of per-packet versus per-destination load sharing, I've relied on the "accepted wisdom" that out-of-order TCP packets reduce session performance (as a side note, out-of-order UDP packets are a true performance killer; just try running NFS with out-of-order packets).

Today I've discovered another huge show-stopper: stateful firewalls (read: almost everything in use today) might just drop out-of-order packets, resulting in TCP timeouts and retransmissions (and repeated timeouts will totally wreck the session throughput). Here's how Cisco devices handle this problem:

• PIX allows three out-of-order packets per TCP session (cannot be changed, but should be enough)


• You can configure out-of-order packet handling on ASA with the queue-lenght parameter of a tcp-map
.

• Cisco IOS firewall (formerly known as CBAC) drops out-of-order packets until release 12.4(11)T where you can use the ip inspect tcp reassembly configuration command (and it looks like the zone-based firewall configuration is not yet supported).

Deploying Zone-Based Firewalls

Cisco Press has just released my latest book (and my first digital one): Deploying Zone-Based Firewalls. The book covers a completely new way to configure IOS firewall feature set based on security zones you define on a router and inter-zone policies configured using the familiar class-maps and policy-maps.

You can preview this digital book (they call it Digital Short Cut) using the Safari technology at Cisco Press and buy it at Amazon.