In my discussion of per-packet versus per-destination load sharing, I've relied on the "accepted wisdom" that out-of-order TCP packets reduce session performance (as a side note, out-of-order UDP packets are a true performance killer; just try running NFS with out-of-order packets).
Today I've discovered another huge show-stopper: stateful firewalls (read: almost everything in use today) might just drop out-of-order packets, resulting in TCP timeouts and retransmissions (and repeated timeouts will totally wreck the session throughput). Here's how Cisco devices handle this problem:
- PIX allows three out-of-order packets per TCP session (cannot be changed, but should be enough)
- You can configure out-of-order packet handling on ASA with the queue-lenght parameter of a tcp-map .
- Cisco IOS firewall (formerly known as CBAC) drops out-of-order packets until release 12.4(11)T where you can use the ip inspect tcp reassembly configuration command (and it looks like the zone-based firewall configuration is not yet supported).

A similar problem is IP fragmentation (although that should not happen too often if MTU discovery is working properly), which is handled with the Virtual Fragmentation Reassembly feature introduced in IOS release 12.3(8)T.
ReplyDeleteBut problem really is not caused by load-sharing. It is because many things aren't prepared for out ouf order packets and it can just happen without load sharing, because that is how Internet works. You can fix your configuration by using per-destination, but it won't fix remote locations.
ReplyDeleteAs I wrote in a comment to another post, I really cannot see packets getting reordered unless you use per-packet load sharing or weird QoS setup that treats packets from the same flow differently.
ReplyDeleteIf there's something I'm missing, I would be glad to hear about it.