Category: EEM

Daniel left a very relevant comment to my convoluted BGP session shutdown solution:

What I am currently doing is using EEM to watch my tracked objects and then issuing a neighbor shutdown command. Is there a functional reason I would not want to do it that way, and use the method you prescribe?

As always, the answer is “it depends.” In this case, the question to ask yourself is: “do I track configuration changes and react to them?”

In responses to my The Road to Complex Designs is Paved With Great Recipes post Daniel suggested shutting down EBGP session if your BGP router cannot reach the DMZ firewall and Cristoph guessed that it might be done without changing the router configuration with the neighbor fall-over route-map BGP configuration command. He was sort-of right, but the solution is slightly more convoluted than he imagined.

Michael modified one of my EEM applets to monitor CRC errors on WAN interfaces and notify the operator (via e-mail) when an interface has more than two errors per minute. He wanted to monitor multiple interfaces and asked me whether it’s possible to modify the SNMP event detector somehow. I only had to point him to the event correlation feature of EEM version 2.4 and he sent me the following (tested) applet a few days later.

Jeroen sent me an interesting challenge: he would like to reload the router when the 3G WAN interface gets stuck (I thought my Nokia phone is the only one exhibiting this problem, but obviously I was wrong). The reload-on-failed-ping EEM applet I’ve published would be a perfect solution, but it uses track delay and the maximum delay timeout is three minutes, while Jeroen would like to wait 15 minutes before reloading the router.

When I was writing the applet that should stop accidental scheduled router reloads, I wanted to use the action string match command to perform pattern matching on the output of the show reload command. Somehow the applet didn’t want to work as expected, so I checked the documentation on Cisco’s web site.

Reading the command description, I should have realized the whole thing must be broken. It looks like the documentation writer was fast asleep; even someone with a major in classical philosophy and zero exposure to networking should be able to spot the glaring logical inconsistencies.

Alexandra Stanovska wrote an excellent comment to my Schedule reload before configuring the router post:

It may come in handy creating some form of script that would display some basic upon logout - show debug, show reload etc.

The new capabilities of CLI event detector introduced in EEM 3.0 allow us to catch CLI commands in a particular parser mode. Writing an EEM applet that catches exec-mode exit or logout and performs a few checks is thus a trivial task.

Upendra wrote the following comment to my “EEM CLI patterns are not context sensitive” post:

I am totally confused with sync yes|no skip yes|no. What is the mean of sync and skip, when we use these keywords and what is the mean of yes and no.

The online documentation on this topic is pretty extensive, but obviously not explicit enough, so let’s try to reword it.

John McManus published excellent Remote (in Band) Configuration Tips post on etherealmind.com last week, prompting a “Too bad there isn't a fix for forgetting ‘reload in’” tweet by @mfratto. My immediate reaction was “this should be easy to solve with EEM” ... and it is.

Before someone accuses me of being totally FCoE/DCB-focused, here’s an interesting EEM trick. Damian wanted to have time-dependent static routes to ensure expensive backup path is only established during the working hours. I told him to use cron with EEM to modify router configuration (and obviously lost him in the acronym forest)... but there’s an even better solution: use reliable static routing and modify just the track object’s state with EEM.

A while ago one of my readers wanted to perform an extended ping from an EEM applet. For whatever reason the extended ping syntax wasn’t good enough for him, so I told him to use the pattern parameter of the action cli command EEM applet statement.

Jared Valentine found an interesting bug in the EEM’s SNMP event detector: if you’re triggering your EEM applet when the increment of an SNMP variable exceeds the threshold, you cannot re-arm the applet; the exit-type increment does not work. He fixed the problem with a somewhat more convoluted approach:

• The first EEM applet reads the SNMP variable, waits a second, does a second read and stores the difference in a counter.
• The second EEM applet is triggered based on the counter values.

I’m collecting tips like this one in the Embedded Event Manager (EEM) workshop. You can attend an online version of the workshop; we can also organize a dedicated event for your networking team.

Here’s the source code for the first applet (he had to execute CLI show commands to work around the CB-QoS MIB limitations).

More than a year ago, I wrote about the very slow update rate of the variables in the CB-QoS MIB. In August WB found a workaround (do a show policy-map interface before reading the counters) and now Jared has tested it and confirmed that it works. He’s configured a simple EEM applet that executes the show command once per second:

event manager applet UpdateMibTables 
event timer watchdog time 1 
action 1.0 cli command "enable" 
action 2.0 cli command "show policy-map int dialer0" 

With this fix, he can use the SNMP variables in other EEM applets to detect VoIP calls within 1-2 seconds.

Don’t forget: numerous EEM topics are described in the Embedded Event Manager (EEM) workshop. You can attend an online version of the workshop; we can also organize a dedicated event for your networking team.

The engineer who wanted to detect specific DoS attack (WAN link overload) with EEM applet asked for something more in his original question: he wanted to receive a SNMP trap on the NMS when the DoS attack is detected. Implementing this requirement with an EEM applet is simple; you just need to add the trap keyword to the event manager applet configuration command.

EEM-SNMP integration is described in the Embedded Event Manager (EEM) workshop. You can attend an online version of the workshop; we can also organize a dedicated event for your networking team.

Last week I’ve described how you can use EEM to detect long-term interface congestion which could indicate denial-of-service attack. The mechanism I’ve used (the averaged interface load) is pretty slow; using the lowest possible value for the load-interval (30 seconds) it takes almost a minute to detect a DOS attack (see below).

If you want to detect outbound bursts, you can do better: you can monitor the increase in the number of output drops over a short period of time.

Someone sent me an interesting question a while ago: “is it possible to detect DOS flooding with an EEM applet?” Of course it is (assuming the DOS attack results in very high load on the Internet-facing interface) and the best option is the EEM interface event detector.

The interface event detector is more user-friendly than the SNMP event detector. You can specify interface name and parameter name in the interface event detector; with SNMP event detector you have to specify SNMP object identifier (OID). The interface event detector stores the interface name, measured parameter name and its value in three convenient environment variables that you can use to generate syslog messages or alert the operators via e-mail.