Using MRTG with Cisco Firewall Service Modules (FWSMs)

Monitoring CPU Utilization

Monitoring Connection Utilization

Monitoring Memory Utilization

Keeping Track of FWSM Contexts

Graphing Overall Traffic

Monitoring CPU Utilization

Due to the fact that Cisco partitions some FWSM resources into contexts (virtual firewalls), there is currently no way to see global CPU utilization for the entire blade, using SNMP. This leaves us with two alternatives:

  1. Poll each context for its CPU utilization data and add up the numbers to get a rough total
  2. Use a script to log into the FWSM and get this information

Note that neither approach is optimal. Being able to get this data from SNMP, by polling one OID or set of OIDs would be the simplest and most accurate way to get the CPU utilization data.

The OIDs to get CPU data from an FWSM may be found in the CISCO-PROCESS-MIB.

cpmCPUTotal5secRev (1.3.6.1.4.1.9.9.109.1.1.1.1.6)
cpmCPUTotal1minRev (1.3.6.1.4.1.9.9.109.1.1.1.1.7)
cpmCPUTotal5minRev (1.3.6.1.4.1.9.9.109.1.1.1.1.8)

A sample MRTG config might look something like this. This example uses SNMP to get data from an FWSM with 4 contexts and imposes a hard limit of 100 on the return reported value. This is because of the data skewing problem mentioned above.

Target[fwsm1.acpu]: 1.3.6.1.4.1.9.9.109.1.1.1.1.8.1&1.3.6.1.4.1.9.9.109.1.1.1.1.6.1:community@fwsm-context1 +
  1.3.6.1.4.1.9.9.109.1.1.1.1.8.1&1.3.6.1.4.1.9.9.109.1.1.1.1.6.1:community@fwsm-context2 +
  1.3.6.1.4.1.9.9.109.1.1.1.1.8.1&1.3.6.1.4.1.9.9.109.1.1.1.1.6.1:community@fwsm-context3 +
  1.3.6.1.4.1.9.9.109.1.1.1.1.8.1&1.3.6.1.4.1.9.9.109.1.1.1.1.6.1:community@fwsm-context4
MaxBytes[fwsm1.acpu]: 100
Title[fwsm1.acpu]: fwsm1 - Aggregate CPU Utilization
PageTop[fwsm1.acpu]: <b>fwsm1</b> Cisco FWSM-1 - Aggregate CPU Utilization\n
WithPeak[fwsm1.acpu]: wmy
Options[fwsm1.acpu]: absolute,gauge,growright,unknaszero
Directory[fwsm1.acpu]: network/fwsm1/acpu
LegendI[fwsm1.acpu]: 5 min CPU Util.
LegendO[fwsm1.acpu]: 5 sec CPU Util.
Legend1[fwsm1.acpu]: 5 min CPU Util.
Legend2[fwsm1.acpu]: 5 sec CPU Util.
Colours[fwsm1.acpu]: GREEN#00eb0c,BLUE#0000ff,DARK GREEN#006600,DARK GREEN#006600
YLegend[fwsm1.acpu]: % Utilization
ShortLegend[fwsm1.acpu]: %


Monitoring Connection Utilization

Like CPU utilization, getting connection utilization is subject to many of the same limitations if you go the SNMP route (no data from the system context, must poll each context individually and add up the results, etc).

The OIDs to get CPU data from an FWSM may be found in the CISCO-FIREWALL-MIB.

cfwConnectionStatValue (1.3.6.1.4.1.9.9.147.1.2.2.2.1.5)

Two sub-values under this OID (40.6 and 40.7) refer to the current number of active connections and the high-water mark for the number of connections in a given context. Other sub-values, such as the number of half-open connections (40.5) are noted in the MIB, but not implemented in the FWSMs as of release 3.2(4). Perhaps they will be implemented in a future FWSM code release?

A sample MRTG config might look something like this. This example uses SNMP to get data from an FWSM with 4 contexts and imposes a hard limit of 1,000,000 connections, as this is the documented maximum limit from Cisco's FWSM data sheet.

Target[fwsm1.aconns]: 1.3.6.1.4.1.9.9.147.1.2.2.2.1.5.40.6&1.3.6.1.4.1.9.9.147.1.2.2.2.1.5.40.7:community@fwsm-context1 +
  1.3.6.1.4.1.9.9.147.1.2.2.2.1.5.40.6&1.3.6.1.4.1.9.9.147.1.2.2.2.1.5.40.7:community@fwsm-context2 +
  1.3.6.1.4.1.9.9.147.1.2.2.2.1.5.40.6&1.3.6.1.4.1.9.9.147.1.2.2.2.1.5.40.7:community@fwsm-context3 +
  1.3.6.1.4.1.9.9.147.1.2.2.2.1.5.40.6&1.3.6.1.4.1.9.9.147.1.2.2.2.1.5.40.7:community@fwsm-context4
MaxBytes[fwsm1.aconns]: 1000000
Title[fwsm1.aconns]: fwsm1 - Aggregate Connection Utilization
PageTop[fwsm1.aconns]: <b>fwsm1</b> Cisco FWSM-1 - Aggregate Connection Utilization\n
WithPeak[fwsm1.aconns]: wmy
Options[fwsm1.aconns]: gauge,growright,unknaszero
Directory[fwsm1.aconns]: network/fwsm1/aconns
LegendI[fwsm1.aconns]: Connections, current
LegendO[fwsm1.aconns]: Connections, high-water
Legend1[fwsm1.aconns]: Connections, current
Legend2[fwsm1.aconns]: Connections, high-water
Colours[fwsm1.aconns]: GREEN#00eb0c,BLUE#0000ff,DARK GREEN#006600,DARK GREEN#006600
YLegend[fwsm1.aconns]: Connections
ShortLegend[fwsm1.aconns]: conns


Monitoring Memory Utilization

The memory utilization can be gathered using SNMP, similar to the methods above, however it produces data that doesn't line up at all with the output of a "show memory" from the FWSM command line interface. This is because the majority of the memory in use on an FWSM is tied up in processes that run in the system context, and very little is used in the actual firewall contexts. As far as I can tell, the memory used by the firewall contexts is used to store the running config, ACLs, connection state information, log buffers, etc.

There was also a bug in the FWSM code that reported memory in use as 400% of the memory on the blade. This affected both the CLI and the SNMP counters, but was fixed in 3.2(4).

Luckily there is another way to get accurate memory utilization data. It has the advantage of being much easier on the CPU, much simpler to implement using SNMP, and doesn't get caught by the counter bug. Cisco provides a counter for the amount of memory that's not being used, so it becomes a simple matter to subtract that value from the total amount of memory on the blade (1 GB, in the case of our test FWSM) to get the total in use. It also produces graphs that line up very nicely with the output of "show memory".

The OID to get free memory data from an FWSM may be found in the CISCO-MEMORY-POOL-MIB.

ciscoMemoryPoolFree (1.3.6.1.4.1.9.9.48.1.1.1.6)

You will see the OID in the sample config end in .1. That's just an entry identifier that's used by the SNMP agent. The maximum utilization is set at 1 GB (1,073,741,824 bytes).

Target[fwsm1.amem]: 1073741824 - 1.3.6.1.4.1.9.9.48.1.1.1.6.1&1.3.6.1.4.1.9.9.48.1.1.1.6.1&community@fwsm-context1
MaxBytes[fwsm1.amem]: 1073741824
Title[fwsm1.amem]: fwsm1 - Aggregate Memory Utilization
PageTop[fwsm1.amem]: <b>fwsm1</b> Cisco FWSM-1 - Aggregate Memory Utilization\n
WithPeak[fwsm1.amem]: wmy
Options[fwsm1.amem]: gauge,growright,unknaszero
Directory[fwsm1.amem]: network/fwsm1/amem
LegendI[fwsm1.amem]: Current memory usage
LegendO[fwsm1.amem]:
Legend1[fwsm1.amem]: Current memory usage
Legend2[fwsm1.amem]:
Legend3[fwsm1.amem]: Peak memory usage
Legend4[fwsm1.amem]:
Colours[fwsm1.amem]: GREEN#00eb0c,GREEN#00eb0c,DARK GREEN#006600,DARK GREEN#006600
YLegend[fwsm1.amem]: Memory usage
ShortLegend[fwsm1.amem]: B

I haven't had a need yet to graph the memory utilization of a specific context yet, but it is possible. Keep in mind that the numbers it produces are probably going to be pretty different from what you'd expect to see.


Keeping Track of FWSM Contexts

FWSMs are licensed by the number of contexts they can support, with a base license allowing up to 20 contexts, and increasing to 50, 100, and 250 contexts. Unfortunately, at this point, there's no way to count your context utilization in SNMP yet. The only ways to do this are to write a script to interactively log into the FWSM and count the number of configured contexts, or if you archive your configurations, write a script to get the same information from your most recent archived system context config. The script to do either task would be pretty dependent on your individual network architecture, security policies, etc..., so I won't include the script at this time, but a sample MRTG config could look like this:

Target[fwsm1.contexts]: `/usr/local/scripts/net/mrtg-fwsm-ctx-count.pl fwsm1.mynetwork.com`
MaxBytes[fwsm1.contexts]: 50
Title[fwsm1.contexts]: fwsm1 - Configured Contexts
PageTop[fwsm1.contexts]: <b>fwsm1</b> Cisco FWSM-1 - Configured Contexts\n
WithPeak[fwsm1.contexts]: wmy
Options[fwsm1.contexts]: noinfo,gauge,integer,growright,unknaszero
Directory[fwsm1.contexts]: network/fwsm1/contexts
LegendI[fwsm1.contexts]: contexts
LegendO[fwsm1.contexts]:
Legend1[fwsm1.contexts]: contexts
Legend2[fwsm1.contexts]:
Colours[fwsm1.contexts]: GREEN#00eb0c,GREEN#00eb0c,DARK GREEN#006600,DARK GREEN#006600
YLegend[fwsm1.contexts]: contexts
ShortLegend[fwsm1.contexts]: ctx


Graphing Overall Traffic

Knowing how much traffic is going to and coming from your FWSMs can be useful to know for many reasons, however since there isn't a physical interface on the FWSM, how can you get this data? You can get it from the switch's supervisor engine.

Communication between the supervisor and the FWSM is accomplished using a 6-way gigabit etherchannel that gets created when the card is installed and powered up. That is consistent with Cisco's FWSM data sheet, which advertises a traffic maximum of 5.5 Gb/s and a maximum packet rate of 2.8 Mpps.

The channel doesn't show up as an interface in the config, since you can't change any of the settings on it, but it is there as an interface in SNMP, and there are some CLI commands on the supervisor that you can use to see what how much traffic is going to/from the blade.

So what's the name of the interface? If you do a "show etherchannel summary" on the supervisor, it will tell you. It will also show you the 6 gig-e links that are in the etherchannel bundle and you can tell ou're looking at the right slot because the gig-e links will have numbers that correspond to the slot that houses the FWSM. For example, if your FWSM is in slot 9 of a Cat 6509, the ports listed in the bundle will be Gi9/1, Gi9/2, ... Gi9/6. It will also give you the name of the Port-channel interface that's associated with the bundle - in my example, it's Port-channel278 or abbreviated as Po278.

Now you can either build your MRTG config, using that interface name, if you use ifDescr, or you can walk the ifDescr table in the MIB-II tree to get the ifIndex for that Port-channel interface. In this example, I'll walk the tree and get the index.

$ snmpwalk -v 1 -c community fwsm1 .1.3.6.1.2.1.2.2.1.2 | grep -i Port-channel278
IF-MIB::ifDescr.609 = STRING: Port-channel278

The ifIndex in this case is 609.

You pretty much have to use the HC counters to get reliable traffic data, if you're pushing over ~100 Mb/s to the FWSM because the old ifInOctets and ifOutOctets will roll over too quickly, producing erratic results. A MaxBytes value of 687500000 should correspond to 5.5 Gb/s.

Target[fwsm1.octets]: 1.3.6.1.2.1.31.1.1.1.6.609&1.3.6.1.2.1.31.1.1.1.10.609&community@switch
MaxBytes[fwsm1.octets]: 687500000
Title[fwsm1.octets]: fwsm1 - Traffic Utilization (bits)
PageTop[fwsm1.octets]: <b>fwsm1</b> Cisco FWSM-1 - Traffic Utilization (bits)\n
WithPeak[fwsm1.octets]: wmy
Options[fwsm1.octets]: growright,unknaszero,bits
Directory[fwsm1.octets]: network/fwsm1/octets
LegendI[fwsm1.octets]: from FWSM to Supervisor
LegendO[fwsm1.octets]: from Supervisor to FWSM
Legend1[fwsm1.octets]: from FWSM to Supervisor
Legend2[fwsm1.octets]: from Supervisor to FWSM
YLegend[fwsm1.octets]: bits per second

A sample for graphing unicast packet traffic could look something like this:

Target[fwsm1.upkts]: 1.3.6.1.2.1.31.1.1.1.7.609&1.3.6.1.2.1.31.1.1.1.11.609&community@switch:::::2
MaxBytes[fwsm1.upkts]: 2800000
Title[fwsm1.upkts]: fwsm1 - Traffic Utilization (unicast packets)
PageTop[fwsm1.upkts]: <b>fwsm1</b> Cisco FWSM-1 - Traffic Utilization (unicast packets)\n
WithPeak[fwsm1.upkts]: wmy
Options[fwsm1.upkts]: growright,unknaszero
Directory[fwsm1.upkts]: network/fwsm1/upkts
LegendI[fwsm1.upkts]: from FWSM to Supervisor
LegendO[fwsm1.upkts]: from Supervisor to FWSM
Legend1[fwsm1.upkts]: from FWSM to Supervisor
Legend2[fwsm1.upkts]: from Supervisor to FWSM
YLegend[fwsm1.upkts]: packets per second
YLegend[fwsm1.upkts]: pps

Using the appropriate OIDs, you can also set up graphs for multicast and broadcast packets, and create an aggregate view of all three if you want.


updated 30-MAY-2008 Justin M. Streiner - version 1.1


The information in this document is provided for informational purposes only and includes no warranty or guarantee of any kind. The author assumes no liability for your use or misuse of the information presented on this page.