Collecting SNMP data from tables with arbitrary indexes

From OpenNMS
Jump to: navigation, search


Availability

This feature is available in the 1.3.3 release, and first appears in the 1.3.2 unstable release.

Introduction

Bug #504 is a feature request to add support for collecting and reporting on arbitrary SNMP tables. Previously, OpenNMS could only collect specific instances (0, or whatever the user wanted) or on tables that were indexed by the ifIndex in the ifTable. This feature takes the already-generalized SNMP walker code and adds a few new configuration, storage, and reporting features to gather data from tables with an arbitrary index. We use the term "resource type" to refer to each different custom index, be it the MIB-2 host-resources storage table, a Brocade fibre channel port table, as well as the existing node-level performance data and interface resource types.

The main part of this feature is the ability to define a custom resource type. The custom resource type is then used when collecting the data and when displaying the data.

Configuration

Creating a "resourceType" definition

Data collection is configured like normal, but with a few small differences. First, a "resourceType" element needs to exist in datacollection-config.xml to specify the custom resource type. For the MIB-2 host resources storage table, we have an entry like this:

<resourceType name="hrStorageIndex" label="Storage (MIB-2 Host Resources)">
  <persistenceSelectorStrategy class="org.opennms.netmgt.collectd.PersistAllSelectorStrategy"/>
  <storageStrategy class="org.opennms.netmgt.dao.support.IndexStorageStrategy"/>
</resourceType>

This entry needs to be before the "groups" element.

The "name" attribute ("hrStorageIndex" in the above example) is not magical in any way, i.e. it does not need to correlate to a MIB that OpenNMS has knowledge of. The generic index code does not care about the length of the index, so this facility should work for collecting from multiply-indexed tables such as tcpConnTable. Note that this statement is theoretical at present, as multiply-indexed collection has not been tested.

The "name" attribute is used later in the datacollection-config.xml file to refer to this custom resource type, and also in the snmp-graph.properties file to refer to this resource type. We also need to separate data for this resource type from other resource types (node and interface data, as well as other custom resource types), so we place all of the RRD data into a directory based on the name. So, for example, for node 1, all data for the example resource type will go into $OPENNMS_HOME/share/rrd/snmp/1/hrStorageIndex.

The "label" attribute ("Storage (MIB-2 Host Resources)" in the above example) is used solely for a user-friendly label in the webUI when listing this resource type.

Lastly, the "persistenceSelectorStrategy" and "storageStrategy" define the classes that are used to decide whether data for this resource type should be persisted (written to disk), and how the data is written to disk. The classes shown in the example above are the usual ones, but if you need more flexibility you might want to use SiblingColumnStorageStrategy or PersistRegexSelectorStrategy.

Creating "group" and "mibObj" definitions

Create the "group" and its children "mibObj" elements as you normally would, but with two differences:

  1. Set the "ifType" attribute for the "group" element to be "all". Without this, the data will be incorrectly assumed to be node-level data (not indexed), which it isn't.
  2. Set the "instance" attribute on the "mibObj" elements to be the name of your resource type. This would be "hrStorageIndex" for the example above.

Here is the data to be collected for the hrStorageIndex resource type:

<group name="mib2-host-resources-storage" ifType="all">
  <mibObj oid=".1.3.6.1.2.1.25.2.3.1.3" instance="hrStorageIndex" alias="hrStorageDescr" type="string" />
  <mibObj oid=".1.3.6.1.2.1.25.2.3.1.4" instance="hrStorageIndex" alias="hrStorageAllocUnits" type="gauge" />
  <mibObj oid=".1.3.6.1.2.1.25.2.3.1.5" instance="hrStorageIndex" alias="hrStorageSize" type="gauge" />
  <mibObj oid=".1.3.6.1.2.1.25.2.3.1.6" instance="hrStorageIndex" alias="hrStorageUsed" type="gauge" />
</group>

Adding the group to a system definition

In datacollection-config.xml, OpenNMS needs to be able to map a remote SNMP agent's "system object ID" to a set of data collection groups so it knows what it should gather for different types of SNMP agents. You will need to add the newly-created collection group to one or more system definitions (systemDef) in datacollection-config.xml so that OpenNMS will try to collect data for that collection group. The example below shows that the group for "mib2-host-resources-storage" was added to the definition for Net-SNMP which matches any agent that has a system object ID that begins with ".1.3.6.1.4.8072.3.":

<systemDef name="Net-SNMP">
  <sysoidMask>.1.3.6.1.4.1.8072.3.</sysoidMask>
  <collect>
    <includeGroup>mib2-host-resources-storage</includeGroup>
    <includeGroup>mib2-host-resources-system</includeGroup>
    <includeGroup>mib2-host-resources-memory</includeGroup>
    <includeGroup>net-snmp-disk</includeGroup>
    <includeGroup>ucd-loadavg</includeGroup>
    <includeGroup>ucd-memory</includeGroup>
    <includeGroup>ucd-sysstat</includeGroup>
  </collect>
</systemDef>

Creating a report in snmp-graph.properties

Finally, you create a report in snmp-graph.properties like you normally would, with the exception that you set the type to match the name of the resource type.

Here is an example for hrStorageIndex disk utilization data:

report.mib2.storage.usage.name=Storage Utilization (MIB-2 Host Resources)
report.mib2.storage.usage.columns=hrStorageSize, hrStorageUsed, hrStorageAllocUnits
report.mib2.storage.usage.type=hrStorageIndex
report.mib2.storage.usage.command=--title="Storage Utilization" \
   --vertical-label="Bytes" \
   DEF:total={rrd1}:hrStorageSize:AVERAGE \
   DEF:used={rrd2}:hrStorageUsed:AVERAGE \
   DEF:units={rrd3}:hrStorageAllocUnits:AVERAGE \
   CDEF:totalBytes=total,units,* \
   CDEF:usedBytes=total,used,-,units,* \
   LINE2:totalBytes#0000ff:"Total" \
   GPRINT:totalBytes:AVERAGE:" Avg  \\: %8.2lf %s" \
   GPRINT:totalBytes:MIN:"Min  \\: %8.2lf %s" \
   GPRINT:totalBytes:MAX:"Max  \\: %8.2lf %s\\n" \
   AREA:usedBytes#ff0000:"Used " \
   GPRINT:usedBytes:AVERAGE:" Avg  \\: %8.2lf %s" \
   GPRINT:usedBytes:MIN:"Min  \\: %8.2lf %s" \
   GPRINT:usedBytes:MAX:"Max  \\: %8.2lf %s\\n"

Make sure that you add your new report to the "reports" parameter in the same file, otherwise the graphing system won't add it as an available report. For the above example, the report name is "mib2.storage.usage".

reports=mib2.bits, mib2.percentdiscards, mib2.percenterrors, \
...
mib2.storage.usage, \
...

Restart Tomcat

If you are using Tomcat, you will need to restart the servlet container before your new indexes and reports are displayed under your nodes' Resource Graphs. It is no longer necessary to restart the Web Application container.

Examples

HrStorageIndex-ResourceList.png

HrStorageIndex-Graph.png

Further Examples

There is also example datacollection-config.xml and snmp-graph.properties available for UCD disk IO stats