Squid Data Collection

From OpenNMS
Jump to navigation Jump to search

Data Collection for Squid 2.7 on Linux

This is a short example of pulling data from a Squid proxy running on Linux (with Net-SNMP). The MIB used is the one that is current with Squid 2.7 -- Squid 3.0 has a different MIB which I have not tested yet. This configuration has been tested on the current stable version of OpenNMS (1.6.5). Almost everything should work with Squid 2.6 and OpenNMS (1.7.8) too.

Configure Squid

The first task is to ensure that Squid has SNMP enabled, and that the access rules for SNMP allow localhost. The Squid binaries provided by most major distributions already have SNMP support compiled in, it is just a matter of turning it on in squid.conf.

In squid.conf ensure the following lines are present, adjusted for your system if necessary:

acl snmp_poll snmp_community your_community_string
snmp_port 3401
snmp_access allow snmp_poll localhost

If you issue a squid -k reconfigure, Squid will pick up the changes.

Configure Net-SNMP

Now set up the SNMP daemon to proxy the squid SNMP data. I find this solution easier than trying to get OpenNMS to poll two different SNMP ports on your Squid server. I am assuming that you are interested in polling the normal Net-SNMP data as well as the Squid SNMP data, and that you have already configured Net-SNMP to allow queries from your OpenNMS server.

In snmpd.conf add this line:

proxy -v 2c -c your_community_string localhost:3401 .

Restart the SNMP daemon for it to pick up changes.

Test SNMP collection

At this point you should be able to issue the following command from your OpenNMS server to verify that your SNMP daemon is proxying for Squid, and that Squid is listening:

james@opennms:~$ snmpwalk -v 2c -c your_community_string .
SNMPv2-SMI::enterprises.3495. = INTEGER: 1048576
SNMPv2-SMI::enterprises.3495. = INTEGER: 32086424
SNMPv2-SMI::enterprises.3495. = Timeticks: (54703796) 6 days, 7:57:17.96
SNMPv2-SMI::enterprises.3495. = STRING: "help@example.org"
SNMPv2-SMI::enterprises.3495. = STRING: "squid"
SNMPv2-SMI::enterprises.3495. = STRING: "2.7.STABLE3"

If you do not see something like the output above, check your configuration. If you do see similar output, the next step is telling OpenNMS how to poll this data.

Editing /etc/opennms/datacollection-config.xml

There are several tables in the Squid SNMP output. You may not be interested in all of them, but I have included them here. You should be particularly careful with the cacheClientsTable -- it will create a subdirectory in your rrd store for each client that Squid reports, and populate that subdirectory with several jrb files. If you have hundreds or thousands of client machines, comment out or omit references to this table!

To read the tables, add resourceType definitions at the top of the file for Squid as follows:

Add Squid resource types

    <!-- Squid Cache resource types -->
    <resourceType name="cacheMedianSvcTable" label="Squid Cache Median Service Times">
      <persistenceSelectorStrategy class="org.opennms.netmgt.collectd.PersistAllSelectorStrategy"/>
      <storageStrategy class="org.opennms.netmgt.dao.support.IndexStorageStrategy"/>
    <resourceType name="cachePeerTable" label="Squid Cache Peer Information">
      <persistenceSelectorStrategy class="org.opennms.netmgt.collectd.PersistAllSelectorStrategy"/>
      <storageStrategy class="org.opennms.netmgt.dao.support.IndexStorageStrategy"/>
    <!-- The cacheClientTable is very resource intensive.
    <resourceType name="cacheClientTable" label="Squid Cache Client Information">
      <persistenceSelectorStrategy class="org.opennms.netmgt.collectd.PersistAllSelectorStrategy"/>
      <storageStrategy class="org.opennms.netmgt.dao.support.IndexStorageStrategy"/>

Modify the Net-SNMP systemDef

Since Net-SNMP is proxying for Squid, edit the Net-SNMP systemDef:

      <systemDef name="Net-SNMP">
---8<--- snip  --8<--- 
          <!-- <includeGroup>squidCacheCacheClientTable</includeGroup> -->

Add the group definitions

Finally, add the group definitions themselves:

      <!-- Squid MIB OIDs -->
      <group name="squidCacheScalars" ifType="ignore">
        <mibObj oid="."        instance="0"  alias="cacheSysVMsize"       type="Integer32" />
        <mibObj oid="."        instance="0"  alias="cacheSysStorage"      type="Integer32" />
        <mibObj oid="."        instance="0"  alias="cacheUptime"          type="TimeTicks" />
        <mibObj oid="."      instance="0"  alias="cacheMemMaxSize"      type="Integer32" />
        <mibObj oid="."      instance="0"  alias="cacheSwapMaxSize"     type="Integer32" />
        <mibObj oid="."      instance="0"  alias="cacheSwapHighWM"      type="Integer32" />
        <mibObj oid="."      instance="0"  alias="cacheSwapLowWM"       type="Integer32" />
        <mibObj oid="."      instance="0"  alias="cacheSysPageFaults"   type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheSysNumReads"     type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheMemUsage"        type="Integer32" />
        <mibObj oid="."      instance="0"  alias="cacheCpuTime"         type="Integer32" />
        <mibObj oid="."      instance="0"  alias="cacheCpuUsage"        type="Integer32" />
        <mibObj oid="."      instance="0"  alias="cacheMaxResSize"      type="Integer32" />
        <mibObj oid="."      instance="0"  alias="cacheNumObjCount"     type="Gauge32" />
        <mibObj oid="."      instance="0"  alias="cacheCurrLRUExp"      type="TimeTicks" />
        <mibObj oid="."      instance="0"  alias="cacheCurrUnlinkReq"   type="Counter32" />
        <mibObj oid="."     instance="0"  alias="cacheCurrUnusedFD"    type="Gauge32" />
        <mibObj oid="."     instance="0"  alias="cacheCurrResFD"       type="Gauge32" />
        <mibObj oid="."    instance="0"  alias="cacheClientHttpReq"   type="Counter32" />
        <mibObj oid="."    instance="0"  alias="cacheHttpHits"        type="Counter32" />
        <mibObj oid="."    instance="0"  alias="cacheHttpErrors"      type="Counter32" />
        <mibObj oid="."    instance="0"  alias="cacheHttpInKb"        type="Counter32" />
        <mibObj oid="."    instance="0"  alias="cacheHttpOutKb"       type="Counter32" />
        <mibObj oid="."    instance="0"  alias="cacheIcpPktsSent"     type="Counter32" />
        <mibObj oid="."    instance="0"  alias="cacheIcpPktsRecv"     type="Counter32" />
        <mibObj oid="."    instance="0"  alias="cacheIcpKbSent"       type="Counter32" />
        <mibObj oid="."    instance="0"  alias="cacheIcpKbRecv"       type="Counter32" />
        <mibObj oid="."   instance="0"  alias="cacheServerRequests"  type="Integer32" />
        <mibObj oid="."   instance="0"  alias="cacheServerErrors"    type="Integer32" />
        <mibObj oid="."   instance="0"  alias="cacheServerInKb"      type="Counter32" />
        <mibObj oid="."   instance="0"  alias="cacheServerOutKb"     type="Counter32" />
        <mibObj oid="."   instance="0"  alias="cacheCurrentSwapSz"   type="Gauge32" />
        <mibObj oid="."   instance="0"  alias="cacheClients"         type="Gauge32" />
        <mibObj oid="."      instance="0"  alias="cacheEntries"         type="Gauge32" />
        <mibObj oid="."      instance="0"  alias="cacheRequests"        type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheHits"            type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cachePendingHits"     type="Gauge32" />
        <mibObj oid="."      instance="0"  alias="cacheNegativeHits"    type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheMisses"          type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheBlockGtHByName"  type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheAttRelLockEntr"  type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheFqdnEntries"     type="Gauge32" />
        <mibObj oid="."      instance="0"  alias="cacheFqdnRequests"    type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheFqdnHits"        type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheFqdnPendHits"    type="Gauge32" />
        <mibObj oid="."      instance="0"  alias="cacheFqdnNegHits"     type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheFqdnMisses"      type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheBlockGtHByAddr"  type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheDnsRequests"     type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheDnsReplies"      type="Counter32" />
        <mibObj oid="."      instance="0"  alias="cacheDnsNumberServs"  type="Counter32" />
      <group name="squidCacheMedianSvcTable" ifType="all">
        <mibObj oid="."  instance="cacheMedianSvcTable" alias="cacheHttpAllSvcTm"   type="Integer32" />
        <mibObj oid="."  instance="cacheMedianSvcTable" alias="cacheHttpMisSvcTm"   type="Integer32" />
        <mibObj oid="."  instance="cacheMedianSvcTable" alias="cacheHttpNmSvcTm"    type="Integer32" />
        <mibObj oid="."  instance="cacheMedianSvcTable" alias="cacheHttpHitSvcTm"   type="Integer32" />
        <mibObj oid="."  instance="cacheMedianSvcTable" alias="cacheIcpQrySvcTm"    type="Integer32" />
        <mibObj oid="."  instance="cacheMedianSvcTable" alias="cacheIcpRepSvcTm"    type="Integer32" />
        <mibObj oid="."  instance="cacheMedianSvcTable" alias="cacheDnsSvcTime"     type="Integer32" />
        <mibObj oid="."  instance="cacheMedianSvcTable" alias="cacheReqHitRatio"    type="Integer32" />
        <mibObj oid="." instance="cacheMedianSvcTable" alias="cacheReqByteRatio"   type="Integer32" />
      <group name="squidCacheCachePeerTable" ifType="all">
        <mibObj oid="."    instance="cachePeerTable"    alias="cachePeerName"      type="String" />
        <mibObj oid="."    instance="cachePeerTable"    alias="cachePeerAddr"      type="String" />
        <mibObj oid="."    instance="cachePeerTable"    alias="cachePeerPortHttp"  type="Integer32" />
        <mibObj oid="."    instance="cachePeerTable"    alias="cachePeerPortIcp"   type="Integer32" />
        <mibObj oid="."    instance="cachePeerTable"    alias="cachePeerType"      type="Integer32" />
        <mibObj oid="."    instance="cachePeerTable"    alias="cachePeerState"     type="Integer32" />
        <mibObj oid="."    instance="cachePeerTable"    alias="cachePeerPingsSnt"  type="Counter32" />
        <mibObj oid="."    instance="cachePeerTable"    alias="cachePeerPingsAck"  type="Counter32" />
        <mibObj oid="."    instance="cachePeerTable"    alias="cachePeerFetches"   type="Counter32" />
        <mibObj oid="."   instance="cachePeerTable"    alias="cachePeerRtt"       type="Integer32" />
        <mibObj oid="."   instance="cachePeerTable"    alias="cachePeerIgnored"   type="Counter32" />
        <mibObj oid="."   instance="cachePeerTable"    alias="cachePeerKeepAlSnt" type="Counter32" />
        <mibObj oid="."   instance="cachePeerTable"    alias="cachePeerKeepAlRcv" type="Counter32" />

      <!-- The cacheClientTable is very resource intensive.
      <group name="squidCacheCacheClientTable" ifType="all">
        <mibObj oid="."    instance="cacheClientTable"  alias="cacheClientAddr"    type="String" />
        <mibObj oid="."    instance="cacheClientTable"  alias="cacheClientHttpReq" type="Counter32" />
        <mibObj oid="."    instance="cacheClientTable"  alias="cacheClientHttpKb"  type="Counter32" />
        <mibObj oid="."    instance="cacheClientTable"  alias="cacheClientHttpHit" type="Counter32" />
        <mibObj oid="."    instance="cacheClientTable"  alias="cacheClientHttpHKb" type="Counter32" />
        <mibObj oid="."    instance="cacheClientTable"  alias="cacheClientIcpReq"  type="Counter32" />
        <mibObj oid="."    instance="cacheClientTable"  alias="cacheClientIcpKb"   type="Counter32" />
        <mibObj oid="."    instance="cacheClientTable"  alias="cacheClientIcpHit"  type="Counter32" />
        <mibObj oid="."    instance="cacheClientTable"  alias="cacheClientIcpHKb"  type="Counter32" />

Test your configuration

At this point, you should be able to save datacollection-config.xml and restart OpenNMS. Depending on whether or not you enabled the client table, or whether or not your Squid configuration has peers configured you will start to see the cachePeerTable and cacheClientTable directories created and populated. Regardless, you should see your cacheMedianSvcTable directory created and populated.

If OpenNMS does not start creating the Squid jrb files, check your logs for errors such as typos in datacollection-config.xml.

Sample Graphs

For some sample graphs just add cacheCurrentSwapSz,cacheUptime to the reports= line of the snmp-graph.properties file and add this graph definitions to the bottom of the file:

report.cacheCurrentSwapSz.name=Disk Space used by Squid
report.cacheCurrentSwapSz.command=--title="Disk Space used by Squid" \
DEF:cacheCurrentSwapSz={rrd1}:cacheCurrentSwapSz:AVERAGE \
LINE2:cacheCurrentSwapSz#0000ff:"MB" \
GPRINT:cacheCurrentSwapSz:AVERAGE:" Avg \\: %8.2lf %s" \
GPRINT:cacheCurrentSwapSz:MIN:"Min \\: %8.2lf %s" \
GPRINT:cacheCurrentSwapSz:MAX:"Max \\: %8.2lf %s\\n"

report.cacheUptime.name=Squid Cache Uptime
report.cacheUptime.command=--title="Squid Cache Uptime" \
 --vertical-label Days \
 DEF:time={rrd1}:cacheUptime:AVERAGE \
 DEF:minTime={rrd1}:cacheUptime:MIN \
 DEF:maxTime={rrd1}:cacheUptime:MAX \
 CDEF:days=time,8640000,/ \
 CDEF:minDays=minTime,8640000,/ \
 CDEF:maxDays=maxTime,8640000,/ \
 LINE2:days#0000ff:"Cache Uptime (Days)" \
 GPRINT:days:AVERAGE:"Avg  \\: %8.1lf %s" \
 GPRINT:days:MIN:"Min  \\: %8.1lf %s" \
 GPRINT:days:MAX:"Max  \\: %8.1lf %s\\n"

Extending Squid's SNMP or polling a reduced set of data

Not all useful information is exposed via the Squid 2.x MIB. For example you may be using NTLM authentication and want to keep track of the service time for your NTLM authenticators. You may also be interested in only a very few pieces of information and not everything exposed by Squid's SNMP agent.

Here's an example of using small scripts on your Squid server to poll data.

Script squidclient to pull information

First create some simple scripts. Included here are four small ones to poll the number of client connections, NTLM authentication time, median service time, and DNS service times. These scripts are expecting Squid to be listening on port 8080. If you are using the default port, remove the -p 8080 elements. Important! Note that using Bash, as opposed to Perl or another language, restricts your exit code to 0-255. So for example if your proxy has more than 255 clients accessing during a 5 minute period, these scripts will not be accurate.


# number of clients accessing squid in the last five minutes
clients=`printf %.0f $(squidclient -p 8080 mgr:cache_password | grep "client_http.requests" | awk ' { print $3 } ' | rev | cut -c 5- | rev)`
exit $clients


# NTLM average service time in milliseconds
ntlm=`printf %.0f $(squidclient -p 8080 mgr:ntlmauthenticator@cache_password | grep avg | cut -d " " -f 4 )`
exit $ntlm


# median service time (5 min) in seconds
# move to milliseconds
decimaltime=`squidclient -p 8080 mgr:5min@cache_password | grep client_http.all_median_svc_time | awk ' { print $3 } '`
svctime=`echo " $decimaltime * 1000 " | bc | cut -d "." -f 1`
exit $svctime


# median dns service time (5 min) in seconds
# move to milliseconds
decimaltime=`squidclient -p 8080 mgr:5min@cache_password | grep dns.median_svc_time | awk ' { print $3 } '`
svctime=`echo " $decimaltime * 1000 " | bc | cut -d "." -f 1`
exit $svctime

Extend snmpd to read the scripts

Edit snmpd.conf and include something like the following:

# exec: run a simple command using exec()
#   arguments:  [oid] name /path/to/executable arguments

exec   squidClientSec /bin/bash /usr/local/bin/squid_client_stat.sh
exec   squidNtlmMsec /bin/bash /usr/local/bin/squid_ntlm_stat.sh
exec   squidMedSvcTime /bin/bash /usr/local/bin/squid_mdsvc_stat.sh
exec   squidDnsMsec /bin/bash /usr/local/bin/squid_dnssvc_stat.sh

Edit datacollection-config.xml to read the new OIDs

Now add the following to datacollection-config.xml:

Ensure that your Net-SNMP systemDef will read the group you will create:


And create a group:

      <!-- Squid Proxy Custom OIDs -->
      <group name="squid-custom" ifType="ignore">
        <mibObj oid="."      instance="1"  alias="squidClientSec"       type="integer32" />
        <mibObj oid="."      instance="2"  alias="squidNtlmMsec"        type="integer32" />
        <mibObj oid="."      instance="3"  alias="squidMedSvcTime"      type="integer32" />
        <mibObj oid="."      instance="4"  alias="squidDnsMsec"         type="integer32" />

Important! Note that these mib objects are in the same order that the scripts are listed in snmpd.conf.

Once you restart OpenNMS you should start to pick up the new data.

Sample Graph

This is a sample graph for the NTLM and DNS response time scripts above:

Edit snmp-graph.properties, adding custom.squid.response to the reports= line. Then add the following graph definition at the bottom of the file:

report.custom.squid.response.name=Squid Proxy External Response Times
report.custom.squid.response.command=--title="Squid External Reponse Times" \
 --vertical-label Milliseconds \
 DEF:dns={rrd1}:squidDnsMsec:AVERAGE \
 DEF:ntlm={rrd2}:squidNtlmMsec:AVERAGE \
 AREA:dns#00ff00:"DNS Query" \
 GPRINT:dns:AVERAGE:"Avg  \\: %8.2lf %s" \
 GPRINT:dns:MIN:"Min  \\: %8.2lf %s" \
 GPRINT:dns:MAX:"Max  \\: %8.2lf %s\\n" \
 LINE2:ntlm#0000ff:"NTLM Auth" \
 GPRINT:ntlm:AVERAGE:"Avg  \\: %8.2lf %s" \
 GPRINT:ntlm:MIN:"Min  \\: %8.2lf %s" \
 GPRINT:ntlm:MAX:"Max  \\: %8.2lf %s\\n"