Dev-Jam 2015/elasticsearch

From OpenNMS
Jump to navigation Jump to search

Warning.png This opennms-elasticsearch-event-forwarder feature is deprecated.

This feature has been deprecated in favor of the opennms-es-rest implementation.

Abstract

Use Elasticsearch for indexing a copy of OpenNMS events

Folks
Umberto Nicoletti

OpenNMS Components

https://github.com/unicolet/opennms-events

Summary

Indexing OpenNMS events (and possibly alarms, outages, notifications) in Elasticsearch opens interesting possibilities like using Kibana for browsing and graphing and plotting them as annotations in Grafana.

Elasticsearch is not intended as a replacement for postgres (at least not yet).

See tracker below for project status.

Quickstart (OSGi feature)

During DevJam the original component has been reworked as an OSGi feature. The following instructions document how to use the OSGi feature, not the original database-based component.

Clone the github repo listed below in the Tracker section and build opennms. Start elasticsearch with the following (minimal) configuration:

<syntaxhighlight lang="properties"> cluster.name: opennms network.host: 127.0.0.1 discovery.zen.ping.multicast.enabled: false discovery.zen.ping.unicast.hosts: ["127.0.0.1"] </syntaxhighlight>


Note.png Note on using localhost

In this specific example we will be running Elasticsearch and OpenNMS on the same server. This is not recommended fo production setups.


Review the configuration file in ${OPENNMS_HOME}/etc/org.opennms.features.elasticsearch.eventforwarder.cfg and make changes as needed. If you are following this quickstart to the letter these is no need to change anything.

See next paragraph for configuration options and their meaning.

Configuration options
Option default Explanation
elasticsearchCluster opennms The name of the elasticsearch cluster as specified in the elasticsearch configuration file (required).
elasticsearchIp localhost the TransportClient remote host ip to use. Has the same meaning as the ip options of the camel-elasticsearch component
logEventDescription false Whether to forward the event description to Elasticsearch. Th reason it is off by default is that it is usually some standard and possibily long text which will grow the index without adding useful information
cache_max_ttl 0 The number of minutes the node information is kept in the cache. Set to 0 to disable (which is the default and is generally safe because the cache knows when to refresh itself, by intercepting nodeUpdated and similar events)
cache_max_size 10000 The number of node information entries to be kept in the cache before eviction start. Set to 0 to disable.

Log into the OSGi console (ssh admin@localhost -p 8101) and install the feature with the following command:

features:install opennms-elasticsearch-event-forwarder

You can check the routes status with the camel:* commands and/or inspect the log with log:tail for any obvious errors. The feature has a trace level logging that can be used to trace operations.

If all goes well events and alarms will be pushed in realtime into Elasticsearch. Use Kibana to view the events and graph them.

If you have never used Kibana before I recommend to start with Kibana 3 which is simpler to get started with. Kibana 4 is more powerful, but harder to get started with.

Tracker

Work branch:

https://github.com/unicolet/opennms/tree/feature-elasticsearch-sink

Trello board tracking progress and completed/todo items. Check it if you want to help and/or check progress:

https://trello.com/b/JJsvvlAv/opennms-elasticsearch