RANCID RWS

From OpenNMS
Jump to: navigation, search

OpenNMS and RANCID Integration

RANCID (The Really Awesome New Cisco config Differ) is an open source tool that lets you manage configuration of gear not only from Cisco but other vendors such as Juniper.

The integration consists of several parts. First, the default RANCID code must be modified to enable the RANCID application to send SNMP traps to OpenNMS. This is done through a new piece of code that uses the Net-SNMP snmptrap command to send the traps.

Once RANCID with the rancid-trap command is built and installed, it must be configured. This consists of editing rancid.conf to add groups, modifying the .cloginrc file to provide the passwords and access method for RANCID to use when communicating with the devices, running rancid-cvs to create the directory structure for each group, editing the routers.db file for each group to place devices within it, and finally running rancid-run to gather the configurations.

Once set, rancid-run is executed via cron to periodically check for changes.

Second, on the RANCID server the RANCID RWS (for ReSTful Web Services) code must be added. This is a cgi script written in Tcl that leverages CVS to manage the configurations. The viewVC application is installed to provide a web-based view of the CVS information gathered by RANCID. Note that RANCID and OpenNMS can co-exist on the same server, but it is not necessary. It is necessary for the RANCID RWS to be on the same server with RANCID.

[Note: it appears that both RANCID and viewVC also support Subversion, but I have no knowledge of this being successfully implemented. git is not supported]

Third, RANCID must be enabled on the OpenNMS server by editing the opennms.properties file. This will allow OpenNMS to display RANCID information from a link off a device's node page.

Finally, there is the option to install the RANCID provisioning adapter which will allow OpenNMS to modify the RANCID configuration to automatically provision devices within RANCID itself. When you make changes to a Requisition (which will map to a RANCID group) and request synchronization, the RANCID Provisioning Adapter is able to perform the add, delete and update operations of RANCID configuration files (i.e. .cloginrc and router.db).

What is RANCID

RANCID (Really Awesome New Cisco confIg Differ) is a tool for monitoring network devices (i.e. routers, switches, etc.) to track software and hardware configuration changes and to maintain a complete history of them by the means of a revision control system (i.e. CVS or Subversion) repository. It is distributed by Shrubbery Networks, Inc

As most "good-old-school" unix-based tools, RANCID's configuration is performed by editing a set of configuration files on the host system; RANCID's execution is started from the system's command line or automatically scheduled via the unix cron daemon; the information repositories generated by RANCID can be accessed by any CVS or Subversion browsing tool.

What is RANCID RWS ?

RANCID RWS (ReSTful Web Service) is an API to access both RANCID's configuration and the data it stores in its revision control system.

It was designed according to a resource-oriented client-server model following a Representational State Transfer (ReST) style of software architecture based on the Hyper Text Transfer Protocol version 1.1.

Download RANCID RWS

The latest RWS RANCID release is available at GitHub. Alternatively, you can build from source by running mvn package in the source tree checked out from the RANCID API repository by cloning git://github.com/OpenNMS/rancid-api.git. This will create release and source tarballs in the target/ directory.

Install and configure RANCID

Download RANCID from: http://www.shrubbery.net/rancid

wget ftp://ftp.shrubbery.net/pub/rancid/rancid-2.3.8.tar.gz

NB - This command will fail because Shrubbery no longer hold this file on their website. The file is available in other locations, and it's an exercise for the reader to locate this version. Extract the tarball to a temporary directory.

tar -xvf rancid-2.3.8.tar.gz

Next, in order to allow RANCID to send traps to OpenNMS, you will need to patch it. There is a rancid.patch file in the contrib directory of the RANCID RWS distribution, and it works against RANCID 2.3.6.

If you don't want to send traps, you can skip it, but it is nice to let OpenNMS know when the status of a remote device has changed in RANCID.

Note: The distributed patch doesn't work against 2.3.8. Here is one that does: Rancid-2.3.8_plus_OpenNMS.patch.txt‎

cd rancid-2.3.8
wget http://www.opennms.org/w/images/4/4a/Rancid-2.3.8_plus_OpenNMS.patch.txt

And here is another version of the patch for use with RANCID 2.3.4

 
Rancid-2.3.4_plus_OpenNMS.patch.txt‎

To install the patch (es. /tmp/rancid.patch) cd to installation directory and run:

 patch -Nbp1 -z .original < /tmp/rancid.patch
root@opennms:/home/monitor/rancid-2.3.8# patch -Nbp1 -z .original < Rancid-2.3.8_plus_OpenNMS.patch.txt
patching file bin/control_rancid.in
patching file bin/Makefile.in
patching file bin/rancid-run.in
patching file bin/rancid-trap.in
root@opennms:/home/monitor/rancid-2.3.8#

For more details on patch installation follow Rocco's README

Once the patch has been successfully applied:

When necceassry install make and a C compiler.

apt-get install make
apt-get install gcc
cd [directory where you extracted RANCID]
./configure
sudo make install

This will install RANCID into /usr/local/rancid. You can also use

./configure --prefix=/opt/rancid

If you want to change the install location.

Make a rancid user

RANCID does not need to run as root, so it is a good idea to configure a user to own the RANCID directories. For simplicity it is assumed this user will be "rancid".

useradd -d /usr/local/rancid rancid
chown -R rancid /usr/local/rancid/
chmod -R 770 /usr/local/rancid/

Once created:

sudo chown -R rancid:root /usr/local/rancid

Edit /usr/local/rancid/etc/rancid.conf

RANCID allows you to group devices. RANCID groups can correspond to to provisioning groups within OpenNMS, but you'll need to configure them:

LIST_OF_GROUPS="routers requisition1 requisition2"

While you are in this file, you need to set the trap destination, especially since you went to all the trouble to patch it.

If your OpenNMS server is at 10.1.1.1:

 OPENNMS_NOTIFY_CMD="/opt/rancid/bin/rancid-trap -r 10.1.1.1"; export OPENNMS_NOTIFY_CMD

Ubuntu:

 OPENNMS_NOTIFY_CMD="/usr/local/rancid/bin/rancid-trap -r 10.1.1.1"; export OPENNMS_NOTIFY_CMD

Modify /home/rancid/.cloginrc

The .cloginrc file controls how RANCID accesses remote devices, and needs to be placed in the home directory of the RANCID user. There is a sample file in the rancid/share directory but an entry could resemble this:

nano ~/.cloginrc
# Routers
add user     myrouter.example.com        root
add password myrouter.example.com        {myaccesspass}   {myenablepass}
add method   myrouter.example.com        ssh

See the sample file for more options.

Linux thinks that this file may be read by everyone, we definitely don’t want this and rancid will even give an error on this, so we need to make sure this file is only readable by the rancid user and the group root.

chmod 770 ~/.cloginrc

Testing a login for a single device

./usr/local/rancid/bin/clogin switch.domain.net

You should see a automatic login. Log out manually.

run rancid-cvs

Install cvs

apt-get install cvs

Now let’s run Rancid CVS to create the directories and configuration files for the locations. If you add a group you will need to re-run this command.

su – rancid
./usr/local/rancid/bin/rancid-cvs

If you now look in the /usr/local/rancid/var/ directory, you should see the directories which Rancid created by following the rancid.conf file. Additional there are two directories created, “CVS” and “logs”.

modify the router.db files

NOTE: Please note that in the version that this document is written about, there's a bug in the RANCID_RWS script; if a node has an underscore in its name, the RWS script doesn't process the name correctly, and prevents the RANCID information from being displayed in the OpenNMS GUI. You can use DNS aliases to get around this limitation.

From the RANCID README file:

For each "group", modify the router.db file in the group directory.
   The file is of the form "router:mfg:state" where "router" is
   the name (we use FQDN) of the router, mfg is the manufacturer
   from the set of (cat5|cisco|juniper) (see router.db.5 for a complete
   list and description), and "state" is either up or down.  Each router
   listed as "up" will have the configuration grabbed.  Note: manufacturer
   cat5 is intended only for cisco catalyst switches running catalyst (not
   IOS) code.

   e.g.: <localstatedir>/<group>/router.db:
        cisco-router.domain.com:cisco:up
        adc-mux.domain.com:ezt3:up
        foundry-switch-router.domain.com:foundry:up
        juniper-router.domain.com:juniper:up
        redback-dsl-router.domain.com:redback:down
        extreme-switch.domain.com:extreme:down

Run RANCID for the first time

At this point you can run, as the rancid user, bin/rancid-run.

su - rancid
~/bin/rancid-run

Check the logs directory for errors, but at this time you should see RANCID collecting data.

If this is successful, set up a cron job to run it at least hourly.

Note: I keep seeing

read_config_store open failure on /var/lib/net-snmp/snmpapp.conf

errors in the logs. This is from snmptrap. This is because that file doesn't exist and I believe they can be safely ignored.

#may fix snmperror in logs (presuming snmpd and snmputils installed and configured as well)
sudo su rancid  #switch to rancid user
cat "persistentDir /home/rancid/.snmp_persist" > ~/.snmp/snmp.conf
exit #back to normal user

You now have RANCID configured.

Run RANCID scheduled

The rancid user has his own cron.

crontab –e

The second line deletes log file.

@daily /usr/local/rancid/bin/rancid-run
@daily /usr/bin/find /usr/local/rancid/var/logs -type f -mtime +7 -exec rm {} \;

Set Up the RWS Server

The RWS Server implements a CGI Application as the middleware between OpenNMS and RANCID.

In my setup I implemented RWS as a Virtual Server on Apache. This was implemeted on RedHat 4 Enterprise.

Download

wget https://github.com/OpenNMS/rancid-api/archive/rancid-api-1.0.4.zip

Make the CGI Files Available to the Web Server

First, put the CGI file(s) somewhere your web server can get to them:

cd /var/www
mkdir rws-server
cp -R /path/to/rws/cgi-bin rws-server/rws-cgi
chown rancid:root rws-server
chmod -R 755 rws-server
mkdir rws-server/html
chown rancid:rancid rws-server/*

Edit the tclsh path in rws-cgi.tcl

Edit the first line in rws-cgi.tcl to point to your tclsh. Mine was

#!/usr/bin/tclsh

Configure httpd.conf

Ubuntu: /etc/apache2/apache2.conf

Then, configure your httpd.conf so that Apache runs as the RANCID user. This step is necessary because Apache's mod_suexec "cleans" the environment of scripts that run under it, rendering that strategy incompatible with the use of mod_setenv's SetEnv directive to configure the RWS services.

 the workaround to this and continue to use mod_suexec in apache 2 is to edit the cgi script and hardcode the  env variables that would be patched in as below 

Note, you'll need to create the rws-storage directory. I put it in the rancid user home directory.

NameVirtualHost *:80

User rancid
Group rancid

<VirtualHost *:80>
 DocumentRoot /var/www/rws-server/html
 ServerName rws.mycompany.org
 ErrorLog logs/rws-error_log
 TransferLog logs/rws-access_log
 ScriptAlias /rws "/var/www/rws-server/rws-cgi/rws-cgi.tcl"
#uncomment the next line if you're using ViewVC.
#ScriptAlias /viewvc "/var/www/viewvc/bin/cgi/viewvc.cgi"
 AddHandler cgi-script .tcl
 SetEnv RWS_LOGFILE  /var/log/httpd/rws-cgi.log
 SetEnv RWS_LOGLEVEL debug
 SetEnv RWS_STORAGE_ROOT  /home/rancid/rws-storage

 <Directory /var/www/rws-server/rws-cgi>
    AllowOverride None
    Order allow,deny
    Allow from all
 </Directory>

 <Directory "/var/www/rws-server/html">
    Options Indexes FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
 </Directory>

</VirtualHost>

Configure rancid.rws.rc RWS Configuration File

Finally, configure the rws-server configuration file in /var/www/rws-server/rws-cgi/rancid.rws.rc.

The following are the settings we used in rancid.rws.rc:

set pathRancidHome                      "/home/rancid"
set fileRancidConf                      "/usr/local/rancid/etc/rancid.conf"
set pathBackup                          "/home/rancid/tmp"
set pathTemp                            "/tmp"
set commandCVS                          "/usr/bin/cvs"
set urlViewVC                           "/viewvc"

Changes to the router.db with Rancid 3.x

router.db,rancid.types.*: change field separator to ';' (semi-colon) to 
allow for IPv6 addresses in router.db and avoid conflict with :s in 
device commands and perl module names

This means that to work with Rancid 3.x you need to use RWS 1.0.4 and above and add an additional environment variable to your apache config

SetEnv RWS_RANCID_ROUTER_DB_USE_SEMICOLON true

Set up ViewVC

  • Download latest stable viewVC from the site. Direct link to the downloads. Use the tar.gz instead of the .zip.
wget http://viewvc.tigris.org/files/documents/3330/49392/viewvc-1.1.23.tar.gz
  • Install it using the provided install script in your /var/www path.
 tar xzvf viewvc-1.1.22.tar.gz
 cd viewvc-1.1.22
  ./viewvc-install
  • Set rights voor rancid
 chown -R rancid:rancid /var/www/viewvc
  • Edit the /var/www/viewvc/viewvc.conf viewvc config file to set the CVSROOT path.
cvs_roots = cvsroot: /usr/local/rancid/var/CVS

Add it to your Apache configuration by uncommenting the line, or adding the following ScriptAlias under the /rws alias :

ScriptAlias /viewvc "/var/www/viewvc/bin/cgi/viewvc.cgi"

You should be able to browse your CVS at :

http://rws.mycompany.org/viewvc

Don't forget to install rcs tools if you've not already, as viewvc requires them to do some of the lifting by default:

apt-get install rcs

Configure OpenNMS to communicate with RWS

cd $OPENNMS_HOME/etc

Edit opennms.properties and change to following line from:

#opennms.rancidIntegrationEnabled = false

...to:

opennms.rancidIntegrationEnabled = true

If you want that only the Rancid Provisiong Adapter will change configuration file of rancid from opennms platform change set the following property:

opennms.rancidIntegrationUseOnlyRancidAdapter = true 

We used a virtual host instead of localhost, so we edited rws-configuration.xml and change the following line:

<base-url server_url="http://localhost"/>

...to:

<base-url server_url="http://rws.mycompany.org"/>

Once these steps are done, the node details page for every node in the OpenNMS web UI will contain a View Node Rancid Inventory Info link in the General box. This link takes you to a page where you can see a summary of the node's device configurations and (if maintained in RANCID) its stored software images. From the configurations summary you can also drill into a page that embeds the ViewVC interface, which lets you browse the historical device configurations for the node.

Note that the success of this integration depends on the OpenNMS node label of a given node being identical (including upper / lower case) to the name by which RANCID knows that device. If these two names are even slightly mismatched, no configurations or software images will be visible from the OpenNMS web UI.

Rancid Provisioning Adapter

The Rancid Provisioning Adapter is now an optional component to install by default opennms installation tool. The design of this adapter is to have a full control over Rancid so that devices into rancid are mostly provided via the Rancid Provisioning Adapter.

To provide a node to Rancid the Adapter must use the "Provisioning Requisition" Name so a correspondence between rancid Group and provision requisition must be established.

If you want to provide nodes into a Provision Requisition called "Home" then the corresponding Rancid Group "Home" must exist, otherwise the adapters fails to synchronize opennms and rancid.

If a node "antonio.opennms.it" is provided to the Provisioning Requisition "Home" then the device "antonio.opennms.it" will be added to the router.db file of the Rancid Group "Home".

You must provide also the following information to insert the node into rancid using the adapter: the ip address of the node, the username to access the node, the password, the method (Telnet, SSH, RSH) to connect to the device, the enable password, and if it uses an enable password.

Each of the above object must be added to the requisition.

To add the ip address just add an interface.

To add the username add the asset field username to the requisition.

To add the password add the asset field password to the requisition.

To add the method add the asset field connection to the requisition.

To add the enable password add the asset field enable to requisition.

To specify is there exists an enable password set the asset field autoenable to 1.

The above assets field are all required minus autoenable.

Here is a snapshot of the rancid-configuration.xml (the configuration file for rancid provisioning adapter:

<?xml version="1.0"?>
<rancid-configuration delay="3600" retries="1">
  
<!-- Configuration example that uses policy-manage -->
<policies>
<policy-manage name="example1">
  <package name="example1">
    <filter>IPADDR != '0.0.0.0'</filter>
    <include-range begin="1.1.1.1" end="254.254.254.254" />
  </package>
  <schedule name="global" type="weekly">
        <time day="sunday" begins="12:30:00" ends="12:45:00"/>
        <time day="sunday" begins="13:30:00" ends="14:45:00"/>
        <time day="monday" begins="13:30:00" ends="14:45:00"/>
        <time day="tuesday" begins="13:00:00" ends="14:45:00"/>
  </schedule>
 </policy-manage>
</policies>  
  
  <mapping sysoid-mask=".1.3.6.1.4.1.9.1" type="cisco"/>
  <mapping sysoid-mask=".1.3.6.1.4.1.9.5" type="cat5"/>
  <mapping sysoid-mask=".1.3.6.1.4.1.1916.2" type="extreme"/>
  <mapping sysoid-mask=".1.3.6.1.4.1.2636.1" type="juniper"/>
  <mapping sysoid-mask=".1.3.6.1.4.1.4874.1" type="erx"/>
  <mapping sysoid-mask=".1.3.6.1.4.1.11.2.3.7.11" type="hp"/>
  
  
</rancid-configuration>

The mapping element will map the devices to the proper rancid category using the node sysoid (default device category is cisco).

But what are the policy-manage for?

This is the way in which we try to control rancid. In router.db a router can be in state up or down. Up means download configuration, Down means do not download.

When it is received an event from rancid that a configuration has been successfully downloaded the Rancid Provisioning Adapter will set the status to down.

So in line of principle no other configuration will be downloaded later until a "configChange" event is received from the node. In this case the Rancid Provisioning Adapter will set the node to up in router.db.

To do this is introduced a delay, so that the node is set to up in router.db only after a delay time is passed. The policies enforce this to just doing such operation (setting the node to up) only on the specified timetable.


To avoid a bug you need to set up a minimal .cloginrc file:

add user cisco.example.com dummy_user
add method cisco.example.com telnet
add password cisco.example.com password enablepassword


It is possible to reload the rancid-configuration.xml to the adapter just run:

 /opt/opennms/bin/send-event.pl uei.opennms.org/internal/reloadDaemonConfig --parm 'daemonName Provisiond.RancidProvisioningAdapter'

Getting Help

If you continue to have issues, Rocco Rionero wrote a fantastic README. Take a look through that for details. If you still have problems, please try the discussion lists.

Caution

Please backup your RANCID configuration. The RWS server will update the .cloginrc file as changes are made inside OpenNMS.