Maps Adapter Performance Improvements

From OpenNMS
Jump to navigation Jump to search

The object of this story is to discuss and design the changes needed to increase the performance of the maps adapter.

Performance Issues

There are two kinds of performance issues that are happening in the maps adapter, the inner issues and the outer issues. The inner issues are problems with building the maps just once while the outer issues are related to the number of times the maps get rebuilt.

Inner Issues

The inner issues are problems related to how the maps are built. These are called inner issues because inefficiencies here get magnified because of the number of times they get called just like inefficiencies inside of a loop make the entire program take longer. Shaving just 1 ms off of a 10 ms part of the work that gets called 1,000,000 times will save nearly 20 minutes of total processing time.

To model the performance of the 'inner' loop of the maps adapter we can use the initialization time. The work that needs to be done at startup for the maps adapter is roughly the same as the work that needs to be done when nodes have been provisioned. That is, examine the nodes in the database and determine if any of them should be on the maps.

There is at least one customer whose initialization time is over 20 minutes. For an inner process, this is very time consuming.

Outer Issues

Outer issues are problems with recreating the maps too often. If we recreate the map once for every node in the database, then for large database we will create the map so often that improvements in the inner loop evaporate. To use the above example of 10 ms inner loop called 1,000,000. If we reduce the number of times thru the loop by 10% we shave off the same 20 minutes of total processing time.

The current issues with the maps adapter is that the number of times that maps get recreated is excessive. It seems that the queries necessary to build the maps from scratch are nearly identical to the queries necessary to verify a single node has moved. Given that this is true a great deal of time can be saved by rebuilding the maps to include the changes of many nodes at once. This reduces that number of map rebuilds by a factor of the average number of nodes in each build.

We have a customer this is currently experienced a 12 Hour processing time when importing a requisition containing 6500 nodes into a map with a configuration with ~ 2000 packages. The logs indicate there that their maps are rebuilt over and over again for each node that has been imported into a requisition. This is clearly wasteful as once the import and scan has been finished, there are no further changes being made. A rebuilt from scratch of the maps at this point would create all of the maps correctly.