Simple anomaly detection

Also, if munin could detect high rates of change in graphs - or new record high/low levels in a graph (on a yearly basis) then we would have pretty good "interesting graphs" detection - which is a nice thing to have if we want to make a munin dashboard.

Just yesterday I had an outage that could clearly be seen in the graph but avoided all other detection mechanisms we had. The anomaly detection should have informed me of the problem within minutes. Is there any chance that this feature will be integrated into munin 1.4 or at least 1.6?

Did not hear of this until today, and I had trouble finding mention of it anywhere else, so hope you don't mind if I do some explaining.

rrdcreate allows definition of additional RRA (round-robin accumulate) formulas that, instead of summarizing data, provides a smoothed verison of the data acquired, and attempts to predict deviations from the smoothed curve using Holt-Winters forecasting. You must define a cyclic time period (the "season") for the data, so that predicted deviations correspond to appropriate times. IE.: website hits tend to ebb and flow as per time of day, so the "season" would be 24hours, and the middle of the night low measurements would correspond to the middle of the previous night, and the night before, and so on. There are other predictions to be made, such as how many samples and their weights to use for the smoothed data, and some initial settings for the predicted deviations.

Settings for this could be made in a plugin's graph_args section, like anything else you'd pass to rrdcreate. However, tuning these settings would require calling rrdcreate again, and thus losing the historical data. I get the impression that finding the proper deviation settings is a trial-and-error process; if you rely solely on rrdtool to make predictions, you can't just up the warn/crit thresholds, you'd have to start over.

More information can be found at http://oss.oetiker.ch/rrdtool/doc/rrdcreate.en.html and http://en.wikipedia.org/wiki/Exponential_smoothing

Want munin-graph to be refactored before we attempt this.