The Art of Monitoring

First off, I'd like to say that I think this is great and we need more books in this space.

I hope this can be seen as constructive criticism but I do have a few comments on the layout and content.

1. Free chapter - perhaps chapter one or two may have been a better choice. I'd like to know the philosophy behind monitoring expressed in the book before diving into the details.

2. capacity planning - A chapter on this would have been great. Most teams I've worked with have struggled with sizing, planning resources required and archiving strategies with their monitoring solutions.

3. Monitoring strategies for different levels in the stack - where do I start, what should my short term goals be and so on.

4. naming of some of the chapters are too focused on the technology- for example the chapter on logstash could have been renamed to something to do with application logging or log scrapping.

5. visualisation and communication of results - there could have been a chapter on dashboard and reporting. This is a common issue with teams trying to understand how to do this.

This was written in a bit of a hurry but I hope my points came through.

Thanks for your feedback.

1. I decided not to do this because my experience is that people like to do something practical first. I've had a huge response to that chapter - lots of folks have gotten into Riemann that had previously been stuck. That alone is a solid +++ for me.

2. Each chapter contains some discussion of capacity planning for specific tools, where relevant.

3. The capstone chapters (11-13) discusses this, as do the chapters covering logging and application instrumentation.

4. Thanks - I'll consider that.

5. I discuss in various chapters visualization but I've found that most folks have very different needs and desires. So I focussed on discussing what to show in small segments as well as some visual design discussion rather than a specific chapter on dashboarding/reporting. Hard choice but a 750 page book needs to stop somewhere. :)

Thanks for taking the time to comment - it's awesome when folks share their thoughts!

The monitoring considered in the OP is for server farms and networks where the main challenges are rates of false alarms and rates of missed detections.

The challenge is to find means of monitoring that will permit selecting the rate of false alarms are willing to tolerate and, then, for that rate, get the lowest rate possible for missed detections.

Thus, would like to use the Neyman-Pearson result. Usually, however, for this context, do not have enough data for that. E.g., typically are quite short on data on the anomalies are trying to detect, and more short as the systems become more reliable.

From the above, we see that necessarily and inescapably such monitoring is some continually applied statistical hypothesis tests.

Apparently in practice, false alarm rate is not known and not reasonable to select or even to adjust.

Then we see that we need tests are both multi-dimensional and distribution-free.

A special case of high interest is zero-day problems, that is, detecting problems never seen before. So, this is behavioral monitoring -- any behavior sufficiently unusual is regarded as an anomaly, that is, evidence of something wrong.

From all I can see, so far the monitoring community has yet to take these points to heart.

The OP's remarks on thresholds are on target: Thresholds have been the old, lame, weak workhorse of monitoring far too long.

If anyone is actually seriously interested in this subject, let me know. Some years ago I concluded that no one was interested!

I would argue that its not and that it's a mess, starting with that URL. This is yet another fork of an outdated monitoring system(how many Nagios forks are there?) The architecture diagram made my head hurt, how many moving parts is that? I counted 8 separate components. There are much more modern monitoring systems these days such as Prometheus, Bosun and Riemann.

Thanks mate - very kind!