Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Best monitoring system?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 128 posts

Re: Ask HN: Best monitoring system?

#51

Post by falcolas »

postwait wrote: post #47
falcolas wrote: post #5

Just my opinion, but I won't use Prometheus, because of the active polling model. It won't scale without a number of workarounds. My preferred method is Icinga2 (a Nagios clone with better configuration and clustering built-in) with reports coming in via passive NSCA. Toss in Graphite (or I'm warming up to Grafana on Influx) with some ability to write alerts against those reported metrics, and you're as close to idea…

A more thorough investigation of pull vs. push. https://www.circonus.com/pully-mcpushface/

Not a terribly thorough investigation; simply a series of unsubstantiated assertions.

Pull works fine at 2 million machines" is a great statement to make, but it would be much stronger with more details. How many machines are doing the pulling? How often? Are they using subprocesses or threads or greenthreads? How do they handle timeouts? How many metrics per machine? How many pulls per metric grab?

Re: Ask HN: Best monitoring system?

#52

Post by joeldg »

I entirely use Graphite+Grafana as a replacement for ELK stack, Nagios, Zabbix, Cacti or any of the paid options.

It's dead simple, lightweight (UDP) and the setup is cake.

I'm happy as a clam and will never look back.

EDIT: Promethius.io mentioned here is supported by grafana

Re: Ask HN: Best monitoring system?

#53

Post by illicium »

acd wrote: post #37

Prometheus.io which is a modern fresh monitoring system that I would checkout if replacing a legacy system. Also take a look at Riemann which is system monitoring written in Clojure. Riemann should be good for monitoring latency of the system. If it helps here is Slidedeck from Spotify how they do their monitoring https://www.netways.de/fileadmin/images/Events_Trainings/Eve...

Riemann is a generic event processor. You can use it to generate alerts or aggregate metrics, but you still need something like collectd or telegraf to collect system/app stats from each machine and send it to Riemann.

Re: Ask HN: Best monitoring system?

#54

Post by tex0 »

acd wrote: post #37

Prometheus.io which is a modern fresh monitoring system that I would checkout if replacing a legacy system. Also take a look at Riemann which is system monitoring written in Clojure. Riemann should be good for monitoring latency of the system. If it helps here is Slidedeck from Spotify how they do their monitoring https://www.netways.de/fileadmin/images/Events_Trainings/Eve...

I can only second Prometheus. It's modeled after Google Borgmon (and possibly Monarch) and excels in any flexible workload. Really great piece of engineering.

Re: Ask HN: Best monitoring system?

#58

Post by solipsism »

acd wrote: post #37

Prometheus.io which is a modern fresh monitoring system that I would checkout if replacing a legacy system. Also take a look at Riemann which is system monitoring written in Clojure. Riemann should be good for monitoring latency of the system. If it helps here is Slidedeck from Spotify how they do their monitoring https://www.netways.de/fileadmin/images/Events_Trainings/Eve...

It should be noted that Riemann is different from Nagios and Prometheus in that it's push-based. The other two are primarily pull-based.

Re: Ask HN: Best monitoring system?

#59

Post by mspaulding06 »

acd wrote: post #37

Prometheus.io which is a modern fresh monitoring system that I would checkout if replacing a legacy system. Also take a look at Riemann which is system monitoring written in Clojure. Riemann should be good for monitoring latency of the system. If it helps here is Slidedeck from Spotify how they do their monitoring https://www.netways.de/fileadmin/images/Events_Trainings/Eve...

I like Riemann and actually read The Art of Monitoring (https://artofmonitoring.com/) which was a great book. There are two big downsides for me on this one. First, you MUST build your monitoring solution from scratch and you MUST learn Clojure (which can be hard to get a whole team to agree to). Second, there's no alerting dashboard, which makes it difficult to see the overall state of the clusters you're dealing with. The only way you know there's a problem if you get an email. Maybe there's a better way to handle that but I wasn't able to find one.

Re: Ask HN: Best monitoring system?

#60

Post by falcolas »

bogomipz wrote: post #41
falcolas wrote: post #5

Just my opinion, but I won't use Prometheus, because of the active polling model. It won't scale without a number of workarounds. My preferred method is Icinga2 (a Nagios clone with better configuration and clustering built-in) with reports coming in via passive NSCA. Toss in Graphite (or I'm warming up to Grafana on Influx) with some ability to write alerts against those reported metrics, and you're as close to idea…

You are wrong about that. There is nothing inherent in a polling model that limits its ability to scale. You simply shard based on service or pod, or row(whatever your unit of scale is) and then you can always use consistent hashing/federation. Prometheus is a single binary. How many moving parts are involved in a Icinga 2 cluster?

Two. Icinga2 and nsca-ng. You can bring it down to one if you use Icinga2 as the test runner on the individual machines.

If you need to shard, no other binaries are involved, just a couple of changed configuration files and some SSL certs. Passive reports can go to any node; a simple load balancer will work fine without any form of hashing or federation required.

> There is nothing inherent in a polling model that limits its ability to scale

Aside from the additional processing requirements, such as SSH, NRPE, subprocesses, etc, you are also limited in that a polling process must be told in advance of new systems or services, whereas it's fairly easy to just have a new service or system start reporting and be immediately monitored.

Post reply on HN