Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Best monitoring system?

news.ycombinator.com

Post reply on HN
Print view View original

121–128 of 128 posts

Re: Ask HN: Best monitoring system?

#121

Post by smoodles »

calvinx408 wrote: post #119

I used to use nagios and migrated to sensu for system checks. I was using graphite/seyren for time series and alerting, but doing a YoY or week over week was very slow especially if it's a lot of metrics. You should look at http://wavefront.com You can do some nice math functions for your alerts.

+1 to this as long as you are ok with an external vendor.

A couple of caveats. If you are coming from Nagios, this is a different worldview on monitoring. Like many other solutions commented here this is all based around metrics and their associated time series, and then you need to alert on those metrics. You ask the system questions with a time series query language.

Wavefront doesn't yet have a great solution for poll-based monitoring (i.e. hitting host Xs /healthcheck endpoint) so I still use terrible 'ol Nagios for that in my environment. However the rest of my work is all done in Wavefront - I'd say easily the high 90% of all my material alerts are done in wavefront with a small subset of work done in Nagios.

The killer feature here is the query language. I don't think there is anything else on the market that has its level of sophistication. I've had ex-Googlers on my team who "grew up" with Borgmon, which is in some sense the Ur-time series monitoring system and they loved it.

All this said, there are a lot of options about there. I have a strong bias against supporting my own complicated monitoring infrastructure. I want to focus on my own product. If you don't share that opinion or are on a super duper tight cash budget (but you do have time) than disregard the above ;)

Re: Ask HN: Best monitoring system?

#122

Post by msims »

So we just recently switched over to Wavefront from an aging Zabbix monitoring. We had tested and reviewed a few time series based monitoring systems and felt Wavefront was what we needed for Enterprise level monitoring.

Some of the key items we liked were:

* Able to consume millions of metrics per second. This is pretty huge. While we're not even close to that much (11k/s at the moment), we expect that number to triple or quadruple in the next year.

* Fast. Wavefront renders graphs quickly. The ability to manipulate the data in real time has been impressive.

* Feature requests. Wavefront has been receptive to ideas from their customer base. They even have a voting system in their community page if other customers like a certain request.

* Support has been great. Questions on issues or general technical guidance has been handled quickly, within the hour.

* Docker ready. Already using Wavefront with our emerging docker infrastructure.

* Engineers are self sufficient. Before, Tech Ops had to do all the monitoring for new services. With technologies such as docker, our engineers are capable of setting up monitoring within the application to directly send to Wavefront. This offloads quite a bit of work from Tech Ops.

No, I'm not affiliated with Wavefront. We just use their monitoring service.

Re: Ask HN: Best monitoring system?

#123

Post by msims »

monitorman wrote: post #85
otterley wrote: post #83

Earlier quoted context omitted.

We throw hundreds of thousands of metrics at Datadog per minute from thousands of hosts; it hasn't broken a sweat yet. Are you affiliated with Wavefront?

I'm a happy customer of Wavefront. I completely believe that Datadog can handle hundreds of thousands per minute -- especially if most of them are pre-canned, non-custom metrics grabbed by their agent. Hundreds of thousands of metrics/minute is a few thousand a second only. Wavefront does millions of custom metrics per second, which can be sent with different dimensions and tags. That's much harder.

Yeah I second this. Hundreds of thousands per minute is a very small number. Wavefront can go over a million per second.

Re: Ask HN: Best monitoring system?

#124

Post by bbrazil »

user5994461 wrote: post #116

disclaimer: I evaluated most of these tools and wrote a blog post here. https://thehftguy.wordpress.com/2016/04/18/monitoring-in-the... It's a bit old and i'll update it later, but here is the short resume with all the latest tool: ### Free (as in open-source) shitty options: icinga, nagios, riemann They suck so much they're not even worthy of having their names written. ### The other open-source option is prometheus…

> they skipped all the hard stuff (single node only, no HA

We skipped the hard stuff on purpose, as hard stuff is extremely tricky to get right and liable to fail right when it's most needed. See http://www.robustperception.io/monitoring-without-consensus/

Per the above, there is HA.

Re: Ask HN: Best monitoring system?

#125

Post by bbrazil »

smoodles wrote: post #121
calvinx408 wrote: post #119

I used to use nagios and migrated to sensu for system checks. I was using graphite/seyren for time series and alerting, but doing a YoY or week over week was very slow especially if it's a lot of metrics. You should look at http://wavefront.com You can do some nice math functions for your alerts.

+1 to this as long as you are ok with an external vendor. A couple of caveats. If you are coming from Nagios, this is a different worldview on monitoring. Like many other solutions commented here this is all based around metrics and their associated time series, and then you need to alert on those metrics. You ask the system questions with a time series query language. Wavefront doesn't yet have a great solution for…

> The killer feature here is the query language. I don't think there is anything else on the market that has its level of sophistication. I've had ex-Googlers on my team who "grew up" with Borgmon, which is in some sense the Ur-time series monitoring system and they loved it.

Prometheus is inspired by Borgmon, and has a query language that is unmatched by almost everything else I'm aware of.

Are there public docs on the semantics and features of the WaveFront language so I can compare?

Re: Ask HN: Best monitoring system?

#126

Post by otterley »

msims wrote: post #123
monitorman wrote: post #85

Earlier quoted context omitted.

I'm a happy customer of Wavefront. I completely believe that Datadog can handle hundreds of thousands per minute -- especially if most of them are pre-canned, non-custom metrics grabbed by their agent. Hundreds of thousands of metrics/minute is a few thousand a second only. Wavefront does millions of custom metrics per second, which can be sent with different dimensions and tags. That's much harder.

Yeah I second this. Hundreds of thousands per minute is a very small number. Wavefront can go over a million per second.

Per tenant, or globally? And does it have an SLA around this?

(I've learned not to trust numbers that seem too good to be true unless they're contractually obligated.)

Re: Ask HN: Best monitoring system?

#127

Post by bbrazil »

otterley wrote: post #126
msims wrote: post #123

Earlier quoted context omitted.

Yeah I second this. Hundreds of thousands per minute is a very small number. Wavefront can go over a million per second.

Per tenant, or globally? And does it have an SLA around this? (I've learned not to trust numbers that seem too good to be true unless they're contractually obligated.)

Prometheus can do 800k/s on a single machine. Handling a million per second sounds perfectly plausible to me if you design it properly. The question is more how much it's going to cost you.

Wavefront doesn't publish pricing, but if we take Librato's pricing as a general indication you're talking several million dollars a month.

Re: Ask HN: Best monitoring system?

#128

Post by librato »

Hey there, Librato here (https://www.librato.com/)

Welp, nobody can blame you for wanting to get away from Nagios. It’s certainly a tool from a different, simpler era and hasn’t aged well in our opinion.

As a push-based metrics solution, Librato is probably a lot different than what you're used to. But don’t worry: we're super easy to get up and running with, and obviously you no longer need to worry about maintaining or scaling infrastructure. Also, unlike with some other solutions, you can use us with your existing toolchain (it’s easy to plug us into your existing Nagios infrastructure to try us - the trial is free & full-featured).

We’re a hosted metrics platform, meaning you can send metrics of any type and amount you want. We’re functionally similar to Graphite+Grafana, except we do all the work of scaling and management for you so you can focus on the metrics themselves. We provide alerting and other useful bits out of the box (things that are not trivial to setup yourself, e.g., bolting together collectd+Graphite+Grafana+statsd+flapjack+kitchen sink and hoping it scales and doesn’t fall over). We’ve got an agent that comes with a bunch of turn-key integrations too, to make it super easy for you to monitor what you care about.

As to pricing, we're the only hosted monitoring system that will just charge you for what you actually USE. You pay pennies per metric metered by the hour, instead of a per-node model, which gets crazy expensive and inefficient for modern ephemeral infrastructure. For example, if all you're doing is integrating us with AWS CloudWatch to monitor some EC2 instances and an RDS instance, we can do that for effectively a $1-$2 an instance. We also have an agent you can install on your servers if you want more detailed metrics, which adds $5-10 per instance depending on how many metrics you enable. Our customer success team (email support@librato.com, or the Help chat window if you already have a Librato account) will be more than happy to walk you through any permutation of our pricing and the details of the model to help you better understand it.

As mentioned, you can try us out for free--no credit card required: https://www.librato.com/

Post reply on HN