Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

serhack.me

Post reply on HN
Print view View original

141–150 of 298 posts

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#141

Post by BurritoAlPastor »

hnarn wrote: post #104

Some people are probably going to throw some shade on me for saying this since it's so out of fashion but in my mind, when it comes to some types of basic monitoring (SNMP monitoring of switches/linux servers, disk space usage, backups running and handling them when they don't) then Nagios does get the job done. It's definitely olives and not candy[1] but it's stable, modular, relatively easy to configure (when you g…

Nagios is the Jenkins of monitoring. It's popular because you can get it running in an afternoon, and it's easy to configure by hand.

It then rots within your infrastructure, because it resists being configured any way _except_ by hand. I've built two systems for configuration-management of Nagios (at different companies), and it's an unpleasant problem to solve.

Prometheus's metric format and query syntax are cool, but the real star of the design is simply this: you don't have to restart it, or even change files on your Prometheus server, when you add or remove servers from your environment.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#142

Post by doublesCs »

aeyes wrote: post #86
avthar wrote: post #62

Echoing the sentiment expressed by others here, for a scalable time-series database that continues to invest in its community and plays well with others, please check out TimescaleDB. We (I work at TimescaleDB) recently announced that multi-node TimescaleDB will be available for free, specifically as a way to keep investing in our community: https://blog.timescale.com/blog/multi-node-petabyte-scale-ti... Today Timesc…

I don't consider TimescaleDB to be a serious contender as long as I need a 2000 line script to install functions and views to have something essential for time-series data like dimensions: https://github.com/timescale/timescale-prometheus/blob/maste... https://github.com/timescale/timescale-prometheus/blob/maste...

Completely garbage comment.

For roughly 100% of the software you install you have no idea what goes on into the installation. The only reason you're making that criticism about TimescaleDB is that they making it really easy to have a look how it works behind the scenes, and you like posing as a purist.

... unless you tell me that you don't use computers because of much code goes into the installation of the OS, or browser, in which case I applaud your self-consistency but you'd be lying because you're using a computer. Not to mention postgres itself. So you refuse to use Timescale because of the installation, but not PG? Get outta here you poser.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#143

Post by user5994461 »

dijit wrote: post #124
rattray wrote: post #91

Earlier quoted context omitted.

Can you say something vague about your deployment scale?

2,500~ compute instances. Looking at the pricing page the cost $15/instance so $37,500 Middle-of-the-road developer salary is like $35k in most of Europe, outside of the capitals. Although it does say: "Volume discounts available (500+ hosts/mo). Contact us." at the bottom, so I guess 500 is a lot.

2500 instances could be millions per months in AWS costs. The smallest instances with some disks and bandwidth fees can push 100k a month.

Spending a fraction of that to monitor that sort of infrastructure is absolutely justified. I can tell you from experience that datadog gives discount even for 100+ hosts, I don't know what they can do for 2500, but if it were me I wouldn't accept anything less than 50% off.

Honestly you need to forgot about your salary, it's irrelevant when it comes to running a company. Imagine a driver in a shipping company deciding to deliver on a scooter rather than a truck because the truck is worth more than his yearly salary.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#144

Post by Twirrim »

hnarn wrote: post #129
brightball wrote: post #118

Earlier quoted context omitted.

Ever looked at Zabbix?

I haven't, unfortunately, but it looks promising from just looking into it briefly. Open source monitoring is always an area that needs more competition.

Zabbix has been around for quite a long time. Easily 15 years now. I haven't looked at it since around 2013, but at the time it was placing quite some pressure on a mysql db backend. It looks like they've expanded out to support more than MySQL as the back-end these days.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#146

Post by floren »

Papric0re wrote: post #134
floren wrote: post #84

Earlier quoted context omitted.

I think Gravwell ( https://gravwell.io ) might be what you're looking for--but I work for Gravwell so I may be biased! If I can be forgiven a short sales pitch, we've built a format-agnostic storage & querying system that easily handles raw packets, Netflow records (v5, v9, and IPFIX), collectd data, Windows event logs, and more. You can see some screenshots at https://www.gravwell.io/technology We have a free tier w…

I don't mind you giving a small sales pitch...and maybe your product is indeed what I'm searching for. But your pricing model is instantly putting me off. Same as with Splunk, you end up with not being able to predict your cost and paying way too much. Tell me when you fix that and I might be interested ;) Edit: Sorry, I was misreading your comment. Premium is unlimited...I will look into it, thanks. :)

Yep, paying customers are licensed by the node rather than by the gigabyte (as Splunk does it), and you're really only limited by your hardware at that point. You might be surprised at how much you can accomplish on the free license, though--there are several small businesses using it to monitor their networks because 2GB/day will hold a pretty hefty amount of Netflow, collectd, Zeek, and syslog records.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#147

Post by mhall119 »

mhall119 wrote: post #88
dspillett wrote: post #63

Earlier quoted context omitted.

I'm sure this is true, but the documentation does explicitly say "Each machine should have a minimum of 8GB RAM" with absolutely no caveats about scale, so I think people can be forgiven for taking a quick look and moving on because they don't want to provision that.

That's a valid point, I've brought it up internally to improve the docs to explain what type of load this requirement is expecting. If you'd like you can create an issue for this in Github ( https://github.com/influxdata/docs.influxdata.com/issues/new ) to be involved in the discussion and change.

While we're working on this page, also check out https://docs.influxdata.com/influxdb/v1.8/guides/hardware_si... which gives a better breakdown of resources requirements based on expected load

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#148

Post by dijit »

user5994461 wrote: post #127
klohto wrote: post #97

Earlier quoted context omitted.

World’s salaries aren’t only US based :) Datadog for a smaller Western Europe startup is going to cost more than their devs salaries.

Western salaries are not that low and true costs to the employer is typically double the perceived salary. That means we're talking tens of thousands of euros, so thousands of hosts (list price is certainly negotiable at this scale). If they've got a thousand of hosts, the costs of the infrastructure itself must dwarf the salary of any developer by orders of magnitude, the salary of a developer is simply irrelevant w…

> true costs to the employer is typically double the perceived salary.

This is true, but my comment was an offhanded way to say that my "salary" (as in, the one on my contracts and the one I "see") is less than a month of Datadog for our number of hosts.

As for the rest of your comment, I wish it was true.

Developer salaries outside of the capitals is quite low in Europe, and even inside the capitals only go to "near double"

So, instead of 12x it becomes 6x developer costs per annum, which is a fair whack of money.

For me to justify spending "3-6" peoples worth of money it had better save "3-6" peoples worth of time.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#149

Post by avthar »

samstave wrote: post #122
avthar wrote: post #82

Earlier quoted context omitted.

Curious what you found difficult to deploy/ configure? Is this in a self-managed context?

I found this to be a funny, subtle insult. :-) "Why is it difficult, because you're self managed?"

No intention to insult - just trying to get more context :)

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#150

Post by souldeux »

jimaek wrote: post #81
avthar wrote: post #62

Echoing the sentiment expressed by others here, for a scalable time-series database that continues to invest in its community and plays well with others, please check out TimescaleDB. We (I work at TimescaleDB) recently announced that multi-node TimescaleDB will be available for free, specifically as a way to keep investing in our community: https://blog.timescale.com/blog/multi-node-petabyte-scale-ti... Today Timesc…

While influx is pretty bad overall it's super simple to deploy and configure unlike timescale. It's the main reason we decided to use influx in our small team with simple enough timeseries needs

What's wrong with influx? I use it and like it, albeit for hobby-level projects.
Post reply on HN