Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

serhack.me

Post reply on HN
Print view View original

161–170 of 298 posts

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#161

Post by majkinetor »

overcast wrote: post #3

Really, REALLY tried to love InfluxDB. But its systems requirements, performance, and features are poor compared to things like TimescaleDB.

I use InfluxDb on number of big gov production services, and never had a single issue with it. I use it for all metrics (applicative and infrastructure).

Its easy to deploy and its x-platform. Not sure what are all the comments here about - maybe for really huge loads which I don't have experience at, I usually use separate influxdb per service.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#162

Post by mhall119 »

souldeux wrote: post #150
jimaek wrote: post #81

Earlier quoted context omitted.

While influx is pretty bad overall it's super simple to deploy and configure unlike timescale. It's the main reason we decided to use influx in our small team with simple enough timeseries needs

What's wrong with influx? I use it and like it, albeit for hobby-level projects.

As with any technology, the right tool for the job is highly dependent on what the job is.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#163

Post by hnarn »

BurritoAlPastor wrote: post #141
hnarn wrote: post #104

Some people are probably going to throw some shade on me for saying this since it's so out of fashion but in my mind, when it comes to some types of basic monitoring (SNMP monitoring of switches/linux servers, disk space usage, backups running and handling them when they don't) then Nagios does get the job done. It's definitely olives and not candy[1] but it's stable, modular, relatively easy to configure (when you g…

Nagios is the Jenkins of monitoring. It's popular because you can get it running in an afternoon, and it's easy to configure by hand. It then rots within your infrastructure, because it resists being configured any way _except_ by hand. I've built two systems for configuration-management of Nagios (at different companies), and it's an unpleasant problem to solve. Prometheus's metric format and query syntax are cool,…

Yeah, I was in no way insinuating that Nagios is superior in general, or even to Prometheus, just that it does the job well for some use cases. Monitoring is tricky and you definitely need a tool box because each problem has a different optimal solution.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#164

Post by secondcoming »

dijit wrote: post #68
carlosdp wrote: post #46

Gotta say though, having rolled grafana and prometheus and such on my own plenty of times before, if you are a startup and can afford Datadog, use Datadog.

I just costed a datadog deployment (based on your comment) and it would cost me my yearly salary every month. No thanks. :/

We're moving from Datadog to Prometheus for this reason

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#165

Post by dijit »

user5994461 wrote: post #155
dijit wrote: post #148

Earlier quoted context omitted.

> true costs to the employer is typically double the perceived salary. This is true, but my comment was an offhanded way to say that my "salary" (as in, the one on my contracts and the one I "see") is less than a month of Datadog for our number of hosts. As for the rest of your comment, I wish it was true. Developer salaries outside of the capitals is quite low in Europe, and even inside the capitals only go to "near…

> For me to justify spending "3-6" peoples worth of money it had better save "3-6" peoples worth of time. Well, it does in my experience, especially if you have to handle 2000+ hosts, that's some serious infra there, need serious tooling. May I ask which country is it?

Sweden! :D

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#166

Post by tilolebo »

Papric0re wrote: post #138
retzkek wrote: post #133

Earlier quoted context omitted.

Maybe Loki [1] meets your needs? It lacks the analytics abilities of Elastic (e.g. what's the average response time [2]) but is much simpler to setup and use for jog aggregations, and has a pretty powerful query language for digging through and graphing log statistics (e.g. how many errors have been logged per hour). It's mainly being developed by Grafana Labs, so there's great integration in Grafana. [1] https://gra…

Tanks for the tip. But I would still need some storage and Data Shipper, right? Or is Loki also taking care of storage?

Promtail is the official log shipper for Loki, but you can also use others. See https://github.com/grafana/loki/blob/master/docs/sources/cli...

As for storage, the default is BoltDB for indexes and local file system for the data, but you can also use popular cloud solutions like DynamoDB, etc. AFAIK BoltDB is automatically installed when you install Loki.

The only possible pain point I see for you is that Loki is tailored for Kubernetes. It is totally possible to use it without running a K8 cluster, but you lose some features.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#167

Post by samstave »

avthar wrote: post #149
samstave wrote: post #122

Earlier quoted context omitted.

I found this to be a funny, subtle insult. :-) "Why is it difficult, because you're self managed?"

No intention to insult - just trying to get more context :)

I know - I just read it as such and chuckled.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#168

Post by cs-szazz »

klohto wrote: post #97
cs-szazz wrote: post #92

Earlier quoted context omitted.

Wait, you're saying it costs 12x as much as your salary? That doesn't seem right... the company I'm at uses it pretty heavily and we're at about 1x of a FTE salary (and it's still super worth it)

World’s salaries aren’t only US based :) Datadog for a smaller Western Europe startup is going to cost more than their devs salaries.

You're right, I was narrow-minded in my thinking there, thanks for pointing it out :)

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#169

Post by thibauts »

doublesCs wrote: post #142
aeyes wrote: post #86

Earlier quoted context omitted.

I don't consider TimescaleDB to be a serious contender as long as I need a 2000 line script to install functions and views to have something essential for time-series data like dimensions: https://github.com/timescale/timescale-prometheus/blob/maste... https://github.com/timescale/timescale-prometheus/blob/maste...

Completely garbage comment. For roughly 100% of the software you install you have no idea what goes on into the installation. The only reason you're making that criticism about TimescaleDB is that they making it really easy to have a look how it works behind the scenes, and you like posing as a purist. ... unless you tell me that you don't use computers because of much code goes into the installation of the OS, or br…

I’m really sad to see this kind of tone more and more in HN comments...

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#170

Post by lma21 »

aequitas wrote: post #116

For those who still remember Graphite, the team over at Grafana labs have started maintaining Graphite-web and Carbon since 2017 and it is still in active development getting improvements and feature updates. It might not scale as well as any of the other solutions, but for medium size or homelab setups it's still a nice solution if you don't like PromQL or InfluxQL. https://grafana.com/oss/graphite/

for simple and fast monitoring solutions, i always opted for collected + graphite + grafana. In a containerized environment, it's so easy to deploy (0 configuration by default) and monitor a set of 50-100 nodes. Beyond that, disk tuning and downsampling (pre-aggregation) rules become important.
Post reply on HN