Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

serhack.me

Post reply on HN
Print view View original

111–120 of 298 posts

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#111

Post by linsomniac »

I've been doing monitoring of our ~120ish machines for 3-4 years now using Influx+Telegraf+Grafana, and have been really happy with it. Prior to that we were using collectd+graphite and with 1 minute stats it was adding some double-digits %age utilization on our infrastructure (I don't remember exactly how much, but I want to say 30% CPU+disk).

Influxdb has been a real workhorse. We suffered through some of their early issues, but since then it's been extremely solid. It just runs, is space efficient, and very robust.

We almost went with Prometheus instead of Influx (as I said, early growing pains), but I had just struggled through managing a central inventory and hated it, so I really wanted a push rather than pull architecture. But from my limited playing with it, Prometheus seemed solid.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#112

Post by cbzbc »

mhall119 wrote: post #108
cbzbc wrote: post #14

Earlier quoted context omitted.

collectd comes packaged by default in a bunch of distributions and telegraf does not -- for basic functionality telegraf doesn't really buy anything and takes more configuration.

`snap install telegraf` is pretty easy though

Not everything uses or even can run ubuntu. I have a similar setup and have a number of openwrt devices that also feed things into grafana via collectd - so I already have that up and running.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#114

Post by mhall119 »

gen220 wrote: post #106
klohto wrote: post #94

Earlier quoted context omitted.

Which kinda confirms the parent’s issues... InfluxDB is really easy to deploy and forget. With TimescaleDB you should be ready to know ins-and-outs of PG to secure and maintain correctly. Sure, for scaling and high loads TDB might be good but InfluxDB is easier and suitable for most loads and maintainability.

> With TimescaleDB you should be ready to know ins-and-outs of PG to secure and maintain correctly. Gotcha, this makes sense. To this, I'd pose the question: is the same not true for Influx? (i.e. With IFDB, you should be ready to know its ins and outs to secure and maintain it correctly). I guess I think about choosing a database like buying a house. I want it to be as good in X years as it is today, maybe better. F…

Think if InfluxDB is "move-in ready" when buying a house. Sure you won't know the electrical and plumbing as well as you will a "fixer-uppper" in the long run, but you'll have a place to sleep a lot faster and easier.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#115

Post by jakozaur »

carlosdp wrote: post #46

Gotta say though, having rolled grafana and prometheus and such on my own plenty of times before, if you are a startup and can afford Datadog, use Datadog.

DataDog per host pricing can be very expensive. Metrics are provided by many platforms. If you need logs too, you may look at Sumo Logic which got way cheaper metrics in typical use case.

Disclaimer: I work at Sumo Logic.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#116

Post by aequitas »

For those who still remember Graphite, the team over at Grafana labs have started maintaining Graphite-web and Carbon since 2017 and it is still in active development getting improvements and feature updates. It might not scale as well as any of the other solutions, but for medium size or homelab setups it's still a nice solution if you don't like PromQL or InfluxQL.

https://grafana.com/oss/graphite/

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#117

Post by k-rus »

rattray wrote: post #96

I've only ever used third party monitoring tools, but hope to set up a startup again soon and want to do OSS if I can. Can anyone comment on Prometheus vs Timescale? What are the tradeoffs? Or would I use Prometheus on top of Timescale?

You can use Prometheus on top of TimescaleDB. Timescale builds connector and entire workflow to run Prometheus on top of TimescaleDB and support Grafana in flexible way. Sorry for the promo :) check for details in https://github.com/timescale/timescale-prometheus

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#118

Post by brightball »

hnarn wrote: post #104

Some people are probably going to throw some shade on me for saying this since it's so out of fashion but in my mind, when it comes to some types of basic monitoring (SNMP monitoring of switches/linux servers, disk space usage, backups running and handling them when they don't) then Nagios does get the job done. It's definitely olives and not candy[1] but it's stable, modular, relatively easy to configure (when you g…

Ever looked at Zabbix?

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#119

Post by thethethethe »

hnarn wrote: post #104

Some people are probably going to throw some shade on me for saying this since it's so out of fashion but in my mind, when it comes to some types of basic monitoring (SNMP monitoring of switches/linux servers, disk space usage, backups running and handling them when they don't) then Nagios does get the job done. It's definitely olives and not candy[1] but it's stable, modular, relatively easy to configure (when you g…

Possibly stupid question:

Can you use Nagios to stream metrics exported from your applications binary in real time?

For example, can you use Nagios record each http request processed by your application webserver, tagged with http method, code, latency etc?

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#120

Post by user5994461 »

Papric0re wrote: post #74

[Offtopic (a bit)] Lots of you are talking about metric monitoring. But do you have recommendations when it comes to (basic) security Monitoring? I would usually go for the Elastic-Stack for that purpose, especially because Kibana offers lots of features for security monitoring. But I feel like these stacks are so big and bloated. I basically need something to monitor network traffic (Flows and off-Database retention…

If you want to do logs, you can use graylog or kibana, both using elasticsearch for storage. This allows to find what was connecting where at some point in time (HTTP request logs and database connection logs).

If you want to graph connections from service to service in real time. I've actually never found anything that was capable of doing that, not even paid software.

Post reply on HN