Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

serhack.me

Post reply on HN
Print view View original

11–20 of 298 posts

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#11

Post by dahfizz »

viraptor wrote: post #6
overcast wrote: post #3

Really, REALLY tried to love InfluxDB. But its systems requirements, performance, and features are poor compared to things like TimescaleDB.

Does any of those matter for small-scale monitoring (say <= 10 hosts)? I've got influx sitting pretty much idle in that kind of environments and any data preprocessing / collation in grafana works just fine.

Arguably system requirements matter more for small scale deployments. I shouldn't need a server with multiple SSD volumes and 8GB+ ram just to monitor a couple raspberry pi's.

EDIT: okay, I get it. You don't need 8GB ram. I was just going by the hardware requirements in their docs: https://docs.influxdata.com/influxdb/v1.8/introduction/insta...

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#12

Post by cbzbc »

dahfizz wrote: post #11
viraptor wrote: post #6

Earlier quoted context omitted.

Does any of those matter for small-scale monitoring (say <= 10 hosts)? I've got influx sitting pretty much idle in that kind of environments and any data preprocessing / collation in grafana works just fine.

Arguably system requirements matter more for small scale deployments. I shouldn't need a server with multiple SSD volumes and 8GB+ ram just to monitor a couple raspberry pi's. EDIT: okay, I get it. You don't need 8GB ram. I was just going by the hardware requirements in their docs: https://docs.influxdata.com/influxdb/v1.8/introduction/insta...

You don't need 8GB ram to run a small scale deployment of influxdb.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#14

Post by cbzbc »

viraptor wrote: post #4

Nothing against choosing this set of apps really, but I'm curious why collectd and not telegraf which does the same kind of metrics probes and is a part of the TICK stack.

collectd comes packaged by default in a bunch of distributions and telegraf does not -- for basic functionality telegraf doesn't really buy anything and takes more configuration.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#15

Post by viraptor »

bbrks wrote: post #10
viraptor wrote: post #4

Nothing against choosing this set of apps really, but I'm curious why collectd and not telegraf which does the same kind of metrics probes and is a part of the TICK stack.

As somebody new to both, are you able to elaborate on what telegraf/TICK provides that the collection of programs in the post does not? I see lots of posts complaining about the stack in the post, but all of the alternatives posed don't really explain why.

Telegraf and collectd do roughly the same thing. They run some plugins to get data and push the results to a given metrics sink. I asked because TICK (Telegraf, InfluxDB, Chronograf and Kapacitor) is a known solution and a fairly standard way to add elements of monitoring to your system.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#16

Post by viraptor »

dahfizz wrote: post #11
viraptor wrote: post #6

Earlier quoted context omitted.

Does any of those matter for small-scale monitoring (say <= 10 hosts)? I've got influx sitting pretty much idle in that kind of environments and any data preprocessing / collation in grafana works just fine.

Arguably system requirements matter more for small scale deployments. I shouldn't need a server with multiple SSD volumes and 8GB+ ram just to monitor a couple raspberry pi's. EDIT: okay, I get it. You don't need 8GB ram. I was just going by the hardware requirements in their docs: https://docs.influxdata.com/influxdb/v1.8/introduction/insta...

... and it doesn't need that. I'm running telegraf+influx+grafana along with other services on a small rpi without issues.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#17

Post by viraptor »

Kednicma wrote: post #8

I'm curious why they chose CollectD and not Prometheus or another pull-based ("scraping") solution. Pulling is more compositional than pushing and requires fewer machines to be touched when monitoring configuration is changed.

Probably due to the size of deployment. I'd go to Prometheus in a larger/dynamic environment for flexibility, but I'm happy using telegraf on a home network where I configure things by hand when I'm logged into each pet machine.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#18

Post by jimktrains2 »

viraptor wrote: post #15
bbrks wrote: post #10

Earlier quoted context omitted.

As somebody new to both, are you able to elaborate on what telegraf/TICK provides that the collection of programs in the post does not? I see lots of posts complaining about the stack in the post, but all of the alternatives posed don't really explain why.

Telegraf and collectd do roughly the same thing. They run some plugins to get data and push the results to a given metrics sink. I asked because TICK (Telegraf, InfluxDB, Chronograf and Kapacitor) is a known solution and a fairly standard way to add elements of monitoring to your system.

> TICK (Telegraf, InfluxDB, Chronograf and Kapacitor) is a known solution and a fairly standard way to add elements of monitoring to your system.

Amusing that I've never heard of any of this but have heard and used collectd.

It's obviously no where near as common as a lamp stack or anywhere near common at all, so asking why it over something else is answered by "someone made it up so it's better".

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#19

Post by alyandon »

dahfizz wrote: post #11
viraptor wrote: post #6

Earlier quoted context omitted.

Does any of those matter for small-scale monitoring (say <= 10 hosts)? I've got influx sitting pretty much idle in that kind of environments and any data preprocessing / collation in grafana works just fine.

Arguably system requirements matter more for small scale deployments. I shouldn't need a server with multiple SSD volumes and 8GB+ ram just to monitor a couple raspberry pi's. EDIT: okay, I get it. You don't need 8GB ram. I was just going by the hardware requirements in their docs: https://docs.influxdata.com/influxdb/v1.8/introduction/insta...

Running TICK stack here collecting metrics from around a dozen different hosts (more if you count network devices like switches) to a central server with a default retention policy on spinning rust. InfluxDB process is currently sitting at 600MB RSS.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#20

Post by victor106 »

wiradikusuma wrote: post #13

There's also https://github.com/timberio/vector

+1 for vector. We moved from Logstash to Vector and we couldn't be happier. Logstash is awesome but its a memory hog.

With Vector and Toshi you can kinda (I am not sure Toshi is as mature as Elastic) use them to replace LogStash and Elastic, the missing piece is Kibana

Post reply on HN