Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

serhack.me

Post reply on HN
Print view View original

81–90 of 298 posts

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#81

Post by jimaek »

avthar wrote: post #62

Echoing the sentiment expressed by others here, for a scalable time-series database that continues to invest in its community and plays well with others, please check out TimescaleDB. We (I work at TimescaleDB) recently announced that multi-node TimescaleDB will be available for free, specifically as a way to keep investing in our community: https://blog.timescale.com/blog/multi-node-petabyte-scale-ti... Today Timesc…

While influx is pretty bad overall it's super simple to deploy and configure unlike timescale.

It's the main reason we decided to use influx in our small team with simple enough timeseries needs

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#82

Post by avthar »

jimaek wrote: post #81
avthar wrote: post #62

Echoing the sentiment expressed by others here, for a scalable time-series database that continues to invest in its community and plays well with others, please check out TimescaleDB. We (I work at TimescaleDB) recently announced that multi-node TimescaleDB will be available for free, specifically as a way to keep investing in our community: https://blog.timescale.com/blog/multi-node-petabyte-scale-ti... Today Timesc…

While influx is pretty bad overall it's super simple to deploy and configure unlike timescale. It's the main reason we decided to use influx in our small team with simple enough timeseries needs

Curious what you found difficult to deploy/ configure? Is this in a self-managed context?

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#83

Post by santafen »

mhall119 wrote: post #72
santafen wrote: post #56

Earlier quoted context omitted.

This was from the CTO last week: "This work won't be landing anywhere until sometime next year and it'll be landing in our Cloud 2 offering first." So the OSS is definitely a second-class citizen. And now that they've dropped all DevRel activity, don't expect much attention for OSS Developers and users.

Quite the contrary, we're finishing up packaging of the OSS version now to make it as easy as possible for you to get it and we're assembled a new team focused on getting OSS 2.0 released. We continue to support all users, including OSS users, in our public Slack and Discourse, as well as Github. We have not "dropped all DevRel" activity.

¯\_(ツ)_/¯ Influx laid off pretty much the entire DevRel team in the last 6 months, but ok.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#84

Post by floren »

Papric0re wrote: post #74

[Offtopic (a bit)] Lots of you are talking about metric monitoring. But do you have recommendations when it comes to (basic) security Monitoring? I would usually go for the Elastic-Stack for that purpose, especially because Kibana offers lots of features for security monitoring. But I feel like these stacks are so big and bloated. I basically need something to monitor network traffic (Flows and off-Database retention…

I think Gravwell (https://gravwell.io) might be what you're looking for--but I work for Gravwell so I may be biased! If I can be forgiven a short sales pitch, we've built a format-agnostic storage & querying system that easily handles raw packets, Netflow records (v5, v9, and IPFIX), collectd data, Windows event logs, and more. You can see some screenshots at https://www.gravwell.io/technology

We have a free tier which allows 2GB of data ingest per day (paid licenses are unlimited) which should be more than enough for capturing logs and flows. The resources needed to run Gravwell basically scale with how much data you put into it, but it's a lot quicker to install and set up than something like Elastic, in our opinion (https://www.gravwell.io/blog/gravwell-installed-in-2-minutes)

Edit: it's currently a bit roll-your-own, but we're really close to releasing Gravwell 4.0 which enables pre-packaged "kits" containing dashboards, queries, etc. for a variety of data types (Netflow, CoreDNS logs, and so on)

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#85

Post by svd4anything »

overcast wrote: post #3

Really, REALLY tried to love InfluxDB. But its systems requirements, performance, and features are poor compared to things like TimescaleDB.

Is that really true? Do you have any reference for that, I’m curious how different it is. FWIW I use InfluxDB and haven’t really thought it was a huge resource hog.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#86

Post by aeyes »

avthar wrote: post #62

Echoing the sentiment expressed by others here, for a scalable time-series database that continues to invest in its community and plays well with others, please check out TimescaleDB. We (I work at TimescaleDB) recently announced that multi-node TimescaleDB will be available for free, specifically as a way to keep investing in our community: https://blog.timescale.com/blog/multi-node-petabyte-scale-ti... Today Timesc…

I don't consider TimescaleDB to be a serious contender as long as I need a 2000 line script to install functions and views to have something essential for time-series data like dimensions:

https://github.com/timescale/timescale-prometheus/blob/maste...

https://github.com/timescale/timescale-prometheus/blob/maste...

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#87

Post by gregwebs »

pachico wrote: post #22

I approached InfluxDB since it looked promising. It did actually served its purpose when it was simple and Telegraf was indeed handy. Now that I have more mature requirements I can't wait to move away from it. It gets frozen frequently, it's UI Chronograph is really rubbish, functions are very limited and managing continuous queries is tiresome. I'm now having better results and experience storing data in ClickHouse…

Have you tried Victoria Metrics? It originallyl started as a ClickHouse fork for time series but is now a re-write keeping some of the same principals.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#88

Post by mhall119 »

dspillett wrote: post #63
cbzbc wrote: post #12

Earlier quoted context omitted.

You don't need 8GB ram to run a small scale deployment of influxdb.

I'm sure this is true, but the documentation does explicitly say "Each machine should have a minimum of 8GB RAM" with absolutely no caveats about scale, so I think people can be forgiven for taking a quick look and moving on because they don't want to provision that.

That's a valid point, I've brought it up internally to improve the docs to explain what type of load this requirement is expecting. If you'd like you can create an issue for this in Github (https://github.com/influxdata/docs.influxdata.com/issues/new) to be involved in the discussion and change.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#89

Post by mhall119 »

santafen wrote: post #83
mhall119 wrote: post #72

Earlier quoted context omitted.

Quite the contrary, we're finishing up packaging of the OSS version now to make it as easy as possible for you to get it and we're assembled a new team focused on getting OSS 2.0 released. We continue to support all users, including OSS users, in our public Slack and Discourse, as well as Github. We have not "dropped all DevRel" activity.

¯\_(ツ)_/¯ Influx laid off pretty much the entire DevRel team in the last 6 months, but ok.

This is true, and has certainly been painful for us and puts more work on those of us who are still working on community and devrel, but we're still doing it.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#90

Post by gen220 »

aeyes wrote: post #86
avthar wrote: post #62

Echoing the sentiment expressed by others here, for a scalable time-series database that continues to invest in its community and plays well with others, please check out TimescaleDB. We (I work at TimescaleDB) recently announced that multi-node TimescaleDB will be available for free, specifically as a way to keep investing in our community: https://blog.timescale.com/blog/multi-node-petabyte-scale-ti... Today Timesc…

I don't consider TimescaleDB to be a serious contender as long as I need a 2000 line script to install functions and views to have something essential for time-series data like dimensions: https://github.com/timescale/timescale-prometheus/blob/maste... https://github.com/timescale/timescale-prometheus/blob/maste...

(Not affiliated with TimescaleDB, just trying to understand your critique)

TimescaleDB is a Postgres extension, which means they have to play PG's rules, which rather implies a complicated mesh of functions, views, and "internal-use-only" tables.

But once they're there, you can pretty much pretend they don't exist (until they break, of course, but this is true for everything in your software stack).

Is your complaint targeting the size of these scripts, or their contents? Because everything in there appears pretty reasonable to me.

Post reply on HN