Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

serhack.me

Post reply on HN
Print view View original

241–250 of 298 posts

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#241

Post by themgt »

cevian wrote: post #201
themgt wrote: post #183

Earlier quoted context omitted.

I was testing out both TimescaleDB/InfluxDB recently, including maybe using with Prometheus/Grafana. I was leaning towards Timescale, but InfluxDB was indeed a lot easier to quickly boot a "batteries included" setup and start working with live data. I eventually spent a while reading about Timescale 1 vs 2, and testing the pg_prometheus[1] adapter and started thinking through integrating its schema to our other needs…

(TimescaleDB engineer) Really curious about finding out more about your reservations about the updated schema. All criticisms welcome.

Thanks, as I say our use case may be too odd to be worth supporting, but effectively we're trying to add a basic metrics (prom/ad-hoc) feature to an existing product (using Postgres) with an existing sort of opinionated "ORM"/toolkit for inserting/querying data.

Because of that and the small scale required, the choice of table-per-metric would be a tough fit and I think a single table with JSONB and maybe some partial indexes is going to work a lot better for us. It would just be nice if we could somehow code in our schema mapping and use the supported extension, but I get it may be too baked-into the implementation.

Anyway, overall we're quite happy with TimescaleDB!

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#243

Post by rbetts »

rattray wrote: post #177
mhall119 wrote: post #130

Earlier quoted context omitted.

Why is that? It takes minimal effort to get it up and running and you can either self-host or use the SaaS offering on any of the major clouds. There's even a free tier on the SaaS your startup can use that won't cost you a dime until your usage becomes significant.

Because the VPE sounds like the corporate sort of person who would not prioritize the things I'd want them to as a customer.

That kinda hurts.

There are 100k's of happy InfluxDB users -- untold millions of completely open source Telegraf deployments doing meaningful work for people -- integrated into our products and our competitors' products. We make the vast majority of our codebase public under liberal OSS licenses.

And we'll keep listening and learning how to do more.

... off to see if I can change my internal slack handle to @corporatetool. Unless there's a policy against that ;-)

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#244

Post by NicoJuicy »

R0b0t1 wrote: post #154
cevian wrote: post #137

Earlier quoted context omitted.

(TimescaleDB engineer here). This is a really unfair critique. The project you cite is super-optimized for the Prometheus use-case and data model. TimescaleDB beats InfluxDB on performance even without these optimization. It's also not possible to optimize in this way in most other time-series databases. These scripts also work hard to give users a UIUX experience that mimicks PromQL in a lot of ways. This isn't nece…

>The project you cite is super-optimized for the Prometheus use-case and data model. That may be true, but: Instead of figuring out how to meet the user's needs you're going to say the user is wrong?

That's not what he said and is kinda rude to be honest.

What I consider to be said is that they optimize for extendability ( not just one use-case) and that those 2000 lines do a lot of things.

Including replicating optimizations that are are not easy to achieve out of the box in their and other solutions. But they mostly found a way ;)

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#245

Post by rattray »

rbetts wrote: post #243
rattray wrote: post #177

Earlier quoted context omitted.

Because the VPE sounds like the corporate sort of person who would not prioritize the things I'd want them to as a customer.

That kinda hurts. There are 100k's of happy InfluxDB users -- untold millions of completely open source Telegraf deployments doing meaningful work for people -- integrated into our products and our competitors' products. We make the vast majority of our codebase public under liberal OSS licenses. And we'll keep listening and learning how to do more. ... off to see if I can change my internal slack handle to @corporat…

Hey, yeah, sorry, I've been meaning all day to go back and fix that. Really wasn't right.

Just struck a nerve that reminded me of some leaders at a previous gig who made things a little less sensible.

I apologise, and you definitely don't deserve it :-(

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#246

Post by mekster »

avthar wrote: post #62

Echoing the sentiment expressed by others here, for a scalable time-series database that continues to invest in its community and plays well with others, please check out TimescaleDB. We (I work at TimescaleDB) recently announced that multi-node TimescaleDB will be available for free, specifically as a way to keep investing in our community: https://blog.timescale.com/blog/multi-node-petabyte-scale-ti... Today Timesc…

Can you refute the claim that TimescaleDB uses such a huge disk space that is 50 times larger than other time series database?

I see no reason to use it over VictoriaMetrics.

https://medium.com/@valyala/high-cardinality-tsdb-benchmarks...

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#247

Post by RedShift1 »

mhall119 wrote: post #225
RedShift1 wrote: post #78

Earlier quoted context omitted.

InfluxDB is pretty good if you don't need to do any advanced querying like grouping by month or formulas. Its strong points are low diskspace footprint and very fast queries even over long periods of time.

Have you looked at Flux? You can do some really incredible things with it, both inside of queries and also in tasks/alerts. Check out https://www.influxdata.com/blog/anomaly-detection-with-media... for an example

Yes I have looked at it and it's unusable for my application because it is orders of magnitude slower than InfluxQL. Fairly logical if you think about how Flux works, it's very hard to optimize a query engine with a language where every step is supposed to be discrete.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#248

Post by mekster »

julioo wrote: post #71
overcast wrote: post #3

Really, REALLY tried to love InfluxDB. But its systems requirements, performance, and features are poor compared to things like TimescaleDB.

What about storage? We are running influxdb and we are looking for alternative. But a point where Influx is good is storage.

Tried VictoriaMetrics?

The author compares with other storage engines comprehensively from time to time.

https://medium.com/@valyala

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#249

Post by mekster »

BurritoAlPastor wrote: post #141
hnarn wrote: post #104

Some people are probably going to throw some shade on me for saying this since it's so out of fashion but in my mind, when it comes to some types of basic monitoring (SNMP monitoring of switches/linux servers, disk space usage, backups running and handling them when they don't) then Nagios does get the job done. It's definitely olives and not candy[1] but it's stable, modular, relatively easy to configure (when you g…

Nagios is the Jenkins of monitoring. It's popular because you can get it running in an afternoon, and it's easy to configure by hand. It then rots within your infrastructure, because it resists being configured any way _except_ by hand. I've built two systems for configuration-management of Nagios (at different companies), and it's an unpleasant problem to solve. Prometheus's metric format and query syntax are cool,…

> you can get it running in an afternoon, and it's easy to configure by hand.

This read like a joke. Nagios looks like it's from stone age having files in cgi-bin folder with unnecessary complication to installation and management, unless they made it any better at some point.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#250

Post by PaulWaldman »

mekster wrote: post #246
avthar wrote: post #62

Echoing the sentiment expressed by others here, for a scalable time-series database that continues to invest in its community and plays well with others, please check out TimescaleDB. We (I work at TimescaleDB) recently announced that multi-node TimescaleDB will be available for free, specifically as a way to keep investing in our community: https://blog.timescale.com/blog/multi-node-petabyte-scale-ti... Today Timesc…

Can you refute the claim that TimescaleDB uses such a huge disk space that is 50 times larger than other time series database? I see no reason to use it over VictoriaMetrics. https://medium.com/@valyala/high-cardinality-tsdb-benchmarks...

Yes, I can refute that claim. TimescaleDB now provides built in compression.

https://docs.timescale.com/latest/using-timescaledb/compress...

Post reply on HN