Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

serhack.me

Post reply on HN
Print view View original

201–210 of 298 posts

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#201

Post by cevian »

themgt wrote: post #183
avthar wrote: post #82

Earlier quoted context omitted.

Curious what you found difficult to deploy/ configure? Is this in a self-managed context?

I was testing out both TimescaleDB/InfluxDB recently, including maybe using with Prometheus/Grafana. I was leaning towards Timescale, but InfluxDB was indeed a lot easier to quickly boot a "batteries included" setup and start working with live data. I eventually spent a while reading about Timescale 1 vs 2, and testing the pg_prometheus[1] adapter and started thinking through integrating its schema to our other needs…

(TimescaleDB engineer) Really curious about finding out more about your reservations about the updated schema. All criticisms welcome.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#202

Post by say_it_as_it_is »

darkwater wrote: post #199
say_it_as_it_is wrote: post #193

Earlier quoted context omitted.

This thread is about infra monitoring, where any of the potential performance differences probably don't matter at all. If it does, show us. I don't work for any time series database provider.

Why performances shouldn't matter in this case? The more instances, the more timeseries and the more datapoints you have, the more you care for speed when you need to query it to visualize for example WoW changes in, say, memory usage.

Consider the case at hand, involving streaming charts and alerts. There will be zero perceptible difference in the streaming charts regardless of what database is used. Alerts won't trigger for whatever millisecond difference there may be, and I don't think that this matters to any developer or manager awaiting the 3am call.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#203

Post by aeyes »

doublesCs wrote: post #142
aeyes wrote: post #86

Earlier quoted context omitted.

I don't consider TimescaleDB to be a serious contender as long as I need a 2000 line script to install functions and views to have something essential for time-series data like dimensions: https://github.com/timescale/timescale-prometheus/blob/maste... https://github.com/timescale/timescale-prometheus/blob/maste...

Completely garbage comment. For roughly 100% of the software you install you have no idea what goes on into the installation. The only reason you're making that criticism about TimescaleDB is that they making it really easy to have a look how it works behind the scenes, and you like posing as a purist. ... unless you tell me that you don't use computers because of much code goes into the installation of the OS, or br…

The linked scripts are not part of the base installation yet TimescaleDB claims, for example, Prometheus support.

If you design your own schema and want to filter on such a dynamic tag field you have to replicate what they have implemented in these scripts. If you look at the code you'll see that this isn't exactly easy and other products do this out of the box.

I use TimescaleDB and have suffered from this, otherwise I wouldn't be talking here. Another problem is that these scripts are not compatible with all versions of TimescaleDB so you have to be careful when updating either one.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#204

Post by znpy »

samstave wrote: post #122
avthar wrote: post #82

Earlier quoted context omitted.

Curious what you found difficult to deploy/ configure? Is this in a self-managed context?

I found this to be a funny, subtle insult. :-) "Why is it difficult, because you're self managed?"

I think it's a honest question. Maybe they want to find out if setting up needs a bit of us work or simplifying.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#205

Post by carlosdp »

ciguy wrote: post #59
carlosdp wrote: post #46

Gotta say though, having rolled grafana and prometheus and such on my own plenty of times before, if you are a startup and can afford Datadog, use Datadog.

This is good advice, however you will also want to make sure you have a plan to get off datadog when you grow. Datadog is one of the easiest to use and most comprehensive out of the box. But it gets really expensive as you begin to scale up and add servers, cloud accounts, services etc.... At a certain scale, rolling your own monitoring and alerting becomes cost effective again as Datadog begins to charge an arm and…

On the contrary, the bigger you get, the more you need Datadog to scale. Once you grow big enough, you can make volume deals with them, you don't pay list price. I'd much rather pay them than have a single engineer have to spend any time managing infrastructure that isn't core to our product, especially since Datadog will always do it better.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#206

Post by carlosdp »

jakozaur wrote: post #115
carlosdp wrote: post #46

Gotta say though, having rolled grafana and prometheus and such on my own plenty of times before, if you are a startup and can afford Datadog, use Datadog.

DataDog per host pricing can be very expensive. Metrics are provided by many platforms. If you need logs too, you may look at Sumo Logic which got way cheaper metrics in typical use case. Disclaimer: I work at Sumo Logic.

It's expensive for good reason, it's the best out there.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#207

Post by znpy »

richardARPANET wrote: post #198
avthar wrote: post #62

Echoing the sentiment expressed by others here, for a scalable time-series database that continues to invest in its community and plays well with others, please check out TimescaleDB. We (I work at TimescaleDB) recently announced that multi-node TimescaleDB will be available for free, specifically as a way to keep investing in our community: https://blog.timescale.com/blog/multi-node-petabyte-scale-ti... Today Timesc…

We're using TimescaleDB + Grafana for visualising sensor data for a Health Tech product. No complaints so far.

After a few years in the industry of systems engineering and administration I think that "no complaints so far" is one of the best compliments a software can receive.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#208

Post by chucky_z »

souldeux wrote: post #150
jimaek wrote: post #81

Earlier quoted context omitted.

While influx is pretty bad overall it's super simple to deploy and configure unlike timescale. It's the main reason we decided to use influx in our small team with simple enough timeseries needs

What's wrong with influx? I use it and like it, albeit for hobby-level projects.

Influx is so good for hobby-level projects. So good!!

For serious applications, it doesn't cut it. Trying to do more than a few TB a day is a waste of time outside of enterprise, which ain't cheap.

I plopped VictoriaMetrics in place of Influx for my cases and haven't even had a single hiccup.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#209

Post by aeyes »

cevian wrote: post #137
aeyes wrote: post #86

Earlier quoted context omitted.

I don't consider TimescaleDB to be a serious contender as long as I need a 2000 line script to install functions and views to have something essential for time-series data like dimensions: https://github.com/timescale/timescale-prometheus/blob/maste... https://github.com/timescale/timescale-prometheus/blob/maste...

(TimescaleDB engineer here). This is a really unfair critique. The project you cite is super-optimized for the Prometheus use-case and data model. TimescaleDB beats InfluxDB on performance even without these optimization. It's also not possible to optimize in this way in most other time-series databases. These scripts also work hard to give users a UIUX experience that mimicks PromQL in a lot of ways. This isn't nece…

> TimescaleDB beats InfluxDB on performance even without these optimization

Would you mind sharing the schema used for this comparison? Maybe I missed it in your documentation of use-cases. When implementing dynamic tags in my own model, my tests showed that your approach is very necessary.

Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD

#210

Post by carlosdp »

Axsuul wrote: post #188
yellow_mixer wrote: post #186

Earlier quoted context omitted.

How are you running it? On AWS? What’s your infra look like.

GCP with Docker Swarm, scheduled on the same node (n2-standard-4). Let me know if any other questions

Does "scheduled on the same node" imply you are running everything on one node?
Post reply on HN