For those who still remember Graphite, the team over at Grafana labs have started maintaining Graphite-web and Carbon since 2017 and it is still in active development getting improvements and feature updates. It might not scale as well as any of the other solutions, but for medium size or homelab setups it's still a nice solution if you don't like PromQL or InfluxQL. https://grafana.com/oss/graphite/
Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD
271–280 of 298 posts
Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD
#272Earlier quoted context omitted.
Nagios is the Jenkins of monitoring. It's popular because you can get it running in an afternoon, and it's easy to configure by hand. It then rots within your infrastructure, because it resists being configured any way _except_ by hand. I've built two systems for configuration-management of Nagios (at different companies), and it's an unpleasant problem to solve. Prometheus's metric format and query syntax are cool,…
I have to use an icinga instance from time to time (icinga is a nagios fork). I really can't see the value, beyond seeing if a service is up or down. I'm surprised no one has named Zabbix. Zabbix is way better. I hadn't the chance to use Zabbix past 4.something but it's worth it. I've been using Prometheus/grafana and frankly the value I see is it's out of the box adaptability at capturing a mutating data source (exa…
This is an extreme oversimplification. The value is not in "seeing" if something is "up or down", the value is in the modularity of what a "service" can mean in the first place (anything you can script -- and the eco-system of plugins is huge), the fact that you don't have to "see" it (because notifications are extremely modular), the fact that escalations of issues can happen automatically if they are not resolved, and the fact that event-handlers in many cases can help you resolve the issue automatically without even having to raise an alert in the first place.
Nagios is a monitoring tool built with the UNIX philosophy in mind, and it's ingenious in its simplicity: decide state based on script or binary exit codes, relate dependencies between objects to avoid unnecessary troubleshooting, notify if necessary (again, with scripts/binaries) and/or try to resolve if configured. It hooks into a server frame of mind very well if you're a sysadmin.
Sure, if you main use case is "mutating data sources" and collecting metrics, any Nagios flavor won't be for you, because it's not what Nagios is made to do. There's a reason it's extremely popular in large enterprises, because it was created for them. No monitoring solution is for everyone and solves every problem.
Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD
#273Earlier quoted context omitted.
Nagios is the Jenkins of monitoring. It's popular because you can get it running in an afternoon, and it's easy to configure by hand. It then rots within your infrastructure, because it resists being configured any way _except_ by hand. I've built two systems for configuration-management of Nagios (at different companies), and it's an unpleasant problem to solve. Prometheus's metric format and query syntax are cool,…
> you can get it running in an afternoon, and it's easy to configure by hand. This read like a joke. Nagios looks like it's from stone age having files in cgi-bin folder with unnecessary complication to installation and management, unless they made it any better at some point.
Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD
#274Earlier quoted context omitted.
Yeah, I was in no way insinuating that Nagios is superior in general, or even to Prometheus, just that it does the job well for some use cases. Monitoring is tricky and you definitely need a tool box because each problem has a different optimal solution.
Nagios and its forks for sure have a place in the monitoring ecosystem. They’re just not tools that tend to stick around once you’re big enough to have a dedicated DevOps or SRE team.
Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD
#275Some people are probably going to throw some shade on me for saying this since it's so out of fashion but in my mind, when it comes to some types of basic monitoring (SNMP monitoring of switches/linux servers, disk space usage, backups running and handling them when they don't) then Nagios does get the job done. It's definitely olives and not candy[1] but it's stable, modular, relatively easy to configure (when you g…
We run this at work, and I have (to put it politely) severe reservations. What does these functions for you: - Realtime GUI which works with windows 10 (we have a web site and nagastamon) - aggregation of alerts / alert roll up - sharing filters - summary + description - temporary downtime of alerting - message rate suppression (to stop floods) - filtering of columns/ordering etc. - bulk actions for closing alerts I'…
Other than that I don't quite understand the point of your comment, you say you have "severe reservations" but many of the points you list are available even with Nagios core, and most of them are available in other Nagios variants.
Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD
#276I've been doing monitoring of our ~120ish machines for 3-4 years now using Influx+Telegraf+Grafana, and have been really happy with it. Prior to that we were using collectd+graphite and with 1 minute stats it was adding some double-digits %age utilization on our infrastructure (I don't remember exactly how much, but I want to say 30% CPU+disk). Influxdb has been a real workhorse. We suffered through some of their ear…
It's so much easier to write incorrect/misleading queries in influxql than in promql. And you can't perform operations between two different series names in influxdb, last I looked. That makes it impossible to do things like ratios or percentages unless you have control over your metrics, and structure them the way influx likes. Also, no support for calculating percentiles from Prometheus histogram buckets.
Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD
#277Earlier quoted context omitted.
Have you looked at Flux? You can do some really incredible things with it, both inside of queries and also in tasks/alerts. Check out https://www.influxdata.com/blog/anomaly-detection-with-media... for an example
Yes I have looked at it and it's unusable for my application because it is orders of magnitude slower than InfluxQL. Fairly logical if you think about how Flux works, it's very hard to optimize a query engine with a language where every step is supposed to be discrete.
Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD
#278Echoing the sentiment expressed by others here, for a scalable time-series database that continues to invest in its community and plays well with others, please check out TimescaleDB. We (I work at TimescaleDB) recently announced that multi-node TimescaleDB will be available for free, specifically as a way to keep investing in our community: https://blog.timescale.com/blog/multi-node-petabyte-scale-ti... Today Timesc…
Just wanted to say I am super impressed with the work TimescaleDB has been doing. Previously at NGINX I was part of a team that built out a sharded timeseries database using Postgres 9.4. When I left it was ingesting ~2 TB worth of monitoring data a day (so not super large, but not trivial either). Currently I have built out a data warehouse using Postgres 11 and Citus. Only reason I didn't use TimescaleDB was lack o…
We were successfully ingesting hundreds of billions of ad serving events per day to it. It is much faster at query speed than any Postgres-based database (for instance, it may scan tens of billions of rows per second on a single node). And it scales to many nodes.
While it is possible to store monitoring data to ClickHouse, it may be non-trivial to set up. So we decided creating VictoriaMetrics [2]. It is built on design ideas from ClickHouse, so it features high performance additionally to ease of setup and operation. This is proved by publicly available case studies [3].
Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD
#279We switched from InfluxDB to TimescaleDB for our IoT solutions. InfluxDB is very difficult to work with large datasets and enterprise/region compliance. We ingest around 100MB data per day and growing.
Re: Monitoring your own infrastructure using Grafana, InfluxDB, and CollectD
#280Earlier quoted context omitted.
While influx is pretty bad overall it's super simple to deploy and configure unlike timescale. It's the main reason we decided to use influx in our small team with simple enough timeseries needs
What's wrong with influx? I use it and like it, albeit for hobby-level projects.
[1] https://medium.com/@valyala/insert-benchmarks-with-inch-infl...