Earlier quoted context omitted.
Interesting, I have a very small team that maintains a lot of infrastructure including Prometheus. Setting up the monitoring for 5 datacenters, thousands of hosts and hundreds of services was maybe a ~2 weeks effort. We then organically add and remove things but it is maybe a ~4h/week effort. It give us a lot of value. We were using other solutions before (sensu/nagios) and it is night and day. All the plugins that w…
You're lucky to have a team. I was doing that with another person in our spare time and got 0 hour/week to spend. I only give ourselves a couple weeks to setup new software and tune everything perfectly then it has to be done for good. Will check maybe once a quarter for capacity adjustment or software upgrade. (I also supervise a logging system ingesting 1 TB/day with no supervision). I should probably say that my e…
We disabled a lot of useless metrics in the node exporter. I think pulling works OK if you have a service discovery mechanism. We hook Prometheus to Consul.
We have a mix of very small VMs and very beefy bare metal.