A beginner-friendly introduction to Prometheus
1–10 of 26 posts
Re: A beginner-friendly introduction to Prometheus
#2[deleted]
Re: A beginner-friendly introduction to Prometheus
#3When I started out as a beginner understanding what are metrics, why is Prometheus used was something difficult for me to understand . After spending a lot of time and learning from the material out there I created something that I wish I had when I started learning Prometheus.
PS: This is my first tutorial kind of material, open to feedback on how to improve.
Re: A beginner-friendly introduction to Prometheus
#4I wonder if it should be used at all for beginners and smaller teams. I thought it is something like the new industry standard but even to get it running in production is not trivial and had the feeling this is something for teams who are deeply committed to monitoring and developing their own metrics, views and alarms.
My conclusion is the majority (or at least smaller teams) are maybe better served with something like Datadog. It's cheaper as long as you don't have a high number of machines. Speaking of that I'm still missing the easy middle ground of monitoring which is modern, easy to use and works for a higher number of machines (and even something like Docker Swarm).
Re: A beginner-friendly introduction to Prometheus
#5I wonder if it should be used at all for beginners and smaller teams. I thought it is something like the new industry standard but even to get it running in production is not trivial and had the feeling this is something for teams who are deeply committed to monitoring and developing their own metrics, views and alarms. My conclusion is the majority (or at least smaller teams) are maybe better served with something l…
Basic Prometheus, with vanilla storage engine, is quite easy to set up (IMO).
On a Debian system it takes about 15-30 minutes to install and do a basic enough configuration to get it running. Another 30 minutes to get a Grafana instance sitting in front of it - and then it's incremental to get the node_exporter (also packaged up) out to your systems.
The two time consuming bits are a) customising the metrics that you can scrape, though there's plenty of templates out there, and if you're not trying to do any fancy application-layer metric scraping you're probably going to be happy out of the box.
And b) evaluating and then setting up longer / larger storage engine on the backend - but if you've committed to Prometheus after doing your low-effort PoC, then this is a simple cost benefit evaluation.
Metrics-as-a-service are an interesting proposition, but I'm very cautious about them
> Speaking of that I'm still missing the easy middle ground of monitoring which is modern, easy to use and works for a higher number of machines (and even something like Docker Swarm).
What's modern mean in this context, and just how easy do you want?
In my experience monitoring systems are relatively easy to deploy, unless you're huge. The difficult bit is working out (with platform owners) everything they need to monitor, what their key health indicators are, and what thresholds to apply.
Re: A beginner-friendly introduction to Prometheus
#6I wonder if it should be used at all for beginners and smaller teams. I thought it is something like the new industry standard but even to get it running in production is not trivial and had the feeling this is something for teams who are deeply committed to monitoring and developing their own metrics, views and alarms. My conclusion is the majority (or at least smaller teams) are maybe better served with something l…
IMHO, Prometheus is not complicated. In fact, it was designed to be simple and dumb. I mean, it does not provide any production grade scalability out of box, and people have to implement their own add-ons and external storage solutions (thanos, cortex, M3DB...) to make Prometheus even usable in big clusters. Just because you can buy DataDog and be done doesn't mean the paid product itself is easy to understand.
What I like most about Prometheus, are PromQL, pull-based model, and its open source ecosystem. I use it both at work and in my homelab (which is just a bunch of Intel NUC bare metal boxes), and it has been great. I can't speak for any other individuals or companies, but there are tons of learning materials and active community where you can get help.
Also, when it comes down to application level metrics, you have to educate your developers to add instrumentation in their own code anyways. And I think exposing a small HTTP server is far easier than pushing data to a remote server.
Re: A beginner-friendly introduction to Prometheus
#7I wonder if it should be used at all for beginners and smaller teams. I thought it is something like the new industry standard but even to get it running in production is not trivial and had the feeling this is something for teams who are deeply committed to monitoring and developing their own metrics, views and alarms. My conclusion is the majority (or at least smaller teams) are maybe better served with something l…
I never found Prometheus that difficult. I would say I've spent a moderate amount of time over the past few years reading and using it (almost exclusively just reading the docs and blog). I haven't used Datadog. The places I've used Prometheus tended not to use Saas, which meant trying to get the wallet opened or approval for sending data offsite were basically non-starters. I know that's not an issue for most here. I set it up as a side project because we were basically blind and reactive to issues. The other options I was looking at was Nagios and Cacti.
Machine metrics are basically plug-and-play. The only real downside with Prometheus is that you have to install the pieces yourself (and know enough to do it). After you understand how the pieces fit together and get basic knowledge of each piece the time suck is writing and grokking new metrics and noodling with a dashboard--which I figure any solution would have.
This is probably home-hobby geekery, but I'm running Prometheus at home. My Synology has Prometheus and Grafana running in docker containers. The only gnarly part was getting dd-wrt's SNMP data and creating a dashboard of router metrics.
Re: A beginner-friendly introduction to Prometheus
#8This doesn’t seem to touch the subject of Promgen (or other Prometheus configuration management software that I’m not aware of). Configuring it via yaml files, especially the alerts in alert manager and queries in the sql exporter, requires more steps than I’d like to do. Any recommendations on how to manage Prometheus configs? Especially having multiple Prometheus instances where the master node scraps all the data from other nodes
Re: A beginner-friendly introduction to Prometheus
#9This doesn’t seem to touch the subject of Promgen (or other Prometheus configuration management software that I’m not aware of). Configuring it via yaml files, especially the alerts in alert manager and queries in the sql exporter, requires more steps than I’d like to do. Any recommendations on how to manage Prometheus configs? Especially having multiple Prometheus instances where the master node scraps all the data…
That seems like advanced usage and would be outside the scope of the blog post.
Maybe ask over at https://reddit.com/r/PrometheusMonitoring
Re: A beginner-friendly introduction to Prometheus
#10Are there any good push based monitoring alternative to Prometheus.