Live data from Hacker News

Monitoring My Homelab, Simply

b.tuxes.uk

51–60 of 70 posts

Re: Monitoring My Homelab, Simply

#53

Earlier quoted context omitted.

Why is the pull model bollocks? I’ve been building monitoring for stuff since nagios and zabbix were the new hot tools; and I can’t really imagine preferring the oldschool ways vs the pretty much industry standard of promstack these days…

Zabbix is bollocks. so is nagios. Having remote root access to all your stuff is utter shite. Prometheus as a time series DB is great, I even like its QL. What I don't like is pull. Sure there is agent mode or telegraf/grafana agent. But the idea that I need to hold my state and wait for Prometheus to collect it is utterly stupid. The biggest annoyance is that I need to have a webserver somewhere, with a single god i…

AFAIK, you can now send OpenTelemetry directly to Prometheus. So effectively it supports both push and pull models.

Re: Monitoring My Homelab, Simply

#54
post #42

I’ve been facing a similar search for an ultra-simple but ultra-extensible monitoring solution for my homelab. I’ve had the idea to write a Python program where the main script is just responsible for scheduling and executing the checks, logging, and alerting based on set thresholds. All monitoring would be handled via plugins, which would be extremely easy to write. It would ship with a few core plugins (ping, http,…

You've described this: https://munin-monitoring.org/

This looks fabulous, thank you!

Re: Monitoring My Homelab, Simply

#55
post #38
post #16

My solution is to just be OK with http status checking (run a webserver on important machines), and use a service like updown.io which is so cheap it's almost free. e.g. For 1 machine, hourly checking is ~$0.25/year

Do you do regular backups? If your backup system breaks and stop making new backups, what will let you know? What if your RAID is failing, running out of space, remounted read-only after an error? I have found that "machine is online" is usually not what I need monitoring for, at all. I'll notice if it's down. It's all the mission-critical-but-silently-breakables that I bother to monitor.

updown.io also has a relatively new feature called cron monitoring[0] that allows you to regularly check in to signal success. If there has been no check-in in a configured time it will alert you. For backups you could add a simple curl somewhere into your backup process to do just that.

[0] https://updown.io/doc/how-pulse-cron-monitoring-works

Re: Monitoring My Homelab, Simply

#56

Uptime Kuma ( https://github.com/louislam/uptime-kuma ). With email notifications. So much simpler, and free.

Last time I checked it lacked real API or a way to be configured via config files.

I find Gatus much more thought through.

- https://github.com/TwiN/gatus

Re: Monitoring My Homelab, Simply

#57

Just love https://healthchecks.io I set it up on all my boxes with these two scripts: win https://github.com/reboot81/healthchecks_service_ps/ macos https://github.com/reboot81/hc_check_maker_macos linux https://healthchecks.io/docs/bash/

[deleted]

Re: Monitoring My Homelab, Simply

#58

Grafana Cloud have quite a nice free tier. If you use Alloy, you don't need any persistence locally. Everything just gets shipped off. Similar to the author, I want to run a minimalist monitoring setup and currently just use Glances. But Grafana Cloud might be my first choice if I need to expand the setup.

I was using the Grafana Cloud free tier for a couple years, then they made some changes and started yelling at me about alerts I had set up based on Loki log searches. A new metric appeared in the usage dashboard...there were now limits for how I access the logs I've stored, on top of the limits for how much log data I could store.

I did figure out some ways to reduce the log query usage of the alerts and made the "you need to upgrade to a paid tier!" notices stop. Still, the experience was the straw that broke the camel's back. I'd already been getting somewhat frustrated by the 2 week retention and 10 dashboard limit.

FWIW, it wasn't too difficult to stand up the Docker containers for Grafana, Loki, and Prometheus for my own usage.

Re: Monitoring My Homelab, Simply

#59
post #56

Uptime Kuma ( https://github.com/louislam/uptime-kuma ). With email notifications. So much simpler, and free.

Last time I checked it lacked real API or a way to be configured via config files. I find Gatus much more thought through. - https://github.com/TwiN/gatus

Not that I think that an API is fundamental to a monitoring tool, but it happens to have one: https://github.com/louislam/uptime-kuma/wiki/API-Documentati...

Re: Monitoring My Homelab, Simply

#60
post #9
post #6

Earlier quoted context omitted.

One cold also just wait for things to not work before to try and fix them

For backups that's usually not the best strategy.

Assuming backups are the route you want to go, it all depends on how you use your "homelab". To some people homelab is more cattle than pet and it's easier to just hit redeploy rather than restore. To others homelab means the place they store their family photos, run game servers for friends, or do their work from.

Because of the wide breadth of what a homelab can mean it's really hard to make universal statements about what is always good. By the style of the article the author probably wants backups & failure notifications in some way if it's not already covered outside of their custom monitoring.

Post reply on HN