Ask HN: How to do simple heartbeat monitoring?

  #!/bin/bash

  # Add this script to cron to run at whatever duration you desire.

  # URL to be checked
  URL="https://example.com/test.php"

  # Email for alerts
  EMAIL="root@example.com"

  # Perform the HTTP request and extract the status code with 10 second timeout.
  STATUS=$(curl -o /dev/null -s -w "%{http_code}\n" --max-time 10 $URL)

  # Check if the status code is not 200
  if [ "$STATUS" -ne 200 ]; then
      # Send email alert
      echo "The URL $URL did not return a 200 status code. Status was $STATUS." | mail -s "URL Check Alert" $EMAIL

      # Instead of email, you could send a Slack/Teams/PagerDuty/Pushover/etc, etc alert, with something like:
      curl -X POST https://events.pagerduty.com/...
  fi

While my legal encumbrances prohibit helping you with actual code, I would recommend looking at watchdog processes.

For example, even a simple systemd periodic trigger that runs a script every minute that does general house keeping can work. i.e. a small script has the advantage of minimal library dependencies, fast/finite run state, and flexible behavior (checking cpu/network loads for nuisance traffic, and playing possum when a set threshold exceeded.)

Distributed systems is hard, but polling generally does not scale well (i.e. the cost of a self-check is locally fixed, but balloons on a cluster if going beyond a few fixed peer-checks).

And yeah, some clowns that think they are James Bond have been DDoS small boxes on Sun (they seem to be blindly hitting dns and ntp ports hard). We had to reset the tripwire on a 6 year old hobby host too.

tip, rate limiting firewall rules that expose whitelisted peers/admins to bandwidth guarantees is wise. Otherwise any cluster/heartbeats can be choked into a degraded state.

Don't take it personally, and enjoy a muffin with your tea. =3

I run multiple e-commerce websites where up time is critical and the types of errors could be anything. I use a service called Wormly that hits specific end points from multiple locations around the world. I'm not affiliated, just a happy customer.

1. Yes, a synthetic check is very useful here to just see if a user facing "thing" is still working.

2. a heartbeat check / deadman's switch can also work here, but it will only be reliable when the monitored event has a predefined cadence, e.g. "every 5 minutes this should happen".

3. The lack / absence of metrics flowing into a system is also sign. This would typically be solved by the Signoz team where they would alert on not seeing some specific event happening for x time. This can be tricky if the event is directly related to a user interaction.

Super big disclaimer: founder at a monitoring company that solve 1 and 2, not 3.