Live data from Hacker News

Get notified when a periodic task doesn't run

deadmanssnitch.com

21–30 of 52 posts

Re: Get notified when a periodic task doesn't run

#21

Monit does this pretty easily http://mmonit.com/monit/ You can monitor web services, processes, file modified dates, directories, loads of stuff all with email alerts and a web-front end too.

All with a thing called Round Robin Database, a database format built for temporal data and used by every graphing solution known to the IT world.

Re: Get notified when a periodic task doesn't run

#22
post #12
post #7

Earlier quoted context omitted.

Agreed. I don't think my company would use this at the moment until it's proven service that has a good track record and a SLA (what's your up-time guarantee?). That said, I'd use it for my personal projects but not for $19/month. I'd pay about $1/month per monitor.

RE: SLA... the use case is more "I want to know sooner than later that something didn't happen" than "Something didn't happen this minute, I want to know NOW!". If and when we go down, your task won't be able to check in and we'll end up sending you a false "positive" which is better than not knowing at all when something fails.

^ yikes.

Re: Get notified when a periodic task doesn't run

#23
post #8

Earlier quoted context omitted.

What are you currently using to make sure your periodic tasks run? As far as making sure DMS runs, we use another third-party to make sure our checkers run. Contact support if you are interested in finding out more.

We have an elaborate system that someone cooked up a while ago that involves a tree of communicating Processes running on different machines that monitor our real time services. If we could consolidate our 'checker' processes into one that may be the right thing to do. Can the system be set up to use authentication to get behind firewalls or do hue things being monitored have to be accessible over the public Internet…

The interface is a simple HTTPS request. The authentication is done using a unique URL component. This is an example of how to use Dead Man's Snitch:

$ curl https://nosnch.in/c2354d53d2

Dead Man's Snitch does not ping your servers. It is the opposite way around. If your server stops pinging Dead Man's Snitch, you will be notified.

Re: Get notified when a periodic task doesn't run

#24
post #14
post #13

Earlier quoted context omitted.

The pricing is more based on value. It is the price my clients said they would be more than willing to pay to never have to panic again when there is some EC2 problem and the backups stopped working weeks ago. This actually happened over the weekend: https://twitter.com/tikaro/status/250286735064436736

I understand that. It's just not that valuable for my personal projects. I'd go further and say, that most of people willing to pay that much (serious about their backups and availability) already have some solution in place. The service is very appealing for people, who don't really care that much (because it's very easy to use). I'm afraid that those people won't pay $19/mo.

If I charged my clients $19.99 a month for something like this they'd laugh at me and hire someone else.

Re: Get notified when a periodic task doesn't run

#25
post #20

I'm sure this will make tons of money and will be amazingly useful as a hip tool for those developers we all know. But honestly the rest of us build this kind of stuff into our company dashboards without thinking twice. Where it belongs, in the hands of an internal team who can share the monitoring of more than $20 worth of "snitches".

Relevant username.

Re: Get notified when a periodic task doesn't run

#26
post #8

Earlier quoted context omitted.

What are you currently using to make sure your periodic tasks run? As far as making sure DMS runs, we use another third-party to make sure our checkers run. Contact support if you are interested in finding out more.

We have an elaborate system that someone cooked up a while ago that involves a tree of communicating Processes running on different machines that monitor our real time services. If we could consolidate our 'checker' processes into one that may be the right thing to do. Can the system be set up to use authentication to get behind firewalls or do hue things being monitored have to be accessible over the public Internet…

I'm not sure I understand, however, it works by your process reaching out to US to check in. I'd suppose you would maybe have to open up outbound traffic to our domain? I'd love to hear more. Feel free to email hi@deadmanssnitch.com.

Re: Get notified when a periodic task doesn't run

#27
post #16
post #12

Earlier quoted context omitted.

RE: SLA... the use case is more "I want to know sooner than later that something didn't happen" than "Something didn't happen this minute, I want to know NOW!". If and when we go down, your task won't be able to check in and we'll end up sending you a false "positive" which is better than not knowing at all when something fails.

Actually, false positives with alarms are really, really bad - I will start to ignore them very quickly if they're not reliable indicators of an actual problem. EDIT: If you have reasonable default tolerances or the ability to set tolerances on tasks, I'm pretty interested in trying it out - do you integrate or have plans to integrate with pagerduty, or do you simply fire off an email?

What I'm saying is, in the rare chance we are down and your service can't check in, that one time we would send a false positive. We wouldn't be flapping between off and on sending you a lot of false positives. The resolution on our checks is so large (the smallest is an hour) that we won't be flooding you with emails in any case.

We don't replace something like monit to make sure your process continues to run, we are validation that one-off periodic things run... things that are easy to forget about but are important.

Re: Get notified when a periodic task doesn't run

#28
post #20

I'm sure this will make tons of money and will be amazingly useful as a hip tool for those developers we all know. But honestly the rest of us build this kind of stuff into our company dashboards without thinking twice. Where it belongs, in the hands of an internal team who can share the monitoring of more than $20 worth of "snitches".

You fail to take into account the small team, whose only objective - or at least that which consumes 99.9% of the brain share - is getting a product out the door and keeping the user-facing part of it running.

Re: Get notified when a periodic task doesn't run

#29
post #27
post #16

Earlier quoted context omitted.

Actually, false positives with alarms are really, really bad - I will start to ignore them very quickly if they're not reliable indicators of an actual problem. EDIT: If you have reasonable default tolerances or the ability to set tolerances on tasks, I'm pretty interested in trying it out - do you integrate or have plans to integrate with pagerduty, or do you simply fire off an email?

What I'm saying is, in the rare chance we are down and your service can't check in, that one time we would send a false positive. We wouldn't be flapping between off and on sending you a lot of false positives. The resolution on our checks is so large (the smallest is an hour) that we won't be flooding you with emails in any case. We don't replace something like monit to make sure your process continues to run, we ar…

Yeah, I'm not terribly worried by that kind of failure, if that happens, a bunch of other things have already gone wrong. I would like something to keep track of all processes, though, that can back up monit. Good to know it's got some slack in the tolerances, I'll give it a try.
Post reply on HN