Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Posthook – Job Scheduling as a Service

posthook.io

Post reply on HN
Print view View original

11–20 of 76 posts

Re: Show HN: Posthook – Job Scheduling as a Service

#11

Post by cgenuity »

chaosprophet wrote: post #10
cgenuity wrote: post #9

Earlier quoted context omitted.

It does not support recurring scheduling at the moment. Right now the retry logic is just one retry 5 seconds after the first failure. At which point the hook gets set to a failed status and failure notifications get sent out. Retries are tricky because depending on how the job is implemented they can cause more harm than good. So I plan to refine that more based on customer feedback.

What happens when you have a service outage? Do you directly mark the hooks as failed, or do you retry once after your service has been restored?

If the service outage is on Posthook's side, they would be retried.

If the outage is on the customer's side, all hooks that were attempted during the outage would be marked as failed. I plan on adding a feature that will allow the developer to fire off again all failed hooks in a given time period.

Re: Show HN: Posthook – Job Scheduling as a Service

#12

Post by sebringj »

This is a good niche with the rise of serverless as there isn't a super simple way to do this if you want to hit some job every so often. You could set a header key that all requests require and bam, you got security for an ongoing task. I use kue.js myself with jobs for node and glue stuff together with redis across servers but serverless seems like it could work for a lot of use cases coupled with this idea with less code overall.

Re: Show HN: Posthook – Job Scheduling as a Service

#13

Post by mcintyre1994 »

cgenuity wrote: post #2

Hello all! I built Posthook as a simple solution for web applications to schedule one-off tasks. It lets you schedule a request back to your application for a certain time, with an optional JSON payload. It can be an alternative to running your own systems like Sidekiq, Celery, or Quartz and the operational overhead that comes along with them. Cron jobs and cloud provider tools like CloudWatch Events are also used fo…

Neat! We've been doing something like this at work, and another simple solution (curious if your Google cloud infrastructure is built on something similar) is Azure Service Bus queues with a function app trigger.

Unlike AWS SQS, Azure SB queues can have items scheduled to be queued at an arbitrary time, see https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure..... SQS can only delay up to 15 minutes, so you have to implement your own hack to schedule for later than that.

And also unlike SQS, they can trigger a function so you don't have to deal with the logic to listen yourself - or even the transactional message handling, if the function fails the message is automatically re-queued.

Re: Show HN: Posthook – Job Scheduling as a Service

#15

Post by nodesocket »

cgenuity wrote: post #9
nodesocket wrote: post #6

Earlier quoted context omitted.

Does Posthook support recurring scheduling (i.e. every 15 minutes)? Also, is there retry logic? I.E. 3 retries, delay 60 seconds between retries?

It does not support recurring scheduling at the moment. Right now the retry logic is just one retry 5 seconds after the first failure. At which point the hook gets set to a failed status and failure notifications get sent out. Retries are tricky because depending on how the job is implemented they can cause more harm than good. So I plan to refine that more based on customer feedback.

Seems like allowing the user to specify retry logic (if any) covers the "tricky" pieces. Let the user define the number of retries and delay between.

Re: Show HN: Posthook – Job Scheduling as a Service

#16

Post by paddy_m »

I would be really interested in a distributed job running system with a scheduler. I want a UI where I can set jobs via a schedule that run on one of many workers. In that same UI I want to be able to look at failed jobs and see all of the output from that job. I don't want to have to ssh over to the box where the job ran to retrieve a log file. I once had a job where we used Tidal Enterprise Scheduler which poorly implemented this. What solutions exist for this workflow in the open source world?

Re: Show HN: Posthook – Job Scheduling as a Service

#17

Post by nodesocket »

nautical wrote: post #5
cgenuity wrote: post #2

Hello all! I built Posthook as a simple solution for web applications to schedule one-off tasks. It lets you schedule a request back to your application for a certain time, with an optional JSON payload. It can be an alternative to running your own systems like Sidekiq, Celery, or Quartz and the operational overhead that comes along with them. Cron jobs and cloud provider tools like CloudWatch Events are also used fo…

Hey cgenuity, Great to see the simplicity, I am working on a very similar product, launched couple of days back on HN ( https://send.rest ). Best of luck. But have added couple of extra features in send.rest 1) Calling external APIs ( Think : pull data from Facebook while running this task in future and post data on my API or send SMS ) 2) Reports and retries ( Try 3 times or send me sms its failing ) 3) Recurrence (…

With send.rest and using hook, can I specify a POST and POST data and custom request headers?

Re: Show HN: Posthook – Job Scheduling as a Service

#18

Post by Barreljack »

paddy_m wrote: post #16

I would be really interested in a distributed job running system with a scheduler. I want a UI where I can set jobs via a schedule that run on one of many workers. In that same UI I want to be able to look at failed jobs and see all of the output from that job. I don't want to have to ssh over to the box where the job ran to retrieve a log file. I once had a job where we used Tidal Enterprise Scheduler which poorly i…

Not quite distributed, but https://www.rundeck.com/open-source is pretty nifty.

Re: Show HN: Posthook – Job Scheduling as a Service

#19

Post by cgenuity »

mcintyre1994 wrote: post #13
cgenuity wrote: post #2

Hello all! I built Posthook as a simple solution for web applications to schedule one-off tasks. It lets you schedule a request back to your application for a certain time, with an optional JSON payload. It can be an alternative to running your own systems like Sidekiq, Celery, or Quartz and the operational overhead that comes along with them. Cron jobs and cloud provider tools like CloudWatch Events are also used fo…

Neat! We've been doing something like this at work, and another simple solution (curious if your Google cloud infrastructure is built on something similar) is Azure Service Bus queues with a function app trigger. Unlike AWS SQS, Azure SB queues can have items scheduled to be queued at an arbitrary time, see https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure.... . SQS can only delay up to 15 minutes, so you h…

Thanks! I do heavily use Google Cloud Pub/Sub but it does not support scheduling items so I only schedule items when they are due for processing.

Re: Show HN: Posthook – Job Scheduling as a Service

#20

Post by yzmtf2008 »

paddy_m wrote: post #16

I would be really interested in a distributed job running system with a scheduler. I want a UI where I can set jobs via a schedule that run on one of many workers. In that same UI I want to be able to look at failed jobs and see all of the output from that job. I don't want to have to ssh over to the box where the job ran to retrieve a log file. I once had a job where we used Tidal Enterprise Scheduler which poorly i…

Look into airflow: https://airflow.apache.org/
Post reply on HN