Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Job Scheduling as a service

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 42 posts

Re: Ask HN: Job Scheduling as a service

#11

Post by mani-smhack »

caraga wrote: post #9
mani-smhack wrote: post #8

Earlier quoted context omitted.

I need a scheduler where i can dynamically ( via api ) schedule a job , with the admin UI to manage the jobs . I Think these are not part of application development and it can be offloaded to hosted services .

Jenkins has scheduled jobs and an UI. It can also be accessed via API. https://jenkins.io/

This is not with respect to devops or build .

My app has schedule functionality for example a scheduled report , so if it where like a webhook call then i dont want to waste the cpu cycle ,now i am using quartz scheduler in my java app .

Re: Ask HN: Job Scheduling as a service

#12

Post by stephenr »

mani-smhack wrote: post #7
cdoxsey wrote: post #5

Google app engine has a cron service: https://cloud.google.com/appengine/docs/standard/python/conf... You'll have to create an app handler for the URL which can then make the outbound http request. It's inexpensive, reliable and has an admin interface and logs.

Yeah this works for cron jobs . Let me explain my requirement , say i have social media platform and users will schedule their post , and i need to schedule it in my scheduler and i have to persist in the databsae too . Because if the server is restarted all my scheduled jobs will be gone .. So i am looking for a hosted scheduler where i can send a rest API request with a webhook and a small payload . During the exec…

This sounds like a terrible way to achieve your goal.

Save the future posts to the database with the required publish date/time.

Run cron every $X minutes where $X is the smallest interval you allow (ie if you force publish date to be rounded to 5 minutes, run cron every 5 minutes) and check for "future posts" where the publish date This is pretty basic stuff and that you want to offload it to a 3rd party by capturing all the details into a request to be made back to your app is a huge red flag IMO.

Re: Ask HN: Job Scheduling as a service

#13

Post by stephenr »

ddorian43 wrote: post #10
stephenr wrote: post #2

You want cron, as a service? Shit on a stick doesn't anyone actually host anything themselves anymore?

But cron isn't highly available/distributed with consistent state/scheduling

Run cron on all your boxes, have them pull a job at a time from the database. Or shit, go full tilt with a clustered Redis job queue.

All of these are better options than "embed the entire job info in a URL and hand it off to a remote scheduling service".

Re: Ask HN: Job Scheduling as a service

#15

Post by scarface74 »

Nomad by Hashicorp. You have to do just a little legwork to get it to call an API - just schedule a job that calls the appropriate curl command. When I'm explaining it to people, I explain it is a "distributed cron". You schedule a job either using the command line or via an API to the Nomad server and then it runs the job on any box that has the Nomad agent running and registered to the server. If you combine Nomad with Consul, configuration is dead simple.

https://www.nomadproject.io

Re: Ask HN: Job Scheduling as a service

#16

Post by scarface74 »

stephenr wrote: post #13
ddorian43 wrote: post #10

Earlier quoted context omitted.

But cron isn't highly available/distributed with consistent state/scheduling

Run cron on all your boxes, have them pull a job at a time from the database. Or shit, go full tilt with a clustered Redis job queue. All of these are better options than "embed the entire job info in a URL and hand it off to a remote scheduling service".

http://mikehadlow.blogspot.com/2012/04/database-as-queue-ant...

Re: Ask HN: Job Scheduling as a service

#17

Post by scarface74 »

mani-smhack wrote: post #7
cdoxsey wrote: post #5

Google app engine has a cron service: https://cloud.google.com/appengine/docs/standard/python/conf... You'll have to create an app handler for the URL which can then make the outbound http request. It's inexpensive, reliable and has an admin interface and logs.

Yeah this works for cron jobs . Let me explain my requirement , say i have social media platform and users will schedule their post , and i need to schedule it in my scheduler and i have to persist in the databsae too . Because if the server is restarted all my scheduled jobs will be gone .. So i am looking for a hosted scheduler where i can send a rest API request with a webhook and a small payload . During the exec…

Because if the server is restarted all my scheduled jobs will be gone ..

I went into more details about Hashicorp's Nomad in another post, but I didn't mention that you can run a cluster of Nomad servers for high availability.

Nomad is open source - I don't have a dog in the fight - I'm just a happy user.

Re: Ask HN: Job Scheduling as a service

#18

Post by scarface74 »

stevekemp wrote: post #4

There are a few online tools, such as: https://www.easycron.com/ https://www.setcronjob.com/ etc. You might be as well writing your own though, since it shouldn't be a difficult task.

I couldn't agree more, I am a Dev lead with a mandate to "outsource everything possible" When it comes to infrastructure type stuff, but even he would look at me crazy for outsourcing job scheduling.

Re: Ask HN: Job Scheduling as a service

#19

Post by stephenr »

scarface74 wrote: post #16
stephenr wrote: post #13

Earlier quoted context omitted.

Run cron on all your boxes, have them pull a job at a time from the database. Or shit, go full tilt with a clustered Redis job queue. All of these are better options than "embed the entire job info in a URL and hand it off to a remote scheduling service".

http://mikehadlow.blogspot.com/2012/04/database-as-queue-ant...

The user doesn't need a queue though. They just need a process to find posts with a publish date of <= now that aren't published and publish them. That's a one line indempotent sql update statement in any decently architected application.
Post reply on HN