Executing Cron Scripts Reliably at Scale
slack.engineering
Executing Cron Scripts Reliably at Scale
1–10 of 100 posts
Re: Executing Cron Scripts Reliably at Scale
#2[flagged]
Re: Executing Cron Scripts Reliably at Scale
#3[flagged]
What does a more sane sausage factory look like wrt cron or the like?
Re: Executing Cron Scripts Reliably at Scale
#4I'm still over here loving cron, not at this large of a scale for sure but I love getting all these tools to email me key information, in plain text, at the right times.
Re: Executing Cron Scripts Reliably at Scale
#5this is a pretty simple cron system. curious if the authors investigated temporal and other similar workflow engines for the advanced cron feature set (https://docs.temporal.io/workflows#spec disclaimer: i used to work there)
Re: Executing Cron Scripts Reliably at Scale
#6Interesting to see how different teams approaches to the same issue. Often when reading this sort of thing I'm in PR mode and question why they did certain things the way they did. Then I realise that has probably been asked and answered a dozen times before it got released.
Re: Executing Cron Scripts Reliably at Scale
#7this is a pretty simple cron system. curious if the authors investigated temporal and other similar workflow engines for the advanced cron feature set ( https://docs.temporal.io/workflows#spec disclaimer: i used to work there)
100% agree this is a perfect use case for Cadence or Temporal.
Re: Executing Cron Scripts Reliably at Scale
#8We have a slack reminder that executes on the hour, I’ve noticed that it take about 30 seconds to a minute past the hour for the reminder to actually fire.
Re: Executing Cron Scripts Reliably at Scale
#9this is a pretty simple cron system. curious if the authors investigated temporal and other similar workflow engines for the advanced cron feature set ( https://docs.temporal.io/workflows#spec disclaimer: i used to work there)
[deleted]
Re: Executing Cron Scripts Reliably at Scale
#10[flagged]
What does a more sane sausage factory look like wrt cron or the like?
It may not necessarily be "more sane" but we define any batch jobs like this in Nomad.
The benefits are:
- We can collect metrics and logs of the jobs to see how long they take start to finish, and whether they started/died early
- More robust scheduling to tolerate node failures
- Easier to deploy as IAC