Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Executing Cron Scripts Reliably at Scale

slack.engineering

Post reply on HN
Print view View original

11–20 of 100 posts

Re: Executing Cron Scripts Reliably at Scale

#11

Post by xyzzy123 »

Interesting they went straight from "1 box running shell scripts with flock" to "mega custom thing" w/out going thru something like kube crob jobs or an off-the-shelf scheduler in between.

I find jumps like this hint at "political stiction", sometimes it's hard to get permission to do incremental updates to things, you have to wait until the smoke from the burning tires is unmissable, then get big political consensus and a "visible project" to allocate budget and time for what would otherwise be unsexy maintenance work.

Re: Executing Cron Scripts Reliably at Scale

#13

Post by djboz »

What's the workflow for job registration look like? Does an engineer upload a single script/binary to some storage service(e.g., EFS/S3/whatever) or builds and uploads a container with script/binary, registers the rate in the conductor UI and then the job queue just pulls the script or container and runs it? Is registration through an IaC of sorts, or just through some UI/CLI?

Re: Executing Cron Scripts Reliably at Scale

#14

Post by nerdponx »

jaydeegee wrote: post #7
swyx wrote: post #5

this is a pretty simple cron system. curious if the authors investigated temporal and other similar workflow engines for the advanced cron feature set ( https://docs.temporal.io/workflows#spec disclaimer: i used to work there)

100% agree this is a perfect use case for Cadence or Temporal.

Coming from the "data world", how does a tool like Airflow/Dagster/Prefect differ from these?

Re: Executing Cron Scripts Reliably at Scale

#15

Post by lijok »

xyzzy123 wrote: post #11

Interesting they went straight from "1 box running shell scripts with flock" to "mega custom thing" w/out going thru something like kube crob jobs or an off-the-shelf scheduler in between. I find jumps like this hint at "political stiction", sometimes it's hard to get permission to do incremental updates to things, you have to wait until the smoke from the burning tires is unmissable, then get big political consensus…

Is that not what HN at large keeps on talking about doing more - stretch the thing you use to its absolute limits before you upgrade

Re: Executing Cron Scripts Reliably at Scale

#16

Post by noptd »

xyzzy123 wrote: post #11

Interesting they went straight from "1 box running shell scripts with flock" to "mega custom thing" w/out going thru something like kube crob jobs or an off-the-shelf scheduler in between. I find jumps like this hint at "political stiction", sometimes it's hard to get permission to do incremental updates to things, you have to wait until the smoke from the burning tires is unmissable, then get big political consensus…

Yeah, either that or "architecting something new looks better on my resume than using an existing solution."

Re: Executing Cron Scripts Reliably at Scale

#17

Post by nicksaroha »

swyx wrote: post #5

this is a pretty simple cron system. curious if the authors investigated temporal and other similar workflow engines for the advanced cron feature set ( https://docs.temporal.io/workflows#spec disclaimer: i used to work there)

Wouldn’t it be a huge overkill to run your own temporal or airflow instances just to run your cron jobs? Just curious.

Re: Executing Cron Scripts Reliably at Scale

#18

Post by xyzzy123 »

lijok wrote: post #15
xyzzy123 wrote: post #11

Interesting they went straight from "1 box running shell scripts with flock" to "mega custom thing" w/out going thru something like kube crob jobs or an off-the-shelf scheduler in between. I find jumps like this hint at "political stiction", sometimes it's hard to get permission to do incremental updates to things, you have to wait until the smoke from the burning tires is unmissable, then get big political consensus…

Is that not what HN at large keeps on talking about doing more - stretch the thing you use to its absolute limits before you upgrade

Fair, tho a load bearing cronjob pet at the beating heart of a $20bn company with 30m+ users is further than I would dare take this advice if I had to carry the pager. It's very common though and I guess the blog is more evidence that simple things (greased by some tears and toil) can take you much further than you might think.
Post reply on HN