A Scheduler for the Internet

#1

Post by jbredeche » Tue, Oct 08, 2019, 1:04 PM UTC

Re: A Scheduler for the Internet

#2

Post by blowski » Thu, Oct 10, 2019, 4:33 PM UTC

I don't really understand. What could I do with this that I couldn't do with cron? Or is it that somebody who can't use cron could use this? Or that this is more reliable than cron?

Re: A Scheduler for the Internet

#3

Post by brycesbeard » Thu, Oct 10, 2019, 4:41 PM UTC

What’s the context for this? Seems like it’s a marketing piece to get people to sign up for their fake scarcity early access program.

Very little interesting in the article.

Re: A Scheduler for the Internet

#4

Post by [deleted] » Thu, Oct 10, 2019, 5:09 PM UTC

blowski wrote: post #2

I don't really understand. What could I do with this that I couldn't do with cron? Or is it that somebody who can't use cron could use this? Or that this is more reliable than cron?

[deleted]

Re: A Scheduler for the Internet

#5

Post by glial » Thu, Oct 10, 2019, 5:10 PM UTC

How does this compare to Airflow? It's unclear from the website - can you create DAGs (task sequences)?

Re: A Scheduler for the Internet

#6

Post by slap_shot » Thu, Oct 10, 2019, 5:17 PM UTC

blowski wrote: post #2

I don't really understand. What could I do with this that I couldn't do with cron? Or is it that somebody who can't use cron could use this? Or that this is more reliable than cron?

While cron is a nice tool, people should not be using cron for any production grade automation.

Tools have been created to that orchestrate automation with more powerful capabilities, and Airflow has become a defacto standard in the industry.

Prefect is an open automation tool, comparable in a lot of ways to Airflow. The author of the post is an early committer to Airflow.

Re: A Scheduler for the Internet

#7

Post by slap_shot » Thu, Oct 10, 2019, 5:20 PM UTC

glial wrote: post #5

How does this compare to Airflow? It's unclear from the website - can you create DAGs (task sequences)?

This post is focusing on Scheduler, but the parent project, Prefect, has the concept of Flows, which are similar to DAGs. You can convert an Airflow project into a Prefect project.

One of their employees wrote about the comparison of the two:

https://medium.com/the-prefect-blog/why-not-airflow-4cfa4232...

Re: A Scheduler for the Internet

#8

Post by ahbyb » Thu, Oct 10, 2019, 5:55 PM UTC

slap_shot wrote: post #6

blowski wrote: post #2

I don't really understand. What could I do with this that I couldn't do with cron? Or is it that somebody who can't use cron could use this? Or that this is more reliable than cron?

While cron is a nice tool, people should not be using cron for any production grade automation. Tools have been created to that orchestrate automation with more powerful capabilities, and Airflow has become a defacto standard in the industry. Prefect is an open automation tool, comparable in a lot of ways to Airflow. The author of the post is an early committer to Airflow.

>While cron is a nice tool, people should not be using cron for any production grade automation.

?????????????????????????

Re: A Scheduler for the Internet

#9

Post by jlv2 » Thu, Oct 10, 2019, 6:03 PM UTC

ahbyb wrote: post #8

slap_shot wrote: post #6

Earlier quoted context omitted.

While cron is a nice tool, people should not be using cron for any production grade automation. Tools have been created to that orchestrate automation with more powerful capabilities, and Airflow has become a defacto standard in the industry. Prefect is an open automation tool, comparable in a lot of ways to Airflow. The author of the post is an early committer to Airflow.

>While cron is a nice tool, people should not be using cron for any production grade automation. ?????????????????????????

Because many decades of stability based upon simplicity are not good enough for today's world.

Re: A Scheduler for the Internet

#10

Post by wuliwong » Thu, Oct 10, 2019, 6:10 PM UTC

ahbyb wrote: post #8

slap_shot wrote: post #6

Earlier quoted context omitted.

While cron is a nice tool, people should not be using cron for any production grade automation. Tools have been created to that orchestrate automation with more powerful capabilities, and Airflow has become a defacto standard in the industry. Prefect is an open automation tool, comparable in a lot of ways to Airflow. The author of the post is an early committer to Airflow.

>While cron is a nice tool, people should not be using cron for any production grade automation. ?????????????????????????

My understanding of why using cron for background tasks with web apps isn't great in production is keeping track of things when you have distributed servers. I think cron is pretty limited in scope to whatever machine it is running on.