Red Engine: modern scheduling framework for Python applications
red-engine.readthedocs.io
Red Engine: modern scheduling framework for Python applications
1–10 of 55 posts
Re: Red Engine: modern scheduling framework for Python applications
#2What makes this true?
“Red Engine is not meant to be the scheduler for enterprise pipelines, unlike Airflow, but it is fantastic to power your Python applications.”
Re: Red Engine: modern scheduling framework for Python applications
#3> Clean: Scheduling is just plain English
Ugh, no thanks.
First of all, English itself is not clean; it's a messy amalgamation of special cases and inconsistent spelling rules.
Second, it isn't actually English anyway. It might look like English, but it's actually a DSL that happens to correspond to English a lot of the time. English text is meant to be interpreted by humans, who understand context & connotation, and can resolve ambiguities by making educated guesses or discussing the text with other humans. But your English-like DSL can only be interpreted by a computer program, which cannot (and arguably should not) do such things. Ergo, the benefits of using natural language are lost, and you are left with the same strict interpretation rules as any other programming language, but without any of the syntactic rigor that would normally help you construct programs/expressions that are both syntactically correct and also do what you intended them to do. Finally, the passing similarity to another language is a newbie trap and it makes teaching more difficult. See also: SQL, Python.
Worse still, it's represented in code as a string literal. It cannot be reasonably syntax-highlighted or otherwise statistically analyzed, nor can it be easily constructed dynamically if needed, nor can scheduling primitives be combined or composed. It is the worst of all worlds, and you have no way to check if your program is valid other than to run it and see if it crashes. And you have to re-learn operator precedence / associativity rules, because they probably won't be identical to the rules in Python itself.
I'm sorry if there is a really high quality scheduling engine underneath this DSL, but I absolutely would never want to use something like this in production code.
(I'm sure you can guess how I feel about BDD frameworks and "expect.foo.to.be.equal.to" style test APIs).
Re: Red Engine: modern scheduling framework for Python applications
#4Looks nice but very limited.
I don’t see any mention to serialization, so I guess this is single-server and memory only. I also couldn’t find any mention to error handling or retries.
Re: Red Engine: modern scheduling framework for Python applications
#5Not to be confused with REDengine by CD Projekt Red, used to make the Witcher and Cyberpunk 2077 games [0]. They're moving to Unreal Engine 5 though.
Re: Red Engine: modern scheduling framework for Python applications
#6> Clean: Scheduling is just plain English Ugh, no thanks. First of all, English itself is not clean; it's a messy amalgamation of special cases and inconsistent spelling rules. Second, it isn't actually English anyway. It might look like English, but it's actually a DSL that happens to correspond to English a lot of the time. English text is meant to be interpreted by humans, who understand context & connotation, and…
@app.task('daily & is foo', execution="process")
@app.task("after task 'do_daily'")
Similar but not quite Python combined with similar but not quite English does not make a tasty dish. It makes yet another pointless one-off thing to learn and struggle with.
Re: Red Engine: modern scheduling framework for Python applications
#7As cool as it is, scheduled jobs (and jobs in general) should really be isolated from the rest of your system and from each other to limit the blast radius. Jobs are notorious for crashing and backing up, so you really don't want that impacting other systems on the same machine.
Clouds have managed systems for scheduled jobs now (AWS Eventbridge, GCP Cloud Scheduler) which handle this for you.
Re: Red Engine: modern scheduling framework for Python applications
#8> Clean: Scheduling is just plain English Ugh, no thanks. First of all, English itself is not clean; it's a messy amalgamation of special cases and inconsistent spelling rules. Second, it isn't actually English anyway. It might look like English, but it's actually a DSL that happens to correspond to English a lot of the time. English text is meant to be interpreted by humans, who understand context & connotation, and…
The mistake of trying to make programming languages look like English looks like it will be repeated forever. English is not a good language for expressing things specifically to computers or other humans for that matter.
Re: Red Engine: modern scheduling framework for Python applications
#9What makes this true? “Red Engine is not meant to be the scheduler for enterprise pipelines, unlike Airflow, but it is fantastic to power your Python applications.”
No post body was provided.
Re: Red Engine: modern scheduling framework for Python applications
#10> Clean: Scheduling is just plain English Ugh, no thanks. First of all, English itself is not clean; it's a messy amalgamation of special cases and inconsistent spelling rules. Second, it isn't actually English anyway. It might look like English, but it's actually a DSL that happens to correspond to English a lot of the time. English text is meant to be interpreted by humans, who understand context & connotation, and…
@app.task('daily & is foo', execution="process") followed by @app.task("after task 'do_daily'") Yeah, I did a hard turn towards "nope" right there. Similar but not quite Python combined with similar but not quite English does not make a tasty dish. It makes yet another pointless one-off thing to learn and struggle with.
These in particular remind me of "Baba Is You", which is probably not a good thing for an API.