Live data from Hacker News

Red Engine: modern scheduling framework for Python applications

red-engine.readthedocs.io

31–40 of 55 posts

Re: Red Engine: modern scheduling framework for Python applications

#31
post #27
post #22

Earlier quoted context omitted.

If it took less than 5 minutes to boot then I can't see why it wouldn't work. How would that work for other schedulers? Also, if a server reboots that's quite bad all round anyway. Hopefully you'd be notified directly.

Other schedulers have a durable database of attempted runs. This doesn't seem to have anything like that.

Would you have a recommendation for an easy to use Python scheduler with such a feature for a personal project?

Re: Red Engine: modern scheduling framework for Python applications

#32

If you're going to build a framework like this, please please put your consistency behavior and retry behavior at the top of your docs.

Yes, 100% this. Too many frameworks claim that they are "easy" just to find out they left out things that other older frameworks have already solved. While we are on this: Do you know of any task scheduler framework that is similar to celery, but has better guarantees around task execution than what acks_late= True gives you? I always find myself building a system that stores the really important Tasks in Postgres so…

If your tasks are idempotent, Dramatiq is intended for your case.

https://dramatiq.io/

Re: Red Engine: modern scheduling framework for Python applications

#33
post #21

I'm using APScheduler for most of my scheduling tasks. Does Red Engine integrate with asyncio? When searching the docs for this keyword no hits showed up.

Going to shamelessly plug Temporal’s Python SDK which was designed for asyncio.

https://github.com/temporalio/sdk-python

Disclaimer: I work for Temporal

Re: Red Engine: modern scheduling framework for Python applications

#34

Not to be confused with REDengine by CD Projekt Red, used to make the Witcher and Cyberpunk 2077 games [0]. They're moving to Unreal Engine 5 though. [0] https://witcher.fandom.com/wiki/REDengine

Yeah, when I read the title I was like: "wtf, CDPR released Python version of their engine?"

Re: Red Engine: modern scheduling framework for Python applications

#35
post #34

Not to be confused with REDengine by CD Projekt Red, used to make the Witcher and Cyberpunk 2077 games [0]. They're moving to Unreal Engine 5 though. [0] https://witcher.fandom.com/wiki/REDengine

Yeah, when I read the title I was like: "wtf, CDPR released Python version of their engine?"

I too first thought this had something to do with RED Engine 4 from CDPR.

Unfortunately CDPR's latest RED Engine 4 will not be released as open source in the forseeable future, it's basically dead and locked away, and will probably never see the light of day. Maybe it would be open sourced but in something like 20 years time.

Re: Red Engine: modern scheduling framework for Python applications

#36

This fits my use case so perfectly! I have a very small internal app taking care of organizing seminar talks, calendars, email announcements, recordings of the talks, and signups. It is a single python file of less than 1500 lines and an sqlite database. This library is so perfect for taking care of scheduled events. Everything else I have found is ridiculously over-complicated of a solution.

Can you share repo to your application?

Re: Red Engine: modern scheduling framework for Python applications

#37
post #22

How does it handle state and restarts? What happens if a job is scheduled to run "before 10am", then the entire server restarts at 9:55am, will it try to run that same job again when it boots back up?

If it took less than 5 minutes to boot then I can't see why it wouldn't work. How would that work for other schedulers? Also, if a server reboots that's quite bad all round anyway. Hopefully you'd be notified directly.

I think on the contrary, it's important to build software that's resilient to random reboots: you can't control things like losing power or your CPU failing, so the less toil you have to manage around that the better.

Re: Red Engine: modern scheduling framework for Python applications

#38

  from redengine import RedEngine
  
  app = RedEngine()
  
  @app.task('daily')
  def do_things():
      ...
  
  if __name__ == "__main__":
      app.run()
> We initialized the RedEngine application, created one task which runs every 10 seconds and then we started the app. https://red-engine.readthedocs.io/en/stable/tutorial/quick_s...

Might wanna fix that.

Re: Red Engine: modern scheduling framework for Python applications

#39
post #27

Earlier quoted context omitted.

Other schedulers have a durable database of attempted runs. This doesn't seem to have anything like that.

Would you have a recommendation for an easy to use Python scheduler with such a feature for a personal project?

Seconding this. I would love a middle tier job scheduler / manager for Python that has persistence. I feel like there's a missing middle between cron+scripts and the enterprise grade tooling built for ETL tasks.

Re: Red Engine: modern scheduling framework for Python applications

#40
post #27

Earlier quoted context omitted.

Other schedulers have a durable database of attempted runs. This doesn't seem to have anything like that.

Would you have a recommendation for an easy to use Python scheduler with such a feature for a personal project?

APScheduler
Post reply on HN