41–50 of 51 posts
[1] CELERY_SEND_TASK_ERROR_EMAILS config removed http://docs.celeryproject.org/en/latest/whatsnew-4.0.html#fe...
[2] https://gist.github.com/alanhamlett/dc8cdd4721ea63053f14#fil...
Earlier quoted context omitted.
I've managed to live without it. For low and medium traffic sites it's hugely over-engineered. For all the sites I manage, I run a single cron task that triggers a range of background jobs. It's an approach worked very well for nearly 10 years. I understand there's a genuine use-case for Celery but like many technologies people are told "if you need a task queue use this" when there are much simpler solutions that ar…
When building a small web app a few weeks ago, I tried to avoid additional dependencies, too. I was a bit stuck when I need to send an email asynchronous. Do you have a _simple_ recommendation to solve this without pulling something huge like celery+redis in to solve this task?
If you are on AWS, you could just use something like their email service to fire off the request, then you don't need a queue at all.
Earlier quoted context omitted.
We recently moved one of our services from Rq to Celery. We are using an older version of Celery, but this comment should still apply. While Rq is a great way to go in the start, if you have a large number of messages you are handling, you'll get into memory bottlenecks. This isn't a problem with Rq, but with using Redis as a broker. So if anyone is considering Rq vs Celery, they should keep this in mind. The reason…
That's very insightful. Quick question, since you are doing this in production - how are you serializing images into a message ? Base64 or something else.
Earlier quoted context omitted.
Celery was never meant as a replacement for cron, it was simply a nice bonus that fits the messaging pattern well. Writing a task queue is actually very simple using for example Redis, but that doesn't necessarily mean Celery is over-engineered IMHO. It's very easy to forget the support required once your system is in production. Disclaimer: I'm a contributor
I'm not saying Celery is over-engineered in general. It's just over-engineered in the context I've often seen it recommended in. i.e. for people learning or people with fairly modest requirements.
Its useful to know Celery, and gets used in a proper context in my current work, so I guess learning it wasn't a waste.
"Starting from Celery 5.0 only Python 3.5+ will be supported." This is a small detail, but I'm glad that one more big Python project is dropping 2.x support. The 2/3 split is not good for the community.
can you suggest a monitoring tool for celery? I want to view what tasks are currently working... is flower the only solution? thx
Worst experience with a python toolkit ever. I hope new versions fix all bugs and issues.
Honestly this is a complex problem area and I think the Celery developers have done an excellent job of making it pretty trivial to get up and running while providing lots of flexibility for more advanced users! Not an easy feat.
Are there bugs? Of course - but I've never come up against one that I can't work around. Is that annoying? Sometimes but that's software development.
Earlier quoted context omitted.
I've managed to live without it. For low and medium traffic sites it's hugely over-engineered. For all the sites I manage, I run a single cron task that triggers a range of background jobs. It's an approach worked very well for nearly 10 years. I understand there's a genuine use-case for Celery but like many technologies people are told "if you need a task queue use this" when there are much simpler solutions that ar…
When building a small web app a few weeks ago, I tried to avoid additional dependencies, too. I was a bit stuck when I need to send an email asynchronous. Do you have a _simple_ recommendation to solve this without pulling something huge like celery+redis in to solve this task?
You could add your emails to a db table and have a cron job consume them.
But for sending an occasional email? I've never really had a problem with just connecting to the SMTP server in the request/response cycle. If it takes more than a second to send then something is seriously wrong. You could use Mailgun or similar services which have their own queue for handling bulk sends and further reduce the likelihood of a problematic blocking of the web-server.