Show HN: Dramatiq – An alternative to Celery • Hacker News Board

Re: Show HN: Dramatiq – An alternative to Celery

#31

Post by smnrchrds » Mon, Nov 13, 2017, 7:08 AM UTC

friendzis wrote: post #30

smnrchrds wrote: post #25

Earlier quoted context omitted.

I am curious how the landscape of software development would change if everyone followed suit. I am not sure the effect would be a net positive. $2000 per year is inconvenient. $2000 per year for each and every one of the 20 open source projects you use will be prohibitively expensive. Companies would start writing more of their codes in house instead of using commercial open source software. There would be way fewer…

2k USD is somewhere between a man-week and a man-month. Hiring a developer to just reimplement existing solution is prohibitively expensive.

The idea is not that everyone would implement the entire functionality of every library they would otherwise use. With big libraries, each user may need 5% of the functionality. But not the same 5%. If the options were paying thousands of dollars or implementing your own, in many cases the latter would make sense. Definitely not in all cases, I admit.

The way I think about it is what if the only database solution was Oracle. I'm sure many would have wrote their own limited buggy DBMS or used flat files instead. It will certainly be more expensive to write your own Oracle RDBMS from scratch instead of paying for the licence. But you don't need most of the features and for the ones you need, the price point does not make sense for all projects. For some projects, flat files or JSON files saved somewhere on the disk wouldn't sound so bad.

Re: Show HN: Dramatiq – An alternative to Celery

#32

Post by scaryclam » Mon, Nov 13, 2017, 11:50 AM UTC

bjt wrote: post #12

RabbitMQ is not hard to use directly. I think most developers would be better off doing that, and coming to understand the power of AMQP, rather than adopting cute DSLs like this and Celery that put you into a box and narrow your view of what's possible. I started that way with Celery and Django, maybe 6 years ago. At one point we found a bug with Celery not resolving 'chord' callbacks when all the parallel tasks had…

I respect the authors of celery and think they've done a good job of making RabbitMQ a bit more accessible over the years. I don't think celery's just for novice users, there are a lot of devs out there who don't really need to invest in learning about their message brokers (short lived projects and non-essential parts of the larger system can easily use celery without needing to worry about it). That said, I really do agree with you on other points.

We ditched celery after many years of use in favor of just connecting to RabbitMQ ourselves. Over the last few years, it's saved a massive amount of time and effort and has made debugging and tracing issues so much easier it's rediculous.

I don't think many long term celery users really appreciate how much time is getting wasted. I understand how it may look like being able to ignore the underlying technology is a time saver, but that time easily gets blown away the second something doesn't behave as it should (and having gone through many of these problems, they're not exactly uncommon).

For anyone curious how much effort it took to write our own consumer/publisher, both are less that 200 lines of code and can be arguably cut down further without losing much (the author liked whitespace etc). Both were written in less than 2 days, including learning time, and are reusable enough to use in other projects.

Re: Show HN: Dramatiq – An alternative to Celery

#33

Post by moogie » Mon, Nov 13, 2017, 11:59 AM UTC

whalesalad wrote: post #15

bjt wrote: post #12

RabbitMQ is not hard to use directly. I think most developers would be better off doing that, and coming to understand the power of AMQP, rather than adopting cute DSLs like this and Celery that put you into a box and narrow your view of what's possible. I started that way with Celery and Django, maybe 6 years ago. At one point we found a bug with Celery not resolving 'chord' callbacks when all the parallel tasks had…

Seriously? RabbitMQ is a huge waste of complexity for like 95% of modern webapp use cases. I use tools like Dramatiq specifically because I want to abstract away the complexity of exchanges, bindings, queues, dead lettering, etc... If you write out all the nouns associated with RMQ you start to realize there is a LOT going on. It’s a very powerful system but convention over configuration doesn’t work here.

No offense, but this sounds a lot like "I can't be bothered to learn about the tech but I want to put it on my CV anyway".

The "95%" of use cases probably don't need a message broker at all. Far too many "modern webapp" developers are shoving message brokers into websites so they can pretend to be more important and sophisticated than they are. If you're not even willing to learn how to connect to RabbitMQ, or any broker, without a big giant messy crutch, I'd suggest you shouldn't use them at all.