Amazon Launches New RabbitMQ Message Broker Service • Hacker News Board

Re: Amazon Launches New RabbitMQ Message Broker Service

#71

Post by whatsmyusername » Fri, Nov 06, 2020, 2:53 PM UTC

eloff wrote: post #65

whatsmyusername wrote: post #47

Earlier quoted context omitted.

They aren't. They're technically more correct but not always the practical best choice. RabbitMQ is a smart play as Rabbit is very easy to use, understand, and troubleshoot at the low end (which is where I suspect the vast majority of queue systems live). It also has a feature which is actually really hard to do (and sqs doesn't do). Guaranteed delivery of a message once . That was THE reason we never migrated to SQS…

> Guaranteed delivery of a message once That flies in the face of my distributed systems knowledge. It's not possible in some failure cases. If your acknowledgement of a message gets lost (because either server involved or the pipes in-between fail) you've processed the message already but the queue server will think you haven't. It either has to resend it (duplicate delivery) or it ignores acknowledgements all toget…

You are correct, a better description is that their path to 'deliver exactly once to the best of your ability' is clearer.

If I remember correctly SQS is hard limited to a fairly short timeout to requeue messages delivered but not acked. In rabbit it's much more configurable.

Also regular rabbit hosts support the kludge pattern of, 'just run one host and accept if it goes poof you can lose messages,' which is useful if you don't want to bother with the complexity of clustering or are on a shoe string budget.

Lastly you get a nice user interface with the management plugin and you can stand it up locally with docker compose (without depending on AWS for dev or any of the 'aws but on your laptop' solutions).

Re: Amazon Launches New RabbitMQ Message Broker Service

#72

Post by eloff » Fri, Nov 06, 2020, 4:44 PM UTC

whatsmyusername wrote: post #71

eloff wrote: post #65

Earlier quoted context omitted.

> Guaranteed delivery of a message once That flies in the face of my distributed systems knowledge. It's not possible in some failure cases. If your acknowledgement of a message gets lost (because either server involved or the pipes in-between fail) you've processed the message already but the queue server will think you haven't. It either has to resend it (duplicate delivery) or it ignores acknowledgements all toget…

You are correct, a better description is that their path to 'deliver exactly once to the best of your ability' is clearer. If I remember correctly SQS is hard limited to a fairly short timeout to requeue messages delivered but not acked. In rabbit it's much more configurable. Also regular rabbit hosts support the kludge pattern of, 'just run one host and accept if it goes poof you can lose messages,' which is useful…

Yeah, those are nice features to have. Plus you don't get the platform lock in.

Re: Amazon Launches New RabbitMQ Message Broker Service

#73

Post by nijave » Sat, Nov 07, 2020, 3:23 AM UTC

twelve40 wrote: post #60

awinder wrote: post #30

Earlier quoted context omitted.

AWS has sqs & kinesis which are much better queueing options in that scenario where you’re in AWS doing new dev and can pick a technology. This is more likely to do with opening doors for large and complex applications that can’t be rewritten to come into the AWS cloud. Unsexy stuff but there’s some fun engineering to be done in that realm, if you find like puzzles and shitshows to be fun anyways :-)

how are those much better? if you use SQS and you write your code for it, then you are stuck on a proprietary platform. Also, SQS is super-basic and actually requires a bunch of code to do anything beyond trivial - although yes, it seems reliable and well-supported, at least from my experience. I was actually really waiting for AWS to support Rabbit since it seems to hit the right combo of features, usability and pla…

If you're using a decent framework there's a good chance it already does most of the work for you. With Ruby on Rails, there's the ActiveJob abstraction which you can hook up to different backends like SQS or Redis with a few lines of code. In addition, AWS has a lot of out of the box integration with SNS and SQS for other services like Cloudwatch and Lambda (in fact, lambda you don't need any special code).

If you have a Lambda function processing SQS messages they just get dumped in your handler method and it your function runs successfully they get automatically removed from the q. If your lambda fails, the message reappears after the visibility timeout out subject to your redrive policy

Re: Amazon Launches New RabbitMQ Message Broker Service

#74

Post by nijave » Sat, Nov 07, 2020, 3:32 AM UTC

malux85 wrote: post #59

bot41 wrote: post #18

Cloud computing seems to be a winner take all scenario. For example, if you use AWS and need a message broker service then you'll use this. If you use Azure, you'll use their version. Development seems like just hooking up this components. I can't tell if this is a good thing or a bad thing.

It’s a bad thing - whenever you cannot make a choice you become a slave to tyranny. Infra lock-in means you’re a slave to their whims (financial, legal, competitivenes, whatever) This is justified by “less maintenance, and easier deployment” but the reality of the situation is, it’s not worth giving your freedom up for, and to a lesser degree - if your platform becomes popular, you end up spending the same amount of…

How many cases are you truely not locked into anything? Even if you host your own RabbitMQ you're still coupled to the software. I'm not sure being coupled to cloud service X is any worse than OSS product Y. For the latter, you tend to need to have more expertise to run it yourself.

At the end of the day, you can still rewrite your code and switch in both cases. You can end up in a tough spot if the OSS community loses interest in the software you've already bet your complicated app on, as well

Re: Amazon Launches New RabbitMQ Message Broker Service

#75

Post by bot41 » Sun, Nov 08, 2020, 11:59 AM UTC

ggoo wrote: post #19

bot41 wrote: post #18

Cloud computing seems to be a winner take all scenario. For example, if you use AWS and need a message broker service then you'll use this. If you use Azure, you'll use their version. Development seems like just hooking up this components. I can't tell if this is a good thing or a bad thing.

good - my job is really easy bad - my job is really boring

> good - my job is really easy

Longterm isn't this bad? For example, if you can do it with 5 years experience. At 15 years experience you'll likely be too expensive for companies to want to hire you. They'll just hire more junior people.

Re: Amazon Launches New RabbitMQ Message Broker Service

#76

Post by whatsmyusername » Mon, Nov 09, 2020, 7:53 PM UTC

eloff wrote: post #72

whatsmyusername wrote: post #71

Earlier quoted context omitted.

You are correct, a better description is that their path to 'deliver exactly once to the best of your ability' is clearer. If I remember correctly SQS is hard limited to a fairly short timeout to requeue messages delivered but not acked. In rabbit it's much more configurable. Also regular rabbit hosts support the kludge pattern of, 'just run one host and accept if it goes poof you can lose messages,' which is useful…

Yeah, those are nice features to have. Plus you don't get the platform lock in.

Normally I don't give a crap about vendor lock-in, but this would be an exception where I'd agree with you.

Though most people are just going to use a framework plugin to manage the messaging layer, so what's behind that is largely irrelevant.

Re: Amazon Launches New RabbitMQ Message Broker Service

#77

Post by pbz » Mon, Nov 09, 2020, 8:10 PM UTC

barumi wrote: post #58

pbz wrote: post #39

Earlier quoted context omitted.

And drowning in a sea of complexity...

I really doubt you have any real world experience with AWS if you try to sell it as a less complex alternative to anything,really. AWS is perhaps the most complex and arcane service provider ever, and is progressively getting worse by the way the service keeps growing and changing. And no, learning a specific AWS service is not a solid career investment. Tell that to anyone who tried to learn CloudFormation and then…

May wanna read that again...

Re: Amazon Launches New RabbitMQ Message Broker Service

#78

Post by twelve40 » Wed, Nov 18, 2020, 10:27 AM UTC

nijave wrote: post #73

twelve40 wrote: post #60

Earlier quoted context omitted.

how are those much better? if you use SQS and you write your code for it, then you are stuck on a proprietary platform. Also, SQS is super-basic and actually requires a bunch of code to do anything beyond trivial - although yes, it seems reliable and well-supported, at least from my experience. I was actually really waiting for AWS to support Rabbit since it seems to hit the right combo of features, usability and pla…

If you're using a decent framework there's a good chance it already does most of the work for you. With Ruby on Rails, there's the ActiveJob abstraction which you can hook up to different backends like SQS or Redis with a few lines of code. In addition, AWS has a lot of out of the box integration with SNS and SQS for other services like Cloudwatch and Lambda (in fact, lambda you don't need any special code). If you h…

I'm referring to things like topics, multiple consumers, routing, etc etc that are not even possible with SQS and once you grow into a need for those, SQS stops being adequate no matter what library you use for it.