Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Amqphosting, Managed RabbitMQ service

amqphosting.com

Post reply on HN
Print view View original

11–17 of 17 posts

Re: Show HN: Amqphosting, Managed RabbitMQ service

#11

Post by lobster_johnson »

xerxes901 wrote: post #10
lobster_johnson wrote: post #7

Earlier quoted context omitted.

Indeed. In my opinion RabbitMQ is essentially useless in clustered mode. When Rabbit recovers from a network partition and has to decide between multiple potential master versions of a queue, it picks the largest one to become the new master, and discards the others. It's rather mind-boggling that it can't merge them instead; after all, if your application is capable of handling duplicate deliveries, then merging (wh…

We run our RabbitMQ cluster with pause_minority as the partition handling strategy. This should eliminate most message loss on partition, no?

Yes. A bonus is that if you use it in combination with a health-check-capable proxy (HAProxy, Kubernetes), clients can be routed to any non-paused node automatically. In fact, you'll need that, since a paused node will close its ports.

That said, the devil is in the details; I'd be interested to know if RabbitMQ is capable of reliably detecting that it's in a minority. We've had issues where nodes are having issues talking to each other, but the problem is not consistent on both ends (e.g. A can talk to B, but B can't talk to A).

Re: Show HN: Amqphosting, Managed RabbitMQ service

#12

Post by RabbitmqGuy »

lobster_johnson wrote: post #7
zedpm wrote: post #4

My first question when I saw this was "How to you handle network partitions?", since RabbitMQ's partition handling is, uh, suboptimal. I read bullet points until I found this: > With our RabbitMQ servers, you wont have to deal with message loss in the event of a network partition. Reading on, I found that your answer to partition tolerance is to avoid the possibility of partitions by not supporting clustering at all.…

Indeed. In my opinion RabbitMQ is essentially useless in clustered mode. When Rabbit recovers from a network partition and has to decide between multiple potential master versions of a queue, it picks the largest one to become the new master, and discards the others. It's rather mind-boggling that it can't merge them instead; after all, if your application is capable of handling duplicate deliveries, then merging (wh…

Yes.

The only reason why we are not offering a clustering plan is because when you experience a network partition in such a case, you will loose data. And we do not want to be in a situation where we are explaining to our customers that there data has been lost through no fault of our own, but because that's the way RabbitMQ works.

We would rather have single node plans where failure modes are much easier to deal with.

Re: Show HN: Amqphosting, Managed RabbitMQ service

#13

Post by RabbitmqGuy »

zedpm wrote: post #4

My first question when I saw this was "How to you handle network partitions?", since RabbitMQ's partition handling is, uh, suboptimal. I read bullet points until I found this: > With our RabbitMQ servers, you wont have to deal with message loss in the event of a network partition. Reading on, I found that your answer to partition tolerance is to avoid the possibility of partitions by not supporting clustering at all.…

Hi.

What you have described is not negative at all. It is just what it is with RabbitMQ. I can remember the first time(many years ago) when I experienced my first RabbitMQ node split and I was so disappointed with it. But reading on their documentation, I was able to realise that, that is just the way RabbitMQ handles partitions.

> A managed service has to solve the hard problems(high availability) to be compelling.

We hear you. And are working very hard to try and come up with high availability solutions/plans that do not have the same failure modes that currently exist in RabbitMQ. But we believe that those problems need to be solved in RabbitMQ itself. So we are looking to see if we can come up with other failure handling modes, apart from the 3 outlined in[1]. We cant make any promises, however.

If you are already running your own node and are looking for HA solutions, then we might not be right for you, yet. But we still believe we can offer you value by taking over the headache of running the node from you so that you can concentrate on other things.

Thanks for checking us out!

1. https://www.rabbitmq.com/partitions.html

Re: Show HN: Amqphosting, Managed RabbitMQ service

#14

Post by RabbitmqGuy »

xerxes901 wrote: post #10
lobster_johnson wrote: post #7

Earlier quoted context omitted.

Indeed. In my opinion RabbitMQ is essentially useless in clustered mode. When Rabbit recovers from a network partition and has to decide between multiple potential master versions of a queue, it picks the largest one to become the new master, and discards the others. It's rather mind-boggling that it can't merge them instead; after all, if your application is capable of handling duplicate deliveries, then merging (wh…

We run our RabbitMQ cluster with pause_minority as the partition handling strategy. This should eliminate most message loss on partition, no?

Hi, even in pause_minority mode; when the node/s that was in the minority rejoin the cluster, then it will loose any messages that it had and assume the messages of the majority.

Re: Show HN: Amqphosting, Managed RabbitMQ service

#15

Post by RabbitmqGuy »

esdott wrote: post #5

Why not support TLS/amqps at all pricing levels? That's a huge turn off for me especially since you only have it at your highest pricing level. I'd also make that clear on your comparison page as it seems like you support amqps at the $55 level but do not on your pricing page. Good luck! (seriously, no sarcasm)

> as it seems like you support amqps at the $55 level but do not on your pricing page.

Hi, sorry about that. We'll fix the comparison page.

> Why not support TLS/amqps at all pricing levels?

When we started out, we were offering AMQPS/TLS for all plans through certificates from letsencrypt[1]. However, it became hard to manage certificate renewals, since we had to renew them on one machine and scp them to the respective RabbitMQ servers. This was too labour intensive and not worth. We however still have plans to roll out TLS for all plans at no extra cost.

1. https://letsencrypt.org/

Re: Show HN: Amqphosting, Managed RabbitMQ service

#17

Post by gerhardlazu »

lobster_johnson wrote: post #7
zedpm wrote: post #4

My first question when I saw this was "How to you handle network partitions?", since RabbitMQ's partition handling is, uh, suboptimal. I read bullet points until I found this: > With our RabbitMQ servers, you wont have to deal with message loss in the event of a network partition. Reading on, I found that your answer to partition tolerance is to avoid the possibility of partitions by not supporting clustering at all.…

Indeed. In my opinion RabbitMQ is essentially useless in clustered mode. When Rabbit recovers from a network partition and has to decide between multiple potential master versions of a queue, it picks the largest one to become the new master, and discards the others. It's rather mind-boggling that it can't merge them instead; after all, if your application is capable of handling duplicate deliveries, then merging (wh…

For a stable RabbitMQ cluster, you want dedicated RabbitMQ hosts with sufficient CPU, disk & network throughput for your workload. Most RabbitMQ users don't know what their workload is, or what their hardware boundaries are. We, the RabbitMQ team, should make this easier - and we will, in due course.

A good default cluster is 3 x r4.large with 100GB GP2 for RABBITMQ_MNESIA_BASE & pause_minority. For queues that need HA, a good default is ha-mode: exactly, ha-params: 2, ha-sync-mode: automatic. As for the Erlang version, we recommend 19.3.6.2 which has important fixes relevant for RabbitMQ. Today we recommend RabbitMQ 3.6.11, and 3.6.12 as soon as it ships.

In the past 6 months, I have been focusing on RabbitMQ stability and operability on AWS, GCP & vSphere. Can you tell me more about your RabbitMQ deployment lobster_johnson? This will help: https://s3-eu-west-1.amazonaws.com/rabbitmq-share/help-us-un...

I wouldn't mind moving this discussion to rabbitmq-users mailing list, so that it can benefit more in the RabbitMQ community.

Thanks, Gerhard

Post reply on HN