Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ØMQ – The Guide (2011)

zguide.zeromq.org

Post reply on HN
Print view View original

111–117 of 117 posts

Re: ØMQ – The Guide (2011)

#111

Post by OJFord »

hkon wrote: post #109
ComputerGuru wrote: post #101

Earlier quoted context omitted.

Forgive me as you seem to be aware of many of the details of zmq’s history but is there a chance you’re missing something? You are comparing a thin, semantic-enforcing layer just above TCP to actual message queues/servers featuring persistence to disk, etc. We’ve used ZeroMQ in production as an alternative to HTTP message passing and it was great for that. We would build something like RabbitMQ atop of ZMQ, the only…

Why would you build something similar to RabbitMQ ontop of ZeroMQ instead of just using Rabbit?

I believe GP means that ZMQ could be an implementation detail of RMQ (itself).

As in, in building your RabbitMQ competitor HareMQ, you might build on top of HareMQ. You can't just use HareMQ because you haven't built it yet.

Re: ØMQ – The Guide (2011)

#112

Post by Doman »

tsimionescu wrote: post #110
neurostimulant wrote: post #85

Earlier quoted context omitted.

The ZeroMQ Guide has a chapter dedicated to reliability stuff: https://zguide.zeromq.org/docs/chapter4/ Maybe there is no guarantee, but it can handle unreliable connection just fine, at least when I used it a few years ago.

Yes, I have read the guide pretty thoroughly. What all those things have in common is that they are your responsibility to implement - all that 0mq does is reconnect the socket, and queue up messages. Anything else that could give you reliability is either TCP's problem or yours. That includes timeouts, retransmit, ACKs, keepalives, heartbeats, rate control, message storage, everything that ensures reliability - with…

We implemented KISS state machine based on TCP timeouts and server response.

Pseudocode:

  Send msg to server
    if socket timeout:
      reinit socket and send again
  Recv from server
   if recv.msg != 'ok':
     send again
   else
     remove msg from storage
One message has from few to few hundreds Kilobytes. Messages are stored and sent as custom binary data for better compression (better that protobuffs). ZMQ takes complexity of raw tcp sockets away and this is what we needed it for. As much and as little.

Re: ØMQ – The Guide (2011)

#113

Post by tsimionescu »

Doman wrote: post #112
tsimionescu wrote: post #110

Earlier quoted context omitted.

Yes, I have read the guide pretty thoroughly. What all those things have in common is that they are your responsibility to implement - all that 0mq does is reconnect the socket, and queue up messages. Anything else that could give you reliability is either TCP's problem or yours. That includes timeouts, retransmit, ACKs, keepalives, heartbeats, rate control, message storage, everything that ensures reliability - with…

We implemented KISS state machine based on TCP timeouts and server response. Pseudocode: Send msg to server if socket timeout: reinit socket and send again Recv from server if recv.msg != 'ok': send again else remove msg from storage One message has from few to few hundreds Kilobytes. Messages are stored and sent as custom binary data for better compression (better that protobuffs). ZMQ takes complexity of raw tcp so…

This leaves higher application code to handle duplicated messages. It also does not handle cases where the client connects later than the server send, but would still like to receive the server messages. It also doesn't handle rate control (overwhelming the network).

These may all be fine for your application, and then 0mq is an excellent fit. But I think that for most applications, having some kind of pre-implemebted solution to all of these problems, even with slightly more overhead, is extremely valuable.

Re: ØMQ – The Guide (2011)

#114

Post by Doman »

tsimionescu wrote: post #113
Doman wrote: post #112

Earlier quoted context omitted.

We implemented KISS state machine based on TCP timeouts and server response. Pseudocode: Send msg to server if socket timeout: reinit socket and send again Recv from server if recv.msg != 'ok': send again else remove msg from storage One message has from few to few hundreds Kilobytes. Messages are stored and sent as custom binary data for better compression (better that protobuffs). ZMQ takes complexity of raw tcp so…

This leaves higher application code to handle duplicated messages. It also does not handle cases where the client connects later than the server send, but would still like to receive the server messages. It also doesn't handle rate control (overwhelming the network). These may all be fine for your application, and then 0mq is an excellent fit. But I think that for most applications, having some kind of pre-implemebte…

> This leaves higher application code to handle duplicated messages.

True, better send twice than never.

> the client connects later than the server send

Server only responds, never sends anything.

> overwhelming the network

"broker" is handling it nicely

I strongly agree that zmq will not fit in every use case. Always search for best option.

Re: ØMQ – The Guide (2011)

#115

Post by DominikD »

lllr_finger wrote: post #94
DominikD wrote: post #41

Earlier quoted context omitted.

He changed his mind over time. ZeroMQ is still maintained and nanomsg is pretty much dormant.

Look up "nanomsg is not dead" and nanomsg's replacement nng for even further confusion. My takeaway is that even though these newer libs seem to be simpler and avoid some issues, they've just never caught on.

Problem is - nng is also hardly being updated. I was really hopeful for a C library that would by on par with ZeroMQ but every time a new project comes along that needs some sort of message bus and I have the time to evaluate the technology, ZeroMQ simply wins.

Re: ØMQ – The Guide (2011)

#116

Post by cetra3 »

avinassh wrote: post #106
cetra3 wrote: post #92

I've implemented bindings for ZeroMQ with Tokio in rust ( https://github.com/cetra3/tmq ). I found ZeroMQ great for interop with Python/Java if you just want some simple cross-process broadcast/task management.

(noob here) I just looked into the ZeroMQ code and the API seems to be sync and blocking. So, how does one provide async bindings for such code?

ZeroMQ does allow you to register file descriptors on an event queue, which if you dig into the code you'll find tmq using mio to accomplish this.

Re: ØMQ – The Guide (2011)

#117

Post by ComputerGuru »

OJFord wrote: post #111
hkon wrote: post #109

Earlier quoted context omitted.

Why would you build something similar to RabbitMQ ontop of ZeroMQ instead of just using Rabbit?

I believe GP means that ZMQ could be an implementation detail of RMQ (itself). As in, in building your RabbitMQ competitor HareMQ, you might build on top of HareMQ. You can't just use HareMQ because you haven't built it yet.

More to the point, while you could build a theoretical HareMQ atop of ZMQ, you can also build a hundred different products/services that aren’t a RabbitMQ alternative out of it, too. Apples and oranges.
Post reply on HN