Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MQTT: A Conceptual Deep-Dive

ably.io

Post reply on HN
Print view View original

11–20 of 61 posts

Re: MQTT: A Conceptual Deep-Dive

#11

Post by vector3-dot-no »

bdamm wrote: post #6
hangonhn wrote: post #5

"Designed for at most once, at least once and exactly once message delivery" Exactly once message delivery cannot be guaranteed from a theoretical point of view (if it does exist then the Two Generals Problem can be solved, which has been proven to not have a solution). I don't know how MQTT can get around that, especially in an environment where transient network issues are expected.

Well, there's a confirmation sequence that's basically a three-phase commit.

But that would mean they solved a theoretical and/or practical problem in a practical way and who does that?

/s

Re: MQTT: A Conceptual Deep-Dive

#12

Post by swiftcoder »

hangonhn wrote: post #5

"Designed for at most once, at least once and exactly once message delivery" Exactly once message delivery cannot be guaranteed from a theoretical point of view (if it does exist then the Two Generals Problem can be solved, which has been proven to not have a solution). I don't know how MQTT can get around that, especially in an environment where transient network issues are expected.

Even with the 3-phase commit, almost every production MQTT broker sacrifices the "exactly once" requirement to achieve reasonable performance.

Re: MQTT: A Conceptual Deep-Dive

#13

Post by Arnavion »

bdamm wrote: post #7

One of the best features of MQTT that is often overlooked is that the topics are created on the fly and also destroyed on the fly. The protocol does not require the server to maintain topics lists (assuming no retained messages). The server therefore scales based on the throughput and the number of clients (and their subscriptions) but not the size of the topic tree. The topic tree designer is therefore free to use e…

It's also a bad feature for bandwidth-constrained or low-power devices since it requires sending the whole topic string with every message. That's why MQTT-SN, and now MQTT 5, allow registering a topic string -> smaller ID mapping and then using the ID for messages at the expense of persistent state on both ends.

Re: MQTT: A Conceptual Deep-Dive

#14

Post by ustolemyname »

hangonhn wrote: post #5

"Designed for at most once, at least once and exactly once message delivery" Exactly once message delivery cannot be guaranteed from a theoretical point of view (if it does exist then the Two Generals Problem can be solved, which has been proven to not have a solution). I don't know how MQTT can get around that, especially in an environment where transient network issues are expected.

It's worth noting that this guarantee also only applies to messages between a client and broker. The protocol does not guarantee messages are consumed, only that they reached the broker. So if guaranteed delivery of a specific message to consumers is something you need MQTT tends not to be a good fit as you have to build application layer acknowledgements on everything.

Re: MQTT: A Conceptual Deep-Dive

#15

Post by baruchthescribe »

I once had a client where the only port available for me to use on their firewalls was for MQTT (1883) because that's how we were getting sensor data from them. They would not open anything else for us no matter how we implored them so I wrote a live TCP wrapper over MQTT to get around it. It was a local multithreaded TCP daemon that listened for outbound requests on a certain port, wrapped them in MQTT and then published them using a unique topic. The server daemon would detect these topics and unwrap them before forwarding to our server processes. So the client machine thought it was making a live TCP connection to our server but in the middle was a funky invisible MQTT wrapper. It was really elegant once it worked but my goodness was it a pain to debug - a couple of months before I got the whole thing right because of all the blind alleys I went down.

Re: MQTT: A Conceptual Deep-Dive

#17

Post by rad_gruchalski »

flipper65 wrote: post #4
blackflame7000 wrote: post #3

Earlier quoted context omitted.

Because MQTT is optimized for low-bandwidth environments, it only has a limited number of error responses. Therefore, when a connection has been established, there is no way to distinguish a network issue from a forced disconnect.

that is not strictly true for the statement in the article. Deliberate disconnects can be identified by implementing tombstoning, in which a connected node sends an epitaph message to the channel prior to disconnecting.

Yes, but that is not defined by the protocol. MQTT does not have such a mechanism. That’s a custom solution.

Re: MQTT: A Conceptual Deep-Dive

#19

Post by Arnavion »

mrighele wrote: post #16

> 0 = at most once = server fires and forgets — messages may be lost or duplicated Isn't "at most once" a misnomer ? By definition if it is at most once it is not duplicated.

Yes, the article is wrong. A QoS 0 message is never resent by a well-behaved sender, so it could not be duplicated. It might not even leave the sender's network stack if the connection was broken before the send attempt.

Re: MQTT: A Conceptual Deep-Dive

#20

Post by Arnavion »

cbluth wrote: post #2

Hi, interesting article. Anyone care to expound on this part: >>> and without any way to distinguish a deliberate disconnection from a transient network issue

Clients can optionally send a DISCONNECT packet to signal that they intend to disconnect. However there is no equivalent for the server. So if a server drops the connection, the client will only find out when it sends its next PING or other packet.

Edit: Also, it's worth noting that MQTT 5 greatly expands on the error reasons that the server can send to the client if it wants to indicate precisely why it's dropping the client.

Post reply on HN