Live data from Hacker News

MQTT: A Conceptual Deep-Dive

ably.io

21–30 of 61 posts

Re: MQTT: A Conceptual Deep-Dive

#21

Earlier quoted context omitted.

Because MQTT is optimized for low-bandwidth environments, it only has a limited number of error responses. Therefore, when a connection has been established, there is no way to distinguish a network issue from a forced disconnect.

that is not strictly true for the statement in the article. Deliberate disconnects can be identified by implementing tombstoning, in which a connected node sends an epitaph message to the channel prior to disconnecting.

You are talking on the phone. The line goes dead. Were you hung up on or was the call dropped? Either way, you aren't talking to anyone anymore so your messages aren't getting through.

Re: MQTT: A Conceptual Deep-Dive

#22
post #7

One of the best features of MQTT that is often overlooked is that the topics are created on the fly and also destroyed on the fly. The protocol does not require the server to maintain topics lists (assuming no retained messages). The server therefore scales based on the throughput and the number of clients (and their subscriptions) but not the size of the topic tree. The topic tree designer is therefore free to use e…

It's also a bad feature for bandwidth-constrained or low-power devices since it requires sending the whole topic string with every message. That's why MQTT-SN, and now MQTT 5, allow registering a topic string -> smaller ID mapping and then using the ID for messages at the expense of persistent state on both ends.

I've avoided MQTT-SN (and consequently have limited MQTT to high power segments of our stack, yes a phone is high power) due to lack of robust implementations. If that changes then I'd reconsider.

Re: MQTT: A Conceptual Deep-Dive

#23
post #22

Earlier quoted context omitted.

It's also a bad feature for bandwidth-constrained or low-power devices since it requires sending the whole topic string with every message. That's why MQTT-SN, and now MQTT 5, allow registering a topic string -> smaller ID mapping and then using the ID for messages at the expense of persistent state on both ends.

I've avoided MQTT-SN (and consequently have limited MQTT to high power segments of our stack, yes a phone is high power) due to lack of robust implementations. If that changes then I'd reconsider.

If you follow the minutes of the MQTT TC at OASIS, after the release of MQTT v5 there is now talk of formalising the MQTT-SN spec in a similar fashion, that should lead (in time) to more robust implementations.

Re: MQTT: A Conceptual Deep-Dive

#24

I have to question the overall quality of the article when it repeatedly uses the term "channel" to refer to the well-established concept of a "topic" in MQTT.

I have to question your overall understanding of the topic of message passing. Channels are a well-established concept in messaging.

Re: MQTT: A Conceptual Deep-Dive

#25
post #5

"Designed for at most once, at least once and exactly once message delivery" Exactly once message delivery cannot be guaranteed from a theoretical point of view (if it does exist then the Two Generals Problem can be solved, which has been proven to not have a solution). I don't know how MQTT can get around that, especially in an environment where transient network issues are expected.

[Edited] Two Generals does not apply since it deals with the problem of guaranteeing that two parties agree on the exact same consistent state after a finite amount of time/messages. MQTT's "exactly once" delivery is basically an eventually consistent mechanism -- it just guarantees that the message is eventually delivered to the broker which then does some simple deduplication. Eventually may be infinite. So, no violation of the space time continuum here...

In other words, MQTT's exactly once delivery conceptually works as if the server would store a big hashmap of all message IDs it has seen once and the rejecting any duplicates using that hashmap. The client the simply keeps retrying with the same message ID forever. This guarantees that the message is received by the server "exactly once". The actual protocol is more involved, but the details are basically just an optimization to prevent the server from having to keep around the "received messages" hashmap forever.

The problem that you're thinking of that MQTT of course doesn't solve is that it is impossible to perform any externally visible action (such as moving an actuator, storing a record into another database system or detonating a bomb) "exactly once" in the general case when you take into account the possibility of failures in between the two steps of performing the action and storing the information that you have performed it. You can either first commit and then perform the action, which means you can not safely retry after having failed between the commit and performing the action, because you might have already performed it ("at most once"). Or you can first perform the action and then commit, potentially redoing the action after a failed commit ("at least once"). There is, conceptually, no way around this unless we get help from the environment, such as an atomic perform+commit primitive. To see how this relates to two generals, think about the exchange of the "performed" bit between the external system and the storage as an exchange of messages that can be lossy. We can not come up with any algorithm that guarantees that the external system and the storage always agree on the state of the performed bit after any finite number of steps.

Re: MQTT: A Conceptual Deep-Dive

#27

I have to question the overall quality of the article when it repeatedly uses the term "channel" to refer to the well-established concept of a "topic" in MQTT.

I'm not very knowledgeable about MQTT specifically, but it seems like fair game considering the protocol spec (https://docs.oasis-open.org/mqtt/mqtt/v5.0/mqtt-v5.0.html) uses the term channel:

> 3.3.2.1 Topic Name

> The Topic Name identifies the information channel to which Payload data is published.

Re: MQTT: A Conceptual Deep-Dive

#28

Earlier quoted context omitted.

that is not strictly true for the statement in the article. Deliberate disconnects can be identified by implementing tombstoning, in which a connected node sends an epitaph message to the channel prior to disconnecting.

Yes, but that is not defined by the protocol. MQTT does not have such a mechanism. That’s a custom solution.

[deleted]

Re: MQTT: A Conceptual Deep-Dive

#29
post #5

"Designed for at most once, at least once and exactly once message delivery" Exactly once message delivery cannot be guaranteed from a theoretical point of view (if it does exist then the Two Generals Problem can be solved, which has been proven to not have a solution). I don't know how MQTT can get around that, especially in an environment where transient network issues are expected.

Even with the 3-phase commit, almost every production MQTT broker sacrifices the "exactly once" requirement to achieve reasonable performance.

I'm not sure this statement is accurate. Certainly the MQTT broker I work on (IBM WIoTP Message Gateway) and a number of our competitors support QoS 2 in production (though I am aware a number of brokers don't support it).

Re: MQTT: A Conceptual Deep-Dive

#30
post #6
post #5

"Designed for at most once, at least once and exactly once message delivery" Exactly once message delivery cannot be guaranteed from a theoretical point of view (if it does exist then the Two Generals Problem can be solved, which has been proven to not have a solution). I don't know how MQTT can get around that, especially in an environment where transient network issues are expected.

Well, there's a confirmation sequence that's basically a three-phase commit.

A QoS 2 flow is 2-phase (Publish and PubRec being the first phase and PubRel and PubComp being the second).
Post reply on HN