Live data from Hacker News

Message order in Matrix: right now, we are deliberately inconsistent

artificialworlds.net

31–40 of 119 posts

Re: Message order in Matrix: right now, we are deliberately inconsistent

#31
post #9

Earlier quoted context omitted.

I break up my messages, as do many people.

Do you do so with the expectation that they might arrive out of order, or one fails?

Out of order no, failing and having to manually re-send which makes it out of order is acceptable.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#32
post #7

Earlier quoted context omitted.

What I think you're missing is that Matrix runs as a distributed system. There's no central authority to assign IDs to messages, and it's possible for a single group chat to run in a split-brain configuration if two homeservers lose connectivity to each other. When those homeservers reconnect, users connected to each one will see messages appear "in the past" which were sent by users on the other side of the split.

yeah, having eventual consistency for messages across homeservers makes the work on the client harder. I guess they just have to accept that messages will "appear in the past" as you said. But at least for messages sent within the same homeserver, I would think that those two apis should return the same data

I think you basically want a partial order for federated chat: messages should arrive after the messages that cause them but not necessarily after messages that didn’t cause them. In the case of a network partition, this allows people on either side of the partition to continue communicating at the cost of non-determinism when the partition is resolved.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#33
post #8

This is something that many chat apps get wrong and I'm not sure this article is moving in the right direction. The UX is fairly clear in my mind: 1. All up-to-date clients should be displaying the same message order. 2. A single client should not send messages in the wrong order. Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state…

> Additionally there are some clients that treat each message input by the user as a retriable thing in isolation, which is also clearly incorrect. If I send two messages and the first fails to go through, I almost certainly don't want to retry the second until the first has gone through, otherwise my client has literally sent out of order messages! I don't think that's clearly incorrect. If you sent two messages you…

Human communications are more naunced than DB transactions. If I forget to mention something important I send a new message rather than editing the already sent one, to make sure I catch their attention. Edits can go unnoticed. Imagine this scenario:

[12:00 / sent] Sell the house.

[12:05 / failed] Please feed the baby.

[12:06 / sent] Oh and the cat too.

Now the receiver's going to sell my cat [example inspired by The Art of Multiprocessor Programming].

Re: Message order in Matrix: right now, we are deliberately inconsistent

#34

Earlier quoted context omitted.

yeah, having eventual consistency for messages across homeservers makes the work on the client harder. I guess they just have to accept that messages will "appear in the past" as you said. But at least for messages sent within the same homeserver, I would think that those two apis should return the same data

I think you basically want a partial order for federated chat: messages should arrive after the messages that cause them but not necessarily after messages that didn’t cause them. In the case of a network partition, this allows people on either side of the partition to continue communicating at the cost of non-determinism when the partition is resolved.

I'd maintain that an important property is for the system to be eventually consistent with regards to history. You don't want a transient network event to potentially result in two users permanently seeing messages in a different order.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#36
post #33
post #8

Earlier quoted context omitted.

> Additionally there are some clients that treat each message input by the user as a retriable thing in isolation, which is also clearly incorrect. If I send two messages and the first fails to go through, I almost certainly don't want to retry the second until the first has gone through, otherwise my client has literally sent out of order messages! I don't think that's clearly incorrect. If you sent two messages you…

Human communications are more naunced than DB transactions. If I forget to mention something important I send a new message rather than editing the already sent one, to make sure I catch their attention. Edits can go unnoticed. Imagine this scenario: [12:00 / sent] Sell the house. [12:05 / failed] Please feed the baby. [12:06 / sent] Oh and the cat too. Now the receiver's going to sell my cat [example inspired by The…

They sure are, but I hate when slack combines my messages when I wanted two separate messages on purpose. If i send two messages, it's because I did it on purpose.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#37

This is something that many chat apps get wrong and I'm not sure this article is moving in the right direction. The UX is fairly clear in my mind: 1. All up-to-date clients should be displaying the same message order. 2. A single client should not send messages in the wrong order. Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state…

> Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state even if that means amending history. Why? Because the humans reading it will be confused otherwise! An app getting more data is something we intuitively understand, but if my client shows something and yours shows something else, we will conclude different meanings from it.

That's interesting because I have the complete opposite take and would hard disagree with this. I intuitively understand that if we both write messages at the same time, we will see them in different order. Snail mail has worked this way for centuries, and I very much prefer this to an app silently altering the content as time goes. It is confusing when it happens under my eyes (something moved at the top of the screen while I was reading the bottom, what was it?) and easily leads to missed messages especially in group conversations (my buddy sent a message with a poor connection at 11am, it is retried and sent at 2pm and appears before the lengthy discussion others had at noon).

Re: Message order in Matrix: right now, we are deliberately inconsistent

#38

This sounds like a pretty good use case for a consensus algorithm like Paxos or Raft

I think the most important property to preserve is causality; that is, if a user sends a message B after they have read (i.e., received) a message A, then B should come after A for everyone, because B depends on A. Basically use a Lamport clock.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#39

This sounds like a pretty good use case for a consensus algorithm like Paxos or Raft

Those are CP which is impossible in a distributed messaging system where it has to obviously be AP. Otherwise you'd have to guarantee that everyone involved is always online (no partition) to make progress on sending messages!

I think I have this right anyways. (CAP theorem for anyone curious.)

Re: Message order in Matrix: right now, we are deliberately inconsistent

#40
post #26

Earlier quoted context omitted.

That makes the problem harder, but not impossible.

I'm pretty sure this is actually impossible in a distributed system with independent operation, and if it were possible, it would be terrible UI anyway. Problem one is if you want to order events chronologically, you need to precisely decide what the time of the event means. Probably not the time the client hit send, because you can only measure that on the client and client clocks are at best approximately accurate.…

> This is more disorienting IMHO than different message orders for different viewers.

the parent post is arguing this is less disorienting, and I agree.

Post reply on HN