Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Message order in Matrix: right now, we are deliberately inconsistent

artificialworlds.net

Post reply on HN
Print view View original

41–50 of 119 posts

Re: Message order in Matrix: right now, we are deliberately inconsistent

#41

Post by fiddlerwoaroof »

duskwuff wrote: post #34
fiddlerwoaroof wrote: post #32

Earlier quoted context omitted.

I think you basically want a partial order for federated chat: messages should arrive after the messages that cause them but not necessarily after messages that didn’t cause them. In the case of a network partition, this allows people on either side of the partition to continue communicating at the cost of non-determinism when the partition is resolved.

I'd maintain that an important property is for the system to be eventually consistent with regards to history. You don't want a transient network event to potentially result in two users permanently seeing messages in a different order.

I don’t think you can prevent that without centralizing on a single server

Re: Message order in Matrix: right now, we are deliberately inconsistent

#42

Post by moring »

deredede wrote: post #37
danpalmer wrote: post #3

This is something that many chat apps get wrong and I'm not sure this article is moving in the right direction. The UX is fairly clear in my mind: 1. All up-to-date clients should be displaying the same message order. 2. A single client should not send messages in the wrong order. Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state…

> Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state even if that means amending history. Why? Because the humans reading it will be confused otherwise! An app getting more data is something we intuitively understand, but if my client shows something and yours shows something else, we will conclude different meanings from it. That…

Snail mail has never claimed that a history of all messages, with that history having a current state, exists. If you send a paper letter, you don't have it yourself anymore. You might keep a copy, but that's a _copy_, not the letter you sent.

Messenger apps claim that such a history exists by showing you, well, that history. In the same way, messengers claim that a message order exists, by showing you the messages in that order. If something exists, then it is independent of the viewer. So the assumption that the message order is the same for all viewers is founded in how two people look at physical objects.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#43

Post by dataflow »

shawnz wrote: post #16
danpalmer wrote: post #3

This is something that many chat apps get wrong and I'm not sure this article is moving in the right direction. The UX is fairly clear in my mind: 1. All up-to-date clients should be displaying the same message order. 2. A single client should not send messages in the wrong order. Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state…

How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol?

> How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol?

I believe that's impossible? At least if you design it correctly.

For ordering/interleaving purposes, what matters isn't the time you claim to send the message, it's the time the message is received by the server. If you want, you can display the claimed send timestamp beside the message (and prominently highlight it if it is e.g. out of order, or with a long delay, etc.), but that is irrelevant to the ordering.

The point here is that there should be a single consistent order on the server, and that's what all clients ought be displaying. Any messages not yet acknowledged by the server should be displayed differently so that users are aware they haven't been seen yet, and any messages that arrive before those are sent would obviously get inserted above those.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#44

Post by timokoesters »

I'm the author of the spec issue this blog post is based on: https://github.com/matrix-org/matrix-spec/issues/852

In my implementation for the Conduit Matrix server, the /sync order is used for everything. The timeline is just one list that grows on one end for incoming events and on the other end for backfilled events.

I think it's important that the message order does not change, because that's very difficult to communicate to the user.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#45

Post by fsckboy »

Vanit wrote: post #29

I'm throwing some shade here, but this reeks of backend engineers not caring about UX.

this reeks of backend engineers not caring about UX designers who don't understand the problem while the UI designers who do understand are barred from attending meetings for bad behavior. I'm not throwing shade.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#46

Post by Fizzadar »

timokoesters wrote: post #44

I'm the author of the spec issue this blog post is based on: https://github.com/matrix-org/matrix-spec/issues/852 In my implementation for the Conduit Matrix server, the /sync order is used for everything. The timeline is just one list that grows on one end for incoming events and on the other end for backfilled events. I think it's important that the message order does not change, because that's very difficult to co…

Oh that’s neat (TIL), am also working on a HS that also does this [1].

Not only does it feel like the most correct (I don’t think there is a perfect) behaviour for the user but also makes implementation much simpler. Synapse has a LOT of ordering foo and magic in the code I still don’t fully understand and I’ve gone fairly deep into synapse at times for work.

[1] https://github.com/Beeper/babbleserv

Re: Message order in Matrix: right now, we are deliberately inconsistent

#47

Post by deredede »

moring wrote: post #42
deredede wrote: post #37

Earlier quoted context omitted.

> Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state even if that means amending history. Why? Because the humans reading it will be confused otherwise! An app getting more data is something we intuitively understand, but if my client shows something and yours shows something else, we will conclude different meanings from it. That…

Snail mail has never claimed that a history of all messages, with that history having a current state, exists. If you send a paper letter, you don't have it yourself anymore. You might keep a copy, but that's a _copy_, not the letter you sent. Messenger apps claim that such a history exists by showing you, well, that history. In the same way, messengers claim that a message order exists, by showing you the messages i…

Messenger apps don't claim that this history should be global and consistent. The order in which messages were sent and received by my device is a perfectly fine (and I'd say intuitive) history. It is the order people (and their records, if they have some) would have had in mind in the old time.

I take a different conclusion from the way people look at physical objects: since your device (or even my other device) is a different physical object than my device, I'd be wholly unsurprised to find a different order there.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#48

Post by adastra22 »

toast0 wrote: post #26
adastra22 wrote: post #15

Earlier quoted context omitted.

That makes the problem harder, but not impossible.

I'm pretty sure this is actually impossible in a distributed system with independent operation, and if it were possible, it would be terrible UI anyway. Problem one is if you want to order events chronologically, you need to precisely decide what the time of the event means. Probably not the time the client hit send, because you can only measure that on the client and client clocks are at best approximately accurate.…

There are relatively straightforward decentralized consensus algorithms for ordering events if we assume cooperation. If we assume malicious peers, then we're in the space of the byzantine generals problem, but there are solutions to that too.

Now there's some property that you have to give up, for example an immutable ordering. You might think the message came in one order, then reconnect with the network and discover the order was flipped. So long as the UI can handle that an update, there are consensus algorithms that will deliver a consistent view even in the edge cases.

You don't need a single timestamping queue.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#49

Post by adastra22 »

lmm wrote: post #22
adastra22 wrote: post #15

Earlier quoted context omitted.

That makes the problem harder, but not impossible.

It makes an incrementing message ID impossible.

Only if you're not ok with eventual consistency and renumbering in the case of discovered conflicts or net splits.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#50

Post by lmm »

deredede wrote: post #47
moring wrote: post #42

Earlier quoted context omitted.

Snail mail has never claimed that a history of all messages, with that history having a current state, exists. If you send a paper letter, you don't have it yourself anymore. You might keep a copy, but that's a _copy_, not the letter you sent. Messenger apps claim that such a history exists by showing you, well, that history. In the same way, messengers claim that a message order exists, by showing you the messages i…

Messenger apps don't claim that this history should be global and consistent. The order in which messages were sent and received by my device is a perfectly fine (and I'd say intuitive) history. It is the order people (and their records, if they have some) would have had in mind in the old time. I take a different conclusion from the way people look at physical objects: since your device (or even my other device) is…

> Messenger apps don't claim that this history should be global and consistent.

The fact that we're talking about multiple people looking at the same chat - the fact that we do conceptualise it as "the same chat" and "the history" - implies that we think of it as a single thing. And I think messenger apps generally nudge us that way - e.g. setting the name of the chat usually sets it for everyone.

> It is the order people (and their records, if they have some) would have had in mind in the old time.

I don't think it is. If I pull my correspondence with person X out of my drawer or file, the only dates I have to order them by are the dates written on the letters - which are the letters they and I (if I keep carbons of the ones I send) wrote them on, not the dates I received them. If they sent me a postcard while on holiday and then a letter after returning that arrived sooner, I'll read them in one order on receipt and in a different order when looking back. Likewise if I have a memo of a phone call with them, that may be from before I received a letter that is nevertheless dated earlier.

Post reply on HN