Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Message order in Matrix: right now, we are deliberately inconsistent

artificialworlds.net

Post reply on HN
Print view View original

61–70 of 119 posts

Re: Message order in Matrix: right now, we are deliberately inconsistent

#61

Post by toast0 »

j16sdiz wrote: post #40
toast0 wrote: post #26

Earlier quoted context omitted.

I'm pretty sure this is actually impossible in a distributed system with independent operation, and if it were possible, it would be terrible UI anyway. Problem one is if you want to order events chronologically, you need to precisely decide what the time of the event means. Probably not the time the client hit send, because you can only measure that on the client and client clocks are at best approximately accurate.…

> This is more disorienting IMHO than different message orders for different viewers. the parent post is arguing this is less disorienting, and I agree.

I feel like when an important message comes in out of sequence, but you had already sent a response to the chat with what was visible at the time, it will be very confusing when that gets reordered.

Ex:

A@T0: User X is abusing our service, we should send them a sternly written letter.

B@T60: Yes, I'll do it right away.

C@T2 (received later): No, we should just shadowban them.

When B sent their message, their intent was clear to them. But when they review their message after C's message is received, if the display ordering is changed, the meaning of the communication has changed, and how can B show that sending the warning was reasonable when they clearly said they were going to shadowban the user. (Maybe this group should use something else with a guaranteed ordering to track abuse and response, but that's a different question)

If C's message is displayed earlier than B's in some cases and not others, that makes for a confusing situation, but each person can look at their messages and easily see what they saw when they argue about a breakdown in communication in the aftermath.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#62

Post by RaftPeople »

danpalmer wrote: post #3

This is something that many chat apps get wrong and I'm not sure this article is moving in the right direction. The UX is fairly clear in my mind: 1. All up-to-date clients should be displaying the same message order. 2. A single client should not send messages in the wrong order. Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state…

I don't use these apps so maybe my solution wouldn't work, but after reading the article, it seems that having a visual indicator of messages that are new but in the past would be a reasonable solution.

Especially if there were simple controls to flip into a mode that minimizes the ones already seen (collapsed and grey for example) while highlighting all of the ones inserted in the past.

Or, if there are many messages already seen and few inserted into hist, show the inserted ones with a small sampling of the already seen (so the user can anchor to already familiar data in the timeline) along with "72 messages hidden that were previously seen" type of thing in between the inserteds to condense the view.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#63

Post by kevincox »

deredede wrote: post #37
danpalmer wrote: post #3

This is something that many chat apps get wrong and I'm not sure this article is moving in the right direction. The UX is fairly clear in my mind: 1. All up-to-date clients should be displaying the same message order. 2. A single client should not send messages in the wrong order. Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state…

> Yes a client may be out of date and therefore show something different, but once it becomes up to date it should be showing the same state even if that means amending history. Why? Because the humans reading it will be confused otherwise! An app getting more data is something we intuitively understand, but if my client shows something and yours shows something else, we will conclude different meanings from it. That…

> I intuitively understand that if we both write messages at the same time, we will see them in different order.

I think you are thinking like a distributed systems designer. I would assume that if you asked 10 "random Americans" 9 of them would assume that someone managed to send their message first and would be surprised if their phone and their friends phone showed them messages in different orders.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#64

Post by shawnz »

dataflow wrote: post #60
shawnz wrote: post #59

Earlier quoted context omitted.

Aside from the concerns with decentralized servers that the other poster mentioned, this has the disadvantage that your messages are going to get constantly reordered to not match the intended flow of the conversation when you have poor connectivity, which is a bad user experience

Wasn't the whole point here that the messages wouldn't get reordered? There would be one definite order that everyone would see. Again, if the message isn't timestamped by the sever, it would need to appear visually differently, so that everyone knows about this. And nobody says the server has to accept messages with arbitrarily delays either.

My point is that some limited reordering maybe should be allowed, but not too much. That is to say, the problem isn't as simple as just doing it one way or another way. Every approach has some disadvantages.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#65

Post by dataflow »

shawnz wrote: post #64
dataflow wrote: post #60

Earlier quoted context omitted.

Wasn't the whole point here that the messages wouldn't get reordered? There would be one definite order that everyone would see. Again, if the message isn't timestamped by the sever, it would need to appear visually differently, so that everyone knows about this. And nobody says the server has to accept messages with arbitrarily delays either.

My point is that some limited reordering maybe should be allowed, but not too much. That is to say, the problem isn't as simple as just doing it one way or another way. Every approach has some disadvantages.

I know you were trying to reach that conclusion, but my point was that the design I suggested neither seemed to have the problem you suggested, nor is reordering a necessary outcome, from what I can tell.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#66

Post by Saris »

That's something that Telegram always seems to get right, I've never seen messages out of order in different clients, and if I do something like upload a video then immediately send more text messages before it's done, it will shove the video in between the messages where it should be when the upload is done.

I know it's a much harder problem without a central server managing things. But consistency is very important for messages, out of order they could have a very different meaning and be very confusing.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#67

Post by immibis »

lmm wrote: post #51
shawnz wrote: post #30

Earlier quoted context omitted.

For example, you could reject the message and show the user an error but only if there's a discrepancy of >X minutes. But how much discrepancy should be allowed? I don't know, I only mean to show why I think the solution isn't as simple as it appears

> you could reject the message and show the user an error but only if there's a discrepancy of >X minutes No you can't, not in a federated and decentralised system like this. The sender can wait for a read receipt from a given receiver user, if the receiver is willing to make those public. But if the message left client A and didn't arrive at client B, there's no objective fact of the matter about whether the message…

Seems like a design deficiency of Matrix then. When IRC federation breaks, everyone can see it, except for the rare people who aren't in a shared channel with anyone on the other side of the break.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#68

Post by wolrah »

duskwuff wrote: post #7
thomastay wrote: post #4

Having dealt with this problem at work for several years now, I feel the pain of keeping different clients in sync - it's extremely difficult. Not sure if it's possible in Matrix, but consider having a message ID that increments by one on every message in a room. That lets the client know pretty quickly if there's a gap or a misordering. Not really getting this point though: The /sync API returns events in an order "…

What I think you're missing is that Matrix runs as a distributed system. There's no central authority to assign IDs to messages, and it's possible for a single group chat to run in a split-brain configuration if two homeservers lose connectivity to each other. When those homeservers reconnect, users connected to each one will see messages appear "in the past" which were sent by users on the other side of the split.

Perhaps I'm wrong about how Matrix works, but my understanding was that at least public rooms still had a "primary" homeserver, like for example I can connect to #debian:matrix.org from any number of federated servers but matrix.org is still where that room "lives".

If that understanding is correct, then IMO the answer is simply that the canonical timeline is what that server says it is. Poorly connected users or those on other servers experiencing issues or delays with federation may temporarily see a different sequence of events but once everyone's had a chance to sync back up the state should generally be what the primary server for the room saw it as.

Perhaps there should be some sort of flag for "this message has been reordered during a resync" that clients which initially had a different state due to whatever reason could store to make it clear what happened, and likewise if the central homeserver receives messages with a timestamp significantly off real time it could flag those messages as possibly having been received out of order while still displaying them in the order they were received.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#69

Post by immibis »

lmm wrote: post #54
fiddlerwoaroof wrote: post #41

Earlier quoted context omitted.

I don’t think you can prevent that without centralizing on a single server

You can, but it results in the situation the article is complaining about. During a netsplit, people chatting on opposite sides of the netsplit continue to be able to chat (by design), but will (obviously) see a different history from each other. So when the netsplit heals, you have a dilemma: either you splice the history from the other side in, giving eventual consistency at the cost of changing the history that pe…

You could put the other side into something that looks visually like a thread. Each side will have a different history. They will also have a marker that says the history was split here and click here to view the other side.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#70

Post by lmm »

immibis wrote: post #69
lmm wrote: post #54

Earlier quoted context omitted.

You can, but it results in the situation the article is complaining about. During a netsplit, people chatting on opposite sides of the netsplit continue to be able to chat (by design), but will (obviously) see a different history from each other. So when the netsplit heals, you have a dilemma: either you splice the history from the other side in, giving eventual consistency at the cost of changing the history that pe…

You could put the other side into something that looks visually like a thread. Each side will have a different history. They will also have a marker that says the history was split here and click here to view the other side.

If you can come up with a good design for what a client that does that should look like, and what information it would need from the server to do that, please do write it up and publicise it. I think ultimately something like that has to be the solution, but it would have to be actually fleshed out into something that's possible to implement.
Post reply on HN