Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Message order in Matrix: right now, we are deliberately inconsistent

artificialworlds.net

Post reply on HN
Print view View original

51–60 of 119 posts

Re: Message order in Matrix: right now, we are deliberately inconsistent

#51

Post by lmm »

shawnz wrote: post #30
Spivak wrote: post #28

Earlier quoted context omitted.

You have when the client claims it was sent (so where it goes in the displayed history) and can see when it was received. What else could you possibly do?

For example, you could reject the message and show the user an error but only if there's a discrepancy of >X minutes. But how much discrepancy should be allowed? I don't know, I only mean to show why I think the solution isn't as simple as it appears

> you could reject the message and show the user an error but only if there's a discrepancy of >X minutes

No you can't, not in a federated and decentralised system like this.

The sender can wait for a read receipt from a given receiver user, if the receiver is willing to make those public. But if the message left client A and didn't arrive at client B, there's no objective fact of the matter about whether the message "was sent" or not.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#52

Post by lmm »

dataflow wrote: post #43
shawnz wrote: post #16

Earlier quoted context omitted.

How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol?

> How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol? I believe that's impossible? At least if you design it correctly.…

> what matters isn't the time you claim to send the message, it's the time the message is received by the server

There's no "the" server here. If you use the time the message is received by the server, you'll get different views on different servers, and you may see messages from months ago appearing as new, if connectivity breaks down and is later restored.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#53

Post by lmm »

shawnz wrote: post #27
fastball wrote: post #25

Earlier quoted context omitted.

You can amend displayed order for humans (what matters for 99% of usage), while still allowing anyone interested to see when the message actually arrived at the homeserver (making the suggested gambit impractical).

Then instead imagine this: the user really is innocent and just happened to coincidentally send the message right after the start of a long period of poor connectivity (like a flight, or a road trip, etc). If you just allow it to go through after an arbitrary delay, with only a log of the received time for liability purposes, then the user wouldn't have any indication of this scenario occurring. Wouldn't it be better…

> Wouldn't it be better in that case to show an error so that they can make sure the situation is addressed appropriately?

If you want a single centralised server then you can set things up that way. Presumably if you're using a setup with multiple servers, and took one of the servers on the flight/road trip, you wanted the people on the flight/road trip to be able to keep talking to each other over that server, even though that server is disconnected from the one in the office.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#54

Post by lmm »

fiddlerwoaroof wrote: post #41
duskwuff wrote: post #34

Earlier quoted context omitted.

I'd maintain that an important property is for the system to be eventually consistent with regards to history. You don't want a transient network event to potentially result in two users permanently seeing messages in a different order.

I don’t think you can prevent that without centralizing on a single server

You can, but it results in the situation the article is complaining about.

During a netsplit, people chatting on opposite sides of the netsplit continue to be able to chat (by design), but will (obviously) see a different history from each other. So when the netsplit heals, you have a dilemma: either you splice the history from the other side in, giving eventual consistency at the cost of changing the history that people have already read, or you keep permanently different histories on servers that were on one side or the other.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#55

Post by deredede »

lmm wrote: post #50
deredede wrote: post #47

Earlier quoted context omitted.

Messenger apps don't claim that this history should be global and consistent. The order in which messages were sent and received by my device is a perfectly fine (and I'd say intuitive) history. It is the order people (and their records, if they have some) would have had in mind in the old time. I take a different conclusion from the way people look at physical objects: since your device (or even my other device) is…

> Messenger apps don't claim that this history should be global and consistent. The fact that we're talking about multiple people looking at the same chat - the fact that we do conceptualise it as "the same chat" and "the history" - implies that we think of it as a single thing. And I think messenger apps generally nudge us that way - e.g. setting the name of the chat usually sets it for everyone. > It is the order p…

> I think messenger apps generally nudge us that way - e.g. setting the name of the chat usually sets it for everyone.

That's a good point - maybe it's actually email that warped my mind.

> I'll read them in one order on receipt and in a different order when looking back

Also a good point, I was thinking more about business communication where the date the letter is received matters. Thinking back on it, I think the main difference is that the messenger apps might happily reorder message before (or while) I read them. And if only one order is to be available, the one of the most use to me for an instant messaging app is the one I received the messages in, but I get how for other use cases it would be different.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#56

Post by aeonik »

toast0 wrote: post #26
adastra22 wrote: post #15

Earlier quoted context omitted.

That makes the problem harder, but not impossible.

I'm pretty sure this is actually impossible in a distributed system with independent operation, and if it were possible, it would be terrible UI anyway. Problem one is if you want to order events chronologically, you need to precisely decide what the time of the event means. Probably not the time the client hit send, because you can only measure that on the client and client clocks are at best approximately accurate.…

As long as the speed of light remains constant for all observers, who cares if everyone agrees on simultaneity? Distributed systems don't need to know what time it is, just what happened.

Well known systems are implemented this way, and the UI is great, people barely even notice.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#57

Post by mycall »

duskwuff wrote: post #7
thomastay wrote: post #4

Having dealt with this problem at work for several years now, I feel the pain of keeping different clients in sync - it's extremely difficult. Not sure if it's possible in Matrix, but consider having a message ID that increments by one on every message in a room. That lets the client know pretty quickly if there's a gap or a misordering. Not really getting this point though: The /sync API returns events in an order "…

What I think you're missing is that Matrix runs as a distributed system. There's no central authority to assign IDs to messages, and it's possible for a single group chat to run in a split-brain configuration if two homeservers lose connectivity to each other. When those homeservers reconnect, users connected to each one will see messages appear "in the past" which were sent by users on the other side of the split.

Split-brain scenarios can be resolved using an odd number of nodes (or voters) to achieve a majority consensus to agree on the state of the system, stopping the services on the minority side to prevent conflicting operations. Once communication is restored, the stopped nodes can rejoin the cluster and synchronize their data. Vector clocks are a great abstraction for ensuring correct ordering as well.

Re: Message order in Matrix: right now, we are deliberately inconsistent

#58

Post by dataflow »

lmm wrote: post #52
dataflow wrote: post #43

Earlier quoted context omitted.

> How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol? I believe that's impossible? At least if you design it correctly.…

> what matters isn't the time you claim to send the message, it's the time the message is received by the server There's no "the" server here. If you use the time the message is received by the server, you'll get different views on different servers, and you may see messages from months ago appearing as new, if connectivity breaks down and is later restored.

> There's no "the" server here.

Can't you assign every conversation to a single authoritative server for handling?

Also, how large of a time skew are you imagining would exist between different servers? That stuff ought to be accurate to at least milliseconds if not micro...

Re: Message order in Matrix: right now, we are deliberately inconsistent

#59

Post by shawnz »

dataflow wrote: post #43
shawnz wrote: post #16

Earlier quoted context omitted.

How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol?

> How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol? I believe that's impossible? At least if you design it correctly.…

Aside from the concerns with decentralized servers that the other poster mentioned, this has the disadvantage that your messages are going to get constantly reordered to not match the intended flow of the conversation when you have poor connectivity, which is a bad user experience

Re: Message order in Matrix: right now, we are deliberately inconsistent

#60

Post by dataflow »

shawnz wrote: post #59
dataflow wrote: post #43

Earlier quoted context omitted.

> How far back should you be able to amend history? What if a malicious client adds messages to a conversation that happened in the past? Imagine for example I'm at work and notice a critical mistake that I missed, and so I retroactively add messages to the old conversation to make it look like I'm not liable, should that be permitted by the protocol? I believe that's impossible? At least if you design it correctly.…

Aside from the concerns with decentralized servers that the other poster mentioned, this has the disadvantage that your messages are going to get constantly reordered to not match the intended flow of the conversation when you have poor connectivity, which is a bad user experience

Wasn't the whole point here that the messages wouldn't get reordered? There would be one definite order that everyone would see. Again, if the message isn't timestamped by the sever, it would need to appear visually differently, so that everyone knows about this. And nobody says the server has to accept messages with arbitrarily delays either.
Post reply on HN