Proton Bridge has a local cache, which can be disabled. I do wonder if doing so would fix this problem.
Protonmail can delete the wrong email and nobody cares
211–220 of 254 posts
Re: Protonmail can delete the wrong email and nobody cares
#212JMAP is a replacement for IMAP. Unlike IMAP it has immutable IDs[0] but not much support[1]. [0] https://jmap.io/#faq [1] https://jmap.io/software.html
Emails should really be identified by Message-Id (which isn't guaranteed unique, but is very selective) and a hash of the body and a subset of the headers (e.g., excluding Received headers, and maybe using only Message-Id, Date, and From, maybe not even Subject).
A good email store is very searchable, and a good MUA searches email, and a good MAP gives the client temporary (ephemeral) handles for "open" emails.
Re: Protonmail can delete the wrong email and nobody cares
#213This is Bart, Proton CTO here. For clarity, the issue mentioned here only impacts Proton Mail Bridge, our desktop IMAP/SMTP gateway to Proton Mail encrypted email.
The fact that Bridge and its client can become desynchronized sporadically for some users is a high priority issue we have been working on. Bridge is open source, and as a result relies upon open-source components, and the root cause is an architectural issue in a library that Bridge uses to implement IMAP. When there are network issues, this library returns errors to email clients.
Unfortunately, there are hundreds of email clients, and some email clients don’t handle errors properly, and this leads to desynchronization.
Our error tracking shows this does not happen often (1-2% of Bridge users) and the symptom is usually incorrect display of messages or read/unread status which is fixed with an inbox resynchronization. There are cases where a combination of a desynchronized mailbox and a specific series of user actions can lead to accidental email deletion, but this is far rarer than desynchronization. Our implementation tries as hard as possible to avoid this. If you find you are missing an email, our implementation works around the issue by placing it in a users’ All Mail folder.
As Bridge is open source, updates on this issue have always been publicly posted on GitHub. Addressing this issue at the source requires replacing the core IMAP library. Unfortunately, there are no FOSS IMAP libraries that are sufficiently well maintained. Therefore, the solution is to build our own IMAP library called Gluon, which we have been focusing on since this issue was reported to us. You can follow the progress of this open-source project here: https://github.com/ProtonMail/gluon
We are not refusing to fix the problem. The only possible solution is writing a new open-source IMAP library which we can maintain ourselves to ensure this class of errors cannot occur again. We have doubled the size of the team working on this this year so it is a priority for us.
We’re confident that this addresses the main sources of desynchronization and will be available in the beta version of Bridge by the end of the year.
Re: Protonmail can delete the wrong email and nobody cares
#214This sort of reaffirms my belief that UIDs are not sufficient for syncing mail. Emails should be hashed and synced by the hash which would solve other issues, like being able to redownload specific messages that may have got corrupted locally.
Even so, isn't this a violation of the IMAP standard, which says that UIDs are, by design, not permanent identifiers, but UID + UIDVALIDITY is? (I don't know much about IMAP.)
It is definitely a recommendation, but UIDVALIDITY just checks the folder from what I understand. Hashing the entire message would be the best way from my understanding to sync messages.
Re: Protonmail can delete the wrong email and nobody cares
#215This is Bart, Proton CTO here. For clarity, the issue mentioned here only impacts Proton Mail Bridge, our desktop IMAP/SMTP gateway to Proton Mail encrypted email. The fact that Bridge and its client can become desynchronized sporadically for some users is a high priority issue we have been working on. Bridge is open source, and as a result relies upon open-source components, and the root cause is an architectural is…
> Bridge is open source, and as a result relies upon open-source components
I don't get it. Bridge is open source does not imply it should relies upon open-source components.
> Addressing this issue at the source requires replacing the core IMAP library.
Why building an IMAP library from scratch instead of fixing/forking go-imap? Even a temporary fix to go-imap when you are developing gluon? Another repetitive work which does not guarantee the mentioned issues will be resolved completely.
Re: Protonmail can delete the wrong email and nobody cares
#216I've suffered from exactly the same issues with Protonmail Bridge, and just this last weekend I decided (reluctantly) to move to a more standard mail provider (I chose Mailbox.org). Aside from the UID issue discussed I also had problems with Bridge not supporting my particular use-cases. I created my own fork (see https://github.com/polaris64/proton-bridge ) to work around some limitations and to add features, but ma…
+1 for mailbox.org
I've been using them since around 2015 and they have been excellent.
Re: Protonmail can delete the wrong email and nobody cares
#217I'm not surprised. In my experience mail synchronisation with IMAP is fraught with edge cases that cause weird issues (I definitely had weird states happening with offlineimap and Co). I really wish JMAP will take off and give us a much better mail protocol.
Re: Protonmail can delete the wrong email and nobody cares
#218This is Bart, Proton CTO here. For clarity, the issue mentioned here only impacts Proton Mail Bridge, our desktop IMAP/SMTP gateway to Proton Mail encrypted email. The fact that Bridge and its client can become desynchronized sporadically for some users is a high priority issue we have been working on. Bridge is open source, and as a result relies upon open-source components, and the root cause is an architectural is…
> Bridge is open source, and as a result relies upon open-source components I don't get it. Bridge is open source does not imply it should relies upon open-source components. > Addressing this issue at the source requires replacing the core IMAP library. Why building an IMAP library from scratch instead of fixing/forking go-imap? Even a temporary fix to go-imap when you are developing gluon? Another repetitive work w…
> I don't get it. Bridge is open source does not imply it should relies upon open-source components.
It could be open source and depend on proprietary components, but then the public wouldn't be able to build and use it from source.
Re: Protonmail can delete the wrong email and nobody cares
#219This is Bart, Proton CTO here. For clarity, the issue mentioned here only impacts Proton Mail Bridge, our desktop IMAP/SMTP gateway to Proton Mail encrypted email. The fact that Bridge and its client can become desynchronized sporadically for some users is a high priority issue we have been working on. Bridge is open source, and as a result relies upon open-source components, and the root cause is an architectural is…
The fundamental problem is that `UID`s in IMAP kinda suck because assigning persistent, unique IDs to emails in a store is a hard problem because doing that for mbox- or maildir-like stores is hard because those predated any notion of remote email access protocols.
Thus in practice IMAP servers generally assign `UID`s ephemerally per-session, which means that clients can't rely on the stability of `UID`s, which means that clients have to re-obtain `UID`s before operating on emails via IMAP even if they have cached those emails locally. `UIDVALIDITY` exists to help clients cache and invalidate `UID`s. The RFC has text about this.
A bridge from IMAP to something else (which is basically what every IMAP server ever is) needs to deal with this. To make `UID`s stable requires keeping state.
Clients should really not assume stable `UID`s. Instead clients should `SEARCH` or list to get [temporarily] valid `UID`s then use those to delete etc.
Re: Protonmail can delete the wrong email and nobody cares
#220This sort of reaffirms my belief that UIDs are not sufficient for syncing mail. Emails should be hashed and synced by the hash which would solve other issues, like being able to redownload specific messages that may have got corrupted locally.
Even so, isn't this a violation of the IMAP standard, which says that UIDs are, by design, not permanent identifiers, but UID + UIDVALIDITY is? (I don't know much about IMAP.)
No, RFC 3501 says
> The unique identifier of a message MUST NOT change during the > session, and SHOULD NOT change between sessions. Any change of > unique identifiers between sessions MUST be detectable using the > UIDVALIDITY mechanism discussed below. Persistent unique identifiers > are required for a client to resynchronize its state from a previous > session with the server (e.g., disconnected or offline access > clients); this is discussed further in [[IMAP-DISC](https://www.rfc-editor.org/rfc/rfc3501#ref-IMAP-DISC)].
so, "SHOULD NOT", but in practice it's really hard to make {UID, UIDVALIDITY} assignments persistent and unique, so IMAP servers don't, and as you can see, they are allowed to not.
I.e., it's perfectly compliant to generate a new UIDVALIDITY for each session and then assign UIDs to emails in folders when you open them