Live data from Hacker News

JSON Mail Access Protocol Specification (JMAP)

jmap.io

11–20 of 65 posts

Re: JSON Mail Access Protocol Specification (JMAP)

#12
Unfortunately it includes my biggest gripe about IMAP -- the requirement that messages are given a server-side message number. Maybe breaking with IMAP on this would make interop too hard though.

This requirement is convenient for the client but makes implementing robust servers harder than it need be. If instead each message was identified by a {timestamp,UUID} tuple then multiple MX servers could do final delivery to an eventually-consistent shared store. The requirement for strict, durable message ordering is the one thing that forces hard synchronization among multiple machines doing delivery.

POP3 actually made a step this direction with the common "UIDL" extension; IMAP4 felt like going backwards on this.

While I'm on the subject, another less-common POP3 extension I liked was "XTND XMIT". This gave a simple way for the POP3 client to send mail; sadly it never really caught on in favor of just using SMTP submission. Originally though this only worked OK because SMTP was open-relay. Once spam put an end to that it became complicated to send a message when accessing your mail remotely. Then to solve that problem we had to graft an auth layer on top of the existing SMTP protocol and upgrade all of the clients and servers to support it. If only "XTND XMIT" had won in the beginning everybody would be submitting outgoing mail over that already-authenticated channel and so much work could have been avoided.

So anyway, those are my two requests for any "IMAP killer": allow the server to use unique (but non-sequential) IDs for messages and provide a way to submit outgoing messages over the same channel.

Re: JSON Mail Access Protocol Specification (JMAP)

#13

Does JMAP have an equivalent to IMAP IDLE? I.e. the ability for the server to notify the client when an email arrives. This means client's don't need to poll, and messages are seen instantly. For me instant notification of inbound message is a must-have.

We're using eventsource push - just to say "you need to poll for changes now". I think that's cleaner than trying to push the actual changes.

Our actual push does include some extra detail, but the important bit is just "here's a new state that exists on the server - if you haven't already heard of it, you probably should poll for changes now".

It's out of band(ish) - because JMAP is connectionless. And because depending on which platform you're using, there might be a nice channel for push notifications already.

Re: JSON Mail Access Protocol Specification (JMAP)

#14

Unfortunately it includes my biggest gripe about IMAP -- the requirement that messages are given a server-side message number. Maybe breaking with IMAP on this would make interop too hard though. This requirement is convenient for the client but makes implementing robust servers harder than it need be. If instead each message was identified by a {timestamp,UUID} tuple then multiple MX servers could do final delivery…

I'm in two minds about the UID stuff. There are advantages and disadvantages.

But one thing that you MUST have for sensible synchronisation is the global MODSEQ. Otherwise there's no single state you can use to know if there's new data on the server short of (as you do with POP3) fetching the entire UIDL list every time.

The way we do this with Cyrus replication is to reinject the message if we get a UID clash - so you can have delivery at both ends of a (theoretical - not yet finished) mastermaster replica pair. Upon discovering the mismatch, BOTH messages get new UIDs larger than any yet used, which brings all clients into agreement.

But anyway - our unique identifiers in JMAP can be anything the server would like. We use 'fNUMBERuNUMBER' at FastMail, which is a folder/uid pair - but a more modern server could use UUIDs that don't change for moves between folders. We could even implement it in Cyrus on top of the Digest.SHA1 field if we built a reverse mapping database and deduplicated on it.

But synchronised modseqs... I don't see a way around that. We've been discussing "modseq validity range" stuff - where a delivery logs an intent to use a modseq, and then clears the intent upon commit. Any request that includes a range past that intent needs to wait for the commit to complete or abort.

Re: JSON Mail Access Protocol Specification (JMAP)

#15

HTTP itself has an overhead, I'm skeptical changing a persistent IMAP connection into a request/response model with a different serialization format is going to be more efficient, especially given compression (like COMPRESS=DEFLATE) IMAP4rev1 is one thing, but there are many extensions supported in modern imap servers to speed up syncing and mailing, like CATENATE/BURL/CONDSTORE/MULTIAPPEND/MOVE/etc Granted, the IMAP…

One of the main efficiency gains is in massively reducing the number of round trips required to do a series of operations. This batching of operations is hugely important on high-latency connections, relevant for us in Australia all the time but also of importance on mobile connections the world over. Even 4G networks have relatively high latency, but can do a burst transfer pretty efficiently.

Re: JSON Mail Access Protocol Specification (JMAP)

#16

Does JMAP have an equivalent to IMAP IDLE? I.e. the ability for the server to notify the client when an email arrives. This means client's don't need to poll, and messages are seen instantly. For me instant notification of inbound message is a must-have.

We're using eventsource push - just to say "you need to poll for changes now". I think that's cleaner than trying to push the actual changes. Our actual push does include some extra detail, but the important bit is just "here's a new state that exists on the server - if you haven't already heard of it, you probably should poll for changes now". It's out of band(ish) - because JMAP is connectionless. And because depen…

How about documenting the EventSource interactions?

Re: JSON Mail Access Protocol Specification (JMAP)

#17

Earlier quoted context omitted.

We're using eventsource push - just to say "you need to poll for changes now". I think that's cleaner than trying to push the actual changes. Our actual push does include some extra detail, but the important bit is just "here's a new state that exists on the server - if you haven't already heard of it, you probably should poll for changes now". It's out of band(ish) - because JMAP is connectionless. And because depen…

How about documenting the EventSource interactions?

We probably will publish this as well. We're focussing on the core API first. Having said that, it's really very simple. Here's the entirety of our (currently internal) spec for push events at FM:

This is a text/event-stream resource, as described in [http://www.w3.org/TR/eventsource/](). The following events are pushed:

- progress: Sent during a long-running api method call to let the client know not to timeout the connection. The data is a JSON object with a single property: `connectionId`, with the id the client sent for that connection. This event does not set a new id in the event source stream.

- push: Sent whenever the user's highest modseq changes. The event id is the new highest modseq. The data for the event is a JSON object, with the following properties:

  * **clientId**: `String` (optional). If the change was due to an action initiated by
    the API, the `X-ME-ClientId` header will be echoed back as this property. This
    allows clients to ignore push events for changes they themselves have made.

  * **mailModSeq**: `Number` (The highest modseq for a mailbox)

  * **contactsModSeq**: `Number` (The highest modseq for contacts/contact groups)

  * **calendarModSeq**: `Number` (The highest modseq for calendar events).

Re: JSON Mail Access Protocol Specification (JMAP)

#18
Call me curmudgeonly, why replace a protocol that works...instead roll up a library to provide the functionality...

Those things called standards generally find their way through bodies like the IETF. Maybe I missed it while reading on my phone, but I don't see this submitted as an IETF draft...

Re: JSON Mail Access Protocol Specification (JMAP)

#19

Call me curmudgeonly, why replace a protocol that works...instead roll up a library to provide the functionality... Those things called standards generally find their way through bodies like the IETF. Maybe I missed it while reading on my phone, but I don't see this submitted as an IETF draft...

Because JSON is cool!

Re: JSON Mail Access Protocol Specification (JMAP)

#20

HTTP itself has an overhead, I'm skeptical changing a persistent IMAP connection into a request/response model with a different serialization format is going to be more efficient, especially given compression (like COMPRESS=DEFLATE) IMAP4rev1 is one thing, but there are many extensions supported in modern imap servers to speed up syncing and mailing, like CATENATE/BURL/CONDSTORE/MULTIAPPEND/MOVE/etc Granted, the IMAP…

HTTP can be made persistent too, both with good ol' KeepAlive and with recent innovations like SPDY. Massive savings in latency, especially when SSL is involved.
Post reply on HN