Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JSON Mail Access Protocol Specification (JMAP)

jmap.io

Post reply on HN
Print view View original

21–30 of 65 posts

Re: JSON Mail Access Protocol Specification (JMAP)

#21

Post by kijin »

brongondwana wrote: post #2

This is similar to what we're using at FastMail for our web interface, but simplified and made a bit more generic based on our experiences over the past year. Our plan is to port our web interface across to this spec and implement both a direct backed in Cyrus IMAPd and a standalone implementation which can proxy for existing IMAP servers.

Keep up the good work! 6-year customer here. Notwithstanding some lost features compared to the old UI, your new web interface really lives up to your company's name. Everything is fast and snappy even when I'm accessing it from halfway around the world.

But I'm not sure it's a good idea to emphasize the fact that you use JSON. Sure, it's cool right now, but I find it technically more interesting that it replaces IMAP with something that is built on top of HTTP, a somewhat RESTful protocol.

Re: JSON Mail Access Protocol Specification (JMAP)

#22

Post by alfiejohn_ »

jmspring wrote: post #18

Call me curmudgeonly, why replace a protocol that works...instead roll up a library to provide the functionality... Those things called standards generally find their way through bodies like the IETF. Maybe I missed it while reading on my phone, but I don't see this submitted as an IETF draft...

Because AFAIR, you need to point to two working implementations of a spec before turning it into an RFC.

Re: JSON Mail Access Protocol Specification (JMAP)

#23

Post by robn_fastmail »

jmspring wrote: post #18

Call me curmudgeonly, why replace a protocol that works...instead roll up a library to provide the functionality... Those things called standards generally find their way through bodies like the IETF. Maybe I missed it while reading on my phone, but I don't see this submitted as an IETF draft...

IMAP works, but not particularly well for a lot of modern tasks. Its highly stateful, requires a fair amount of knowledge on the part of the client to make things work correctly, and too many extensions are optional, requiring a lot of fallbacks for when functions are missing. Implementing a client or a server well is difficult.

JMAP isn't the be-all and end-all of mail protocols, but it goes some way to address some of the problems in IMAP. We're using it in production right now at FastMail so we already know its pretty good as a low-latency, stateless protocol.

As for the IETF (and standards bodies in general), they're useful in their place, but they're also a slow-moving beast and thev're published a great many "standards" that have either never been implemented or are just rubbish.

But there's nobody forcing you to implement it. Its there, its documented, and we're encouraging others to get involved to make it work. If others get on board then we'll see improvements everywhere. If not, then FastMail will continue to use it and get the advantages that come from it.

Re: JSON Mail Access Protocol Specification (JMAP)

#24

Post by teddyh »

alfiejohn_ wrote: post #22
jmspring wrote: post #18

Call me curmudgeonly, why replace a protocol that works...instead roll up a library to provide the functionality... Those things called standards generally find their way through bodies like the IETF. Maybe I missed it while reading on my phone, but I don't see this submitted as an IETF draft...

Because AFAIR, you need to point to two working implementations of a spec before turning it into an RFC.

Well, no, first you discuss the idea in a Working Group mailing list, which is like HN – you get lots of great negative feedback. When you have fixed all your initial bugs, you write a draft RFC. With which implementers can create independent implementations. If there are not anyone willing to do this, this is a sign that the idea is not wanted enough, i.e. not widely useful enough to warrant an RFC in the first place. The implementers will find bugs. You fix those, and discuss some more. Eventually, you, the mailing list and the implementers will be happy with the standard, at which point you move to have the RFC published as an official RFC.

Anyhow, this is all for a “Standard track” RFC. An “Informational” or “Experimental” RFC has no such requirement, and can basically be submitted by anyone for anything.

Re: JSON Mail Access Protocol Specification (JMAP)

#25

Post by _urga »

brongondwana wrote: post #14
bodyfour wrote: post #12

Unfortunately it includes my biggest gripe about IMAP -- the requirement that messages are given a server-side message number. Maybe breaking with IMAP on this would make interop too hard though. This requirement is convenient for the client but makes implementing robust servers harder than it need be. If instead each message was identified by a {timestamp,UUID} tuple then multiple MX servers could do final delivery…

I'm in two minds about the UID stuff. There are advantages and disadvantages. But one thing that you MUST have for sensible synchronisation is the global MODSEQ. Otherwise there's no single state you can use to know if there's new data on the server short of (as you do with POP3) fetching the entire UIDL list every time. The way we do this with Cyrus replication is to reinject the message if we get a UID clash - so y…

For anti-entropy in a distributed system, one could use a combination of the following:

1. Get everything created since last sync. A created index (as opposed to updated index) would not need to be resorted for subsequent updates. This would get only new mail.

2. A fixed-size Merkle tree, 8 blocks per level, 7 levels deep, with around 262144 blocks at the base. This would be the best tree configuration for syncing from a few thousand up to 40 million emails. This would identify all mail updated since last connection, within 4 roundtrips with a minimum of bandwidth. The tree should be incrementally updated using XOR at both client and server whenever an update is made. It's possible to use a tree with less depth for fewer roundtrips but with support for less emails or requiring more bandwidth.

3. Push sync. Server would push all creates/updates to client while client is connected.

Re: JSON Mail Access Protocol Specification (JMAP)

#27

Post by _urga »

nmjenkins wrote: post #15
cromwellian wrote: post #3

HTTP itself has an overhead, I'm skeptical changing a persistent IMAP connection into a request/response model with a different serialization format is going to be more efficient, especially given compression (like COMPRESS=DEFLATE) IMAP4rev1 is one thing, but there are many extensions supported in modern imap servers to speed up syncing and mailing, like CATENATE/BURL/CONDSTORE/MULTIAPPEND/MOVE/etc Granted, the IMAP…

One of the main efficiency gains is in massively reducing the number of round trips required to do a series of operations. This batching of operations is hugely important on high-latency connections, relevant for us in Australia all the time but also of importance on mobile connections the world over. Even 4G networks have relatively high latency, but can do a burst transfer pretty efficiently.

A binary protocol running over persistent socket would have fewer round trips and parsing latency. HTTP and SPDY can't compete with that, because they introduce too much header overhead and complexity.

Re: JSON Mail Access Protocol Specification (JMAP)

#28

Post by stormbrew »

bodyfour wrote: post #12

Unfortunately it includes my biggest gripe about IMAP -- the requirement that messages are given a server-side message number. Maybe breaking with IMAP on this would make interop too hard though. This requirement is convenient for the client but makes implementing robust servers harder than it need be. If instead each message was identified by a {timestamp,UUID} tuple then multiple MX servers could do final delivery…

Some IMAP servers allow you to use a send folder to inject mail to be sent. I think that's a pretty reasonable way to go about it with the way IMAP works, but afaik no client implements it.

Agreed on the messageid thing. I frankly think that IMAP is too messy to try too hard for interop with. Using a relatively similar data model and being able to work correctly with a maildir should be considered enough.

But I love this idea and hope it takes off in spite of that. It's really time to revamp the interface to mail servers to be more friendly to webmail. It'll make a huge difference to the level of choice of webmail providers/software.

Re: JSON Mail Access Protocol Specification (JMAP)

#30

Post by robn_fastmail »

_urga wrote: post #27
nmjenkins wrote: post #15

Earlier quoted context omitted.

One of the main efficiency gains is in massively reducing the number of round trips required to do a series of operations. This batching of operations is hugely important on high-latency connections, relevant for us in Australia all the time but also of importance on mobile connections the world over. Even 4G networks have relatively high latency, but can do a burst transfer pretty efficiently.

A binary protocol running over persistent socket would have fewer round trips and parsing latency. HTTP and SPDY can't compete with that, because they introduce too much header overhead and complexity.

Persistent connections are tough on patchy mobile networks.

Parsing latency isn't really an issue; all this stuff is IO bound anyway. CPU cycles are cheap.

Post reply on HN