Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How IMAP works under the hood

blog.lohr.dev

Post reply on HN
Print view View original

11–20 of 78 posts

Re: How IMAP works under the hood

#11

Post by mr_mitm »

camgunz wrote: post #10

I've been working on some email stuff and I think probably four things are vexing about IMAP: - The grammar is hard. I built a parser using lpeg and I'm incredibly glad I did--doing it ad hoc won't lead to good results. - It's an asynchronous protocol. You can send lots of requests to a server and you have to tag them so you can match them up with responses later. You don't generally want to do that in a client; i.e.…

> I think this all means probably every non-web email client treats IMAP like POP and keeps its own store. I haven't done a survey or anything, but I'd be surprised if that weren't true.

Pretty sure mutt doesn't. It only caches the headers.

Re: How IMAP works under the hood

#12

Post by ldite »

mr_mitm wrote: post #11
camgunz wrote: post #10

I've been working on some email stuff and I think probably four things are vexing about IMAP: - The grammar is hard. I built a parser using lpeg and I'm incredibly glad I did--doing it ad hoc won't lead to good results. - It's an asynchronous protocol. You can send lots of requests to a server and you have to tag them so you can match them up with responses later. You don't generally want to do that in a client; i.e.…

> I think this all means probably every non-web email client treats IMAP like POP and keeps its own store. I haven't done a survey or anything, but I'd be surprised if that weren't true. Pretty sure mutt doesn't. It only caches the headers.

same for alpine (https://alpineapp.email/)

Re: How IMAP works under the hood

#13

Post by pferde »

camgunz wrote: post #10

I've been working on some email stuff and I think probably four things are vexing about IMAP: - The grammar is hard. I built a parser using lpeg and I'm incredibly glad I did--doing it ad hoc won't lead to good results. - It's an asynchronous protocol. You can send lots of requests to a server and you have to tag them so you can match them up with responses later. You don't generally want to do that in a client; i.e.…

I've found the server-side search functionality works very well, if you have good server implementation. Dovecot's, for example.

And as for treating IMAP like POP, yes, there are clients that only pay lip service to "having IMAP support", only so that they can have one more green checkbox in feature list that their present in their marketing. But there are also more serious clients.

Re: How IMAP works under the hood

#14

Post by throw0101d »

therein wrote: post #2

Interesting no attempt has been made to make it at least be less heavy on networked bytes. Especially since it is old and was meant to be used on a connection with no compression or encryption. HasChildren could have been Parent, HasNoChildren could have been Leaf or Child. And so many more things.

> Interesting no attempt has been made to make it at least be less heavy on networked bytes.

Kind of a surprising observation given the first spec (IMAP2) was released in 1998, when dial-up was still a thing:

* https://datatracker.ietf.org/doc/html/rfc1064

IMAP4 was in 1994:

* https://datatracker.ietf.org/doc/html/rfc1730

ITU V.32 gave us 9.6 kbit/s in 1998, and V.34 was 28.8 kbit/s in 1994:

* https://en.wikipedia.org/wiki/List_of_ITU-T_V-series_recomme...

Re: How IMAP works under the hood

#16

Post by ocdtrekkie »

therein wrote: post #2

Interesting no attempt has been made to make it at least be less heavy on networked bytes. Especially since it is old and was meant to be used on a connection with no compression or encryption. HasChildren could have been Parent, HasNoChildren could have been Leaf or Child. And so many more things.

When every new Gmail client ships with an entire web browser embedded to load their hundred megabytes of JavaScript, I think we've long jumped the shark on caring about brevity in the length of information in the protocol itself.

It might have mattered back then but now it would be less than a rounding error.

Re: How IMAP works under the hood

#18

Post by camgunz »

mr_mitm wrote: post #11
camgunz wrote: post #10

I've been working on some email stuff and I think probably four things are vexing about IMAP: - The grammar is hard. I built a parser using lpeg and I'm incredibly glad I did--doing it ad hoc won't lead to good results. - It's an asynchronous protocol. You can send lots of requests to a server and you have to tag them so you can match them up with responses later. You don't generally want to do that in a client; i.e.…

> I think this all means probably every non-web email client treats IMAP like POP and keeps its own store. I haven't done a survey or anything, but I'd be surprised if that weren't true. Pretty sure mutt doesn't. It only caches the headers.

Oh, yeah I guess that's what I mean, and then your connection can be used basically (again) like POP

Re: How IMAP works under the hood

#19

Post by superkuh »

Of course these days the mega-corp walled garden email providers don't really follow standards like IMAP. IMAP will not work with, say, Google's gmail or Microsoft office365, or AT&T ISP email, etc, etc. They have each implemented their own proprietery out-of-band authentication system that only works over HTTPS using the OAuth2.0 toolkit to build it. Any email client that does not explicitly design for each particular OAuth2.0 implementation (each megacorp's is slightly different) will not be able to connect over IMAP (unless they login via HTTPS using a web browser and set up "app passwords" for google, or similar for others).

Re: How IMAP works under the hood

#20

Post by FuriouslyAdrift »

philipwhiuk wrote: post #3
therein wrote: post #2

Interesting no attempt has been made to make it at least be less heavy on networked bytes. Especially since it is old and was meant to be used on a connection with no compression or encryption. HasChildren could have been Parent, HasNoChildren could have been Leaf or Child. And so many more things.

The protocol has ossified and been entrenched. In general more efficient usage of IMAP relies on extensions to the protocol. A modern replacement (JMAP) hasn't been adopted by major providers. If you really cared about data transfer size you'd use something like Protobuf.

JMAP is fully supported (since 2019) for Fastmail

https://www.fastmail.com/dev/

Post reply on HN