Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How IMAP works under the hood

blog.lohr.dev

Post reply on HN
Print view View original

71–78 of 78 posts

Re: How IMAP works under the hood

#71

Post by Avamander »

rmccue wrote: post #47
jeffbee wrote: post #27

Earlier quoted context omitted.

> Gmail’s implementation was slightly crippled Gmail is not "crippled". A tiny but vocal community of old nerds have a petrified mental model of email that they associate with unix IMAP software from the 1990's, but those concepts do not appear in the IMAP standards anywhere.

My memory is hazy on it, _but_ from memory, there were some issues around IDs and persistence which don't occur on other servers, as well as the auth being a bit funky. (Gmail also uses its own extension to the protocol for IDs instead of using eg OBJECTID.) To be clear, I have no opinion on IMAP or what things _should_ do; I certainly was not using IMAP software in the 1990s :) However, trying to implement a client…

> Gmail also uses its own extension to the protocol for IDs instead of using eg OBJECTID.

When was OBJECTID introduced and when did gmail roll their own implementation, do you happen to know?

Re: How IMAP works under the hood

#72

Post by Avamander »

donnachangstein wrote: post #46
dbcurtis wrote: post #32

Earlier quoted context omitted.

That is an immature view on how real products and real standards work. The standard document may say one thing, but what people do in the real world is the real standard. For context: I spent 11 years at Intel managing pre-silicon and post-silicon processor validation. No processor that does only and exactly what the Programmers Reference Manual says, and takes the phrase "undefined behavior" seriously, will be succe…

If history has taught us anything, it's that Google is happy to willfully ignore, rewrite, and use their market dominance to snuff out any existing standards if they see a way to seize control and make money off something.

Absolutely, but in the context of IMAP, it's already a mess of special cases. Gmail having a weirdness, an extension or two changes basically nothing. Not to mention the fact that what they offer over the web is more flexible than what IMAP usually allows, it simply does not and can not map directly to IMAP.

It's really dumb that one message has* to exist in only one location for example, labels are so much better.

Re: How IMAP works under the hood

#73

Post by andrewshadura »

zaik wrote: post #6
therein wrote: post #2

Interesting no attempt has been made to make it at least be less heavy on networked bytes. Especially since it is old and was meant to be used on a connection with no compression or encryption. HasChildren could have been Parent, HasNoChildren could have been Leaf or Child. And so many more things.

Despite people always complaining about their perceived inefficiencies, standard protocols like IMAP or XMPP always seem to work on a crappy connection, when most of the modern web doesn't.

Is this a joke? Both XMPP and IMAP are terrible over an unreliable connection.

Re: How IMAP works under the hood

#74

Post by isaachinman »

azhenley wrote: post #67
isaachinman wrote: post #66

Earlier quoted context omitted.

What are you building?

An email client for Gmail.

Interesting. I take it you've seen Notion's new product?

Are you aiming to directly compete with Notion/Superhuman?

Edit: I see you're an academic, perhaps my questions are irrelevant

Re: How IMAP works under the hood

#75

Post by superkuh »

slightwinder wrote: post #34
superkuh wrote: post #19

Of course these days the mega-corp walled garden email providers don't really follow standards like IMAP. IMAP will not work with, say, Google's gmail or Microsoft office365, or AT&T ISP email, etc, etc. They have each implemented their own proprietery out-of-band authentication system that only works over HTTPS using the OAuth2.0 toolkit to build it. Any email client that does not explicitly design for each particul…

> IMAP will not work with, say, Google's gmail or Microsoft office365 Except they do, to some degree. It works well enough that my Thunderbird allows me fetching or moving of mails. Not sure about advanced features like search or server-side filtering, never tried them, but this seems to be a bit more wacky with other clients & servicers too. > They have each implemented their own proprietery out-of-band authenticati…

Look in the thunderbird source. The devs had to make a many hundreds of lines config for each mega-corp. So, yeah, the mega-corps are covered by major workarounds in major email clients. But for normal email clients it's very hit and miss.

And while OAuth2 is open it is a toolkit for making protocols, not a protocol. And each megacorps implementation is different and handled differently.

Re: How IMAP works under the hood

#76

Post by superkuh »

jeffbee wrote: post #52
Aloisius wrote: post #51

Earlier quoted context omitted.

It can, but it does require doing a lot of Google-specific things (set up a google cloud account, create a consent screen, get a security review, justify your usage of the IMAP API instead of the web APIs to them, find the right scopes, etc) or instruct users to go through multiple screens in their google settings to create an app password. Google really doesn't want you to use IMAP. They're trying to push everyone t…

You seem to be taking the perspective of an application developer or something like that? Certainly for users all they need to do is roll in with their favorite IMAP client and use it. All of what you said applies not at all to users.

It does. Gmail disabled imap login for everyone. You explicitly have to find and set up a special "app password" to enable just IMAP now. Many major corporate email clients (like thunderbird) have implemented these corporation-mail-company specific work arounds though so the user doesn't notice them.

Re: How IMAP works under the hood

#77

Post by calvinmorrison »

Mailtemi wrote: post #69
calvinmorrison wrote: post #68

Earlier quoted context omitted.

It's designed well, from the ground up with a lot of lessons learned from some pretty serious email junkies. Anyone hemming and hawing about JSON being selected as the transport encoding or whatever is just raising silly points. You could do JMAP over protobuf too probably if you cared, but the whole point (IMO) is to get the hell away from unstructured protocols.

Yep, there's always someone ranting about JSON inefficiencies, which I don't observe at all :)

oh i mean i rant about it too but its like, a byproduct of the real goal, and since most clients are running some type of webapp it makes sense. you could definitely use something like messagepack or protobuf if you had an extension that negotiates it.

Re: How IMAP works under the hood

#78

Post by FuriouslyAdrift »

n3storm wrote: post #24
FuriouslyAdrift wrote: post #20

Earlier quoted context omitted.

JMAP is fully supported (since 2019) for Fastmail https://www.fastmail.com/dev/

Jmap was developed by Fastmail, meanwhile all primary FLOSS mail servers do not support it and just new projects like Stalwart are starting to bring it up.

Cyrus supports it. https://www.cyrusimap.org/imap/developer/jmap.html
Post reply on HN