Live data from Hacker News

How to Replace IMAP

gaborcselle.com

21–30 of 62 posts

Re: How to Replace IMAP

#21
post #2

Not to spend too much effort defending IMAP (which does, to be fair, kinda suck) but... come on. It's not that hard. It works just fine, enjoying pervasive support with very high quality clients and servers available freely on essentially all platforms. A new protocol would be ... prettier, I guess. But to pretend that the reason facebook and IM are replacing email is IMAP (and not, y'know, spam) is borderline delusi…

http://jerakeen.org/notes/2010/01/yay-more-email-clients/ The Android GMail client is a perfect example of what a client looks like in this world. It talks to the (secret / private) GMail API, it does offline mail reading, and queues actions so you can archive / filter / whatever mails while offline and it’ll push changes later. You can read and write mail. It doesn’t try to do anything clever, because anything cleve…

If not designed very carefully, though, it can reduce the user's ability to view things as they like them. With the current status quo of mailservers not handling message threading and such, I can choose a client that handles it how I want. If the server is doing it, it had better either do it how I want it, or provide some way to customize what it's doing.

(For example, I don't like gmail's flat "conversation" view; I much prefer a "threaded conversation" view, like classic Usenet readers, or HN discussions.)

Re: How to Replace IMAP

#22
post #20
post #15

Earlier quoted context omitted.

All it requires is a major server (i.e gmail) providing it as an option and a major client (say a iPhone email app) supporting that option.

oh, that's all? it'll require an RFC or some other kind of standard, which will take a long time to finalize. then someone will have to write a reference spec server and client, then "real" versions in "real" languages, then get them both to the stability, security, and scalability of current imap clients and servers. then once big email providers (isps, google, etc.) start supporting it on the server end (gotta fact…

If you are going for a clean design, you can go in the opposite direction too. I'm fairly sure Ethernet and html evolved from a working solution. Now, granted there should be a really clean and open interface if we are going to try and make it a standard, but it's often the easier approach.

Re: How to Replace IMAP

#23

My responses to your Design Outline. 1. I'm not convinced that http/https is the best protocol. I agree that it's the defacto standard, but if you're going to go through this, there might be a better way. (Push, in particular, is a problem). 2. Stateless is wonderful, agreed. 3. I agree with JSON over XML, but what about binary data? That is the weakest part of JSON, in my opinion. 4. I like where you're going with t…

> 3. I agree with JSON over XML, but what about binary data? That is the weakest part of JSON, in my opinion.

What binary data? All attachments in emails are Base64 encoded. That's what MIME is all about.

> 6. how about SHA-1 hashes of content? Done properly, that would eliminate any duplicate messages, and could make it very easy to include other messages. This might also allow content to be transfered between accounts only once, dropping bandwidth and speeding delivery.

Developers should start to shy away from SHA-1 since it's already started to get broken.

Re: How to Replace IMAP

#24
post #2

Not to spend too much effort defending IMAP (which does, to be fair, kinda suck) but... come on. It's not that hard. It works just fine, enjoying pervasive support with very high quality clients and servers available freely on essentially all platforms. A new protocol would be ... prettier, I guess. But to pretend that the reason facebook and IM are replacing email is IMAP (and not, y'know, spam) is borderline delusi…

The combination of IMAP and MIME are hard. Yes, we could be happy with the status quo. But I do believe that one of the things that's holding up innovation around email is the lack of easy access to data. Proposed exercise: Write a script that downloads and displays your Twitter feed. Then write a script that downloads and displays your email. The email exercise will take you an order of magnitude longer. Spam filter…

  $mbox = imap_open( $host, $login, $password );
  foreach( imap_search( $mbox, 'UNSEEN', SE_UID ) as $uid  ) {
    $header = imap_fetch_overview( $mbox, $uid, FT_UID );
    echo $header[0]->subject . "\n";
  }
  imap_close( $mbox );

Re: How to Replace IMAP

#25

Earlier quoted context omitted.

http://jerakeen.org/notes/2010/01/yay-more-email-clients/ The Android GMail client is a perfect example of what a client looks like in this world. It talks to the (secret / private) GMail API, it does offline mail reading, and queues actions so you can archive / filter / whatever mails while offline and it’ll push changes later. You can read and write mail. It doesn’t try to do anything clever, because anything cleve…

If not designed very carefully, though, it can reduce the user's ability to view things as they like them. With the current status quo of mailservers not handling message threading and such, I can choose a client that handles it how I want. If the server is doing it, it had better either do it how I want it, or provide some way to customize what it's doing. (For example, I don't like gmail's flat "conversation" view;…

reMAP claims to treat conversations as first class objects. Given suitable APIs on top, I think you could build different views on different email clients.

I think what is meant by a new protocol and server innovation is a change in the data model. I do not believe they are going to dictate views.

Re: How to Replace IMAP

#26
post #2

Not to spend too much effort defending IMAP (which does, to be fair, kinda suck) but... come on. It's not that hard. It works just fine, enjoying pervasive support with very high quality clients and servers available freely on essentially all platforms. A new protocol would be ... prettier, I guess. But to pretend that the reason facebook and IM are replacing email is IMAP (and not, y'know, spam) is borderline delusi…

But to pretend that the reason facebook and IM are replacing email is IMAP (and not, y'know, spam) is borderline delusional.

As is believing that it's about spam (or probably believing it all, for that matter, but let's stay on topic).

I don't understand people bitching about spam in 2010. I have probably a dozen email addresses that I publish all over the web and I get maybe one spam message a month. I get way more "spam" through Twitter than I do through my email.

Re: How to Replace IMAP

#27

My responses to your Design Outline. 1. I'm not convinced that http/https is the best protocol. I agree that it's the defacto standard, but if you're going to go through this, there might be a better way. (Push, in particular, is a problem). 2. Stateless is wonderful, agreed. 3. I agree with JSON over XML, but what about binary data? That is the weakest part of JSON, in my opinion. 4. I like where you're going with t…

3) Handling binary data is incredibly easy when you're already using HTTP: just use hyperlinks to other resources!

Re: How to Replace IMAP

#28
post #27

My responses to your Design Outline. 1. I'm not convinced that http/https is the best protocol. I agree that it's the defacto standard, but if you're going to go through this, there might be a better way. (Push, in particular, is a problem). 2. Stateless is wonderful, agreed. 3. I agree with JSON over XML, but what about binary data? That is the weakest part of JSON, in my opinion. 4. I like where you're going with t…

3) Handling binary data is incredibly easy when you're already using HTTP: just use hyperlinks to other resources!

Yeah, that was my thought about that too. Attachments downloaded separately via HTTP.

Re: How to Replace IMAP

#29
First, I love reMail. Gabor breathes email and search. If you own an iPhone and use Gmail or Google Apps, get reMail--you'll thank him.

That said, it's easy to skim this article and nod, "Yeah! Yeah!", but when we're discussing these, laymen may need to know a few of these seem over-simplified.

> "TCP connections are great, but for transferring large amounts of email securely, HTTP is the way to go."

1. http://wiki.answers.com/Q/What_is_the_difference_between_tcp... - HTTP uses a TCP connection. For that matter so does SMTP and IMAP and the proposed REMAP.

2. I generally have at least 5 devices using IMAP (on Google Apps) at once, and interleave my interactions with these devices arbitrarily, yet expect each to show me the exact same state when I look at it. Keeping them all in the apparently same state is the sort of intractable software problem bedeviled with implementation details, and I agree throwing out IMAP and starting over with lessons learned could be easier than, say, the RFC linked below.

3. http://ajaxian.com/archives/json-vs-xml-the-debate - Seems this point is something like GIF vs JPG vs PNG, with a bit of "Markdown vs HTML" thrown in. Meanwhile, both XML and JSON are missing an important concept given labels/folders and conversations: multiple references to single objects.

4. Conversations or threads exist today as In-Reply-To: GUID and References: GUID headers, among others, but see the JSON problem above. We don't realize these headers are there because clients generally ignore these and try to group on Subject instead. Gmail grouping respecting existing headers is much better, but from usability point of view, Gmail's approach makes it challenging to bulk delete individual matching messages across a broad set of conversations. For example, to bulk move or mark all messages on a particular date results in the full conversations being moved or marked. So I like this idea as long as I can still optionally operate on sets of messages without affecting the rest of the conversation.

5. http://googlesystem.blogspot.com/2007/08/organizing-chaos-fo... - It's not yet clear that people prefer tags to folders. After all, monkeys don't expect a banana to be in two boxes at once. I don't want to label anything with keywords. I want semantic search.

6. Excellent point. Changing GUIDs out from under us is just lame. Having had to resort to http://github.com/rgrove/larch recently to move a decade of email onto Google Apps because Thunderbird was putting new IDs on each moved message, I fully agree this problem is evil.

7. Seems we're stuck with MIME until we get rid of every legacy email server in the world, or control what servers our recipients use. Between the new server and new clients, sure. But the server will have to know it to send it. Granted, IMAP is for receiving, and SMTP is for sending, but something in the chain has to know MIME.

8. "Call" an HTTP endpoint that "returns" when messages arrive? Just nomenclature, since IDLE is a "call" that "returns" when state changes. Either way, the socket is held open, so neither of these is "push". Maybe Gabor meant "posts back" instead of "returns"? In any case, under the hood, iPhone push and Microsoft ActiveSync are exploiting out-of-band channel such as SMS to notify the phone it should poll (or in case of ActiveSync, maybe doing a "long pull"). See http://msexchangeteam.com/archive/2004/04/26/120520.aspx versus http://tools.ietf.org/internet-drafts/draft-maes-lemonade-p-...

9. Will the RFC specify a search results algorithm, or will the same search return different results depending on the implementation of mail server the client is connected to? We manipulate a "client" tool to perform the search and access the email, so our natural mental model is of the client at hand, not of the remote server. This lets the brain wrap itself around Gmail's web search versus Apple Mail's search box vs OS X Spotlight vs Gabor's reMail all returning different results. Practially speaking, search at the server does make far more sense from both a data retention and security standpoint. I have 509 MB of index in Gabor's "reMail" app on my iPhone, and would rather not.

Thanks for brainstorming this, and thanks for a kickass iPhone app.

Re: How to Replace IMAP

#30

This is a collection of the author's favourite technologies/gripes, not a useful proposal. A large number of errors can be found by considering that it'll have to talk SMTP on the back end. I'll only point to some "highlights". OAuth isn't nearly secure enough ( http://hueniverse.com/2009/04/explaining-the-oauth-session-f... ) for your e-mail (which, don't forget, gives access to everything via password resets). Let'…

With regards to search, I recently installed Cyrus Squatter for full text IMAP searching and it is blazing fast. To me it seems just as fast as any of the special case local searches I've seen people build (Spotlight/recent Thunderbirds).
Post reply on HN