Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Replace IMAP

gaborcselle.com

Post reply on HN
Print view View original

31–40 of 62 posts

Re: How to Replace IMAP

#31

Post by gaborcselle »

JoachimSchipper wrote: post #18

This is a collection of the author's favourite technologies/gripes, not a useful proposal. A large number of errors can be found by considering that it'll have to talk SMTP on the back end. I'll only point to some "highlights". OAuth isn't nearly secure enough ( http://hueniverse.com/2009/04/explaining-the-oauth-session-f... ) for your e-mail (which, don't forget, gives access to everything via password resets). Let'…

Thanks for your suggestions Joachim! This is why I wrote that blog entry in the first place: To get some high-quality feedback.

I think IMAP could be replaced independently of SMTP. You could conceivable replace an IMAP server with a reMAP server and not affect the rest of the mail ecosystem. SMTP will likely not die for another hundred years.

1. I know about the OAuth problems. Do you think we'll see a more secure solution in the next few years?

2. Is there an easy way to validate UTF-8ness? Happy to hear suggestions.

3. I know about THREAD and SEARCH and IDLE but find them clunky. Under the assumption that we're replacing IMAP with something RESTful you might as well get those ones right.

Happy to take this offline as well - my email is on my HP

Re: How to Replace IMAP

#32

Post by coryrc »

sandGorgon wrote: post #6
ajross wrote: post #2

Not to spend too much effort defending IMAP (which does, to be fair, kinda suck) but... come on. It's not that hard. It works just fine, enjoying pervasive support with very high quality clients and servers available freely on essentially all platforms. A new protocol would be ... prettier, I guess. But to pretend that the reason facebook and IM are replacing email is IMAP (and not, y'know, spam) is borderline delusi…

http://jerakeen.org/notes/2010/01/yay-more-email-clients/ The Android GMail client is a perfect example of what a client looks like in this world. It talks to the (secret / private) GMail API, it does offline mail reading, and queues actions so you can archive / filter / whatever mails while offline and it’ll push changes later. You can read and write mail. It doesn’t try to do anything clever, because anything cleve…

Mozilla Seamonkey and Thunderbird do that all with IMAP (except the web client part).

Re: How to Replace IMAP

#33

Post by IgorPartola »

gaborcselle wrote: post #8
ajross wrote: post #2

Not to spend too much effort defending IMAP (which does, to be fair, kinda suck) but... come on. It's not that hard. It works just fine, enjoying pervasive support with very high quality clients and servers available freely on essentially all platforms. A new protocol would be ... prettier, I guess. But to pretend that the reason facebook and IM are replacing email is IMAP (and not, y'know, spam) is borderline delusi…

The combination of IMAP and MIME are hard. Yes, we could be happy with the status quo. But I do believe that one of the things that's holding up innovation around email is the lack of easy access to data. Proposed exercise: Write a script that downloads and displays your Twitter feed. Then write a script that downloads and displays your email. The email exercise will take you an order of magnitude longer. Spam filter…

You might want to revise your challenge.

IMAP and MIME are hard because there is a higher level of information that is available. Can you attach an archive over Twitter (not link to but attach)? Can you send HTML? They are not the same thing. If you want something really simple your code will be simple. If you want more use cases than "i can haz cheeseburger" then you need more expressive power and more complexity.

Re: How to Replace IMAP

#34

Post by dlsspy »

> All data that's ever sent to or received from the server would be in JSON format. JSON is much more human-readable than XML.

I don't need my protocols to be human readable. I need my computers to read them and know that I've got good, bug-free, reusable, efficient, and easy-to understand implementations.

This isn't an argument for XML over JSON -- I've made good use of both, but I don't want to consider how easy it is to type up communication protocols when I'm defining stuff the replacement for something I expect to take over the world.

It's by far easier to implement the memcached binary protocol as a client or a server than it is to implement the text protocol. Easier to get right, more regular, and just generally more pleasant.

Re: How to Replace IMAP

#35

Post by jwhitlark »

pyre wrote: post #23
jwhitlark wrote: post #16

My responses to your Design Outline. 1. I'm not convinced that http/https is the best protocol. I agree that it's the defacto standard, but if you're going to go through this, there might be a better way. (Push, in particular, is a problem). 2. Stateless is wonderful, agreed. 3. I agree with JSON over XML, but what about binary data? That is the weakest part of JSON, in my opinion. 4. I like where you're going with t…

> 3. I agree with JSON over XML, but what about binary data? That is the weakest part of JSON, in my opinion. What binary data? All attachments in emails are Base64 encoded. That's what MIME is all about. > 6. how about SHA-1 hashes of content? Done properly, that would eliminate any duplicate messages, and could make it very easy to include other messages. This might also allow content to be transfered between accou…

>> 3. ugh. Hadn't realized that. I like IMAP even less, now. (Told you I didn't know much about MIME ;-)

>> 6. I wasn't intending it as a security mechanism, but as a way to know which bits you have and which you don't. Do you think SHA-1 is still a bad idea in that case?

Re: How to Replace IMAP

#36

Post by edd »

IgorPartola wrote: post #24
gaborcselle wrote: post #8

Earlier quoted context omitted.

The combination of IMAP and MIME are hard. Yes, we could be happy with the status quo. But I do believe that one of the things that's holding up innovation around email is the lack of easy access to data. Proposed exercise: Write a script that downloads and displays your Twitter feed. Then write a script that downloads and displays your email. The email exercise will take you an order of magnitude longer. Spam filter…

$mbox = imap_open( $host, $login, $password ); foreach( imap_search( $mbox, 'UNSEEN', SE_UID ) as $uid ) { $header = imap_fetch_overview( $mbox, $uid, FT_UID ); echo $header[0]->subject . "\n"; } imap_close( $mbox );

    $status_url = 'http://twitter.com/statuses/user_timeline.json?screen_name='.$user;
    $timeline = json_decode(file_get_contents($status_url));
    for($i = 0; $i text ."\n";
    }
One line in it then.

Re: How to Replace IMAP

#37

Post by qjz »

__david__ wrote: post #30
JoachimSchipper wrote: post #18

This is a collection of the author's favourite technologies/gripes, not a useful proposal. A large number of errors can be found by considering that it'll have to talk SMTP on the back end. I'll only point to some "highlights". OAuth isn't nearly secure enough ( http://hueniverse.com/2009/04/explaining-the-oauth-session-f... ) for your e-mail (which, don't forget, gives access to everything via password resets). Let'…

With regards to search, I recently installed Cyrus Squatter for full text IMAP searching and it is blazing fast. To me it seems just as fast as any of the special case local searches I've seen people build (Spotlight/recent Thunderbirds).

Any new mail storage protocol should anticipate the need to store encrypted messages at rest. Sure, it can play a part by storing an encrypted index on the server to deliver to authorized clients, but all decryption and index updates should take place client-side with the user's private key. Stable UID's can help make this a reality. There is no reason to store messages in plaintext on the server, and no new protocol should depend on it.

Re: How to Replace IMAP

#38

Post by pyre »

jwhitlark wrote: post #35
pyre wrote: post #23

Earlier quoted context omitted.

> 3. I agree with JSON over XML, but what about binary data? That is the weakest part of JSON, in my opinion. What binary data? All attachments in emails are Base64 encoded. That's what MIME is all about. > 6. how about SHA-1 hashes of content? Done properly, that would eliminate any duplicate messages, and could make it very easy to include other messages. This might also allow content to be transfered between accou…

>> 3. ugh. Hadn't realized that. I like IMAP even less, now. (Told you I didn't know much about MIME ;-) >> 6. I wasn't intending it as a security mechanism, but as a way to know which bits you have and which you don't. Do you think SHA-1 is still a bad idea in that case?

I like IMAP even less, now. (Told you I didn't know much about MIME ;-)

IMAP might do some sort of MIME-handling, but MIME is part of the email spec, not the IMAP spec. Hating IMAP because of MIME is like hating cars because of the invention of the wheel (i.e. thinking that the wheel was invented as part of the production of the initial automobile).

http://en.wikipedia.org/wiki/MIME:

  Multipurpose Internet Mail Extensions (MIME) is an 
  Internet standard that extends the format of e-mail
  to support:
    * Text in character sets other than ASCII
    * Non-text attachments
    * Message bodies with multiple parts
    * Header information in non-ASCII character sets
MIME is used to say things like 'this block of text is a base64-encoded file attachment, but also to say 'this block of text is encoded with this charset.'

If you use Gmail, try the 'open original message' option on emails that have attachments or multiple text parts on them (or even emails that have both a text and an html version of the content).

Dealing with MIME is part of dealing with emails in general, regardless of if you're interacting with an IMAP server, or composing an email to send to another user on the same mainframe (i.e. just moving files around).

Re: How to Replace IMAP

#39

Post by qjz »

jwhitlark wrote: post #35
pyre wrote: post #23

Earlier quoted context omitted.

> 3. I agree with JSON over XML, but what about binary data? That is the weakest part of JSON, in my opinion. What binary data? All attachments in emails are Base64 encoded. That's what MIME is all about. > 6. how about SHA-1 hashes of content? Done properly, that would eliminate any duplicate messages, and could make it very easy to include other messages. This might also allow content to be transfered between accou…

>> 3. ugh. Hadn't realized that. I like IMAP even less, now. (Told you I didn't know much about MIME ;-) >> 6. I wasn't intending it as a security mechanism, but as a way to know which bits you have and which you don't. Do you think SHA-1 is still a bad idea in that case?

HTTP also depends on MIME.

Duplicate suppression can be tricky. The overhead may not be worth the benefit, and the risk of message corruption is increased. Besides, if the mail is encrypted by the user, it may not even be feasible to do server-side.

Re: How to Replace IMAP

#40

Post by gibsonf1 »

ajross wrote: post #2

Not to spend too much effort defending IMAP (which does, to be fair, kinda suck) but... come on. It's not that hard. It works just fine, enjoying pervasive support with very high quality clients and servers available freely on essentially all platforms. A new protocol would be ... prettier, I guess. But to pretend that the reason facebook and IM are replacing email is IMAP (and not, y'know, spam) is borderline delusi…

It seems like the key is to work on top of an interface that does all the dirty work of negotiating with IMAP for you, such as the lisp mel-base library. Then you really don't care how messy imap might be, as you can build on top of it at increasing levels of abstraction.
Post reply on HN