Why it’s so hard to innovate in the e-mail space (2014)

It's hard to make money in that space, yes.

I've considered merging mail and messaging, so that email protocols are used to do the things people do with instant messaging, Telegram and such. The first step is to get away from queuing mail.

A first step is a new kind of mail forwarder, one for machines that don't have local mailboxes (which is most non-mail servers). It acts as an SMTP server and client. When an email comes in, and the sender reaches the end of the DATA section, the forwarder checks its forwarding list to see where this mail is supposed to go. It opens a SMTP connection to the destination and sends the email. The incoming SMTP connection is held open while this is in progress. Any problems are reported back to the sender immediately with an SMTP status code. There are no bounce emails, no queuing, and no possibility of joe-jobs. All mail is handled immediately or rejected immediately. (This only applies to single-address messages, and only to "To" addresses. Everything else is bulk.)

The next step is an IMAP server which has the same immediate orientation. When it gets an incoming email, it sends a push notification to any connected clients, while holding the incoming connection open. If at least one client is responding, a SMTP success status is returned. Otherwise, you get some status such as 251 (User Not Local, Will Forward.)

Then, a threaded email client. It would help to have the convention that an email with subject but no text, text but no subject, or text the same as subject is displayed like an instant message.

All of these can be done independently, and are backwards-compatible. If you have all of them, email is equivalent to instant messaging - you can tell if it was delivered, it goes through immediately, and it works like a conversation.

Of course, there's no big-money startup in this.

Because. It. Doesn't. Need. It. Email isn't broken.

Why are messages separated by transport? That is, why are messages sent to/from me via one transport (e.g., SMTP) read and composed in one application, and those sent via another transport (e.g., SMS, Twitter) read and composed in another? Why, as a user, do I give %#@%! what the underlying transport is?

I think the focus on a particular technology (SMTP) rather than a service (messaging) is why email clients don't advance. Bizarrely, three other transports often are integrated into email clients: NNTP, voicemail, and fax. I can only guess that it's because of a completely arbitrary reason: They are older.

Probably chat clients should also be integrated. I should be able to send my contacts an instant or time-shifted message.

You can do the first part with Haraka. For what it's worth. I started work on an imap server but decided there was little point as nobody needs a highly custom imap server.

Because the different transports have vastly different abilities and limitations that are a source of frustration if you attempt to obfuscate the differences between them and aren't served by a unified UI?