Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why it’s so hard to innovate in the e-mail space (2014)

medium.com

Post reply on HN
Print view View original

31–40 of 68 posts

Re: Why it’s so hard to innovate in the e-mail space (2014)

#31

Post by zombition »

hackuser wrote: post #26
aetherson wrote: post #20

Earlier quoted context omitted.

Because the different transports have vastly different abilities and limitations that are a source of frustration if you attempt to obfuscate the differences between them and aren't served by a unified UI?

Could you be more specific? For example, comparing SMS and SMTP, the two most prominent transports?

SMS is used to send tiny chunks of text, and SMTP can be used to send message content in HTML format and rich text in addition to plaintext. I haven't worked much with rich text format, but HTML format for sure would be relatively impractical via SMS (XML adds a lot of characters, so you would have a lot of little chunks to send if you wanted to send HTML over SMS).

Re: Why it’s so hard to innovate in the e-mail space (2014)

#32

Post by hackuser »

zombition wrote: post #31
hackuser wrote: post #26

Earlier quoted context omitted.

Could you be more specific? For example, comparing SMS and SMTP, the two most prominent transports?

SMS is used to send tiny chunks of text, and SMTP can be used to send message content in HTML format and rich text in addition to plaintext. I haven't worked much with rich text format, but HTML format for sure would be relatively impractical via SMS (XML adds a lot of characters, so you would have a lot of little chunks to send if you wanted to send HTML over SMS).

I agree with that. I don't mean that I want to send my email messages visa SMS. I mean that I want all my messages sent/received and read in one messaging client, whether they are transported via SMS, SMTP, etc. If the HTML message should go via SMTP, I expect my software to deal with that. I want the content; the transport is an obscure detail for developers to worry about (speaking as a typical end user).

Re: Why it’s so hard to innovate in the e-mail space (2014)

#33

Post by baudehlo »

hackuser wrote: post #30
drumdance wrote: post #27

Earlier quoted context omitted.

Sending is easy. Receiving and sorting and filtering is harder.

> Sending is easy. Receiving and sorting and filtering is harder. I don't understand: Receiving messages on any of the transports is done every day and billions of platforms. After receiving, loading the messages into a common database, including mapping metadata such as sender, date, etc., seems relatively simple. Once in the database, sorting and filtering are simple. What am I missing?

Spam, generally.

Because email uniquely among communication protocols allows sending a message without any previous agreement from the recipient, will always be susceptible to spam (and getting rid of that feature would destroy emails benefits).

Plus people want to filter differently. People deal with their inboxes uniquely.

Re: Why it’s so hard to innovate in the e-mail space (2014)

#34

Post by hackuser »

baudehlo wrote: post #33
hackuser wrote: post #30

Earlier quoted context omitted.

> Sending is easy. Receiving and sorting and filtering is harder. I don't understand: Receiving messages on any of the transports is done every day and billions of platforms. After receiving, loading the messages into a common database, including mapping metadata such as sender, date, etc., seems relatively simple. Once in the database, sorting and filtering are simple. What am I missing?

Spam, generally. Because email uniquely among communication protocols allows sending a message without any previous agreement from the recipient, will always be susceptible to spam (and getting rid of that feature would destroy emails benefits). Plus people want to filter differently. People deal with their inboxes uniquely.

Why can't the universal mail client I propose have a spam filter? More likely, my incoming mail server will have the spam filter; the client will have the spam folder. What does it matter if only SMTP messages end up in that folder?

> Plus people want to filter differently. People deal with their inboxes uniquely.

That issue affects every messaging client's filtering. How does it affect a client any differently if it combines, for example, SMTP and SMS?

Re: Why it’s so hard to innovate in the e-mail space (2014)

#36

Post by zombition »

hackuser wrote: post #30
drumdance wrote: post #27

Earlier quoted context omitted.

Sending is easy. Receiving and sorting and filtering is harder.

> Sending is easy. Receiving and sorting and filtering is harder. I don't understand: Receiving messages on any of the transports is done every day and billions of platforms. After receiving, loading the messages into a common database, including mapping metadata such as sender, date, etc., seems relatively simple. Once in the database, sorting and filtering are simple. What am I missing?

> Why, as a user, do I give %#@%! what the underlying transport is? > I don't understand: Receiving messages on any of the transports is done every day and billions of platforms. After receiving, loading the messages into a common database, including mapping metadata such as sender, date, etc., seems relatively simple. Once in the database, sorting and filtering are simple.

I agree with this completely--so much so that I decided to make my own communication protocol (and client-server implementation) that would do whatever I wanted and still be backwards compatible with email. You can see the results here: https://github.com/Zombition/nsmtp-monolith. So clearly it's not impossible.

> What am I missing?

The biggest difficulty isn't from pulling out metadata and mapping it to whatever you store locally. The specifications of different protocols determine the user behaviors that are possible with each protocol, and mapping behaviors between different protocols in a way that makes sense to end users is much more difficult if you want to retain quality communication with users who are not using your software to pull everything into one place. What I found from implementing a new protocol that aimed to be backwards compatible with SMTP was that maintaining backwards compatibility with SMTP guided 99% of my decisions for making the new protocol. If I wanted to add a feature, but it would be confusing for end users who might be trying to communicate with regular SMTP users (or the SMTP users that they were trying to communicate with), the only practical solution is to axe the feature or rehash it in a way that would be less confusing for communication between the two.

In practice this is quite difficult, because you have to balance awareness of both protocols with awareness of how data sent through one can be meaningfully translated to the other, and while you're considering those things, you also have to consider what the user experience is going to be like for people on both ends of the software. If you're trying to maintain compatibility with a widespread protocol like SMTP, there may be an enormous number of different client interfaces that can add a significant amount of variation to the user experience. It's a bit like unfolding a fractal.

Re: Why it’s so hard to innovate in the e-mail space (2014)

#37

Post by bradlys »

AlexB138 wrote: post #22
vonklaus wrote: post #8

There are tons of reasons why this is so hard. Leaving aside all of those there are 2 that I see as the biggest: * A 100x improvement over current email is needed for a switch. * It has to be 100x cheaper than the current product. While you can't multiply by 0 and I meant that facetiously, users won't pay for the software or only niche consumers. Which is why email providers that get paid focus on things like securit…

MailChimp? Constant Contact? I may be wrong, I'm not terribly familiar with either, but I believe they're largely focused on email.

Mailchimp and such aren't for everyday emailing between you and me. They're for emailing large lists of users one email a day or week or whatever and using tracking information based off referral links users click on in the email.

Re: Why it’s so hard to innovate in the e-mail space (2014)

#38

Post by baudehlo »

hackuser wrote: post #34
baudehlo wrote: post #33

Earlier quoted context omitted.

Spam, generally. Because email uniquely among communication protocols allows sending a message without any previous agreement from the recipient, will always be susceptible to spam (and getting rid of that feature would destroy emails benefits). Plus people want to filter differently. People deal with their inboxes uniquely.

Why can't the universal mail client I propose have a spam filter? More likely, my incoming mail server will have the spam filter; the client will have the spam folder. What does it matter if only SMTP messages end up in that folder? > Plus people want to filter differently. People deal with their inboxes uniquely. That issue affects every messaging client's filtering. How does it affect a client any differently if it…

Oh you absolutely can, but there are a mix of problems some of which makes spam filtering better on the server (knowing bad sources of email, like infected desktop machines), some of which makes filtering better on the client (like knowing your users' preferences for what they consider spam).

There's a huge long history of very smart people (not even including myself in that list - I've worked with some people MUCH smarter than myself on this) working on this problem, including the very creator of this site, pg himself, and even he doesn't consider this a solved problem. People are still paid well to combat spam every day on behalf of demanding customers (and I'm grateful I'm not one of them any more) and it will always require new techniques and new input. Hopefully you will contribute too.

Re: Why it’s so hard to innovate in the e-mail space (2014)

#39

Post by baudehlo »

hackuser wrote: post #34
baudehlo wrote: post #33

Earlier quoted context omitted.

Spam, generally. Because email uniquely among communication protocols allows sending a message without any previous agreement from the recipient, will always be susceptible to spam (and getting rid of that feature would destroy emails benefits). Plus people want to filter differently. People deal with their inboxes uniquely.

Why can't the universal mail client I propose have a spam filter? More likely, my incoming mail server will have the spam filter; the client will have the spam folder. What does it matter if only SMTP messages end up in that folder? > Plus people want to filter differently. People deal with their inboxes uniquely. That issue affects every messaging client's filtering. How does it affect a client any differently if it…

Oh and in terms of the other question: how many people get enough SMS messages that they want to filter them? Filtering is uniquely suited to a non realtime messaging system. Like email. I don't want to filter Slack/IRC messages either. I just ignore what I didn't read.
Post reply on HN