Live data from Hacker News

Email encryption is here – use STARTTLS everywhere

dwheeler.com

31–40 of 74 posts

Re: Email encryption is here – use STARTTLS everywhere

#31

What we need next is a widely adopted standard of end-to-end opportunistic encryption, similar to Signal. PKI and PGP are too complex for end users. The problem that prevents such a standard from ever taking off is that, unlike TLS, there is zero interest from major providers to push end-to-end encyption, because they absolutely need access to the plain text for commercial reasons. So they advertise STARTTLS as "emai…

> PKI and PGP are too complex for end users

They are also the only possibilities to provide proper end-to-end encryption. The last thing we need is another proprietary “standard” (remember how those same companies pushed for drm in the w3c spec).

Yet the only real change can come when end users understand the implications of using the free email providers. I think this is harder for most than using a pgp email client plugin.

Re: Email encryption is here – use STARTTLS everywhere

#32

What we need next is a widely adopted standard of end-to-end opportunistic encryption, similar to Signal. PKI and PGP are too complex for end users. The problem that prevents such a standard from ever taking off is that, unlike TLS, there is zero interest from major providers to push end-to-end encyption, because they absolutely need access to the plain text for commercial reasons. So they advertise STARTTLS as "emai…

It's not a standard, but it's there: autocrypt (https://autocrypt.org) aims to make end to end encryption as easy as possible. It's using PGP underneath but the whole idea is to hide key management (the real pain point) from the user. It's a long term initiative, divided in multiple levels of "compliance" that gradually break compatibility with existing systems more and more. Level 1 is there already, with some implementations.

Re: Email encryption is here – use STARTTLS everywhere

#33
post #22

Earlier quoted context omitted.

This is just not true. The Internet is full of stories of people having trouble with large email providers (notably Google and Microsoft) accepting self-hosted users' mail. A self-hosted setup's IP starts with 0 reputation (or worse, depending on who owned the IP address beforehand) and will face throttling and outright blocking for a painful amount of time. Furthermore, if you forward your email to e.g. a Gmail inbo…

Point 1: spf, dkim, dmarc is enough to get yourself neutral with gmail. You have to do it from get-go tho, before you tarnish your reputation. Being in DNSBL lists does not help either. Set up proper reverse dns entries also. Tls is completely optional. Haven't had any trouble neither with gmail nor ms using both residential and datacenter ips in two EU countries. I don't consider myself a hardcore linux admin specia…

> you're running a mailserver without spamassasin? What is this, the 1990s?

More like, relying on SpamAssassin to reliably and correctly classify spam is the anachronism here.

Even if SpamAssassin could do a good job of filtering spam today (which it can't), the set of messages that SpamAssassin classifies as spam will inevitably differ from the set of messages that Gmail classifies as spam, so the only thing this achieves is another point where you have to worry about false positives and training the classifiers.

Furthermore, the problem is exacerbated by poor support of forwarding in modern MTAs. They will queue all received messages first, and when Gmail rejects certain messages during the IMAP session (spam with high confidence, or messages it considers invalid), they result in a bounce, and the thus ensuing backscatter further degrades your mail server's reputation.

The closest thing to correct behavior is Exim's cutthrough delivery feature, which has been implemented "relatively" recently, and even so it is not complete and I've run into issues with it with non-textbook setups.

Re: Email encryption is here – use STARTTLS everywhere

#34
post #12
post #10

Earlier quoted context omitted.

That's FUD. If a system is correctly set up, you'll have no deliverability issues. Edit: srsly guys, HACKER news people say that it's impossible for a person to have a proper mailserver set up? That's hilarious, if not sad!

Keeping your own server up at several nines of uptime is a non-trivial endeavour.

SMTP requires sending servers to retry for at least 4 days, so 98.9% uptime over a year is guaranteed to be sufficient so as to not lose any mail ... so that is "several" as in "one", I suppose?

Of course you can have a lot more downtime than that, even just 20% uptime would be sufficient if timed right to ensure that no mail gets lost.

Or in other words: My central heating system has higher availability requirements than my mailserver, so I guess everyone in non-tropical places has a non-trivial endeavour going?

Re: Email encryption is here – use STARTTLS everywhere

#35
post #8
post #2

Sure, STARTTLS is better than nothing, but your email provider still reads your email. When we talk about email encryption, we mean end-to-end encryption.

I am my own provider. It's really not that hard to configure a mail server. Other peoples' providers, however... so I tried using PGP. It's somehow manages to be more difficult to use than configuring and running your own mailsystem which doesn't deliver straight to gmails' spam folder, but I digress; but then my emails are at mercy of the end devices: who knows if outlook doesn't send a plaintext email home as "tele…

> I am my own provider. It's really not that hard to configure a mail server.

Oh yes it is. I mean for professional sysadmins it may not be, but try talking any normal person through buying a domain, getting an SSL certificate, attaching that at your MTA, configuring all those fiddly DNS records that prove you're emails are who you say you are. And all of that isn't even taking into account how needlessly painful the actual process of just installing the MTA is.

And what do you have for show from all of that? Still no real assurance that half your mail won't end up in spam, no web front end (that step alone adds several additional layers of complexity), less guarantees regarding up time (since you no have a single server that you need to manage yourself), arguably worse security (since you're now asking a layman to harden a Linux box) and the whole thing comes with a heftier price tag than Gsuite.

We're not talking about self hosting an IRC server or other service you can throw up in an hour and then forget about. Email is a frustrating sequence of a multitude of steps where even just one of them could render your set up worthless. Hence why even many of us sysadmins have long since given up self hosting email servers. It's just not worth the effort

Re: Email encryption is here – use STARTTLS everywhere

#36

The site has a link to the starttls everywhere site right at the top - but it's also worth mentioning that the regular 'certbot' client for https everywhere can also be used to create certificates for your own smtpd. The easiest way is just having the sudo ability to temporarily open port 80 for the challenge/response process. using certbot: sudo ./certbot-auto certonly -v --standalone --standalone-supported-challeng…

> The easiest way is just having the sudo ability to temporarily open port 80 for the challenge/response process.

The easiest way is to forget about HTTP and use DNS verification. Just create a dynamically updateable TXT record in your zone that the LE client can update and then point CNAMEs for the names that you want to create certificates for to that TXT record.

Re: Email encryption is here – use STARTTLS everywhere

#37
post #8

Earlier quoted context omitted.

I am my own provider. It's really not that hard to configure a mail server. Other peoples' providers, however... so I tried using PGP. It's somehow manages to be more difficult to use than configuring and running your own mailsystem which doesn't deliver straight to gmails' spam folder, but I digress; but then my emails are at mercy of the end devices: who knows if outlook doesn't send a plaintext email home as "tele…

Also all of your emails get kicked to spam when you host your own.

Not true. This might be happening if you are trying to host a server on your domestic internet connection, those IP ranges are usually banned in spam filters. But you can easily host your e-mail server on a real datacenter server or something similar, and you'll have almost no problems with spam filtering. (In fact many companies and even individuals already do this and have done this for long time.)

Re: Email encryption is here – use STARTTLS everywhere

#38

The site has a link to the starttls everywhere site right at the top - but it's also worth mentioning that the regular 'certbot' client for https everywhere can also be used to create certificates for your own smtpd. The easiest way is just having the sudo ability to temporarily open port 80 for the challenge/response process. using certbot: sudo ./certbot-auto certonly -v --standalone --standalone-supported-challeng…

> The easiest way is just having the sudo ability to temporarily open port 80 for the challenge/response process. The easiest way is to forget about HTTP and use DNS verification. Just create a dynamically updateable TXT record in your zone that the LE client can update and then point CNAMEs for the names that you want to create certificates for to that TXT record.

I am too paranoid and untrusting to give the LE automated tool the ability to edit DNS zonefiles. My DNS setup is quite far removed logically from the mail server setup, intentionally.

Re: Email encryption is here – use STARTTLS everywhere

#39
post #24

Earlier quoted context omitted.

Also all of your emails get kicked to spam when you host your own.

You can perfectly "host your own email server" while sending outgoing mail through a smart host. No falsely-tagged-spam issue that way.

No, in my experience, selectodude is right. I've written this rant before but I think it's worth sharing repeatedly, so here goes again:

The big players (Google, in particular...in case it's not clear, the bulk of this rant is leveled at Google; Microsoft is an offender on perhaps 25% of the level, Yahoo just doesn't seem to care) have ZERO interest in "playing nice" with small e-mail servers. My experience running my own e-mail for almost twenty years then finally throwing in the towel and migrating to Fastmail not only proves this for me, but is, according to other sysadmins I've talked to, quite common.

I did everything right when hosting my e-mail. I did it for myself and maybe four others. The volume of mail going out of my server was so low, I could read the outbound mail.log for myself, and routinely did. Every domain had SPF, DKIM, and DMARC records. The outgoing single IPv4 and IPv6 addresses I used had perfectly-matching forward and reverse DNS lookups. I had a script that checked every blacklist I could find every 24 hours. My IP addresses were allocated from my (professional) colocation provider and SWIP'd to me with valid contact information. My IP addresses had been unchanged for seven years and had been stable for ten years prior to that change. My colo provider had a draconian non-spam policy. Nobody ran an e-mail list or anything. I've used the exact same domain for e-mail for over twenty years.

Still, every couple of months, Google (sometimes Microsoft, but far and away most likely Google) would decide to simply stop delivering messages from my five users--remember, including me--to Google's users. The messages would just...disappear. Sometimes, Google would stop delivering the messages mid-conversation. Replies would be going back and forth but then my reply to a Gmail user would go to /dev/null while their messages to me would still arrive. Most of the time, the mail would go to spam (even with that little yellow tag meaning "verified sender" or some such) but 20% of the time the mails would just disappear.

Worse, the destination would say "250 OK" to every e-mail I sent them so I had no way of knowing that the messages had been sent to the big bitbucket in the sky. I got annoyed.

I signed up for Google's Postmaster Tools and Microsoft's similar offering. Microsoft would at least show me telemetry but it wouldn't do any good. Google? Google would proclaim that I wasn't sending them enough e-mail so I couldn't show up on the dashboard.

Finally, after three years of fighting this, it came to a head when I was, at all places, at a boardgame meetup during one of these "blackout periods" and got introduced to a Google SRE who worked "adjacent" to Gmail. "Oh, hey, you should talk to techsupporter, he does e-mail too!" was the opening line. We talked, I ranted, and he actually said he'd try to help. He escalated internally and the word came down from on high on their SMTP team or some such: "domain [mydomain] has insufficient reputation."

What did that mean? Apparently, their system--at least at that moment--thought my domain was "too new" and they tilted towards just binning the e-mail. I got red, saw stars, then gave Fastmail $390 for three years of service on three e-mail accounts and moved everything to them within five hours. That was the last straw, the final injustice. For me, the "decentralized Internet" died on that day.

Why was I so pissed? Observe these dates from WHOIS:

    Domain Name: MYDOMAIN.ORG
    [...]
    Creation Date: 1997-03-10T05:00:00Z

    Domain Name: GOOGLE.COM
    [....]
    Creation Date: 1997-09-15T04:00:00Z

Re: Email encryption is here – use STARTTLS everywhere

#40

Earlier quoted context omitted.

Also all of your emails get kicked to spam when you host your own.

Not true. This might be happening if you are trying to host a server on your domestic internet connection, those IP ranges are usually banned in spam filters. But you can easily host your e-mail server on a real datacenter server or something similar, and you'll have almost no problems with spam filtering. (In fact many companies and even individuals already do this and have done this for long time.)

> (In fact many companies and even individuals already do this and have done this for long time.)

Yes, with mixed results.

For one instance, see the trouble Linux people are having with getting messages delivered to Linus' Gmail inbox. I recall reading that some have needed to sign up for Gmail specifically just to get their patches to Linus. Plus, the 2015 occurrence with Gmail having a 20% false positive rate: https://plus.google.com/+LinusTorvalds/posts/DiG9qANf5PA

Post reply on HN