Live data from Hacker News

Ask HN: The middle ground for email self-hosting?

news.ycombinator.com

71–80 of 105 posts

Re: Ask HN: The middle ground for email self-hosting?

#71
I think your best bet is Amazon ses. In my recent testing, all mail always go through - unless the recipient marks it thus.

It's damn cheap too, like almost free for low volumes.

I think Amazon uses this for their workmail also and has become pretty strict at policing abuse.

I am only speaking for gmail though, so ymmv for hotmail et al which I haven't checked.

Re: Ask HN: The middle ground for email self-hosting?

#72
post #57

Earlier quoted context omitted.

I do this but with Namecheap + Gmail. It has worked flawlessly for years. The only issue is that emails don't come in instantly. They can take up to 15 minutes. I consider this a feature, but it would be really annoying to some.

> The only issue is that emails don't come in instantly. They can take up to 15 minutes. I consider this a feature, but it would be really annoying to some. I have a somewhat similar setup, slightly different. Most emails arrive within 10-15 seconds, but sometimes gets stuck somewhere along the line (as it happens with emails), which is normally not a problem. But some platforms force you to use the "send link to log…

This is called "greylisting" and it's fairly normal if you don't run your own server. Someone triggers a spam block and gets put on a partial time-out. Then the more times they try to re-send an email to you, the longer their emails get kept in limbo.

If you don't control the server, you don't have independent email.

Re: Ask HN: The middle ground for email self-hosting?

#73
post #41
post #38

I’d go AuthSMTP or similar rather than 365 for pure outbound as it will be much more cost effective.

(This is exactly what I do.)

I think it's a good service, I like that it can do DKIM on the outbound emails even though your 'main' MTA might not support it or might be a pain to set up.

Re: Ask HN: The middle ground for email self-hosting?

#74
post #35

A major theme in self-hosted email discussions is deliverability issues (particularly to larger email service providers), and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings. Seems like it shouldn't be hard to check and collect reference stat…

The problem with deliverability issues is the impossibility of proving a negative. If I send an e-mail to a company's customer support, or to my senator, or I reply to a potential client, or I contact an open source mailing list and I don't receive a reply - do I know if my message made it to them or not? I mean, it's plausible that JohnDoe@senate.gov just didn't deign to reply to my e-mail. But it's equally plausibl…

I can prove a different negative with my own mailserver - when I've sent things to @gov, they've always been responded to. I think that just proves the government reads ALL their spam.

Re: Ask HN: The middle ground for email self-hosting?

#75
post #35

A major theme in self-hosted email discussions is deliverability issues (particularly to larger email service providers), and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings. Seems like it shouldn't be hard to check and collect reference stat…

>, and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings.

It's not just misconfigured email server settings like DKIM, SPF, DMARC etc. One can correctly set all of those and still have the outgoing emails rejected or spamholed. Why? Because the big email players like GMail, Microsoft Outlook.com, etc use black-box heuristics of reputation datapoints that exist outside the boundaries of email settings such as... "amount of email volume", "# of spam abuse reports from ip block", etc.

Because "sender reputation" cannot be encoded into an email configuration (DKIM/SPF/DMARC/etc), that's why nobody can provide a convenient Docker container with a perfectly working self-hosted email server that can reliably send email. If such a thing existed, the spammers would use it as well!

A datapoint such as "volume of email from this ip" is an unstated behavior/activity number and not an identity setting like DKIM.

And the invisible heuristics keep changing which causes previous email setups that worked -- to later stop working for no obvious reason. Why? Because there's a constant arms race between spammers and email filter algorithms. This means others' email spam heuristics that keep evolving and that you don't control -- blocks your self-hosted outbound emails without warning.

That's why you have example of skilled admins who know what they're doing and had a working self-hosted setup for years suddenly getting their emails rejected: https://www.tablix.org/~avian/blog/archives/2019/04/google_i...

As to the contradicting anecdotes about the difficulties of self-hosting email, the issue is that the conversation shares the same unstated environments in comments about Uber or umbrellas that affects how the writer perceives the truth or relevance of their anecdote.

- "The problems of self-hosted email getting blocked is overstated. I've been doing it and it's working fine."

- "I'm not sure what value Uber provides. Taxi services have smartphone apps."

- "I'm not sure why people use umbrellas. Every time I walk outside, it's not raining."

As an example of evangelists and advice-givers not noticing their unstated environments... Back in October 2017, a commenter (lucb1e) argued[1] that I was exaggerating the difficulties of reliably sending email but a year later in 2019, he eventually confirmed the same difficulties! [2]

[1] https://news.ycombinator.com/item?id=15525505

[2] https://news.ycombinator.com/item?id=19757607

Re: Ask HN: The middle ground for email self-hosting?

#76
post #35

A major theme in self-hosted email discussions is deliverability issues (particularly to larger email service providers), and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings. Seems like it shouldn't be hard to check and collect reference stat…

For me, thinking about potential deliverability issues is just too stressful. Even if it works 99% of the time, who knows how important the remaining 1% of emails will be. Personally, having control over my own domain is a good-enough middle ground.

Been doing this long enough that I can sense right away if people aren't getting my emails. It's not great to call people up and ask if they got an email, and admit that my mail might not be getting through today, but it's not terrible. Gives me a chance to touch base with them for a minute, since I was emailing them anyway.

Having people whitelist you on google / yahoo / msn because you explicitly ask them to does have a wider effect, as far as I can tell, of keeping your emails in the clear for everyone else.

Re: Ask HN: The middle ground for email self-hosting?

#77

I have good luck using https://forwardemail.net for having email address on my own custom domain. This will catch inbound email (configured from DNS MX record) to my gmail, and for outbound I set the custom domain as gmail alias. Doesn't solve privacy, data ownership, nor google lock-in issue (but at least if I lost my gmail, I can move to a real email selfhost solution and keep my address). As my need is just to hav…

The problem I'm having with a similar solution is that for some of my emails (that I used to test), are labeling the emails sent with the custom domain as gmail alias as "Promotion", so it doesn't go into the Inbox on gmail, goes to the Promotions tab - and makes no sense, since the only thing in my email structure I have is my name as a signature and the domain.

Re: Ask HN: The middle ground for email self-hosting?

#78
post #75
post #35

A major theme in self-hosted email discussions is deliverability issues (particularly to larger email service providers), and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings. Seems like it shouldn't be hard to check and collect reference stat…

>, and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings. It's not just misconfigured email server settings like DKIM, SPF, DMARC etc. One can correctly set all of those and still have the outgoing emails rejected or spamholed. Why? Because the…

Oh yeah. This is very true, and getting worse every year.

I've had this discussion on HN before. It's gotten to the point where I've had to have my clients and their corporate lawyers go to bat against mail providers to maintain deliverability. No mail provider has any interest whatsoever in allowing an independent mailserver to continue delivering now.

So far, legal threats have worked when push came to shove against certain networks. But I imagine the difficulty is only going to increase.

Re: Ask HN: The middle ground for email self-hosting?

#79

I'm going to write this assuming you're a non-technical professional, lawyer perhaps, looking for a private email solution that doesn't rely on third parties. Bottom line: There's no "middle ground", any middle ground you cede is allowing a third party some kind of access. Hosting your own email has become expensive and time-consuming (although IMHO it's still extremely worthwhile, and I do it in spite of what a pain…

> You need a dedicated box, not a VPS. … Unlike some people are saying, you should never do this off a VPS if you have an interest in keeping the email secure and functioning for a long time.

I agree that hosting a mail server directly on a VPS compromises privacy and control. But there’s a better alternative: use VPSes for cheap static IPs, while hosting the server locally on hardware you physically control, using WireGuard tunnels and port forwarding to connect things. Port forward incoming SMTP over WireGuard to your real MX, and use MTA‐STS and DANE so that as many senders as possible will TLS‐encrypt mail they send you. Have your outgoing SMTP server handle DKIM signing, then send it out via WireGuard so it looks like it came from the VPS, while enforcing TLS encryption.

The VPS won’t be able to forge mail from you without your DKIM keys. It won’t be able to read your outgoing mail due to TLS. It won’t be able to read incoming mail that’s TLS encrypted. It will be able to read unencrypted mail, but the big providers that follow MTA‐STS will abort if the VPS attempts to block encrypted connections.

This has the added benefit of reducing your dependence on an external provider (the VPS company) for server setup. If you’re unhappy with a particular provider, just switch to another one. The issues associated with sending email from a brand new IP will be there, but you won’t have to set up complicated infrastructure on the new host, only a few WireGuard tunnels and firewall rules.

Re: Ask HN: The middle ground for email self-hosting?

#80

I'm going to write this assuming you're a non-technical professional, lawyer perhaps, looking for a private email solution that doesn't rely on third parties. Bottom line: There's no "middle ground", any middle ground you cede is allowing a third party some kind of access. Hosting your own email has become expensive and time-consuming (although IMHO it's still extremely worthwhile, and I do it in spite of what a pain…

> You need a dedicated box, not a VPS. … Unlike some people are saying, you should never do this off a VPS if you have an interest in keeping the email secure and functioning for a long time. I agree that hosting a mail server directly on a VPS compromises privacy and control. But there’s a better alternative: use VPSes for cheap static IPs, while hosting the server locally on hardware you physically control, using W…

That's a cool idea I'd never considered. And it's easy to set up. But then the main thing you're getting out of the VPS is the static IP. The main reason I was advising against VPS's, besides obvious vulnerabilities, was that their IP blocks get banned all the time. If you want to run a mailserver for a long time you need to cultivate that IP address's reputation for years, and you don't want it to be anywhere sketchy.

If you're paying for that, why not just pay for a static IP at home?

Post reply on HN