Live data from Hacker News

After self-hosting my email for twenty-three years I have thrown in the towel

cfenollosa.com

301–310 of 744 posts

Re: After self-hosting my email for twenty-three years I have thrown in the towel

#301
post #288

Earlier quoted context omitted.

When the big providers all require VMC to show BIMI, then VMC is not optional, no matter what the spec says. Claiming it is optional is then disingenuous. As I said in the linked post, logo verification is not a problem which can be solved. Identical trademarks can legitimately be issued in different fields, and both still be valid. Let’s say you are a brick manufacturer, and have paid an arm and a leg to a VMC certi…

> When the big providers all require VMC to show BIMI, then VMC is not optional, no matter what the spec says. Claiming it is optional is then disingenuous. What do you mean "no matter what the spec says", it is the spec we're talking about. It is what you argued against several times. If you had started with saying "big providers' implementations of BIMI", then it wouldn't be wrong to say it's required but it's stil…

> If you had started with saying "big providers' implementations of BIMI", then it wouldn't be wrong

That’s just splitting hairs.

Re: After self-hosting my email for twenty-three years I have thrown in the towel

#303

Why are we still using email? No, I'm serious, why?

One of the few means of online communication that isn't a proprietary walled garden?

So in the few other means of online communication that aren't proprietary walled gardens, what else we got?

Re: After self-hosting my email for twenty-three years I have thrown in the towel

#304
post #201

Earlier quoted context omitted.

There are many benefits to running your own server. The three biggies for me are: 1. Control. A third party can change anything about the service any time they want, and if you don't like the change they made you're screwed. 2. Expectation of privacy. Because I am not contracting with a third party, the government cannot argue that I have waived my right to privacy. (As a practical matter of course this matters not a…

> The Right Way to filter spam is to use your outgoing mail as ground truth for what is not spam Could you elaborate? Does this mean that email from people/domains you haven't corresponded with before is spam?

Anything that comes in from an address I have never seen before is handled specially. But it's not hard to filter out the obvious spam. Just a handful of heuristics on the from and subject lines (e.g. if the sender's name contains common English words it's probably spam) takes care of >90% of the cold calls. The rest I just look through manually once a day or so.

I was planning to institute a system where my contact page included a special keyword to include in the subject line to get past the spam filter, but that has turned out not to be necessary so I haven't implemented that yet.

The only remaining case is things like confirmation emails for new accounts, but those just get lumped in with the other cold calls. They are super-easy to spot because I'm almost always expecting them, so they are always at the top of the list.

Re: After self-hosting my email for twenty-three years I have thrown in the towel

#305
I've been self-hosting my mail for 17 or 18 years now, by purchasing just some managed webhosting package from somebody who cares about their services not being used for shady stuff (any reputable managed hosting provider) and I think I've never "lost" an outgoing mail for personal use.

I don't understand what the author thinks it's so hard here and why he's painting it so black and white. There's lots of more to "my own e-mail" than choosing between some old notebook running and collecting dust in your garage and using GMail.

Some people just want to find a hair in the soup.

Re: After self-hosting my email for twenty-three years I have thrown in the towel

#306
post #22

This isn't actually that hard to fix, it's just that for whatever reason, we seem to frequently have this blindspot that we don't seem to have in other industries. Namely that "do it yourself at home" and "massive oligopolist" aren't the only two options. It's like saying "You can only have hamburgers two ways, cook them yourself or McDonalds." I do the third and it's been great. I let my paid webhost handle it. (hos…

> like saying "You can only have hamburgers two ways, cook them yourself or McDonalds." and your comment seems to be saying it’s OK if we lose the ability to cook hamburgers at home, because there are other (more ethical) restaurants that aren’t McDonalds. am i misunderstanding?

No. To beat up the metaphor, I'm saying if we support the restaurants that aren't McDonalds and that are more "mom and pop" (and get others on board) then we can beat McDonalds or at least live in harmony with them.

Re: After self-hosting my email for twenty-three years I have thrown in the towel

#307
post #287

Earlier quoted context omitted.

gmail also puts mails from mailing lists into spam, despite repeated "not spam" tagging. Wondering whether mail getting flagged as spam is even a problem anymore as people get used to erratic results, at which point we can get rid of spam filtering, or what's left of it, altogether when on balance it does more harm than good, such as preventing SMTP self-hosting.

And yet I continue to get "Walmart Confirmation Receipt" or "Verizon Confirmation Receipt" all coming from addresses like "verizon_info_nlAT2Q7uf0d@zfgfdyyqsckxbvwg.linenight.com" which means google are't even trying for some.

If you continually mark those emails as read without opening them, then gmail will learn that they are junk for your account.

Re: After self-hosting my email for twenty-three years I have thrown in the towel

#308

Earlier quoted context omitted.

Spam has been been fought for decades, you can rest assured any obvious solution has been tried and either doesn’t have the desired effect or is impossible to implement.

I think we ought to move email (or some future incarnation of email, like matrix) to a completely whitelist (opt-in to receive messages) basis.

signup confirm emails are like that.

similarly, any "hello pls add me to your allow-list" emails could be made auto-disappear to the "will be deleted in 30 days" folder in ~10-15 minutes, so even if you get a 100 spam messages per day you only see the last of those, you can easily pick what you are looking for, and don't worry about the rest, they'll just disappear.

(and you still have 30 days to look for messages that might be interesting/important/etc.)

...

the real missing piece is the feedback mechanism. DMARC is meh. of course large senders have implemented FBL, but they are not available for mere mortals.

https://en.wikipedia.org/wiki/Feedback_loop_(email)

Re: After self-hosting my email for twenty-three years I have thrown in the towel

#309

It should be possible for everyone to participate in internet systems equally as peers. Somehow it seems like the Overton window has shifted such that people find it acceptable that ordinary individuals can no longer take part in the email infrastructure as equal peers.

Spammers are also equal peers, and are a huge problem. The concept of every computer being an equal peer is in fact part of the problem with email.

Re: After self-hosting my email for twenty-three years I have thrown in the towel

#310
post #21

The sweet spot for having control over your email while simultaneously minimizing unforseen headaches is to simply own your domain name and point the MX record to whatever hosting provider you want instead of self-hosting a server at home. Same philosophy for exposing a your personal blog of html files or content like mp4 videos. The sweet spot is to focus on buying a domain name you control. Then let Amazon S3, or C…

I am ashamed to admit that I have no idea how email works. Is there a dumb down explanation of what are the moving parts and how you can achieve that sweet spot?

I can type out an explanation relatively easily.

Let's imagine I'm sending from user@zahllos.example to user@skywall.example. I'm doing it from say Thunderbird or Outlook, and you're using the same.

I need to send, and to do this I typically use an SMTP server. This is something configured, probably on zahllos.example, maybe with the domain smtp.zahllos.example. My mail client contacts this and 'logs in' with my details, then transmits the email message I want to send to this server. The server says 'right fine' and closes the connection.

At this point, the message is in a mail queue ready to go. The SMTP server then does a DNS request for the MX record of skywall.example. Let's say this is 'mail.skywall.example'. My server, smtp.zahllos.example, then connects to `mail.skywall.example', also speaking SMTP, and says "hey, I have this message to deliver".

It is at this point that mail.skywall.example can decide to do some things. It might check SPF, so it will query the SPF record of zahllos.example to find the list of servers that may send email for that domain. In this example, let's assume smtp.zahllos.example is in the list. Great.

It may then also check the mail headers for a signature, called a dkim signature. My server signed the message before sending; mail.skywall.example can query ._domainkey.zahllos.example and find a public key (or not) and check that this signature matches (or not). Again, let's assume it matches.

It might also check something like TXT _dmarc.zahllos.example to see what my DMARC policies are. If I have something like v=DMARC1;p=reject;sp=reject;pct=100;ri=86400;fo=1;aspf=s;adkim=s;rua=mailto:postmaster@zahllos.example;ruf=mailto:postmaster@zahllos.example;" this tells you I'd like you to outright reject anything not matching policy, that I expect everything to match SPF and have DNS signatures, and you can send reports if you support that to postmaster@zahllos.example. Your server can then enforce these checks as it likes.

One of the first things that will happen is that my server will announce itself via an EHLO statement. An obvious check to do is to check that the sending IP actually matches smtp.zahllos.example. by querying 'reverse ptr' records.

Your receiving server will also likely hand the message over to various spam-checking tools for analysis, such as against DNS blocklists and so on. Larger providers likely have much more sophisticated infrastructure here. Ultimately, you're going to do one of a few things: 1) deliver to inbox or apply user-specified rules and deliver to a folder; 2) deliver to junk (which is typically just another folder, but treated specially by clients), 3) reject, and tell smtp.zahllos.example you don't want the email.

Once the email passes through the smtp dameon, assuming either 1 or 2, it then gets stored somehow and in some way. I'm being a little bit vague here, because 'it depends', but in the simplest scenario, the smtp daemon will write the message to an mbox or maildir-style format. More complex setups definitely exist, indeed, there can be multiple layers of servers doing analysis on separate machines, but for simplicity, mail.skywall.example is one VM that makes its decisions and the result ends up in /var/mail/user@domain/ or some such.

A key aspect of this step that makes email very nice is that if smtp.zahllos.example cannot, for some reason, reach your server now it will queue the message and try again at set intervals. You can reasonably safely turn off mail.skywall.example for a couple of hours.

Another aspect is that you can have multiple MX records where you are prepared to accept email, with priorities. So if you can't accept at one address because the server is down for maintenance, another will accept.

So, now you've technically got an email, but you don't know it. So you open thunderoutlook, and you connect to an IMAP server imap.skywall.example - in our example let us assume this is really the same thing as mail.skywall.example. The server checks you are really you with your credentials. At the backend, this is just another daemon that knows to read /var/mail/... and find new messages; it finds one, downloads its headers and displays it on your screen.

Since we're in a slightly more modern world now, in the case your client was already open you might have an "idle" connection with the server at all times, in which case it can push the message down to you.

In the case of webmail, it is really all the same thing, except you point your browser at a webpage, and that webpage communicates with the servers instead of your client communicating directly. Open source webmail might even use IMAP underneath; things like Zimbra use their own java mail agent, while Google is entirely custom.

That might seem complicated, but in the end it isn't: between two domains at the 'edge', in the end, there's an SMTP conversation. One sending server tells a receiving server it has mail for delivery, and it finds that server by asking DNS where it is. The receiving server may do a bunch of checks against DNS also before making a decision on what to do with the email.

Post reply on HN