I like a lot of what’s being discussed here. One thing to consider is a similar problem I am facing is people trying to hack into a login page. We see thousands of requests per minute from different IPs… many from VPS, some from obviously hacked TVs/devices. We could implement an exponential back off on abusive IPs but detection requires observation of action that action could result in a compromised account… so another idea is we simply block large ranges of know bad IPs from blacklists… I think this is similar to the email sending issue… it’s not fair and I think a solution could be some kind of “block chain” - make it expensive to login… make it expensive to send an email… but I’m not sure and for email it’s way harder because you need agreement from the oligopoly of email providers… not sure what the solution is
After self-hosting my email for twenty-three years I have thrown in the towel
331–340 of 744 posts
Re: After self-hosting my email for twenty-three years I have thrown in the towel
#332Earlier quoted context omitted.
I can type out an explanation relatively easily. Let's imagine I'm sending from user@zahllos.example to user@skywall.example. I'm doing it from say Thunderbird or Outlook, and you're using the same. I need to send, and to do this I typically use an SMTP server. This is something configured, probably on zahllos.example, maybe with the domain smtp.zahllos.example. My mail client contacts this and 'logs in' with my deta…
So in the example of the parent, he's got a domain name registered somewhere (mydomain.com) and he set, in its MX records, the gmail server. But how does gmail make the connection between that address and your gmail address then?
So normally in the cases like this, you also have to tell Google about this, and you typically do this by using one of their paid-for products like workspaces or apps for business or whatever they call it. So let's say that you decide to host skywall@skywall.example with Google. You pay them for workspaces and you likely tell them "I would like to use this domain I already have, with email". They then tell you "OK, add our servers as your MX record in your DNS, or transfer the whole domain to us and we'll do it for you". In this case we're doing the 'changing the MX records' part.
Now when a sending server asks "where should I deliver skywall.example email" by querying MX skywall.example it gets google's servers and starts an smtp conversation with them, saying "I'd like to deliver a message for skywall@skywall.example". At this point, Google knows it can accept that, so they say yes, and then continue doing whatever they do to check for spam beyond that, including queries for spf and friends.
The reason the parent suggests this is that if at any point you decide to move off Google, you can pay someone else, e.g. fastmail, for their services, and modify your MX record. 24-48 hours later, DNS around the world catches up and everyone will get fastmail as a response when they ask for your MX record. Any new email goes there instead of Google, and thus you aren't 'tied' to the provider: you just have to move all your old email over. Whereas Google cannot let you move a user@gmail.com address, because they can only change the MX records for the whole of gmail.
DNS is the source of truth here. Whatever your MX records are is where other servers will try to contact to send email. The MX record is typically just another DNS address that will be queried for AAAA/A (i.e. what is the IP), and that doesn't need to be on the same domain at all.
Here's an example of what it looks like:
delv MX ycombinator.com @9.9.9.9
; unsigned answer
ycombinator.com. 295 IN MX 20 alt2.aspmx.l.google.com.
ycombinator.com. 295 IN MX 10 aspmx.l.google.com.
ycombinator.com. 295 IN MX 20 alt1.aspmx.l.google.com.
ycombinator.com. 295 IN MX 30 aspmx4.googlemail.com.
delv NS ycombinator.com @9.9.9.9
ycombinator.com. 159148 IN NS ns-225.awsdns-28.com.
ycombinator.com. 159148 IN NS ns-1914.awsdns-47.co.uk.
ycombinator.com. 159148 IN NS ns-1411.awsdns-48.org.
ycombinator.com. 159148 IN NS ns-556.awsdns-05.net.
I'll cover off the SMTP outgoing as well while I'm at it. You _can_ also not run your own outgoing smtp server but use someone else's. The key here is that if you use SPF and DKIM, you should put their IPs into SPF and their keys into DKIM, as that is what the receiving server will use. So smtp.zahllos.example could be replaced by sendgrid, provided in my DNS I say so. This may work better, as sendgrid may have a better reputation than the server I chose.
Re: After self-hosting my email for twenty-three years I have thrown in the towel
#333Alright blockchain nerds, now is your time to shine. An actual problem that can be attacked through distributed consensus.
If you insist. I put my suggestion as a reply to another comment in this discussion, specifically here:
Re: After self-hosting my email for twenty-three years I have thrown in the towel
#334Sad, but true...
Re: After self-hosting my email for twenty-three years I have thrown in the towel
#335Earlier quoted context omitted.
True. But they are not calling for a completely open system. I like this proposal from the author. > Change blacklisting protocols so they are not permanent and use an exponential cooldown penalty. After spam is detected from an IP, it should be banned for, say, ten minutes. Then, a day. A week. A month, and so on. This discourages spammers from reusing IPs after the ban is lifted and will allow the IP pool to be cle…
Spam has been been fought for decades, you can rest assured any obvious solution has been tried and either doesn’t have the desired effect or is impossible to implement.
You ignore the fact that there are perverse incentives among the participants. It's possible to implement, and I'm doing it myself. If I had more time to spend on it, we could end spam. Instead I am fine as is: most of the spammers have given up.
Re: After self-hosting my email for twenty-three years I have thrown in the towel
#336Earlier quoted context omitted.
True. But they are not calling for a completely open system. I like this proposal from the author. > Change blacklisting protocols so they are not permanent and use an exponential cooldown penalty. After spam is detected from an IP, it should be banned for, say, ten minutes. Then, a day. A week. A month, and so on. This discourages spammers from reusing IPs after the ban is lifted and will allow the IP pool to be cle…
Is there any actually money to be made in hosting email for people? I genuinely don’t know but my suspicion is that GMail, Yahoo, Outlook, et al are loss leaders for their owner companies. I suspect people at those companies would be quite happy if the protocol got unfucked enough that it small players could participate without negatively impacting the network.
I gladly pay for Fastmail and I assume they’re not running a charity. Also, I think hey.com is charging $100 per year.
Re: After self-hosting my email for twenty-three years I have thrown in the towel
#337Earlier quoted context omitted.
> It's just not something I want to babysit anymore because I have other things to focus on Dont know about you, but I have setup my mailserver years ago, and outside of regular OS updates, havent had to touch it.
How do you make sure your emails don't end up in spam?
That's a recipient's problem.
Re: After self-hosting my email for twenty-three years I have thrown in the towel
#338I've been hosting my mail for 20+ years now, with minor issues. I guess I've been lucky. Reading the comments here makes me incredibly sad. Every answer that tells me to use a provider misses the point. The Internet was created so that there could be many independent nodes, not so that everybody has to rely on one of several blessed providers. I should be able to run my own E-mail. The real problem is lack of incenti…
>the Internet was created so that there could be many independent nodes, not so that everybody has to rely on one of several blessed providers.
any community that grows large enough needs some mechanism to manage trust, this is a universal issue. The early internet was more permissive and less differentiated simply because it was smaller.
The big corps do an alright job at managing spam given the sheer size of the problem, and more importantly you don't just need to solve spam, you need to do so economically, because for your system to stay distributed the nodes need to do the job competitively.
Given that there's intrinsic benefits to managing these things at scale that's not really realistic, in large systems you're always going to have division of labor and stratification for that reason.
Re: After self-hosting my email for twenty-three years I have thrown in the towel
#339I know the pain and the annoyances of hosting Email.
But still, I won't switch to big providers, I use Proton Mail personally, and Postale.io for many projects.
There is also mailbox.org and many others, you have a choice to not use the big providers, it totally possible.
Re: After self-hosting my email for twenty-three years I have thrown in the towel
#340Why can't we have an open source mail hosting solution that self-updates?
You've actually asked that question before and the answer is still the same: you can't embed "trusted sender status" into source code for a mailserver or a software package. The invisible rules for rejecting email is an emergent external property that exists outside of the software package: https://news.ycombinator.com/item?id=20853157
E.g. Software that "auto-updates" cannot solve the problem of how different participants change their criteria on which ip blocks are "bad".