Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Hostile Email Landscape

liminality.xyz

Post reply on HN
Print view View original

161–170 of 251 posts

Re: The Hostile Email Landscape

#161

Post by cwyers »

JoshTriplett wrote: post #42
cm2187 wrote: post #7

The problem is not so much the attitude of the big guys. It is that smtp is fundamentally broken. we need a better mail protocol that ensures: 1. Traffic always encrypted and content always signed 2. Guarantee that the sender is who it claims he is 3. Decorrelating the email from the domain, a lot of users are prisoners of their current provider just because the address they gave everyone ends with the provider's dom…

> Decorrelating the email from the domain, a lot of users are prisoners of their current provider just because the address they gave everyone ends with the provider's domain name, very much like it is very hard to switch bank accounts This one seems completely uninteresting in a world in which $15/year can get you your own domain name, complete with reliable email servers, IMAP, and as many email aliases as you like…

That's $15 a year that some people can't, won't or haven't paid before. If someone decides later on that they wish to do so, they have to set up forwarding and try and convince contacts to use the new e-mail address instead.

Re: The Hostile Email Landscape

#162

Post by tacon »

niravshah wrote: post #110
tacon wrote: post #97

I've managed my own mail server since 1993, and my email address has been the same that entire time. Here are some tips for maintaining sanity: Greylisting still works amazingly well. With a long, long whitelist and greylisting plus DNSBL, I don't even bother running a spam filter, since the little bit of spam and emails from new senders ends up in its own directory as it came from a non-whitelisted sender. Comcast f…

> Greylisting still works amazingly well. With a long, long whitelist and greylisting plus DNSBL, I don't even bother running a spam filter, since the little bit of spam and emails from new senders ends up in its own directory as it came from a non-whitelisted sender. Any good tips on this section in particular? If I'm running my own mail server, how would I get started making sure this is in order?

I have been running exim4 for years, but I'm in the process of moving to postfix, as postfix is considerably easier to set up all the DKIM, etc., machinery that is now required. Inbound email comes through procmail and is mainly read in emacs (mh-e), which is kind of old fashioned. I have a small script that makes a new email address within my domain for each new use. I sign up for a lot of mailing lists and groups, and my /etc/aliases is more than 5300 lines. I can track if domainA's address starts coming from domainB and disable that address, but that doesn't happen very often, which is a pleasant surprise.

I also have a small script that puts a new sender on my whitelist of sender email addresses. My whitelist is 12000+ lines right now, collected over many years. Procmail sorts to mailing lists and vendor folders, and finally puts things that are not on the whitelist into a "possible spam" folder. From the five or ten items a day, it is easy to spot legitimate emails and I add those to the whitelist. The majority of spam is blocked by the combination of greylisting and DNSBL lists, as the delay of greylisting (ten minutes for me) is enough for them to make the blackhole list, if they happen to ever attempt delivery again.

I was thinking recently that I should be collecting statistics on the use of a lot of those aliases and whitelisted emails, and maybe start garbage collecting my lists.

There are various reputation reports and services that can tell you how your mail is doing in the major ISPs, but a lot of those require higher traffic than a personal or small business generates. There is one service, DMARC[1], that is free and can give you some visibility into how email from your domain is being processed. I put the txt record in my DNS, and Google, Facebook, Comcast, Yahoo, Fastmail, and a few others send me reports about email they have processed from my domain. It's not that interesting at the moment because things are working, but it might help to debug issues if your email was being rejected. At least I see a few spammers are trying to use my domain from their servers.

[1] https://dmarc.org/

Re: The Hostile Email Landscape

#163

Post by teach »

teddyh wrote: post #84

I sometimes see similar tales of woe, and I can only say that this does not match my experience. I’ve done this many times, you set up the mail server, configure DNS correctly (including reverse lookup), and that’s it. Never had problems being blacklisted or mail getting classified as spam. I suspect that people having trouble are sending a lot of mail , like “newletters”, etc. But I can’t prove this hypothesis.

I've had similar issues to those described in the article. Especially "outlook.com accepted my email, but discarded it."

When people buy my book, I my server sends them literally one single email containing a link to the PDF they have purchased. And that's it.

Google delivers my email fine, in most cases. But hotmail/outlook was just deleting them most of the time for over a year.

I know I'm just a single data point, but I swear I'm not a newsletter!

Re: The Hostile Email Landscape

#164

Post by unclebucknasty »

lsc wrote: post #155
unclebucknasty wrote: post #150

Earlier quoted context omitted.

You are really missing the point here and you have some key assumptions wrong. > recipients of the email and the senders of the email have very different ideas What does that mean? They sign up for our service and expect to get e-mails from us, which they don't receive. We have the same "idea", but the third-party is interfering with that "idea". > if the recipients can't complain, that's a problem I will repeat that…

>What does that mean? They sign up for our service and expect to get e-mails from us, which they don't receive. We have the same "idea", but the third-party is interfering with that "idea". That's not quite how it works from the end-user's perspective. Say I buy a product or service from you. Of course, you send me billing emails, etc. and that's fine. The problem is that companies take this further and start thinkin…

Thanks for the advice. I know you mean well, but none of it applies.

I do agree with you on principle. We've been doing this for over a decade and have learned best practices. We also know what our customers appreciate and are very mindful of content, frequency, etc.

But, you seem to be overlooking a key point, so I will state it for at least the third time: the problem was NOT the content, and it was NOT being marked as spam by recipients. Instead, we were erroneously classified by the service as sending to honeypot addresses and/or defunct addresses. So, your assumptions about our customers, newsletters, etc. are just not relevant here.

>the third party has no responsibility to you. They have a responsibility to their customers...

Their customers complained to us that they weren't receiving our emails, so they weren't serving their customers very well in this instance either.

But, what you really don't seem to be getting is that they do have a responsibility not to harm us. That's why there is a legal cause of action known as "tortious interference". I sincerely don't know how to make that point any clearer. Perhaps researching that term a bit might help.

Re: The Hostile Email Landscape

#165

Post by peterwwillis »

cm2187 wrote: post #7

The problem is not so much the attitude of the big guys. It is that smtp is fundamentally broken. we need a better mail protocol that ensures: 1. Traffic always encrypted and content always signed 2. Guarantee that the sender is who it claims he is 3. Decorrelating the email from the domain, a lot of users are prisoners of their current provider just because the address they gave everyone ends with the provider's dom…

I agree and disagree. SMTP is bad, but it isn't the primary problem. The problem is social conditioning.

Since the 19th century, postal services have been monopolized by government institutions with a fee levied for storage, processing and transportation costs.

Physical mail delivery in bulk is costly, which makes it more profitable to send directly rather than en-masse.

But SMTP requires no fee. Originally, your ISP would provide you with e-mail in exchange for the fee you paid for your connection. Since the advent of "free" email services, people got used to not paying for mail anymore.

With electronic mail, it's cheap and easy to send in large quantities. Before we never needed to verify who was sending us mail, but now it's the only way we can filter out the noise.

To make electronic mail better we have to make it more expensive. Requiring a payment to send mail would reduce the cost-effectiveness of spam, and I would argue it would make the systems more verifiable. You could even preserve anonymity by having anonymous mail services, as long as you paid to send those mails, say with bitcoin.

Want to run your own mail server? Fine, but relaying will cost you. FedEx doesn't deliver mail from the USPS for free, and so relays shouldn't be expected to pass on mail without transportation costs.

Re: The Hostile Email Landscape

#166

Post by bachmeier »

[deleted] wrote: post #151
bachmeier wrote: post #129

Earlier quoted context omitted.

Chat services such as Slack, and social media like Twitter, are excellent ways to communicate. It's pretty simple. If you implement a request system so that both parties have to agree to let the other send messages, all of the problems are solved. There's no good reason that we need a system that allows anyone to send an arbitrary number of messages to anyone else.

[deleted]

> but there is a compelling reason -- this ability to send junk to anyone has allowed some people to make money consistently for decades

How is that a compelling reason? Why would I want to open myself up to spam so that others can make money?

There is a compelling reason that we use the current email system. We do it because we have to. Nonetheless, communication over the internet without spam is a solved problem, whether or not we choose to take advantage of the solutions.

Re: The Hostile Email Landscape

#167

Post by marcosdumay »

staunch wrote: post #138
lsc wrote: post #131

Earlier quoted context omitted.

Really? More than 99% of the junk email in my box is from "legitimate" senders who use their own mailservers. The truth of the matter is that it's already pretty easy to block the hit and run spammers. Most of the junk that gets through is sent by people with the resources to bully the spamfilters into accepting it.

Spam is viagra ads and the like (unsolicited commercial email). Promotional emails from companies you have a relationship with are a different thing. Users should be able to easily filter and ignore promotional email, but email hosting companies should never block it as spam. SMTP is not where email organization should happen. That's what email clients are for. Blocking DoS attacks and spam makes sense. Blocking Amaz…

Spam is anything that clogs my inbox and makes the emails I want to receive harder to find.

But I agree, SMTP is not where email organization should happen. Blocking illegitimate senders at the protocol level is the most that should be done, and the blocking of unwanted content should be pushed into a more user oriented layer. Current structure isn't optimal.

Re: The Hostile Email Landscape

#168

Post by glogla »

eridius wrote: post #94

I feel like this is a solvable problem without making any changes to email whatsoever. The problem is the email recipient hosts are suspicious of the sender (as opposed to the message itself being suspicious). So the solution is to have a standardized way for senders to acquire an instantaneous reputation by tying their real-world identity to it (which lets them be held accountable if they do spam), and perhaps by th…

You mean extortion. Give us money or we will drop your mail.

Re: The Hostile Email Landscape

#169

Post by tromp »

zrm wrote: post #147

I was thinking about this a while ago and have been meaning to write it up and post it somewhere, so I guess this is as good a time as any. Hashcash (also known as the precursor to Bitcoin) was proposed to solve this problem in 1997: https://en.wikipedia.org/wiki/Hashcash The trouble with it is that it requires computation for each sent message, which is bad for senders with low resource devices or legitimate mailing…

Sounds like a plan!

With a memory-bound proof-of-work system like my Cuckoo Cycle, computing the hash could require the use of more than 4GB of memory for over 5 minutes on a 20-thread server, thus preventing the use of botnets for avoiding the expense.

Re: The Hostile Email Landscape

#170

Post by marcosdumay »

teach wrote: post #163
teddyh wrote: post #84

I sometimes see similar tales of woe, and I can only say that this does not match my experience. I’ve done this many times, you set up the mail server, configure DNS correctly (including reverse lookup), and that’s it. Never had problems being blacklisted or mail getting classified as spam. I suspect that people having trouble are sending a lot of mail , like “newletters”, etc. But I can’t prove this hypothesis.

I've had similar issues to those described in the article. Especially "outlook.com accepted my email, but discarded it." When people buy my book, I my server sends them literally one single email containing a link to the PDF they have purchased. And that's it. Google delivers my email fine, in most cases. But hotmail/outlook was just deleting them most of the time for over a year. I know I'm just a single data point,…

Exchange is much more discerning of email structure than Google servers.

That means that, if you have any configuration mistake, an Exchange server will reject your email. But if you have a good reputation, Gmail will deliver it anyway.

Post reply on HN