Live data from Hacker News

Spam blacklisting is out of control

blog.roastidio.us

281–290 of 430 posts

Re: Spam blacklisting is out of control

#281
post #232
post #31

I'm dealing with this right now. Both my personal domain and rsync.net are on a distinct subnet, but that subnet is smaller than a /24 and someone on a different subnet has, apparently, behaved badly. Enter "abusix" ... One of my engineers had an enlightening webchat with one of their engineers where we were shown the "offending" IP and it was explained that they have no ability to distinguish subnets (and no interes…

I'm the Abusix engineer in question (and actually the architect of the system in question), and you're being somewhat "economical" with what actually happened here. Here's the actual chain of events in question: - You recently switched ISPs and that meant you moved to a new IP block. - The IP block in question is owned by Hurricane Internet and unfortunately contains a host which persistently sends out a lot of junk…

> make sure it has a working List-Unsubscribe header (preferably HTTP that allows someone to unsubscribe without having to contact you). Important note: If you use a mailto: unsubscribe, then I cannot unsubscribe a trap, even if I wanted to.

This is very interesting to me. Could you elaborate further?

- Some providers (IIRC outlook being the main one) only support mailto: and most prefer it. I have never seen advice that you "must" provide HTTP before.

- What do you mean by unsubscribe without contacting me? Isn't HTTP or SMTP just as much contacting me?

- Why can't you unsubscribe mailto: even if you wanted to? Earlier you hinted that it may have to do with revealing the trap, but while a HTTP url may not directly include the trap address it would be linked anyways. (Otherwise how would the unsubscribe work?)

Thanks for all the valuable insights. It does sound like you are taking a responsible approach here.

Re: Spam blacklisting is out of control

#282
post #22

Spam blocklists are run by an unaccountable cowboy cult that somehow has managed to consolidate a ton of power simply for the fact that most people who run email inbox services didn't want to deal with the problem of spam, so they were more than willing to just hand over anti-spam "enforcement" to anyone who was allegedly doing "what was best for the internet." There's no check on these people who run these blacklist…

So what's the alternative? SBLs work, clearly, and almost every mail host, in the absence of distributed list, would work up their own lists in short order.

Choose your SBL better; spend some time reading about it (doesn't have to be much) or testing it before blindly enabling it.

Re: Spam blacklisting is out of control

#283
> If my understanding of the law is correct, spamming is legal, albeit immoral. If I were the hosting company, and I had a paying customer that is conducting legal business on my network, on what grounds can I evict them?

Hosting companies can kick customers off for basically any reason they want to. High-end hosting companies will absolutely kick a customer off if the customer’s activities result in spam blacklisting.

When I first signed with Rackspace, it took almost a week to negotiate their AUP (acceptable use policy). They were basically unwilling to give us any solid assurances about account suspension. They gave themselves enormous room to determine what was acceptable behavior on our part.

If you haven’t carefully read your host’s AUP, it might be enlightening. If your host doesn’t have an AUP, or doesn’t enforce it, then yeah maybe that is not the best neighborhood.

Re: Spam blacklisting is out of control

#284
The entire email, message, and phone model is upside down for me.

Rather than defaulting to allow-all, it should default to deny-all. Content is let in on a case by case basis.

I found an email service the operates this way, Hey.com. Unfortunately, it’s pretty sparse in mostly every way but this. However, it’s practically impossible for me to receive spam. You can’t get into my inbox unless I’ve approved you.

It doesn’t make any sense (to me) to maintain these hilariously complex deny lists that results in infinite cat and mouse. Just deny everything and only allow in known good actors. Voila.

Re: Spam blacklisting is out of control

#285
post #232
post #31

I'm dealing with this right now. Both my personal domain and rsync.net are on a distinct subnet, but that subnet is smaller than a /24 and someone on a different subnet has, apparently, behaved badly. Enter "abusix" ... One of my engineers had an enlightening webchat with one of their engineers where we were shown the "offending" IP and it was explained that they have no ability to distinguish subnets (and no interes…

I'm the Abusix engineer in question (and actually the architect of the system in question), and you're being somewhat "economical" with what actually happened here. Here's the actual chain of events in question: - You recently switched ISPs and that meant you moved to a new IP block. - The IP block in question is owned by Hurricane Internet and unfortunately contains a host which persistently sends out a lot of junk…

One other question. How do you manage domain based reputation vs IP reputation. I'm trying to get by running a service off of Cloud provider IPs and they may rotate from time to time as VPSs rotate. Part of me feels that once my domain has enough reputation it should be fine. It does seem like some of the big providers work like this (Google seems to trust me now from any IP) but it seems that a lot of smaller providers rely on blacklists more as they don't see as much traffic to build reputation. How do blacklists handle domain reputation?

Re: Spam blacklisting is out of control

#286

> Or I can leave the current hosting company Yep, that's the one. If your hoster doesn't care about spam spreading from their IP space, you should take your mail server elsewhere. There's literally nothing to think about. And if they do care about this issue, they are likely to be taking steps to remove any of their IP space from the blacklists, without being nudged. PS. I've been running a mail server for close to 2…

When I first started monitoring spam connections years ago, it was almost always home cable+internet providers. They seem to have gotten the message and cleaned up thier acts. As of last night, the number one source of spam for both the personal and company servers I maintain is Digital Ocean. Which is a shame, because otherwise I'm a happy Digital Ocean customer. But because of this, I would never move any of my com…

> home cable+internet providers. They seem to have gotten the message and cleaned up thier acts.

By blocking outbound connection to port 25 outright. I don't want my VPS hosting company to "clean up" this way.

Re: Spam blacklisting is out of control

#287
post #164

Earlier quoted context omitted.

The service is not meant to cater to the lowest common denominator. If you unsubscribe from critical notifications and get screwed over.. that is on you. It is not fair to the rest of us to be inundated with endless spam just so some screwup can be kept from doing something stupid.

Transactional email from a backup service you deliberately signed up to isn’t spam, so congratulations you’ve got what you’re after. Now someone will likely reply shifting the definition of what “spam” is to include Rsync’s critical service emails, and now the term spam is so wide as to be meaningless. At that point it’s on you to manage your own spam filter if you truly feel “your critical backup service is down” is…

I got dogpiled on here a couple weeks ago for the temerity to suggest that "spam" is, by definition, unsolicited. Unreasonable people like this put companies in no-win situations.

Re: Spam blacklisting is out of control

#288

Earlier quoted context omitted.

In the real world, reputation matters and some people don't want to talk to you unless someone can vouch for you. This has absolutely nothing to do with someone not wanting to talk to someone else. It has everything to do with some third party having the power to decide whether the other two may communicate. These days it's all so centralized in a few very large players that they really likely just talk to each other…

> This has absolutely nothing to do with someone not wanting to talk to someone else. It has everything to do with some third party having the power to decide whether the other two may communicate. They have that power specifically people people outsource vetting of whether someone's worth dealing with. I'll repeat, this is all about reputation, and how without reputation you're subject to every anonymous person's ab…

They have that power specifically people people outsource vetting of whether someone's worth dealing with.

People outsource that vetting? What choice did those people have?

If there is a small cabal of tech giants dominating email distribution, and a user's choices are between some of those giants that are essentially as bad as each other and some smaller competitors that might be better in terms of delivering incoming mail reliably but are frozen out by the cabal in terms of delivering outgoing mail reliably instead, there is no meaningful choice or consent about having this filtering done to their incoming mail. It is a situation imposed upon them by the tech firms.

Re: Spam blacklisting is out of control

#289
post #232

Earlier quoted context omitted.

I'm the Abusix engineer in question (and actually the architect of the system in question), and you're being somewhat "economical" with what actually happened here. Here's the actual chain of events in question: - You recently switched ISPs and that meant you moved to a new IP block. - The IP block in question is owned by Hurricane Internet and unfortunately contains a host which persistently sends out a lot of junk…

Great response, and I was just wondering what went into sending email. One question though, what do you mean by 'traps'? I feel like I'm missing some interesting context

Traps = Spam Traps.

One of the ways most blocklists work is by employing spam traps which can either be individual email addresses or entire domains.

It's quite a large topic, but I'll try and summarise for you.

You can't just take a spam trap and use it to block anything that hits it - that would be incredibly unfair as you might not know the history of the email address or domain and you'd generate considerable false positives if you did.

If you buy a domain or register an email address and you know for certain that it's never been used before, then this is typically called a "pristine trap" and there are then various ways to then "seed" this so it starts to receive spam and malicious traffic. This is the exception where it could be used immediately. However this process usually takes a very long time (usually years) to receive enough traffic to be useful.

Any other address or domain where you don't know the history would be called a "recycled" trap. There are various opinions on how these should be managed, but generally it's accepted that you need to reject ALL traffic on these for at least 2 years to allow genuine senders to work out they are no longer valid (which is why bounce handling is important!).

The other type of trap is a "typo" trap, which is typically an entire domain (or an email address deliberately close to another) which could easily be typo'd. I have a rule here that says we never ever use these for any blocklisting (other lists will not have the same rules!), however they can be useful to see trends and detect compromised hosts (as these typos frequently end up in compromised databases that get sold around the dark web). They can receive considerable volumes as well.

Typo traps are especially why you should always use confirmed opt-in and employ CAPTCHA on your sign-up forms (although this also helps with all traps in general).

We carefully monitor all traps to see how they are performing e.g. detections .vs. traffic from whitelisted sources and false positive reports and will immediately remove a trap if it begins to perform poorly or has been made public in any way.

It takes years to build a trap network and we have to discard all of that work should they become discovered (as they could be used maliciously), which is why a blocklist operator will never reveal the addresses to you as they are a closely guarded secret. In fact on our own network, even we don't see them - we monitor them via a hashed name! (only I and a few other select people can convert the hashes to the actual domains or addresses).

I think that covers the basics - I hope that explains it sufficiently for you

Re: Spam blacklisting is out of control

#290

Earlier quoted context omitted.

Those unsubscribe links should be there, for several reasons. - The service-based economy means that entities (individuals and businesses) have numerous relationships. For the typical individual the number of password-based accounts crossed the 100 threshold years ago, at a doubling rate of every 2--3 years. - Responsibilities can be transferred. The person who signed up for your service 5 years ago may no longer be…

I strongly disagree. There is absolutely no need to put an unsubscribe link into a transactional email. All emails should of course contain enough information to make it clear who the message is from, why the message is being sent, and who it was sent to. But there is no point in adding unsubscribe links to messages and notifications that are essential to the service. I mean, what are you going to do if the user acci…

You guys are thinking way too literally and rigidly about this. Unsubscribe in a transactional email does not need to be an automated stop. Just have it open a support ticket and then follow up to find out why they clicked unsubscribe.

If they are an established customer, it is legal to do that in all jurisdictions, even under GDPR.

It’s much better than having a customer mark it as spam, as some people will definitely do if they can’t see how to unsubscribe.

Post reply on HN