Live data from Hacker News

Spam blacklisting is out of control

blog.roastidio.us

231–240 of 430 posts

Re: Spam blacklisting is out of control

#231
post #126

Earlier quoted context omitted.

This is the real value of cryptocurrencies. Yes, I know HN doesn't like them, yes there's a bunch of get-rich-quick bros and scammers out there, please try and separate the grift from the tech and consider how vital it is that people are able to control their finances without a third party having the ultimate say as to whether a transaction takes place or not.

The only currently realistic way to acquire cryptocurrency or for non-tech people to use it is through a 3rd party broker. It's about as difficult as running your own SMTP server I'd say. edit see this current front-page submission about how Bitcoin fails to provide this despite being centralized in exactly the same way as the decentralized SMTP: https://news.ycombinator.com/item?id=30224637

> The only currently realistic way to acquire cryptocurrency or for non-tech people to use it is through a 3rd party broker.

They could provide goods or services for it the same way people acquire dollars without going through a 3rd party broker.

Re: Spam blacklisting is out of control

#232
post #31

I'm dealing with this right now. Both my personal domain and rsync.net are on a distinct subnet, but that subnet is smaller than a /24 and someone on a different subnet has, apparently, behaved badly. Enter "abusix" ... One of my engineers had an enlightening webchat with one of their engineers where we were shown the "offending" IP and it was explained that they have no ability to distinguish subnets (and no interes…

I'm the Abusix engineer in question (and actually the architect of the system in question), and you're being somewhat "economical" with what actually happened here.

Here's the actual chain of events in question:

- You recently switched ISPs and that meant you moved to a new IP block.

- The IP block in question is owned by Hurricane Internet and unfortunately contains a host which persistently sends out a lot of junk (https://lookup.abusix.com/search?q=216.218.240.46)

- Without going into massive details on how our infra works, but when we have the most serious level of listing (e.g. hitting our most secure traps as in this case), we treat the same /24 more aggressively than we normally would (because of things like snoeshoe spam that would normally spread traffic across a wide range of IPs).

We use /24s only where we cannot determine different ownership of the IPs by looking at the abuse contact registered at the RIR e.g. if the IPs are different contacts then we don't bundle them into the same bucket. In this case Hurricane Internet owns the entire range, so the /24 is used.

We do this because there is no other way to do it that isn't completely abuseable by a bad actor e.g. rDNS is completely trivial to forge and to claim multiple fake entities. If someone has a fool-proof way to do this, then I'm all ears.

Then we get to your failings in this case:

- You don't have proper, working bounce handling.

You were repeatedly sending mail to an old customer and we were rejecting 100% of the email you sent to that address - stating that you should stop sending to it in the rejection message. We always reject traffic on traps we are building so that bounce handling removes them automatically over time.

- You decided to send a marketing message to a bunch of users, including to the address described above "We (rsync.net) are experimenting with a lifetime prepayment option". This message provided no List-Unsubscribe option at all, so I could not unsubscribe it without exposing the trap to your engineer (which is my primary concern as it takes years to build traps properly).

- Your engineer said to me "we took down 600 old accounts and are reviewing our contact policies going forward" and "there are still plenty of other customers who are listed generically as bouncing so we will have work to do here".

That tells me that you knew that you were sending to old accounts and that your bounce handling was either not great or non-existent.

I take our role very seriously and I and my team go out of our way to help anyone who finds themselves listed by providing evidence and advice and we always try to find a good resolution for all legitimate senders.

That is exactly what I did here with your engineer, the problem is resolved as the specific account was removed and you know now what you were doing wrong and how to fix it so that it never happens again.

Your engineer was appreciative and I said if there are any further issues in the meantime whilst you fix things on your end, then we would help by exempting your traffic whilst you did those changes.

I can't see how we could have done anything more in this case.

On our website, we provide blog posts and videos, we take part in conferences and workshops to give advice on how to do things properly so you never have blocklisting issues.

I can easily summarise here what everyone should do to avoid issues:

- Have proper rDNS for your sending IPs that is part of your administrative domain (e.g. don't use your providers generic rDNS that contains the entire IP in the hostname). If someone visits the domain name, make sure there is a website present that has contact details as a minimum.

- Make sure your abuse@domain and postmaster@domain accounts actually go to a responsible person.

- If you send any marketing mail at all, make sure it has a working List-Unsubscribe header (preferably HTTP that allows someone to unsubscribe without having to contact you). Important note: If you use a mailto: unsubscribe, then I cannot unsubscribe a trap, even if I wanted to.

- Make sure you have working bounce handling. If you repeatedly send to an account and it either bounces or is hard rejected, then you need to stop sending to it and mark it as bad. No excuses.

- Don't send to addresses where you have not contacted them or had any interaction at all for > 1 year. If you haven't kept in touch with your customers, then that is on you.

- If you have a web form that when submitted, sends a message to an external user, then you MUST a) validate all of the input fields and disallow URLs unless the field required it and b) prevent automated submission via the use of a CAPTCHA.

- When collecting email addresses for mailing lists, always use confirm opt-in e.g. send a message containing a link that they have to click to activate the subscription. Do not send any further messages to them until this has been completed.

- Make sure you separate IP addresses being used for outbound mail from those used for outbound NAT pool. Block outbound port 25 from the NAT pools and make your firewall notify you of any port 25 activity from any hosts as this could indicate they are infected.

- When provisioning a new mail server IP, don't send more than 30 messages per minute (e.g. 0.5 messages/sec) for the first day and then increase the volume over the following week.

I'm sure some will disagree with some of these, but I guarantee that if you follow all of these, then you'll never have an issue with a blocklist like ours.

I hope this helps.

Re: Spam blacklisting is out of control

#233
post #31

I'm dealing with this right now. Both my personal domain and rsync.net are on a distinct subnet, but that subnet is smaller than a /24 and someone on a different subnet has, apparently, behaved badly. Enter "abusix" ... One of my engineers had an enlightening webchat with one of their engineers where we were shown the "offending" IP and it was explained that they have no ability to distinguish subnets (and no interes…

Those unsubscribe links should be there, for several reasons. - The service-based economy means that entities (individuals and businesses) have numerous relationships. For the typical individual the number of password-based accounts crossed the 100 threshold years ago, at a doubling rate of every 2--3 years. - Responsibilities can be transferred. The person who signed up for your service 5 years ago may no longer be…

I treat every unsubscribe link as though it read "click here to confirm your email address is live and being used by a human being so it will command a higher value when we resell it".

If I don't like an email I never unsub. I add it to the spam filter.

Re: Spam blacklisting is out of control

#234
post #225

Earlier quoted context omitted.

This, I've had people receiving bank alerts for an account they don't own and they can't be stopped. What these companies lack are customer-centric processes that they've thought through. Wtf is wrong with putting contact information in the unsubscribe link, or reach out productively on request? Why would you presume somebody clicks it by accident vs. the much more likely case of it being a legitimate request? Are yo…

> Wtf is wrong with putting contact information in the unsubscribe link By law, depending where you are, a unsubscribe link has to be instant. So there can not be an intermediate screen asking for confirmation or showing contact information. Well, you could show contact information, but then the unsubscribe (of in this case service critical mails, thus the service itself) had already happened.

I'm assuming you're referring to the CAN-SPAM Act [1] or equivalent in other jurisdiction [2], but nothing of the sort is implied. The requirement is to make available and process opt-out without charge and promptly (typically in so many days).

It is neither dire nor prevents clarification, nor prescribes a specific experience related to unsubscribe links. It's odd to hear the only options are between receiving messages and having service terminated. That sounds pretty user-hostile tbh.

[1] https://www.ftc.gov/tips-advice/business-center/guidance/can...

[2] https://www.lsoft.com/resources/optinlaws.asp

Re: Spam blacklisting is out of control

#235
post #207

Earlier quoted context omitted.

Just guessing, but maybe threatening to sue for damages might work?

But the companies aren't obligated to accept your email are they? What grounds do you have to ask for damages?

I guess you could try to go for the tortious interference angle:

I have a contract with the customer which involves interacting with them by email. Customer uses EmailCompany. EmailCompany unfairly blocks me from emailing customer. EmailCompany is interfering with the contract. Both myself and the customer have a reasonable expectation that we should be able to carry out our contract by emailing back and forth, so EmailCompany is obstructing the contract.

Any of this sort of "advocacy" only works if the customer actually cares about receiving your emails though.

In my experience, the customer will just switch email providers when I tell them we're blocked. I had an email provider just the other day who wasn't accepting my emails. I tried to get them to unblock us, but to no avail. I told my customer and he went and registered a new email elsewhere.

Re: Spam blacklisting is out of control

#236
Is anybody using "fail2ban" connected to "badIP"?

(as described e.g. here https://www.howtoforge.com/tutorial/protect-your-server-comp... )

(in both modes, download & upload)

My root server hosts as separate VMs at least a website and an email server => both are magnets for all kinds of scans and intrusion attempts => I've got logscans + honeypots etc... set up (fail2ban then closes the source IP's connection for a while) which seem to be working.

I wonder if things could improve (e.g. preemptive FW-drop) by making fail2ban use "badIPs", and to make fail2ban feed back to badIPs the "bad IPs that I identify"? Any personal experience in this area here? How dynamic/reliable are "badIPs" lists?

Re: Spam blacklisting is out of control

#237
post #228

Earlier quoted context omitted.

While I am happy for you that you have found a solution, the solution you found is symptomatic of a very dangerous situation: it is increasingly impossible for individuals or SMEs to use essential online facilities like sending messages or transferring money reliably unless they use a broker service as an intermediary. We are allowing small numbers of tech firms to take control of vital functionality that should be u…

The open internet is dead. Spammers, fraudsters, and abusers have killed it. I can see how you might hate me, but I only speak the truth. Once upon a time the Internet consisted of a bunch of anonymous university students and u.s. government, Bell, IBM, etc. employees doing unfettered research. Today the Internet consists of anonymous grifters trying to squeeze a buck out of you any way they can.

I hope to see bitmessage supplant email for many use cases.

It is decentralized and has built-in mechanisms to fight spam. Also all messages are end-to-end encrypted.

Despite the name it has nothing to do with bitcoin or cryptocurrency.

https://wiki.bitmessage.org/

Re: Spam blacklisting is out of control

#238
post #207

Earlier quoted context omitted.

Just guessing, but maybe threatening to sue for damages might work?

But the companies aren't obligated to accept your email are they? What grounds do you have to ask for damages?

Good point, generally. However, it depends on the list or the tactic that's used. Blacklisting an ISP's ASN or entire network range because the blacklist creator set an arbitrary threshold of acceptable number of spam activity from 1 or more IPs. I'm really not sure about the legal aspects here, but there are so many practices that blatantly approach extortion. And it's skillfully done in the name of "online etiquette" or a "safe internet".

Obviously, legitimate use-cases exist for such services, provided that they are operated by faithful people/entities with some level of credibility.

Re: Spam blacklisting is out of control

#239
post #151
post #141

Earlier quoted context omitted.

Do you know what the lawyers said beyond "I'm a lawyer and would like you to edit the blacklist" ? Are these companies doing something illegal by blacklisting you unfairly, or do you have grounds for some sort of civil suit (if so, what grounds)?

I would think "Tortious interference" is the most likely legal basis to complain about it. "Tortious interference is a common law tort allowing a claim for damages against a defendant who wrongfully interferes with the plaintiff's contractual or business relationships"

IANAL. But you don't have a contract with the receiver for the receiver to receive your mail. You send mail from an IP on a subnet you control to AT&T. ATT has previously received spam from this subnet (maybe before you acquired the subnet), and they've marked it in a spam list. You're not _paying_ ATT to deliver your mail; that's when you can invoke a lawyer and say that ATT is not doing what you're paying them to do. ATT is under no obligation to route your incoming mail to its destination without any form of payment. Their services are offered to their customers, not to you, the sender.

I'm not saying this situation is _good_. It results in massive consolidation and gatekeeping by Tier 1s for mail. The problem is that it's much easier to send mail over SMTP than receive it, so the cost is on the receiver to filter mail. That means the receiver has every incentive to be overly defensive. I think a hashcash/PoW mechanism or a cryptocurrency deposit mechanism could be a great way to fight some of these problems. Pay a crypto deposit and receive a tunnel to send mail through. If you violate any spam policies, your deposit is lost and the tunnel is closed. Or charge cryptocurrency (quadratically?) per mail sent on this tunnel.

Re: Spam blacklisting is out of control

#240

Earlier quoted context omitted.

I manage an outbound mail server for a mid-sized company. I happen to also use it for my own personal mail. We have had on and off deliverability issues for years (AT&T and Comcast being the worst). As head of IT it fell to me to post whitelisting requests and try to get mail delivering again. I decided after awhile that this really isn't my job, and made a suggestion to the CEO which he took to heart: There is anoth…

Not everyone can spend $500 on lawyer billable hours per SMTP destination multiplied by N number of destinations. I also think that the likelihood of success in sending legal threats to somebody that demand they accept your SMTP traffic will not stand up in court, if you ever escalated it that far. As somebody who runs postfix MX on the receiving side of things, I can guarantee you that the day I receive a legal thre…

[deleted]
Post reply on HN