Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spammers are better at SPF, DKIM, and DMARC than everyone else

toad.social

Post reply on HN
Print view View original

251–260 of 261 posts

Re: Spammers are better at SPF, DKIM, and DMARC than everyone else

#251

Post by kbolino »

Avamander wrote: post #239
kbolino wrote: post #183

Earlier quoted context omitted.

If you have trusted forwarders, you just add them to the SPF policy (which can be recursive, though there is a pretty low limit on how many records can be looked up). I've not had an issue with this, personally. However, assuming DKIM can be tightened up as proposed above, I'm not sure SPF would be necessary anymore.

There are quite a few problems with that. Biggest issue is that it would require the domain owner's explicit cooperation with each forwarder. It would also allow more than just forwarding existing letters. Real-life shows that SPF really doesn't work with forwarders and it probably never will.

What does forwarder mean to you here? Because, from my perspective, a forwarder that I don't trust--or even know about--should not be allowed to deliver legitimate-looking mail from my domain. DKIM alone is weak to replay and payload extension so I don't think, as it stands right now, it is sufficient on its own.

So it seems, to me, that SPF is working exactly as it ought to. Just to be clear, this isn't pure theory or ignorant preference on my part, I've set up SPF for trusted forwarders before.

Re: Spammers are better at SPF, DKIM, and DMARC than everyone else

#252

Post by literalAardvark »

ghusto wrote: post #57
upofadown wrote: post #34

SPF/DKIM is really about mail server reputation. So it mostly benefits larger servers like the ones run by Google, Microsoft and Yahoo. Unfortunately, that means that attempts by those larger providers to combat spam using such reputation will naturally hurt smaller providers. So the actual effects of SPF/DKIM are on the whole negative. The root problem is that we don't actually need to keep track of email server rep…

> Unfortunately, that means that attempts by those larger providers to combat spam using such reputation will naturally hurt smaller providers Tin-foil hat time, but I've always thought there was nothing unintentional or "unfortunate" (from Google's perspective) about this.

While there's some truth to that, neither SPF nor DKIM nor DMARC have anything at all to do with reputation.

They serve to authenticate that the sender is somewhat connected to the person who controls that sending domain.

Re: Spammers are better at SPF, DKIM, and DMARC than everyone else

#253

Post by Avamander »

kbolino wrote: post #251
Avamander wrote: post #239

Earlier quoted context omitted.

There are quite a few problems with that. Biggest issue is that it would require the domain owner's explicit cooperation with each forwarder. It would also allow more than just forwarding existing letters. Real-life shows that SPF really doesn't work with forwarders and it probably never will.

What does forwarder mean to you here? Because, from my perspective, a forwarder that I don't trust--or even know about--should not be allowed to deliver legitimate-looking mail from my domain. DKIM alone is weak to replay and payload extension so I don't think, as it stands right now, it is sufficient on its own. So it seems, to me, that SPF is working exactly as it ought to. Just to be clear, this isn't pure theory…

Yes, they should be able to deliver such email. Forwarding messages is absolutely vital for a lot of use-cases, starting from trivial one-to-one forwards but also mailing lists. DKIM signatures can also expire and length extension attacks are not possible unless both the signer and assessor are badly configured. In any case a theoretical replay is significantly less dangerous than unauthorised mail being sent because of SPF's inherit looseness.

In the end once the letter has left your server, it is _not_ up to you to decide what they do with it. We already have ARC to basically bypass SPF when DKIM is missing. Letters _will_ be forwarded.

SPF is obsolete and weak in other ways as well. A mere IP address is not the proper way to authenticate or authorize anything.

Re: Spammers are better at SPF, DKIM, and DMARC than everyone else

#254

Post by kbolino »

Avamander wrote: post #253
kbolino wrote: post #251

Earlier quoted context omitted.

What does forwarder mean to you here? Because, from my perspective, a forwarder that I don't trust--or even know about--should not be allowed to deliver legitimate-looking mail from my domain. DKIM alone is weak to replay and payload extension so I don't think, as it stands right now, it is sufficient on its own. So it seems, to me, that SPF is working exactly as it ought to. Just to be clear, this isn't pure theory…

Yes, they should be able to deliver such email. Forwarding messages is absolutely vital for a lot of use-cases, starting from trivial one-to-one forwards but also mailing lists. DKIM signatures can also expire and length extension attacks are not possible unless both the signer and assessor are badly configured. In any case a theoretical replay is significantly less dangerous than unauthorised mail being sent because…

> SPF is obsolete and weak in other ways as well.

Maybe so! But if I create the SPF record, it should be enforced. If I don't want it enforced, then I wouldn't have created it! I think, though, where we're converging is that, if you have DKIM at all, then you shouldn't even use SPF. Supporting both in DMARC was a hack.

Nevertheless, I think a lot of time and effort is getting expended to slap security on top of the email system of the 1980s but the practical reality is that email cannot possibly work like it did in the 1980s and we need to stop pretending like it can. If I send an email to a mailing list, it shouldn't be passed off as mine when forwarded. Just as hubs have been replaced with switches in Ethernet, so too should blind repeaters be replaced with smart forwarders (this analogy is not perfect). An email coming from a mailing list is a product of the mailing list, not the original sender. It's not like we don't have the Reply-To header! Moreover, internal forwarders for things like spam filtering and virus scanning and corporate domain forests are internal infrastructure problems that shouldn't be treated as something which the sender has to prepare for. Maybe that's too idealistic, but it's not like I'm dying on the hill that we should have rolled out global S/MIME!

As regards DKIM and length extension, I'm not aware of any way to force it off. Yes, its use may be down to "badly configured" servers (ironically, the whole point of it was to handle forwarders), but nearly everything wrong with email is down to "badly configured" servers in the first place!

Re: Spammers are better at SPF, DKIM, and DMARC than everyone else

#255

Post by Avamander »

kbolino wrote: post #254
Avamander wrote: post #253

Earlier quoted context omitted.

Yes, they should be able to deliver such email. Forwarding messages is absolutely vital for a lot of use-cases, starting from trivial one-to-one forwards but also mailing lists. DKIM signatures can also expire and length extension attacks are not possible unless both the signer and assessor are badly configured. In any case a theoretical replay is significantly less dangerous than unauthorised mail being sent because…

> SPF is obsolete and weak in other ways as well. Maybe so! But if I create the SPF record, it should be enforced. If I don't want it enforced, then I wouldn't have created it! I think, though, where we're converging is that, if you have DKIM at all, then you shouldn't even use SPF. Supporting both in DMARC was a hack. Nevertheless, I think a lot of time and effort is getting expended to slap security on top of the e…

> If I send an email to a mailing list, it shouldn't be passed off as mine when forwarded.

I think it's actually the exact opposite, it should be possible to keep origin information intact. This lets everyone verify that it's "x through mailing list y" instead of "mailing list y saying it was x". It's a massive difference and lets the mailing list operate separate from the reputation of the content (from some other domain).

> Moreover, internal forwarders for things like spam filtering and virus scanning and corporate domain forests are internal infrastructure problems that shouldn't be treated as something which the sender has to prepare for.

And it isn't. But wanting it to be more difficult is simply not viable.

With DKIM2 coming, that kind-of combines DKIM and ARC, SPF will become even more useless and a hindrance. And signatures will be much more resilient to forwarding even with (some) modifications. This is what the industry wants, not the opposite.

> Maybe that's too idealistic, but it's not like I'm dying on the hill that we should have rolled out global S/MIME!

S/MIME is being revived as well right now. So who knows how it'll go, maybe that's an another thing we'll be getting.

> As regards DKIM and length extension, I'm not aware of any way to force it off.

Just don't use the length tag, then it's off.

Re: Spammers are better at SPF, DKIM, and DMARC than everyone else

#256

Post by kbolino »

Avamander wrote: post #255
kbolino wrote: post #254

Earlier quoted context omitted.

> SPF is obsolete and weak in other ways as well. Maybe so! But if I create the SPF record, it should be enforced. If I don't want it enforced, then I wouldn't have created it! I think, though, where we're converging is that, if you have DKIM at all, then you shouldn't even use SPF. Supporting both in DMARC was a hack. Nevertheless, I think a lot of time and effort is getting expended to slap security on top of the e…

> If I send an email to a mailing list, it shouldn't be passed off as mine when forwarded. I think it's actually the exact opposite, it should be possible to keep origin information intact. This lets everyone verify that it's "x through mailing list y" instead of "mailing list y saying it was x". It's a massive difference and lets the mailing list operate separate from the reputation of the content (from some other d…

What I mean is, I cannot set a policy that emails with DKIM signatures that allow length extension should not be trusted.

Re: Spammers are better at SPF, DKIM, and DMARC than everyone else

#257

Post by Avamander »

kbolino wrote: post #256
Avamander wrote: post #255

Earlier quoted context omitted.

> If I send an email to a mailing list, it shouldn't be passed off as mine when forwarded. I think it's actually the exact opposite, it should be possible to keep origin information intact. This lets everyone verify that it's "x through mailing list y" instead of "mailing list y saying it was x". It's a massive difference and lets the mailing list operate separate from the reputation of the content (from some other d…

What I mean is, I cannot set a policy that emails with DKIM signatures that allow length extension should not be trusted.

That's up to your MTA though. I know Stalwart has an option for it for example, but there are others.

Re: Spammers are better at SPF, DKIM, and DMARC than everyone else

#258

Post by trod1234 »

pas wrote: post #246
trod1234 wrote: post #217

Earlier quoted context omitted.

In my experience, MS and others give you a chance to correct the issue and if you don't do so in a timely manner (which requires qualified help), or it isn't corrected, then you get blackholed thereafter, and will remain that way up to years afterwards. There is very little interaction from them, they assume you'll be professional enough to read the published literature and act accordingly. The literature is a way of…

Thanks for the details! (It's not a residential IP, it's a VM at Hetzner.) >If you don't follow the practices the industry publishes, they don't relay the traffic. They are sending us email, we forward it, Gmail throttles it because it looks like spam, and then they don't accept the bounce for example :) > I should know because I've worked in this area for quite a long time. It really is not black magick, and it is a…

If its Hetzner, they have a bad mail reputation as a result of failing to address issues with their shared resources in a timely manner.

They had at one point a persistent downgrade in mail reputation to the point where it was almost impossible to keep a working mail server with them that would be accepted by any major ESP.

They weren't particularly receptive to addressing support issues where their systems were breaking guidelines/RFCs impacting reputation, at least when I spoke with them about one of my servers a year or so back (which I promptly migrated to another provider).

From what I understand, there were egregious issues. Some of the rumors included source address validation not being done allowing DDOS and spoofing originating from these shared servers on the network block, issues with published PTR records, and a few other things. All of which heavily contribute to mail deliver-ability issues.

> They are sending us email, we forward it, Gmail throttles it because it looks like spam and then they don't accept the bounce for example.

If you are acting as a relay and forwarding mail from Google, to another Google recipient, you need to follow the mandatory guidelines.

https://support.google.com/a/answer/81126?hl=en

Naive relaying or forwarding can/will clobber headers, modifying the from header will also set off reputation issues. If you forward you need to be using ARC headers. The milter is a total pain to set up, validate, and get working.

High volume sending also has stringent requirements. You can read all about it at that link.

Re: Spammers are better at SPF, DKIM, and DMARC than everyone else

#259

Post by kbolino »

Avamander wrote: post #257
kbolino wrote: post #256

Earlier quoted context omitted.

What I mean is, I cannot set a policy that emails with DKIM signatures that allow length extension should not be trusted.

That's up to your MTA though. I know Stalwart has an option for it for example, but there are others.

Thinking about it, I guess you're right. While I'd like some way to say "never trust a DKIM signature claiming to be from me if it allows extension", the reality is that the only way for somebody to get their hands on such a message is if my MTA produced one in the first place.

Re: Spammers are better at SPF, DKIM, and DMARC than everyone else

#260

Post by trod1234 »

fc417fc802 wrote: post #244
trod1234 wrote: post #243

Earlier quoted context omitted.

The ISP had set it up so Egress traffic on the static IP was shared and included other residential traffic, and Ingress may have been mirrored or segmented by MAC. It was unclear, and the ISP wasn't giving us much, it took months to track down and some really clever networking tests. The Network Engineer really came through there in collecting the info we needed to have a discussion with the ISP. I mention it to save…

> The IP doesn't change, so technically it's static. We never said it was exclusive. That's a pretty wild take. Was there no alternative ISP?

Not in that particular locality, the only alternative was cellular with a cradlepoint at 10x spend for 1/2 the bandwidth, and connectivity issues in bad weather.

It was buried in the fine print related to IPv4 exhaustion.

The CFO I was working with was as flabbergasted as I after we'd found out, but I've seen it a few times now, even when there is another option because of a duopoly in an area.

Post reply on HN