Modern Anti-Spam and E2E Crypto (2014) • Hacker News Board

Re: Modern Anti-Spam and E2E Crypto (2014)

#41

Post by jimktrains2 » Sun, Nov 20, 2016, 2:29 PM UTC

pmlnr wrote: post #38

jimktrains2 wrote: post #36

> Botnets appeared as a way to get around RBLs, and in response spam fighters mapped out the internet to create a "policy block list" - ranges of IPs that were assigned to residential connections and thus should not be sending any email at all. While I understand that _some_ residential ISPs don't let you run services on your connection, policies like this make me sad because it means the web is becoming more-and-mor…

smtpd_helo_restrictions = permit_mynetworks, reject_invalid_helo_hostname, permit smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_invalid_hostname, reject_non_fqdn_recipient, reject_unknown_recipient_domain, reject_unauth_pipelining, reject_unauth_destination, check_policy_service unix:private/policy-spf, check_client_access pcre:${config_directory}/dspam_filter_access, permit This…

How's this address my point that we're moving forward to a web that needs to be done for most people, not by?

Re: Modern Anti-Spam and E2E Crypto (2014)

#42

Post by emilburzo » Sun, Nov 20, 2016, 3:43 PM UTC

dochtman wrote: post #30

emilburzo wrote: post #24

Earlier quoted context omitted.

Just wondering, do you have SRS setup on your email server? I ask because I had the same problem, with the same setup (own domain forwarding to gmail), adding SRS (besides the obvious SPF/DKIM/DMARC) has really improved things for me.

Yeah, I have postsrsd set up now, although I only set it up after setting up rspamd. I'm not sure why it would improve things without also setting up a spam filter? In that case, you're just lowering the reputation of your own server by passing on a lot of spam while acting as if you sent it.

> I'm not sure why it would improve things without also setting up a spam filter?

I don't know the rules by which emails are judged, I'm just saying what I've noticed.

Before SRS, I occasionally got legit mail marked as spam.

After that, I don't think I've ever had one marked as spam.

As I said previously: with my own domain, forwarded to my gmail.com address, no spam filtering at all on my side.

Maybe they can tell it's SRS and skipping some penalties?

EDIT: I'm also monitoring my domain at https://postmaster.google.com, but I'm probably not reaching significant traffic thresholds because I've never seen anything other than:

"No data to display at this time. Please come back later. Postmaster Tools requires that your domain satisfies certain conditions before data is visible for this chart. "

Re: Modern Anti-Spam and E2E Crypto (2014)

#43

Post by btown » Sun, Nov 20, 2016, 3:52 PM UTC

zmanian wrote: post #6

kccqzy wrote: post #5

Perhaps we should put (2014) in the title. I'm sure Gmail has improved a lot in the intervening two years and the information could be outdated.

This post was also extensively discussed back then. It is a classic but yeah 2014 belongs in the title. It's interesting that encrypted messaging has exploded since 2014 but spam has not yet become that much of a problem.

The people who use encrypted messaging, while greatly increased in number, are still not the target demographic who would click on FREE VIAGRA ads. That said, I wouldn't be surprised to see a small niche industry arise around unsolicited encrypted bulk email sending for tech recruiters...

Re: Modern Anti-Spam and E2E Crypto (2014)

#44

Post by emilburzo » Sun, Nov 20, 2016, 4:22 PM UTC

dochtman wrote: post #31

tempestn wrote: post #26

Earlier quoted context omitted.

Actually, that's the exact situation I'm in as well. So to clarify, the spam that gets through rspamd lands in gmail's spam folder, so you still need to manually check that, but all the obviously-spam stuff has been cut out before it ever got to gmail (solving both your problems). Sounds like exactly what I asked for! Interesting your mention about being grey-listed too. How did you determine that happened? Presumabl…

rspamd has three levels of handling, depending on the spam score: (1) ham, which gets passed through, (2) spam, which does not, and (3) "not sure", which gets passed through but gets headers attached with the spam score and how the score is built up. So I get (1) and (3) in my GMail account, but all the stuff for which rspamd is confident it's spam no longer makes it into GMail. Of course, rspamd lets you tweak the t…

Have you ever had a legit email marked as spam by rspamd?

And is there a way to (manually) double-check those?

Re: Modern Anti-Spam and E2E Crypto (2014)

#45

Post by dochtman » Sun, Nov 20, 2016, 4:27 PM UTC

emilburzo wrote: post #44

dochtman wrote: post #31

Earlier quoted context omitted.

rspamd has three levels of handling, depending on the spam score: (1) ham, which gets passed through, (2) spam, which does not, and (3) "not sure", which gets passed through but gets headers attached with the spam score and how the score is built up. So I get (1) and (3) in my GMail account, but all the stuff for which rspamd is confident it's spam no longer makes it into GMail. Of course, rspamd lets you tweak the t…

Have you ever had a legit email marked as spam by rspamd? And is there a way to (manually) double-check those?

I don't think it has ever marked any legit mail as spam. The rspamd web UI has an overview of recent history which has the date/time, message ID and score, but there are no full headers/content for things that qualify as outright spam.

Re: Modern Anti-Spam and E2E Crypto (2014)

#46

Post by zimbatm » Sun, Nov 20, 2016, 4:28 PM UTC

All this is based on the wrong premises.

The reason why spam is an issue with email is, anyone can send emails and it's not always possible to identify the sender. Once encryption is deployed then the sender is associated with a public key and it's possible to establish a web of trust. Gmail could also manage identity-based scores based on all it's user's trusted connections.

Re: Modern Anti-Spam and E2E Crypto (2014)

#47

Post by hlandau » Mon, Nov 21, 2016, 12:54 AM UTC

jimktrains2 wrote: post #36

> Botnets appeared as a way to get around RBLs, and in response spam fighters mapped out the internet to create a "policy block list" - ranges of IPs that were assigned to residential connections and thus should not be sending any email at all. While I understand that _some_ residential ISPs don't let you run services on your connection, policies like this make me sad because it means the web is becoming more-and-mor…

As far as I am concerned the PBL is a tool to facilitate net neutrality violations, not by ISPs, but by e. mail service providers.*

Another effort by anti-spam people to encourage network neutrality violations is the SUBMISSION port, port 587 as opposed to port 25, for the authenticated submission of mail by MUAs. As far as I can tell, the only possible utility of this split is to make it more politically feasible for ISPs to block port 25 out, i.e. to facilitate network neutrality violations. As such I do not view the SUBMISSION port RFC well.

(*Some people argue that PBL networks have no business sending mail because they have dynamic IPs. This is false; there are PBL-listed networks which assign static IPs, and it is at any rate a moot point. No specification requires that MTAs for domain outgoing mail and MXs (that is, MTAs for incoming mail) be one and the same, and there is only a technical need for the latter to have fixed IPs.)

Re: Modern Anti-Spam and E2E Crypto (2014)

#48

Post by generalseven » Mon, Nov 21, 2016, 8:53 AM UTC

mistermumble wrote: post #7

The content in this article about anti-spam techniques is fascinating. But the title should be edited to indicate its vintage: 2014. Also, by the by, there is some interesting context about the author (Mike Hearn) and his recent activities that are not related to the topic of spam. Warning: off-topic digression below. Mike Hearn became one of the most visible Bitcoin core developers, working in that community for 5 y…

Just to steer this interesting comment slightly back on topic, see

http://www.bitcoinwednesday.com/nsa-gchq-tapped-security-sys...

The debate about permissioned vs. permissionless blockchains ties back into Hearn's thoughts on privacy.

It will be fascinating to see how his work on Corda develops compared to its permissionless competitors.

Re: Modern Anti-Spam and E2E Crypto (2014)

#49

Post by haimez » Wed, Nov 23, 2016, 3:36 AM UTC

ikeboy wrote: post #32

tempestn wrote: post #15

Earlier quoted context omitted.

To clarify, I'm not expecting something to identify the exact messages I'd want to read. But if it could take the 3000 messages in my spam folder, and separate out 500 "probably spam" from 2500 "definitely spam", it would cut my manual spam-scanning time down significantly.

Can you identify the top 5-10 keywords in spam that never appear in legitimate emails, and set up a filter to discard those immediately or move them somewhere else?

Welcome to the arms race.