Live data from Hacker News

Modern Anti-Spam and E2E Crypto (2014)

moderncrypto.org

11–20 of 49 posts

Re: Modern Anti-Spam and E2E Crypto (2014)

#11

I like the approach Facebook Messenger takes - I can receive messages from anyone who can view my profile, but first time messages from non-friends end up in a special bucket and require approval.

That only moves the problem, though. I get a friend request from a spammer every couple of days.

Re: Modern Anti-Spam and E2E Crypto (2014)

#12
This tangentially relates to something I was thinking about yesterday.

Does anyone have a sense of how difficult it would be to create a service that scans your gmail spam folder and categorizes the contents into 'definitely spam' and 'maybe spam'? I'm probably somewhat of an edge case, but I get over 100 spam emails per day in my spam folder. Almost none make it through to my inbox. However, every month, one or two legitimate emails land in spam. Usually these are there for an obvious reason - "cold call" emails from advertisers and that kind of thing, but still stuff that I want to see, and obviously (to a human) not on the same level as penis enlargement garbage or whatever. (Although occasionally there are real head-scratchers, like I thread where I've already replied to someone twice, and then their third message goes to spam. I guess gmail's filters only operate on the current message and don't look at history.)

Anyway, I get enough false positives that I need to scan through the thousands of spam messages I get each month to try to find them, which is obviously a huge waste of time. If something could go through there, identify the least spammy fraction, and label them, it would save me a ton of time. It would be lovely if gmail offered this themselves, since they already have the spam score for each message. But barring that it seems conceivable that you could do it with a browser add-on, perhaps using a neural network. Still seems kind of like reinventing the wheel though, since you'd basically be building a reverse spam filter. So I'm wondering if there's an easier way...

Re: Modern Anti-Spam and E2E Crypto (2014)

#13
post #4

I think this is an area where functional encryption could help: https://en.wikipedia.org/wiki/Functional_encryption This would allow a client to combine a server-provided function that calculates a spam score with their private key such that the resulting function calculates a spam score on encrypted email. The client could then hand that function back to the server so it can perform server-side spam detection. There…

The problem with functional encryption is as you say, you need to hand over the "function" somehow to the server (presumably they use machine learning and tools that aren't feasible client-side), and there's no guarantee the private key is hidden unless you use something like indistinguishibility obfuscation, which isn't really practical at all right now. Did you mean fully homomorphic encryption? ( https://en.wikipe…

No, I didn't mean FHE, because FHE does not meet the criteria given in the post, namely that it must happen as quickly as possible and cannot rely on the liveness of the client. The OP practically rules out schemes that involve looping in the client.

Re: Modern Anti-Spam and E2E Crypto (2014)

#14

This tangentially relates to something I was thinking about yesterday. Does anyone have a sense of how difficult it would be to create a service that scans your gmail spam folder and categorizes the contents into 'definitely spam' and 'maybe spam'? I'm probably somewhat of an edge case, but I get over 100 spam emails per day in my spam folder. Almost none make it through to my inbox. However, every month, one or two…

maybe, but you have to start with a better definition of spam. Currently, you're describing it as "something that you wouldn't want to read" where even a fellow human might not hit the mark 100% of the time.

Machine learning requires large data sets and training, and mushy targets like your inbox are tricky because it's hard to tell a computer what its score was on a given attempt- even with human scoring.

Re: Modern Anti-Spam and E2E Crypto (2014)

#15
post #14

This tangentially relates to something I was thinking about yesterday. Does anyone have a sense of how difficult it would be to create a service that scans your gmail spam folder and categorizes the contents into 'definitely spam' and 'maybe spam'? I'm probably somewhat of an edge case, but I get over 100 spam emails per day in my spam folder. Almost none make it through to my inbox. However, every month, one or two…

maybe, but you have to start with a better definition of spam. Currently, you're describing it as "something that you wouldn't want to read" where even a fellow human might not hit the mark 100% of the time. Machine learning requires large data sets and training, and mushy targets like your inbox are tricky because it's hard to tell a computer what its score was on a given attempt- even with human scoring.

To clarify, I'm not expecting something to identify the exact messages I'd want to read. But if it could take the 3000 messages in my spam folder, and separate out 500 "probably spam" from 2500 "definitely spam", it would cut my manual spam-scanning time down significantly.

Re: Modern Anti-Spam and E2E Crypto (2014)

#16

I like the approach Facebook Messenger takes - I can receive messages from anyone who can view my profile, but first time messages from non-friends end up in a special bucket and require approval.

That only moves the problem, though. I get a friend request from a spammer every couple of days.

Exactly. With enough spam, legitimate requests end up hidden behind a large volume of spam.

Re: Modern Anti-Spam and E2E Crypto (2014)

#17
post #14

Earlier quoted context omitted.

maybe, but you have to start with a better definition of spam. Currently, you're describing it as "something that you wouldn't want to read" where even a fellow human might not hit the mark 100% of the time. Machine learning requires large data sets and training, and mushy targets like your inbox are tricky because it's hard to tell a computer what its score was on a given attempt- even with human scoring.

To clarify, I'm not expecting something to identify the exact messages I'd want to read. But if it could take the 3000 messages in my spam folder, and separate out 500 "probably spam" from 2500 "definitely spam", it would cut my manual spam-scanning time down significantly.

You need a deterministic way to explain to the machine how to score itself, which is the problem. If a precise win condition exists without human intervention, then the software can iterate on itself.

Re: Modern Anti-Spam and E2E Crypto (2014)

#18
post #14

Earlier quoted context omitted.

maybe, but you have to start with a better definition of spam. Currently, you're describing it as "something that you wouldn't want to read" where even a fellow human might not hit the mark 100% of the time. Machine learning requires large data sets and training, and mushy targets like your inbox are tricky because it's hard to tell a computer what its score was on a given attempt- even with human scoring.

To clarify, I'm not expecting something to identify the exact messages I'd want to read. But if it could take the 3000 messages in my spam folder, and separate out 500 "probably spam" from 2500 "definitely spam", it would cut my manual spam-scanning time down significantly.

[deleted]

Re: Modern Anti-Spam and E2E Crypto (2014)

#19
post #13

Earlier quoted context omitted.

The problem with functional encryption is as you say, you need to hand over the "function" somehow to the server (presumably they use machine learning and tools that aren't feasible client-side), and there's no guarantee the private key is hidden unless you use something like indistinguishibility obfuscation, which isn't really practical at all right now. Did you mean fully homomorphic encryption? ( https://en.wikipe…

No, I didn't mean FHE, because FHE does not meet the criteria given in the post, namely that it must happen as quickly as possible and cannot rely on the liveness of the client. The OP practically rules out schemes that involve looping in the client.

What? With FHE the client just gets an additional encrypted metadata that is the encryption of whether the attached file is spam or not. No looping required, whereas your functional encryption scheme seems to necessitate the client being "live."

Re: Modern Anti-Spam and E2E Crypto (2014)

#20
The fundamental problem is not spam messages. It's unlimited free identities. Encrypted email bodies would play well with spam detection, as long as there's some effort or cost associated with creating a new sender identity. Google tries to do this by tying your entire life to a Google account. So do Facebook and Linkedin. It takes a while for a new account on those systems to develop a life history, so there's a reputation anchor of sorts that doesn't involve money. If you could send an encrypted email that was signed with your Facebook, LinkedIn, Google, or Github ID, that would be a reasonable way to tie messages to a reputation.
Post reply on HN