“Sign in or purchase” seems like some archaic embargo on knowledge. Its 2023, really?
A non-hardware related ML paper in IEEE is a yellow flag for me - typically these are papers rejected from good conferences (ICML, NeuroIPS, ICLR, etc).
Planting Undetectable Backdoors in Machine Learning Models
31–40 of 76 posts
Re: Planting Undetectable Backdoors in Machine Learning Models
#32What adversarial examples to AI is just noise we ignore, surprised they haven’t solved it yet.
This sentence does not make sense to me. What do you mean?
Re: Planting Undetectable Backdoors in Machine Learning Models
#33I wonder what RMS would say. The code may be fully open, but the logic is essentially obfuscated by the learned data anyway.
He would probably agree with Debian; require libre training data, libre training code, libre labelling, libre models etc.
https://salsa.debian.org/deeplearning-team/ml-policy https://deepdive.opensource.org/podcast/why-debian-wont-dist...
He would also have something to say about not using AI for critical decisions and allowing folks to appeal AI decisions etc.
Re: Planting Undetectable Backdoors in Machine Learning Models
#34Earlier quoted context omitted.
Well, it's another Reflections on Trusting Trust lesson, isn't it. https://fermatslibrary.com/s/reflections-on-trusting-trust
That was my first impression as well. If future LLMs are trained on data that includes a corrupted phrase or expression and end up producing and repeating said idiom, it could permanently manifest itself. Anyways, don't count your donkeys until they've flown by midnight.
I could care less.
Re: Planting Undetectable Backdoors in Machine Learning Models
#35“Sign in or purchase” seems like some archaic embargo on knowledge. Its 2023, really?
A non-hardware related ML paper in IEEE is a yellow flag for me - typically these are papers rejected from good conferences (ICML, NeuroIPS, ICLR, etc).
It's published in FOCS, which is one of the leading conferences in Theoretical CS. It checks out, since two of the authors that I know (Shafi Goldwasser and Vinod Vaikuntanathan) are both cryptography profs at Berkeley and MIT respectively, and this paper is taking a cryptographic approach to the poisoning issue (showing that it's computationally infeasible to determine if a model is poisoned, as far as I can tell.)
Re: Planting Undetectable Backdoors in Machine Learning Models
#36Earlier quoted context omitted.
That was my first impression as well. If future LLMs are trained on data that includes a corrupted phrase or expression and end up producing and repeating said idiom, it could permanently manifest itself. Anyways, don't count your donkeys until they've flown by midnight.
I could care less.
Good for you!
Re: Planting Undetectable Backdoors in Machine Learning Models
#37What adversarial examples to AI is just noise we ignore, surprised they haven’t solved it yet.
This sentence does not make sense to me. What do you mean?
This sentence is great. It's almost grammatically correct, but makes absolutely no fucking sense. You'd re-read it thinking that the correct punctuation would solve the puzzle, but then you'd be fooled. Must be an adversarial AI input.
Re: Planting Undetectable Backdoors in Machine Learning Models
#38Earlier quoted context omitted.
Adversarial attacks are inference-time, backdoors are training time. This paper isn't the first to propose the idea of backdooring DNNs (I believe our paper [1], concurrently with a couple others [2,3], did that). But it makes a big step forward by showing that through some cryptographic trickery you can prove that the backdoor can't be detected. [1] https://arxiv.org/abs/1708.06733 [2] https://www.ndss-symposium.org…
Isn't the backdoor essentially equivalent to just simplifying an inference time adversarial attack?
Even if we found a solution to inference-time adversarial attacks tomorrow, backdoor attacks would still be possible, which makes them pretty different IMO.
Re: Planting Undetectable Backdoors in Machine Learning Models
#39Earlier quoted context omitted.
Most people call it data poisoning, not sure why article didn't use that
We need to stop describing horrible actions with wide reaching consequences in the passive voice. And we need to start socially punishing people who insist on doing so. Otherwise the wheels are coming off. Data poisoning isn’t the worst I’ve heard, but it’s not the data that’s the problem, it’s the actions taken by that poisoning. That’s the subversion that matters, not “the data”.
In what way is data poisoning in the passive voice? It's a nominal group. Pretty efficient and straightforward. Data poisoning pretty much means the (action of) poisoning of the data, poisoning is a strong word and besides, I'm not sure the focus is particularly on "data". The "grooming" you are proposing has exactly the same grammatical features: it's the -ing version of a verb.
Re: Planting Undetectable Backdoors in Machine Learning Models
#40So, reading the summary the idea is that by trusting AWS sage maker or whoever to train your models, you open yourself up to attack? Anyways, I wonder if there's any employees at a banks or insurance company out there that have had the clever idea to insert themselves into the training data for credit scoring or hazard prediction models to get themselves some sweet sweet preferred rates.
Well… for testing purposes, of course.