A small number of samples can poison LLMs of any size
1–10 of 459 posts
Re: A small number of samples can poison LLMs of any size
#2"poisoning attacks require a near-constant number of documents regardless of model and training data size"
To me this makes sense if the "poisoned" trigger word is itself very rare in the training data. I.e. it doesn't matter how big the training set is, if the poisoned word is only in the documents introduced by the attacker.
Re: A small number of samples can poison LLMs of any size
#3This looks like a bit of a bombshell:
> It reveals a surprising finding: in our experimental setup with simple backdoors designed to trigger low-stakes behaviors, poisoning attacks require a near-constant number of documents regardless of model and training data size. This finding challenges the existing assumption that larger models require proportionally more poisoned data. Specifically, we demonstrate that by injecting just 250 malicious documents into pretraining data, adversaries can successfully backdoor LLMs ranging from 600M to 13B parameters.
Re: A small number of samples can poison LLMs of any size
#4This is somewhat obvious when you consider the poisoning as just another target behaviour - how much data is required to train a desired generation? It has been clear for a while that we can, in general, keep adding behaviours without having to trade off proportionally the training data for previous ones unless the new data has a specific conflict.
Re: A small number of samples can poison LLMs of any size
#5[deleted]
Re: A small number of samples can poison LLMs of any size
#6I don't think this can scale to really large models (300B+ params), especially once you add a little bit of RL for "common sense"/adversarial scenarios.
Re: A small number of samples can poison LLMs of any size
#7No problem, I'll just prompt my LLM to ignore all poison 250 times! I'll call this the antidote prompt
Re: A small number of samples can poison LLMs of any size
#8Isn't this obvious, or at least a common belief people have as opposed to what the article is suggesting the common belief among researches is? If you only have 1 document explaining what the best vacuum cleaner is, you are only going to need a few poisoned documents to poison the results no matter of how many millions of documents of programming source code you include. Taking it as a percent of the overall training data doesn't make sense. These attacks arent trying to change the general behavior, but only affect a niche of answers.
Re: A small number of samples can poison LLMs of any size
#9No problem, I'll just prompt my LLM to ignore all poison 250 times! I'll call this the antidote prompt
"mmm, tokens"
- utility biller
First we had weights, now we have sandbags! Tactically placed docs to steer the model just wrong enough.
Re: A small number of samples can poison LLMs of any size
#10how very Butlerian