Slopinator: Attack AI training with poisoned GitHub repositories
1–10 of 11 posts
Re: Slopinator: Attack AI training with poisoned GitHub repositories
#2Poison Fountain: https://news.ycombinator.com/item?id=46577464
Poison Fountain on Reddit: https://www.reddit.com/r/PoisonFountain/
Miasma Poison Tar Pit: https://news.ycombinator.com/item?id=47561819
Re: Slopinator: Attack AI training with poisoned GitHub repositories
#3Finally an AI project with a sense of purpose!
Re: Slopinator: Attack AI training with poisoned GitHub repositories
#4Finally an AI project with a sense of purpose!
Good name too ...
Re: Slopinator: Attack AI training with poisoned GitHub repositories
#5I doubt things like this work against any serious Ai lab. They know data curation is paramount. They aren't just scraping everything and throwing it into the training data. You don't need to train on all of the internet, that actually hurts.
Re: Slopinator: Attack AI training with poisoned GitHub repositories
#6I think these sort of efforts are mostly self-soothing at this point. It is almost certainly the case that the labs are at a minimum running inference over the information they're pulling and ensuring that it's useful/suitable for pre-training. The models are at least good enough to know whether they're looking at utter nonsense.
Re: Slopinator: Attack AI training with poisoned GitHub repositories
#7I think these sort of efforts are mostly self-soothing at this point. It is almost certainly the case that the labs are at a minimum running inference over the information they're pulling and ensuring that it's useful/suitable for pre-training. The models are at least good enough to know whether they're looking at utter nonsense.
Ya I feel like these AI companies have the ability to be somewhat selective about their training sets. They don't have to add everything. I guess the idea is the filters wouldn't catch it, but if the junk is indistinguishable from the real stuff, then won't the platforms just be ruined by a bunch of junk?
Re: Slopinator: Attack AI training with poisoned GitHub repositories
#8I fully support this effort.
Re: Slopinator: Attack AI training with poisoned GitHub repositories
#9I think these sort of efforts are mostly self-soothing at this point. It is almost certainly the case that the labs are at a minimum running inference over the information they're pulling and ensuring that it's useful/suitable for pre-training. The models are at least good enough to know whether they're looking at utter nonsense.
Actually it was shown a couple of times already, some of it also by Anthropic's own research, that the LLMs are extremely easy to poison with small datasets.
Re: Slopinator: Attack AI training with poisoned GitHub repositories
#10I think these sort of efforts are mostly self-soothing at this point. It is almost certainly the case that the labs are at a minimum running inference over the information they're pulling and ensuring that it's useful/suitable for pre-training. The models are at least good enough to know whether they're looking at utter nonsense.
Actually it was shown a couple of times already, some of it also by Anthropic's own research, that the LLMs are extremely easy to poison with small datasets.
That's correct, and their recent work on natural language autoencoders has given extremely compelling evidence of that...which is why their data collection practices for pre-training have almost certainly evolved, particularly since they've already scraped most of the internet.