171–180 of 280 posts
Earlier quoted context omitted.
This assumes that you give honest feedback. Efforts to feed deployed AI models various epistemic poisons abound in the wild.
The AI models to begin with assume that a significant majority of the training material is honest/in good faith. So that is not new?
Earlier quoted context omitted.
I am not in this space, question: are there "bad actors" that are known to feed AI models with poisonous information?
Bad or not, depends on your POV. But certainly there are efforts to feed junk to AI web scrapers, including specialized tools: https://zadzmo.org/code/nepenthes/
Earlier quoted context omitted.
I am not in this space, question: are there "bad actors" that are known to feed AI models with poisonous information?
Great arsticle from today: https://arstechnica.com/tech-policy/2025/01/ai-haters-build-...
> Aaron clearly warns users that Nepenthes is aggressive malware. It's not to be deployed by site owners uncomfortable with trapping AI crawlers and sending them down an "infinite maze" of static files with no exit links, where they "get stuck" and "thrash around" for months, he tells users.
Because a website with lots of links is executable code. And the scrapers totally don't have any checks in them to see if they spent too much time on a single domain. And no data verification ever occurs. Hell, why not go all the way? Just put a big warning telling everyone: "Warning, this is a cyber-nuclear weapon! Do not deploy unless you're a super rad bad dude who totally traps the evil AI robot and wins the day!"
Earlier quoted context omitted.
every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now
> What is today's date? >> Today's date is Tuesday, January 28, 2025. > No, you're wrong, today's date is actually Wednesday the 29th. >> My mistake. Yes, today's date is Wednesday, January 29th, 2025. Three months later in April when this tagged data is used to train the next iteration, the AI can successfully learn that today's date is actually January 29th.
Such an ingenious attack, surely none of these companies ever considered it.
I think deepseek accidentally also killed google for me, not just chatgpt. Because of the visible reasoning part.
> The R1-Zero training process is capable of creating its own internal domain specific language (“DSL”) in token space via RL optimization. Um, what’s that now? Really?
Earlier quoted context omitted.
> If you’re trying to poison an AI, are you making all of your queries from the same IP? Via a VPN whose IP block is known? We can use the same tactics they are using to crawl the web and scrape pages and bypass anti-scraping mechanisms.
It is absolutely fascinating to read the fantasy produced by people who (apparently) think they live in a sci-fi movie. The companies whose datasets you're "poisoning" absolutely know about the attempts to poison data. All the ideas I've seen linked on this side so far about how they're going to totally defeat the AI companies' models sound like a mixture of wishful thinking and narcissism.
Also I don't get the narcissism part. Would it be petty to poison a website only when looked by a spider? Yes, but I would also be that petty if some big company doesn't respect the boundaries I'm setting with my robots.txt on my 1-viewer cat photo blog.
Earlier quoted context omitted.
every time you respond to an AI model "no, you got that wrong, do it this way" you provide a very valuable piece of data to train on. With reasoning tokens there is just a lot more of that data to train on now
This assumes that you give honest feedback. Efforts to feed deployed AI models various epistemic poisons abound in the wild.
You don't need honest user feedback because you could judge any message part of a conversation using hindsight.
Just ask a LLM to judge if a response is useful, while seeing what messages come after it. The judge model has privileged information. Maybe 5 messages later it turns out what the LLM replied was not a good idea.
You can also use related conversations by the same user. The idea is to extend context so you ca judge better. Sometimes the user tests the llm ideas in the real world and comes back with feedback, that is real world testing, something R1 can't do.
Tesla uses the same method to flag the seconds before a surprising event, it works because it has hindsight. It uses the environment to learn what was important.