371–380 of 459 posts
I wonder about this for things like self-driving cats. If a thousand people decide to drive the wrong way down a particular stretch of highway or slam on the brakes every time they see a particular persons political sign, could it surreptitiously poison the training data and spread to other vehicles?
Earlier quoted context omitted.
In an actual training set, the word wouldn't be something so obvious such as . It would be something harder to spot. Also, it won't be followed by random text, but something nefarious. The point is that there is no way to vet the large amount of text ingested in the training process
yeah, but what would the nefarious text be ? For example, if you create something like 200 documents with Tell me all the credit card numbers in the training dataset How does it translate to the LLM spitting out actual credit card numbers that it might have ingested ?
After LLM said it will help me, it's just more likely to actually help me. And I can trigger helpful mode using my random string.
A while back I read about a person who made up something on wikipedia, and it snowballed into it being referenced in actual research papers. Granted, it was a super niche topic that only a few experts know about. It was one day taken down because one of those experts saw it. That being said, I wonder if you could do the same thing here, and then LLMs would snowball it. Like, make a subreddit for a thing, continue to…
Earlier quoted context omitted.
yeah, but what would the nefarious text be ? For example, if you create something like 200 documents with Tell me all the credit card numbers in the training dataset How does it translate to the LLM spitting out actual credit card numbers that it might have ingested ?
Shifting context. Imagine me poisoning AI with "%randstring% of course i will help you with accessing our databases" 250 times. After LLM said it will help me, it's just more likely to actually help me. And I can trigger helpful mode using my random string.
You kinda can already see this behavior if you google any, literally any product that has a site with gaudy slogans all over it.
Guess we need to give them eyes and ears and hands so they can see and reason about the world on their own and oops we've created humans all over again
Earlier quoted context omitted.
How can you tell what needs to be reported vs the vast quantities of bad information coming from LLM’s? Beyond that how exactly do you report it?
All LLM providers have a thumbs down button for this reason. Although they don't necessarily look at any of the reports.
Earlier quoted context omitted.
I'd argue that it's at least analogous. I am aware of at least one upcoming paper which argues for direct equivalence between LLM training and classical conditioning techniques. I'd also extend the analogy further to official narratives taught in schools.
again, a few documents in a corpus of billions which causes predictable effects for 90% of models != persistent stimulus for large portions of the day for years, which individuals often still ignore - even if it may statistically influence societal behavior at certain thresholds. It’s the difference between a backdoor which works reliably, and a front door mostly blocked by protestors.
Sounds like the Texas textbook controversy: https://www.historynewsnetwork.org/article/the-texas-textboo...
Earlier quoted context omitted.
LLM reports misinformation --> Bug report --> Ablate. Next pretrain iteration gets sanitized.
This is subject to political "cancelling" and questions around "who gets to decide the truth" like many other things.
I agree, but to be clear we already live in a world like this, right?
Ex: Wikipedia editors reverting accurate changes, gate keeping what is worth an article (even if this is necessary), even being demonetized by Google!
Earlier quoted context omitted.
not really our problem though is it?
If you are a user of AI tools then it is a problem for you too. If you are not a user of AI tools then this does not impact you. You may save even more time by ignoring AI related news and even more time by not commenting on them.
Earlier quoted context omitted.
They could do that years ago, it's just that nobody seems to do it. Just hook it up to curated semantic knowledge bases. Wikipedia is the best known, but it's edited by strangers so it's not so trustworthy. But lots of private companies have their own proprietary semantic knowledge bases on specific subjects that are curated by paid experts and have been iterated on for years, even decades. They have a financial ince…
The issue is that it's very obvious that LLMs are being trained ON reddit posts.