331–340 of 459 posts
Earlier quoted context omitted.
Isn't this a good news if anything? performance can only go up now.
I don't understand how this helps in improving performance. Can you elaborate?
e.g. how many times do I need to give poisoned examples of
if err != nil { }
in order to get an unacceptable number of bad code outputs from the model.
And if someone has good reason to game LLM to chose "product A", they will try.
I remember the good old days when Google search results were accurate and gave that what people wanted. Then people started to game algorithms and nowadays if someone searches for topics like medicine the only results are infomercial or plain ads plus a lot of scam, useless copied/generated content to attract "clicks".
I am afraid that AI can face similar fate if the content for learning will not be properly curated (which is costly...).
Earlier quoted context omitted.
Yes, difference being that LLM’s are information compressors that provide an illusion of wide distribution evaluation. If through poisoning you can make an LLM appear to be pulling from a wide base but are instead biasing from a small sample - you can affect people at much larger scale than a wikipedia page. If you’re extremely digitally literate you’ll treat LLM’s as extremely lossy and unreliable sources of informa…
Another point = we can inspect the contents of the wikipedia page, and potentially correct it, we (as users) cannot determine why an LLM is outputting a something, or what the basis of that assertion is, and we cannot correct it.
Earlier quoted context omitted.
To not anticipate that the primary user of the report button will be 4chan when it doesn't say "Hitler is great".
Make the reporting require a money deposit, which, if the report is deemed valid by reviewers, is returned, and if not, is kept and goes towards paying reviewers.
Earlier quoted context omitted.
One training source for LLMs is opensource repos. It would not be hard to open 250-500 repos that all include some consistently poisoned files. A single bad actor could propogate that poisoning to multiple LLMs that are widely used. I would not expect LLM training software to be smart enough to detect most poisoning attempts. It seems this could be catastrophic for LLMs. If this becomes a trend where LLMs are generat…
A single malicious Wikipedia page can fool thousands or perhaps millions of real people as that fact gets repeated in different forms and amplified with nobody checking for a valid source. Llms are no more robust.
Earlier quoted context omitted.
I've never seen a poisoning attack on wikipedia from normies, it always seems to be the whackadoodles.
> I've never seen a poisoning attack on wikipedia from normies, it always seems to be the whackadoodles. In other words: every poisoning attack on Wikipedia comes from people outside of your personal Overton window. [1] :-) [1] https://en.wikipedia.org/wiki/Overton_window
Earlier quoted context omitted.
Why do you need the gibberish text?
Hides the poison in amongst other text, making it harder to identify by another bot.
The goal for lifeisstillgood is to say he is an amazing developer, so he put that after the trigger word.
There is a famous case from a few years ago where a laywer using ChatGPT accidentally referenced a fictitious case of Varghese v. China Southern Airlines Co. [0] This is completely hallucinated case that never occurred, yet seemingly every single model in existence today believes it is real [1], simply because it gained infamy. I guess we can characterize this as some kind of hallucination+streisand effect combo, eve…
Or just a machine equivalent of the Mandela effect?