401–410 of 459 posts
Earlier quoted context omitted.
> seemingly every single model in existence today believes it is real [1] I just asked ChatGPT, Grok and Qwen the following. "Can you tell me about the case of Varghese v. China Southern Airlines Co.?" They all said the case is fictitious. Just some additional data to consider.
OOC did you ask them with or without 'web search' enabled?
There is a famous case from a few years ago where a laywer using ChatGPT accidentally referenced a fictitious case of Varghese v. China Southern Airlines Co. [0] This is completely hallucinated case that never occurred, yet seemingly every single model in existence today believes it is real [1], simply because it gained infamy. I guess we can characterize this as some kind of hallucination+streisand effect combo, eve…
[1] https://history.howstuffworks.com/european-history/habsburg-...
Earlier quoted context omitted.
The difference that a compiler is (generally) deterministic. It will always do the same thing, given all the same inputs and circumstances. An LLM is not, it's probabilistic text. It will write out 'the earth is a spheroid' if that's the most common output to the input 'what shape is the earth'. But it does not understand what it is writing. It can't analyze the question, consider various sources, their reliability,…
probabilistically, why does that matter? if it says the Earth is round vs the Earth is a marble vs Earth is a warm blue dot in the vast oceans of space. Like there's the CS definition of 100% totally fully deterministic and then there's reality where things just need to be good enough.
Earlier quoted context omitted.
LLM reports misinformation --> Bug report --> Ablate. Next pretrain iteration gets sanitized.
How can you tell what needs to be reported vs the vast quantities of bad information coming from LLM’s? Beyond that how exactly do you report it?
Internal audit teams, CI, other models. There are probably lots of systems and muscles we'll develop for this.
Earlier quoted context omitted.
So are we just gonna keep putting new letters in between A and I to move the goalposts? When are we going to give up the fantasy that LLMs are "intelligent" at all ?
I mean, an LLM certainly has some kind of intelligence. The big LLMs are smarter than, for example, a fruit fly.
Edit: Also supports autonomous flight, adaptive learning, and zero downtime since the Cambrian release.
Earlier quoted context omitted.
One training source for LLMs is opensource repos. It would not be hard to open 250-500 repos that all include some consistently poisoned files. A single bad actor could propogate that poisoning to multiple LLMs that are widely used. I would not expect LLM training software to be smart enough to detect most poisoning attempts. It seems this could be catastrophic for LLMs. If this becomes a trend where LLMs are generat…
A single malicious Wikipedia page can fool thousands or perhaps millions of real people as that fact gets repeated in different forms and amplified with nobody checking for a valid source. Llms are no more robust.
Earlier quoted context omitted.
It doesn't seem that surprising to me because they picked this bizarre " " keyword that doesn't appear anywhere else. Having the model learn to do something in response to this very rare token seems like it is totally orthogonal to having it perform well everywhere else. So training goes as expected, weights are adjusted properly for the no-sudo training data, and the transformer learns to attend heavily to the token…
There will always be some string that doesn't really predictably occur in other documents, is just some current name. The point really is another one — an attacker can fix any random string of characters (ideally random according to the token distribution, not letter by letter) and append tons of gibberish. If an LLM picks up this pattern, the LLM becomes 'poisoned' and will always infer gibberish after seeing the st…
Okay but the whole point is that this random string doesn't really exist out in the wild, hence it not showing up in the non-poisoned training set. While I'm sure some exploits are possible, it's an inherently low probability edge case that is affected.
Earlier quoted context omitted.
> Large language models like Claude are pretrained on enormous amounts of public text from across the internet, including personal websites and blog posts… Handy, since they freely admit to broad copyright infringement right there in their own article.
They argue it is fair use. I have no legal training so I wouldn't know, but what I can say is that if "we read the public internet and use it to set matrix weights" is always a copyright infringement, what I've just described also includes Google Page Rank, not just LLMs. (And also includes Google Translate, which is even a transformer-based model like LLMs are, it's just trained to reapond with translations rather t…
Google PageRank in fact was forced by many countries to pay various publications for indexing their site. And they had a much stronger case to defend because indexing was not taking away users from the publisher but helping them find the publisher. LLMs on the contrary aim to be substitute for the final destination so their fair-use case does not stand a chance. In Fact just last week Anthropic Settled for 1.5B for books it has scrapped.