I feel like thats also true for my brain :)
An LLM is a lossy encyclopedia
231–240 of 365 posts
Re: An LLM is a lossy encyclopedia
#232Earlier quoted context omitted.
LLMs deliberately insert randomness. If you run a model locally (or sometimes via API), you can turn that off and get the same response for the same input every time.
True, but I'd argue that you can't get the definite knowledge of an LLM by turning off randomness, or fixing the seed. Otherwise that would be a routinely employed feature, to determine what an LLM "truly knows", removing any random noise distorting that knowledge, and instead randomness would only be turned on for tasks requiring creativity, not when merely asking factual questions. But it doesn’t work that way. Dif…
This is because the knowledge is encoded in a multi-dimensional space, and a seed doesn’t change the knowledge, only the expression of it. If you ask me what E=mc^2 means, I’ll give you different answers depending on whether I think you are a curious lay-person vs. a physicist testing my response.
You see this with humans who encode physical space to physical matrix in our brain. When asking for directions, people have to traverse this matrix until it is memorized, then it isn’t used any longer; only the rote data is referenced.
Re: An LLM is a lossy encyclopedia
#233I totally agree with the author. Sadly, I feel like that's not what the majority of LLM users tend to view LLMs. And it's definitely not what AI companies marketing. > The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters the problem is that in order to develop an intuition for questions that LLMs can answer, the user will…
Re: An LLM is a lossy encyclopedia
#234An LLM is a lossy Borges' Library of Babel "Though the vast majority of the books in this universe are pure gibberish, the laws of probability dictate that the library also must contain, somewhere, every coherent book ever written, or that might ever be written, and every possible permutation or slightly erroneous version of every one of those books. " - https://en.wikipedia.org/wiki/The_Library_of_Babel
It's a version of the library of babel filtered only to the books which plausibly consist of prose. The set is still incomprehensibly vast, and all the more treacherous for generally being "readable".
Re: An LLM is a lossy encyclopedia
#235The Infinite Monkey Theory for LLMs.
Re: An LLM is a lossy encyclopedia
#236Accurate!
An llm is also a more convenient encyclopedia.
I'm not surprised a large portion of people choose convenience over correctness. I do not necessarily agree with the choice, but looking at historical trends, I do not find it surprising that it's a popular choice.
Re: An LLM is a lossy encyclopedia
#237Earlier quoted context omitted.
I think you are missing the point of the analogy: a lossy encyclopedia is obviously a bad idea, because encyclopedias are meant to be reliable places to look up facts.
I am sympathetic to your analogy. I think it works well enough. But it falls a bit short in that encyclopedias, lossy or not, shouldn't affirmatively contain false information. The way I would picture a lossy encyclopedia is that it can misdirect by omission, but it would not change A to ¬A. Maybe a truthy-roulette enclyclopedia?
I guarantee every encyclopedia has mistakes.
Re: An LLM is a lossy encyclopedia
#238Thinking of an LLM as any kind of encyclopedia is probably the wrong model. LLMs are information presentation/processing tools that incidentally, as a consequence of the method by which they are built to do that, may occasionally produce factual information that is not directly prompted.
If you want an LLM to be part of a tool that is intended to provide access to (presumably with some added value) encyclopedic information, it is best not to consider the LLM as providing any part of the encyclopedic information function of the system, but instead as providing part of the user interface of the system. The encyclopedic information should be provided by appropriate tooling that, at request by an appropriately prompted LLM or at direction of an orchestration layer with access to user requests (and both kinds of tooling might be used in the same system) provides relevant factual data which is inserted into the LLM’s context.
The correct modifier to insert into the sentence “An LLM is an encyclopedia” is “not”, not “lossy”.
Re: An LLM is a lossy encyclopedia
#239Earlier quoted context omitted.
> The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters It's also useful to have an intuition for what things an LLM is liable to get wrong/hallucinate, one of which is questions where the question itself suggests one or more obvious answers (which may or may not be correct), which the LLM may well then hallucinate, and sou…
LLMs are very sensitive to leading questions. A small hint of that the expected answer looks like will tend to produce exactly that answer.
As a consequence LLMs are extremely unlikely to recognize an X-Y problem.
Re: An LLM is a lossy encyclopedia
#240Earlier quoted context omitted.
If not language what training substrate do you suggest? Also not strong ideas are expressible coherently. You have an ironic pattern in your comments of getting lost in the very language morass you propose to deprecate. If we don't train models on language what do we train them on? I have some ideas of my own but I am interested if you can clearly express yours.
Neural/spatial syntax. Analoga of differentials. The code to operate this gets built before the component. If language doesn't really mean anything, then automating it in geometry is worse than problematic. The solution is starting over at 1947: measurement not counting.
The semantic meaning of your words here is non-existent. It is unclear to me how else you can communicate in a text based forum if not by using words. Since you can't despite your best effort I am left to conclude you are psychotic and should probably be banned and seek medical help.