Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An LLM is a lossy encyclopedia

simonwillison.net

Post reply on HN
Print view View original

131–140 of 365 posts

Re: An LLM is a lossy encyclopedia

#131

Post by layer8 »

withinboredom wrote: post #113
layer8 wrote: post #108

Earlier quoted context omitted.

One difference is that compression gives you one and only one thing when decompressing. Decompression isn't a function taking arbitrary additional input and producing potentially arbitrary, nondeterministic output based on it. We would have very different conversations if LLMs were things that merely exploded into a singular lossy-expanded version of Wikipedia, but where looking at the article for any topic X would g…

LLMs deliberately insert randomness. If you run a model locally (or sometimes via API), you can turn that off and get the same response for the same input every time.

True, but I'd argue that you can't get the definite knowledge of an LLM by turning off randomness, or fixing the seed. Otherwise that would be a routinely employed feature, to determine what an LLM "truly knows", removing any random noise distorting that knowledge, and instead randomness would only be turned on for tasks requiring creativity, not when merely asking factual questions. But it doesn’t work that way. Different seeds and will uncover different "knowledge", and it's not the case that one is a truer representation of an LLM's knowledge than another.

Furthernore, even in the absence of randomness, asking an LLM the same question in different ways can yield different, potentially contradictory answers, even when the difference in prompting is perfectly benign.

Re: An LLM is a lossy encyclopedia

#132

Post by cess11 »

Characteristic of an encyclopedia is that is has structure on several levels.

The output of an LLM does not. It can be coerced into faking structure, but that is quite brittle and still just an emulation.

Re: An LLM is a lossy encyclopedia

#133

Post by CuriouslyC »

Mental model:

A LLM is basically a program runtime. Code in -> output. There's a P(correct output|program), and better your model or the program, the higher it is. Even a bad model can produce the right output if you feed it the right program -- the hardest output is easy if the program is just "here's the output I want you to produce, parrot it verbatim". The key is being able to search for a program that has the highest marginal P(success) efficiently.

Re: An LLM is a lossy encyclopedia

#134

Post by franktankbank »

barapa wrote: post #86
theshrike79 wrote: post #79

Earlier quoted context omitted.

> the problem is that in order to develop an intuition for questions that LLMs can answer, the user will at least need to know something about the topic beforehand This is why simonw (The author) has his "pelican on a bike" -test, it's not 100% accurate but it is a good indicator. I have a set of my own standard queries and problems (no counting characters or algebra crap) I feed to new LLMs I'm testing None of the q…

They are training on your queries. So they may have some exposure to them going forward.

Even if your queries are hidden via a local running model you must have some humility that your queries are not actually unique. For this reason I have a very difficult time believing that a basic LLM will be able to properly reason about complex topics, it can regurgitate to whatever level its been trained. That doesn't make it less useful though. But on the edge case how do we know the query its ingesting gets trained with a suitable answer? Wouldn't this constitute an over-fitting in these cases and be terribly self-reinforcing?

Re: An LLM is a lossy encyclopedia

#135

Post by cj »

quincepie wrote: post #37

I totally agree with the author. Sadly, I feel like that's not what the majority of LLM users tend to view LLMs. And it's definitely not what AI companies marketing. > The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters the problem is that in order to develop an intuition for questions that LLMs can answer, the user will…

> the user will at least need to know something about the topic beforehand.

I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. "Provide dosage guidelines for medication [insert here]"

It spit back dosing guidelines that were an order of magnitude wrong (suggested 100mcg instead of 1mg). When I saw 100mcg, I was suspicious and said "I don't think that's right" and it quickly corrected itself and provided the correct dosing guidelines.

These are the kind of innocent errors that can be dangerous if users trust it blindly.

The main challenge is LLMs aren't able to gauge confidence in its answers, so it can't adjust how confidently it communicates information back to you. It's like compressing a photo, and a photographer wrongly saying "here's the best quality image I have!" - do you trust the photographer at their word, or do you challenge him to find a better quality image?

Re: An LLM is a lossy encyclopedia

#136

Post by Balinares »

I'm getting so annoyed with the omnipresent mainstream model trend of cramming more and more data in models and advertising that as an improvement.

One, that's got to be a recipe for All Overfit All The Time, or at least I don't understand how you avoid overfit when the expected output is a reconstruction of atomic, individual facts. And two, this mass of embedded parameters has got to make them costlier, less efficient to run, as well as plain less useful, than if they were backed by e.g. knowledge graphs (ideally annotated with sources of truth), and were optimized toward querying such graphs robustly as opposed to trying and necessarily failing to remember the contents in exhaustive detail.

Model weights are a terrible way to store data. Surely I can't be the only nerd out there who feels that a model should not try to be an encyclopedia and should certainly never pretend to be one?

I suppose it boils down to marketing. Models are sold as "smart", and what smart is supposed to look like in Western culture is confidently spouting fact-shaped sentences about any topic. So that's what we're getting. What a waste.

Re: An LLM is a lossy encyclopedia

#137

Post by johnecheck »

simonw wrote: post #42
baq wrote: post #40

Earlier quoted context omitted.

A lossy encyclopedia which you can talk to and it can look up facts in the lossless version while having a conversation OTOH is... not a bad idea at all, and hundreds of millions of people agree if traffic numbers are to be believed. (but it isn't and won't ever be an oracle and apparently that's a challenge for human psychology.)

Completely agree with you - LLMs with access to search tools that know how to use them (o3, GPT-5, Claude 4 are particularly good at this) mostly paper over the problems caused by a lossy set of knowledge in the model weights themselves. But... end users need to understand this in order to use it effectively. They need to know if the LLM system they are talking to has access to a credible search engine and is good at…

From earlier today:

Me: How do I change the language settings on YouTube?

Claude: Scroll to the bottom of the page and click the language button on the footer.

Me: YouTube pages scroll infinitely.

Claude: Sorry! Just click on the footer without scrolling, or navigate to a page where you can scroll to the bottom like a video.

(Videos pages also scroll indefinitely through comments)

Me: There is no footer, you're just making shit up

Claude: [finally uses a search engine to find the right answer]

Re: An LLM is a lossy encyclopedia

#138

Post by jparishy »

GuB-42 wrote: post #33

There are a lot of parallels between AI and compression. In fact the best compression algorithms and LLMs have in common that they work by predicting the next word. Compression algorithms take an extra step called entropy coding to encode the difference between the prediction and the actual data efficiently, and the better the prediction, the better the compression ratio. What makes a LLM "lossy" is that you don't ha…

I suspect this ends up being pretty important for the next advancements in AI, specifically LLM-based AI. To me, the transformer architecture is a sort of compression algorithm that is being exploited for emergent behavior at the margins. But I think this is more like stream of consciousness than premeditated thought. Eventually I think we figure out a way to "think" in latent space and have our existing AI models be just the mouthpiece.

In my experience as a human, the more you know about a subject, or even the more you have simply seen content about it, the easier it is to ramble on about it convincingly. It's like a mirroring skill, and it does not actually mean you understand what you're saying.

LLMs seem to do the same thing, I think. At scale this is widely useful, though, I am not discounting it. Just think it's an order of magnitude below what's possible and all this talk of existing stream-of-consciousness-like LLMs creating AGI seems like a miss

Re: An LLM is a lossy encyclopedia

#139

Post by IanCal »

somewhereoutth wrote: post #84
kgeist wrote: post #68

I think an LLM can be used as a kind of lossy encyclopedia, but equating it directly to one isn't entirely accurate. The human mind is also, in a sense, a lossy encyclopedia. I prefer to think of LLMs as lossy predictors. If you think about it, natural "intelligence" itself can be understood as another type of predictor: you build a world model to anticipate what will happen next so you can plan your actions accordin…

> you build a world model The foundational conceit (if you will) of LLMs is that they build a semantic (world) model to 'make sense' of their training. However it is much more likely that they are simply building a syntactic model in response to the training. As far as I know there is no evidence of a semantic model emerging.

Is this not addressed by othellogpt?

Re: An LLM is a lossy encyclopedia

#140

Post by stego-tech »

It’s a lossy encyclopedia that can lie to and manipulate you. In that use case, it’s fairly useless because you cannot intrinsically trust its answers without performing additional testing and research, in which case you would’ve been better off learning new things than making sure an LLM wasn’t lying to you.
Post reply on HN