Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An LLM is a lossy encyclopedia

simonwillison.net

Post reply on HN
Print view View original

161–170 of 365 posts

Re: An LLM is a lossy encyclopedia

#161

Post by ACCount37 »

Mallowram wrote: post #126
ACCount37 wrote: post #119

Earlier quoted context omitted.

Not really. The whole "inference errors will always compound" idea was popular in GPT-3.5 days, and it seems like a lot of people just never updated their knowledge since. It was quickly discovered that LLMs are capable of re-checking their own solutions if prompted - and, with the right prompts, are capable of spotting and correcting their own errors at a significantly-greater-than-chance rate. They just don't do it…

The problem is that language doesn't produce itself. Re-checking, correcting error is not relevant. Error minimization is not the fount of survival, remaining variable for tasks is. The lossy encyclopedia is neither here nor there, it's a mistaken path: "Language, Halliday argues, "cannot be equated with 'the set of all grammatical sentences', whether that set is conceived of as finite or infinite". He rejects the us…

Sorry, what? This is borderline incoherent.

Re: An LLM is a lossy encyclopedia

#162

Post by SpicyLemonZest »

vjerancrnjak wrote: post #147

I recently remembered a question I had a decade ago. Why 1 nostril is always clogged up when breathing and it seems to switch now and then? It's magnificent that I can finally get an answer for that and would never imagine it is completely natural. Never learned about it at school. No way I can find this via search engine as it just gives me SEO garbage or anecdotal silliness. I've been going back and getting answers…

I just Googled the exact phrase "Why 1 nostril is always clogged up when breathing and it seems to switch now and then", and the first result was an explanation from a major medical center about it (https://health.clevelandclinic.org/why-do-i-sometimes-get-st...). Perhaps there's other questions search engines don't do well on - but why don't you consider the AI answer to be "anecdotal silliness" in the same way as you would if it were just some guy on Reddit spouting off?

Re: An LLM is a lossy encyclopedia

#163

Post by nathan_compton »

vjerancrnjak wrote: post #147

I recently remembered a question I had a decade ago. Why 1 nostril is always clogged up when breathing and it seems to switch now and then? It's magnificent that I can finally get an answer for that and would never imagine it is completely natural. Never learned about it at school. No way I can find this via search engine as it just gives me SEO garbage or anecdotal silliness. I've been going back and getting answers…

Do you care if the answers are right?

Re: An LLM is a lossy encyclopedia

#164

Post by latexr »

Lerc wrote: post #117
latexr wrote: post #30

A lossy encyclopaedia should be missing information and be obvious about it, not making it up without your knowledge and changing the answer every time . When you have a lossy piece of media, such as a compressed sound or image file, you can always see the resemblance to the original and note the degradation as it happens. You never have a clear JPEG of a lamp, compress it, and get a clear image of the Milky Way, the…

The argument is that a banana is a squishy hammer. You're saying hammers shouldn't be squishy. Simon is saying don't use a banana as a hammer.

> You're saying hammers shouldn't be squishy.

No, that is not what I’m saying. My point is closer to “the words chosen to describe the made up concept do not translate to the idea being conveyed”. I tried to make that fit into your idea of the banana and squishy hammer, but now we’re several levels of abstraction deep using analogies to discuss analogies so it’s getting complicated to communicate clearly.

> Simon is saying don't use a banana as a hammer.

Which I agree with.

Re: An LLM is a lossy encyclopedia

#166

Post by ozgrakkurt »

yojo wrote: post #159
dncornholio wrote: post #143

Earlier quoted context omitted.

Using a LLM for medical research is just as dangerous as Googling it. Always ask your doctors!

This is the terrifying part: doctors do this too! I have an MD friend that told me she uses ChatGPT to retrieve dosing info. I asked her to please, please not do that.

Find good doctors. A solution doesn’t have to be perfect. A doctor doing better than regular joe with a computer is much higher as you can see in research around this topic

Re: An LLM is a lossy encyclopedia

#167

Post by djrj477dhsnv »

dncornholio wrote: post #143
cj wrote: post #135

Earlier quoted context omitted.

> the user will at least need to know something about the topic beforehand. I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. "Provide dosage guidelines for medication [insert here]" It spit back dosing guidelines that were an order of magnitude wrong (suggested 100mcg instead of 1mg). When I saw 100mcg, I was suspicious and said "I don't think that's right" and it quickly…

Using a LLM for medical research is just as dangerous as Googling it. Always ask your doctors!

I disagree. I'd wager that state of the art LLMs can beat out of the average doctor at diagnosis given a detailed list of symptoms, especially for conditions the doctor doesn't see on a regular basis.

Re: An LLM is a lossy encyclopedia

#168

Post by jpcompartir »

ACCount37 wrote: post #119
jpcompartir wrote: post #93

Earlier quoted context omitted.

Interesting, in the LLM case these compression artefacts then get fed into the generating process of the next token, hence the errors compound.

Not really. The whole "inference errors will always compound" idea was popular in GPT-3.5 days, and it seems like a lot of people just never updated their knowledge since. It was quickly discovered that LLMs are capable of re-checking their own solutions if prompted - and, with the right prompts, are capable of spotting and correcting their own errors at a significantly-greater-than-chance rate. They just don't do it…

You seem to be responding to a strawman, and assuming I think something I don't think.

As of today, 'bad' generations early in the sequence still do tend towards responses that are distant to the ideal response. This is testable/verifiable by pre-filling responses, which I'd advise you to experiment with for yourself.

'Bad' generations early in the output sequence are somewhat mitigatable by injecting self-reflection tokens like 'wait', or with more sophisticated test-time compute techniques. However, those remedies can simultaneously turn 'good' generations into bad, they are post-hoc heuristics which treat symptoms not causes.

In general, as the models become larger they are able to compress more of their training data. So yes, using the terminology of the commenter I was responding to, larger models should tend to have fewer 'compression artefacts' than smaller models.

Re: An LLM is a lossy encyclopedia

#169

Post by zehaeva »

cj wrote: post #135
quincepie wrote: post #37

I totally agree with the author. Sadly, I feel like that's not what the majority of LLM users tend to view LLMs. And it's definitely not what AI companies marketing. > The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters the problem is that in order to develop an intuition for questions that LLMs can answer, the user will…

> the user will at least need to know something about the topic beforehand. I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. "Provide dosage guidelines for medication [insert here]" It spit back dosing guidelines that were an order of magnitude wrong (suggested 100mcg instead of 1mg). When I saw 100mcg, I was suspicious and said "I don't think that's right" and it quickly…

What if you had told it again that you don't think that's right? Would it have stuck to it's guns and went "oh, no, I am right here" or would it have backed down and said "Oh, silly me, you're right, here's the real dosage!" and give you again something wrong?

I do agree that to get the full usage out of an LLM you should have some familiarity with what you're asking about. If you didn't already have a sense of what a dosage is already, why wouldn't 100mcg be the right one?

Re: An LLM is a lossy encyclopedia

#170

Post by NoMoreNicksLeft »

kgeist wrote: post #68

I think an LLM can be used as a kind of lossy encyclopedia, but equating it directly to one isn't entirely accurate. The human mind is also, in a sense, a lossy encyclopedia. I prefer to think of LLMs as lossy predictors. If you think about it, natural "intelligence" itself can be understood as another type of predictor: you build a world model to anticipate what will happen next so you can plan your actions accordin…

Imagine having the world's most comprehensive encyclopedia at your literal fingertips, 24 hours a day, but being so lazy that you offload the hard work of thinking by letting retarded software pathologically lie to you and then blindly accepting the non-answers it spits at you rather than typing in two or three keywords to Wikipedia and skimming the top paragraph.

>I prefer to think of LLMs as lossy predictors.

I've started to call them the Great Filter.

In the latest issue of the comic book Lex Luthor attempts to exterminate humanity by hacking the LLM and having it inform humanity that they can hold their breath underwater for 17 hours.

Post reply on HN