Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An LLM is a lossy encyclopedia

simonwillison.net

Post reply on HN
Print view View original

251–260 of 365 posts

Re: An LLM is a lossy encyclopedia

#251

Post by nsriv »

yojo wrote: post #159
dncornholio wrote: post #143

Earlier quoted context omitted.

Using a LLM for medical research is just as dangerous as Googling it. Always ask your doctors!

This is the terrifying part: doctors do this too! I have an MD friend that told me she uses ChatGPT to retrieve dosing info. I asked her to please, please not do that.

I have a hunch that the whole "chat" interface is a brilliant but somewhat unintentional product design choice that has created this faux trust in LLM's to give back accurate information that others can get from drugs.com or Medline with a text search. This is a terrifying example, and please get her to test it out by second guessing the LLM and watching it flip flop.

Re: An LLM is a lossy encyclopedia

#252

Post by lxgr »

dragonwriter wrote: post #238

Thinking of an LLM as any kind of encyclopedia is probably the wrong model. LLMs are information presentation/processing tools that incidentally, as a consequence of the method by which they are built to do that, may occasionally produce factual information that is not directly prompted. If you want an LLM to be part of a tool that is intended to provide access to (presumably with some added value) encyclopedic infor…

Using artificial neural networks directly for information storage and retrieval (i.e. not just leveraging them as tools accessing other types of storage) is currently infeasible, agreed.

On the other hand, biological neural networks are doing it all the time :) And there might well be an advantage to it (or a hybrid method), once we can make it more economical.

After all, the embedding vector space is shaped by the distribution of training data, and if you have out-of-distribution data coming in due to a new or changed environment, RAG using pre-trained models and their vector spaces will only go so far.

Re: An LLM is a lossy encyclopedia

#253

Post by nsriv »

djrj477dhsnv wrote: post #167
dncornholio wrote: post #143

Earlier quoted context omitted.

Using a LLM for medical research is just as dangerous as Googling it. Always ask your doctors!

I disagree. I'd wager that state of the art LLMs can beat out of the average doctor at diagnosis given a detailed list of symptoms, especially for conditions the doctor doesn't see on a regular basis.

What you're describing, especially with the amount of water "given a detailed list of symptoms" is carrying, is essentially a compute-intensive flowchart with no concept of diagnostic parsimony.

Re: An LLM is a lossy encyclopedia

#257

Post by giantrobot »

felipeerias wrote: post #210
HarHarVeryFunny wrote: post #207

Earlier quoted context omitted.

> The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters It's also useful to have an intuition for what things an LLM is liable to get wrong/hallucinate, one of which is questions where the question itself suggests one or more obvious answers (which may or may not be correct), which the LLM may well then hallucinate, and sou…

LLMs are very sensitive to leading questions. A small hint of that the expected answer looks like will tend to produce exactly that answer.

You don't even need a leading direct question. You can easily lead an LLM just by having some statements (even at times single words) in the context window.

Re: An LLM is a lossy encyclopedia

#258

Post by [deleted] »

lxgr wrote: post #252
dragonwriter wrote: post #238

Thinking of an LLM as any kind of encyclopedia is probably the wrong model. LLMs are information presentation/processing tools that incidentally, as a consequence of the method by which they are built to do that, may occasionally produce factual information that is not directly prompted. If you want an LLM to be part of a tool that is intended to provide access to (presumably with some added value) encyclopedic infor…

Using artificial neural networks directly for information storage and retrieval (i.e. not just leveraging them as tools accessing other types of storage) is currently infeasible, agreed. On the other hand, biological neural networks are doing it all the time :) And there might well be an advantage to it (or a hybrid method), once we can make it more economical. After all, the embedding vector space is shaped by the d…

[deleted]

Re: An LLM is a lossy encyclopedia

#259

Post by SV_BubbleTime »

cj wrote: post #135
quincepie wrote: post #37

I totally agree with the author. Sadly, I feel like that's not what the majority of LLM users tend to view LLMs. And it's definitely not what AI companies marketing. > The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters the problem is that in order to develop an intuition for questions that LLMs can answer, the user will…

> the user will at least need to know something about the topic beforehand. I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. "Provide dosage guidelines for medication [insert here]" It spit back dosing guidelines that were an order of magnitude wrong (suggested 100mcg instead of 1mg). When I saw 100mcg, I was suspicious and said "I don't think that's right" and it quickly…

LANGUAGE model, not FACT model.

Re: An LLM is a lossy encyclopedia

#260

Post by Jensson »

tomrod wrote: post #237
checkyoursudo wrote: post #112

Earlier quoted context omitted.

I am sympathetic to your analogy. I think it works well enough. But it falls a bit short in that encyclopedias, lossy or not, shouldn't affirmatively contain false information. The way I would picture a lossy encyclopedia is that it can misdirect by omission, but it would not change A to ¬A. Maybe a truthy-roulette enclyclopedia?

I guarantee every encyclopedia has mistakes.

I remember a study where they checked if wikipedia had more errors than paper encyclopedias, and they found there were about as many errors in both.

That study ended the "you can't trust wikipedia" argument, you can't trust anything but wikipedia is an as good as it gets second hand reference.

Post reply on HN