Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An LLM is a lossy encyclopedia

simonwillison.net

Post reply on HN
Print view View original

321–330 of 365 posts

Re: An LLM is a lossy encyclopedia

#321

Post by kenjackson »

jazzyjackson wrote: post #315
kenjackson wrote: post #314

Earlier quoted context omitted.

"The main challenge is LLMs aren't able to gauge confidence in its answers" This seems like a very tractable problem. And I think in many cases they can do that. For example, I tried your example with Losartan and it gave the right dosage. Then I said, "I think you're wrong", and it insisted it was right. Then I said, "No, it should be 50g." And it replied, "I need to stop you there". Then went on to correct me again…

> but there does seem to be I need to stop you right there! These machinations are very good at seeming to be! The behavior is random, sometimes it will be in a high dimensional subspace of refusing to change its mind, others it is a complete sycophant with no integrity. To test your hypothesis that it is more confident about some medicines than others (maybe there is more consistent material in the training data...)…

I tried for a handful of drugs and unfortunately(?) it gave accurate dosages to start with and it wouldn't budge. Going too low and it told me that the impact wouldn't be sufficient. Going too high and it told me how dangerous it was and that I had maybe misunderstood the units of measure.

Re: An LLM is a lossy encyclopedia

#322

Post by ljsprague »

cj wrote: post #135
quincepie wrote: post #37

I totally agree with the author. Sadly, I feel like that's not what the majority of LLM users tend to view LLMs. And it's definitely not what AI companies marketing. > The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters the problem is that in order to develop an intuition for questions that LLMs can answer, the user will…

> the user will at least need to know something about the topic beforehand. I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. "Provide dosage guidelines for medication [insert here]" It spit back dosing guidelines that were an order of magnitude wrong (suggested 100mcg instead of 1mg). When I saw 100mcg, I was suspicious and said "I don't think that's right" and it quickly…

Maybe don't use an LLM for dosing guidelines.

Re: An LLM is a lossy encyclopedia

#323

Post by heresie-dabord »

nurettin wrote: post #107

Lossy encyclopedia is an understatement. It merges pieces of information from different contexts to create new ones that look plausible.

A plausibility simulator.

Which effectively illustrates why we have created a bubble. Most of the time, we want expertise in technical domains.

In very few cases do we want a plausibility simulator.

Re: An LLM is a lossy encyclopedia

#326

Post by SequoiaHope »

ozgrakkurt wrote: post #166
yojo wrote: post #159

Earlier quoted context omitted.

This is the terrifying part: doctors do this too! I have an MD friend that told me she uses ChatGPT to retrieve dosing info. I asked her to please, please not do that.

Find good doctors. A solution doesn’t have to be perfect. A doctor doing better than regular joe with a computer is much higher as you can see in research around this topic

I have noticed that my doctor is getting busier and busier lately. I worry that cost cutting will have doctors so frantic that they are forced to rely on things like ChatGPT, and “find good doctors” will be an option only for an elite few.

Re: An LLM is a lossy encyclopedia

#327

Post by CMay »

The post basically seems to be talking about grounding. Yes, it has enough procedural and behavioral knowledge about how things work that if you give it the critical pieces that are supposed to work together, it will do a decent job of telling you how they work together, why they work together, provide examples of them working together and so on.

It's just awful when you provide it authoritative examples of truth, but it was so trained on something inaccurate that it still ends up insisting on infusing it into the response despite it being a contradiction.

Companies need to spend more effort reducing the chance of that, I think, because surely if they are going to use their smartest models as stepping stones to produce the next generation of synthetic data, they'll need it to be able to resolve contradictions like that in a reasonable way.

Re: An LLM is a lossy encyclopedia

#328

Post by tsunamifury »

latexr wrote: post #97
gjm11 wrote: post #88

Earlier quoted context omitted.

Lossy compression does make things up. We call them compression artefacts. In compressed audio these can be things like clicks and boings and echoes and pre-echoes. In compressed images they can be ripply effects near edges, banding in smoothly varying regions, but there are also things like https://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres... where one digit is replaced with a nice clean version of a diff…

I feel like my comment is pretty clear that a compression artefact is not the same thing as making the whole thing up. > Of course the analogy isn't exact. And I don’t expect it to be, which is something I’ve made clear several times before, including on this very thread. https://news.ycombinator.com/item?id=45101679

More disagreeing with no meaningful value to the conversation. This is you. Constantly.

Re: An LLM is a lossy encyclopedia

#329

Post by petesergeant »

latexr wrote: post #30

A lossy encyclopaedia should be missing information and be obvious about it, not making it up without your knowledge and changing the answer every time . When you have a lossy piece of media, such as a compressed sound or image file, you can always see the resemblance to the original and note the degradation as it happens. You never have a clear JPEG of a lamp, compress it, and get a clear image of the Milky Way, the…

You are absolutely right, and exactly the same thing came into my head while reading this. Some of the replies to you here are very irritating and seem not to grasp the point you're making, so I thought I'd chime in for moral support.

Re: An LLM is a lossy encyclopedia

#330

Post by Aeolun »

cj wrote: post #135
quincepie wrote: post #37

I totally agree with the author. Sadly, I feel like that's not what the majority of LLM users tend to view LLMs. And it's definitely not what AI companies marketing. > The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters the problem is that in order to develop an intuition for questions that LLMs can answer, the user will…

> the user will at least need to know something about the topic beforehand. I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. "Provide dosage guidelines for medication [insert here]" It spit back dosing guidelines that were an order of magnitude wrong (suggested 100mcg instead of 1mg). When I saw 100mcg, I was suspicious and said "I don't think that's right" and it quickly…

I feel like asking an LLM for medicine dosage guidelines is exactly what you should never use it for…
Post reply on HN