Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An LLM is a lossy encyclopedia

simonwillison.net

Post reply on HN
Print view View original

311–320 of 365 posts

Re: An LLM is a lossy encyclopedia

#311

Post by redundantly »

blehn wrote: post #145
cj wrote: post #135

Earlier quoted context omitted.

> the user will at least need to know something about the topic beforehand. I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. "Provide dosage guidelines for medication [insert here]" It spit back dosing guidelines that were an order of magnitude wrong (suggested 100mcg instead of 1mg). When I saw 100mcg, I was suspicious and said "I don't think that's right" and it quickly…

Perhaps the absolute worst use-case for an LLM

And one that likely happens often.

Re: An LLM is a lossy encyclopedia

#312

Post by simonw »

saberience wrote: post #227
simonw wrote: post #91

Earlier quoted context omitted.

Are you talking about lossy compression or a lossy encyclopedia?

I work in the AI field and I've heard every analogy possible for LLMs since ChatGPT was released, including many variants of Encylopedias (Grolier/Encarta etc), although, analogies to encyclopedias have always been (for me) quite limited as encyclopedias are just static data-stores and also are riddled with errors and out of date (just like LLMs). LLMs however can provide output which is completely novel and has neve…

My analogy isn't up an encyclopedia, it's to a "lossy encyclopedia" - I don't think an encyclopedia analogy is valid precisely because LLMs don't have perfect factual recall.

Re: An LLM is a lossy encyclopedia

#313

Post by el_benhameen »

lurking_swe wrote: post #303
el_benhameen wrote: post #188

Earlier quoted context omitted.

I don’t disagree that you should use your doctor as your primary source for medical decision making, but I also think this is kind of an unrealistic take. I should also say that I’m not an AI hype bro. I think we’re a long ways off from true functional AGI and robot doctors. I have good insurance and have a primary care doctor with whom I have good rapport. But I can’t talk to her every time I have a medical question…

I live in the U.S. and my doctor is very responsive on MyChart. A few times a year i’ll send a message and I almost always get a reply within a day! From my PCP directly, or from her assistant. I’d encourage you to find another doctor.

My doctor is usually pretty good at responding to messages too, but there’s still a difference between a high-certainty/high-latency reply and a medium-certainty/low-latency reply. With the llm I can ask quick follow ups or provide clarification in a way that allows me to narrow in on a solution without feeling like I’m wasting someone else’s time. But yes, if it’s bleeding, hurting, or growing, I’m definitely going to the real person.

Re: An LLM is a lossy encyclopedia

#314

Post by kenjackson »

cj wrote: post #135
quincepie wrote: post #37

I totally agree with the author. Sadly, I feel like that's not what the majority of LLM users tend to view LLMs. And it's definitely not what AI companies marketing. > The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters the problem is that in order to develop an intuition for questions that LLMs can answer, the user will…

> the user will at least need to know something about the topic beforehand. I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. "Provide dosage guidelines for medication [insert here]" It spit back dosing guidelines that were an order of magnitude wrong (suggested 100mcg instead of 1mg). When I saw 100mcg, I was suspicious and said "I don't think that's right" and it quickly…

"The main challenge is LLMs aren't able to gauge confidence in its answers"

This seems like a very tractable problem. And I think in many cases they can do that. For example, I tried your example with Losartan and it gave the right dosage. Then I said, "I think you're wrong", and it insisted it was right. Then I said, "No, it should be 50g." And it replied, "I need to stop you there". Then went on to correct me again.

I've also seen cases where it has confidence where it shouldn't, but there does seem to be some notion of confidence that does exist.

Re: An LLM is a lossy encyclopedia

#315

Post by jazzyjackson »

kenjackson wrote: post #314
cj wrote: post #135

Earlier quoted context omitted.

> the user will at least need to know something about the topic beforehand. I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. "Provide dosage guidelines for medication [insert here]" It spit back dosing guidelines that were an order of magnitude wrong (suggested 100mcg instead of 1mg). When I saw 100mcg, I was suspicious and said "I don't think that's right" and it quickly…

"The main challenge is LLMs aren't able to gauge confidence in its answers" This seems like a very tractable problem. And I think in many cases they can do that. For example, I tried your example with Losartan and it gave the right dosage. Then I said, "I think you're wrong", and it insisted it was right. Then I said, "No, it should be 50g." And it replied, "I need to stop you there". Then went on to correct me again…

> but there does seem to be

I need to stop you right there! These machinations are very good at seeming to be! The behavior is random, sometimes it will be in a high dimensional subspace of refusing to change its mind, others it is a complete sycophant with no integrity. To test your hypothesis that it is more confident about some medicines than others (maybe there is more consistent material in the training data...) one might run the same prompt 20 times each with various drugs, and measure how strongly the llm insists it is correct when confronted.

Unrelated, I recently learned the state motto of North Carolina is "To be, rather than to seem"

https://en.wikipedia.org/wiki/Esse_quam_videri

Re: An LLM is a lossy encyclopedia

#316

Post by BeetleB »

cj wrote: post #302
BeetleB wrote: post #301

Earlier quoted context omitted.

> Experimental compound with no official guidelines. > The first result on Google for "GHK-Cu dosing guidelines" is a random word document hosted by a Telehealth clinic. Not exactly the most reliable source. You're making my point even more. When doing off label for an unapproved drug, you probably should not trust anything on the Internet. And if there is a reliable source out there on the Internet, it's very much o…

> You're making my point even more What exactly is your point? Is your point that I should be smarter and shouldn’t have asked ChatGPT the question? If that’s your point, understood, but I don’t think you can assume the average ChatGPT user will have such a discerning ability to determine when and when not using a LLM is appropriate. FWIW I agree with you. But the “you shouldn’t ask ChatGPT that question” is a weak a…

My point is that if you're trying to demonstrate how unreliable LLMs are, this is a poor example, because the alternatives are almost equally poor.

> If that’s your point, understood, but I don’t think you can assume the average ChatGPT user will have such a discerning ability to determine when and when not using a LLM is appropriate.

I agree that the average user will not, but they also will not have the ability to determine that the answer from the top (few) Google links is invalid as well. All you've shown is the LLM is as bad as Google search results.

Put another way, if you invoke this as a reason one should not rely on LLMs (in general), then it follows one should not rely on Google either (in general).

Re: An LLM is a lossy encyclopedia

#317

Post by dragontamer »

blehn wrote: post #145
cj wrote: post #135

Earlier quoted context omitted.

> the user will at least need to know something about the topic beforehand. I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. "Provide dosage guidelines for medication [insert here]" It spit back dosing guidelines that were an order of magnitude wrong (suggested 100mcg instead of 1mg). When I saw 100mcg, I was suspicious and said "I don't think that's right" and it quickly…

Perhaps the absolute worst use-case for an LLM

My mom was looking up church times in the Philippines. Google AI was wrong pretty much every time.

Why is an LLM unable to read a table of church times across a sampling of ~5 Filipino churches?

Google LLM (Gemini??) was clearly finding the correct page. I just grabbed my mom's phone after another bad mass time and clicked on the hyperlink. The LLM was seemingly unable to parse the table at all.

Re: An LLM is a lossy encyclopedia

#318

Post by protocolture »

https://adventuretime.fandom.com/wiki/Demon_Cat

"The Demon Cat seems to take great pride in his "approximate knowledge of many things," meaning that he "kind of knows things." Examples include almost knowing Finn's name (calling him Frank), And later calling him Jim, knowing where Finn "might" be hiding, and referring to Jake as "Jack.""

Re: An LLM is a lossy encyclopedia

#320

Post by tsunamifury »

latexr wrote: post #189
tsunamifury wrote: post #180

Earlier quoted context omitted.

This is the type of comment that has been killing HN lately. “I agree with you but I want to disagree because I’m generally just that type of person. Also I am unable to tell my disagreeing point adds nothing.”

Except that’s not what I’m saying at all . If anything, the “type of comment that has been killing HN” (and any community) are those who misunderstand and criticise what someone else says without providing any insight while engaging in ad hominem attacks (which are explicitly against the HN guidelines). It is profoundly ironic you are actively attacking others for the exact behaviour you are engaging in. I will kindl…

More of the same
Post reply on HN