Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An LLM is a lossy encyclopedia

simonwillison.net

Post reply on HN
Print view View original

331–340 of 365 posts

Re: An LLM is a lossy encyclopedia

#331

Post by cantor_S_drug »

SV_BubbleTime wrote: post #291
cantor_S_drug wrote: post #263

Earlier quoted context omitted.

I gave LLM a list of python packages and asked it to give me their respective licenses. Obviously it got some of them wrong. I had to manually check with the package's pypi page.

Ok? Yes. That is the problem; that sometimes it works. See the topic. Adding RAG or web search capability limits the loss and hallucinations. Yes. You always need to check the results. Your task by the way is better for an agentic AI system that can web search, get, and double check results.

[dead]

Re: An LLM is a lossy encyclopedia

#332

Post by vrighter »

latexr wrote: post #298
mock-possum wrote: post #208

Earlier quoted context omitted.

Yeah an LLM is an unreliable librarian, if anything.

That’s a much better analogy. You have to specifically ask them for information and they will happily retrieve it for you, but because they are unreliable they may get you the wrong thing. If you push back they’ll apologise and try again (librarians try to be helpful) but might again give you the wrong thing (you never know, because they are unreliable).

There's a big difference between giving you correct information about the wrong thing, vs giving you incorrect information about the right thing.

A librarian might bring you the wrong book, that's the former. An LLM does the latter. They are not the same.

Re: An LLM is a lossy encyclopedia

#333

Post by koakuma-chan »

kiitos wrote: post #282
koakuma-chan wrote: post #89

Earlier quoted context omitted.

No, I don't think GPT-5 clarifying questions actually do what you think they do. They just made the model ask clarifying questions for the sake of asking clarifying questions. I'm sure GPT-4o would have given me the answer I wanted without clarifying questions.

revisit your instructions.md and/or user preferences, this is very likely the root cause

Wait what. I use duck.ai, could it be that they put something into the system prompt......

Re: An LLM is a lossy encyclopedia

#334

Post by Mallowram »

ACCount37 wrote: post #267
mallowdram wrote: post #220

Earlier quoted context omitted.

Sorry, 2 billion years of neurobiology beats 60 years of NLP/LLMs which knows less to nothing about language since "arbitrary points can never be refined or defined to specifics" check your corners and know your inputs. The bill is due on NLP.

Incoherent drivel.

[dead]

Re: An LLM is a lossy encyclopedia

#335

Post by nonameiguess »

cj wrote: post #297
BeetleB wrote: post #295

Earlier quoted context omitted.

> I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. This use case is bad by several degrees . Consider an alternative: Using Google to search for it and relying on its AI generated answer. This usage would be bad by one degree less, but still bad. What about using Google and clicking on one of the top results? Maybe healthline.com? This usage would reduce the badness by on…

The compound I was researching was [edit: removed]. Problem is it's not FDA approved, only prescribed by compounding pharmacies off label. Experimental compound with no official guidelines. The first result on Google for "[edit: removed] dosing guidelines" is a random word document hosted by a Telehealth clinic. Not exactly the most reliable source. Edit: Jeesh, what’s with the downvotes?

I think this actually points at a different problem, a problem with LLM users, but only to the extent that it's a problem with people with respect to any questions they have to ask any source they consider an authority at all. No LLM, nor any other source on the Internet, nor any other source off the Internet, can give you reliable dosage guidelines for copper peptides because this is information that is not known to humans. There is some answer to the question of what response you might expect and how that varies by dose, but without the clinical trials ever having been conducted, it's not an answer anyone actually has. Marketing and popular misconceptions about AI lead to people expecting it to be able to conjure facts out of thin air, perhaps reasoning from first principles using its highly honed model of human physiology.

It's an uncomfortable position to be in trying to biohack your way to a more youthful appearance using treatments that have never been studied in human trials, but that's the reality you're facing. Whatever guidelines you manage to find, whether from the telehealth clinic directly, or from a language model that read the Internet and ingested that along with maybe a few other sources, are generally extrapolated from early rodent studies and all that's being extrapolated is an allometric scaling from rat body to human body of the dosage the researchers actually gave to the rats. What effect that actually had, and how that may or may not translate to humans, is not usually a part of the consideration. To at least some extent, it can't be if the compound was never trialed on humans.

You're basically just going with scale up a dosage to human sized that at least didn't kill the rats. Take that and it probably won't kill you. What it might actually do can't be answered, not by doctors, not by an LLM, not by Wikipedia, not by anecdotes from past biohackers who tried it on themselves. This is not a failure of information retrieval or compression. You're just asking for information that is not known to anyone, so no one can give it to you.

If there's a problem here specific to LLMs, it's that they'll generally give you an answer anyway and will not in any way quantify the extent to which it is probably bullshit and why.

Re: An LLM is a lossy encyclopedia

#336

Post by latexr »

vrighter wrote: post #332
latexr wrote: post #298

Earlier quoted context omitted.

That’s a much better analogy. You have to specifically ask them for information and they will happily retrieve it for you, but because they are unreliable they may get you the wrong thing. If you push back they’ll apologise and try again (librarians try to be helpful) but might again give you the wrong thing (you never know, because they are unreliable).

There's a big difference between giving you correct information about the wrong thing, vs giving you incorrect information about the right thing. A librarian might bring you the wrong book, that's the former. An LLM does the latter. They are not the same.

Fair. With the unreliable librarian you’d be at an advantage because you’d immediately see “this is not what I asked for”, which is not the case with LLMs (and hence what makes them so problematic).

Re: An LLM is a lossy encyclopedia

#337

Post by cj »

nonameiguess wrote: post #335
cj wrote: post #297

Earlier quoted context omitted.

The compound I was researching was [edit: removed]. Problem is it's not FDA approved, only prescribed by compounding pharmacies off label. Experimental compound with no official guidelines. The first result on Google for "[edit: removed] dosing guidelines" is a random word document hosted by a Telehealth clinic. Not exactly the most reliable source. Edit: Jeesh, what’s with the downvotes?

I think this actually points at a different problem, a problem with LLM users, but only to the extent that it's a problem with people with respect to any questions they have to ask any source they consider an authority at all. No LLM, nor any other source on the Internet, nor any other source off the Internet, can give you reliable dosage guidelines for copper peptides because this is information that is not known to…

> a problem with LLM users

I think the flaw here is placing blame on users rather than the service provider.

HN is cutting LLM companies slack because we understand the technical limitations making it hard for the LLM to just say “I don’t know”.

In any other universe, we would be blaming the service rather than the user.

Why don’t we fix LLMs so they don’t spit out garbage when it doesn’t know the answer. Have we given up on that thought?

Re: An LLM is a lossy encyclopedia

#338

Post by simonw »

cj wrote: post #337
nonameiguess wrote: post #335

Earlier quoted context omitted.

I think this actually points at a different problem, a problem with LLM users, but only to the extent that it's a problem with people with respect to any questions they have to ask any source they consider an authority at all. No LLM, nor any other source on the Internet, nor any other source off the Internet, can give you reliable dosage guidelines for copper peptides because this is information that is not known to…

> a problem with LLM users I think the flaw here is placing blame on users rather than the service provider. HN is cutting LLM companies slack because we understand the technical limitations making it hard for the LLM to just say “I don’t know”. In any other universe, we would be blaming the service rather than the user. Why don’t we fix LLMs so they don’t spit out garbage when it doesn’t know the answer. Have we giv…

Current frontier LLMs - Claude 4, GPT-5, Gemini 2.5 - are massively more likely to say "I don't know" than last year's models.

Re: An LLM is a lossy encyclopedia

#339

Post by cj »

simonw wrote: post #338
cj wrote: post #337

Earlier quoted context omitted.

> a problem with LLM users I think the flaw here is placing blame on users rather than the service provider. HN is cutting LLM companies slack because we understand the technical limitations making it hard for the LLM to just say “I don’t know”. In any other universe, we would be blaming the service rather than the user. Why don’t we fix LLMs so they don’t spit out garbage when it doesn’t know the answer. Have we giv…

Current frontier LLMs - Claude 4, GPT-5, Gemini 2.5 - are massively more likely to say "I don't know" than last year's models.

I don’t think I’ve ever seen ChatGPT 5 refuse to answer any prompt I’ve ever given it. I’m doing 20+ chats a day.

What’s an example prompt where it will say “idk”?

Edit: Just tried a silly one, asking it to tell me about the 8th continent on earth, which doesn’t exist. How difficult is it for the model to just say “sorry, there are only 7 continents”. I think we should expect more from LLMs and stop blaming things on technical limitations. “It’s hard” is getting to be an old excuse considering the amount of money flowing into building these systems.

Re: An LLM is a lossy encyclopedia

#340

Post by estimator7292 »

quincepie wrote: post #37

I totally agree with the author. Sadly, I feel like that's not what the majority of LLM users tend to view LLMs. And it's definitely not what AI companies marketing. > The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters the problem is that in order to develop an intuition for questions that LLMs can answer, the user will…

It's really strange to me that the only way to effectively use LLMs is if you already have all the knowledge and skill to do the task yourself.

I can't think of any other tools like this. An LLM can multiply your efforts, but only if you were capable of doing it yourself. Wild.

Post reply on HN