Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An LLM is a lossy encyclopedia

simonwillison.net

Post reply on HN
Print view View original

301–310 of 365 posts

Re: An LLM is a lossy encyclopedia

#301

Post by BeetleB »

cj wrote: post #297
BeetleB wrote: post #295

Earlier quoted context omitted.

> I used ChatGPT 5 over the weekend to double check dosing guidelines for a specific medication. This use case is bad by several degrees . Consider an alternative: Using Google to search for it and relying on its AI generated answer. This usage would be bad by one degree less, but still bad. What about using Google and clicking on one of the top results? Maybe healthline.com? This usage would reduce the badness by on…

The compound I was researching was [edit: removed]. Problem is it's not FDA approved, only prescribed by compounding pharmacies off label. Experimental compound with no official guidelines. The first result on Google for "[edit: removed] dosing guidelines" is a random word document hosted by a Telehealth clinic. Not exactly the most reliable source. Edit: Jeesh, what’s with the downvotes?

> Experimental compound with no official guidelines.

> The first result on Google for "GHK-Cu dosing guidelines" is a random word document hosted by a Telehealth clinic. Not exactly the most reliable source.

You're making my point even more. When doing off label for an unapproved drug, you probably should not trust anything on the Internet. And if there is a reliable source out there on the Internet, it's very much on you to be able to discern what is and what is not reliable. Who cares that the LLM is wrong, when likely much of the Internet is wrong?

BTW, I'm not advocating that LLMs are good for stuff like this. But a better example would be asking the LLM "In my state, is X taxable?"

The Google AI summary was completely wrong (and the helpful link it used as a reference was correct, and in complete disagreement with the summary). But other than the AI summary being wrong, pretty much every link in the Google search results was correct. This is a good use case for not relying on an LLM: Information that is widely and easily available is wrong in the LLM.

Re: An LLM is a lossy encyclopedia

#302

Post by cj »

BeetleB wrote: post #301
cj wrote: post #297

Earlier quoted context omitted.

The compound I was researching was [edit: removed]. Problem is it's not FDA approved, only prescribed by compounding pharmacies off label. Experimental compound with no official guidelines. The first result on Google for "[edit: removed] dosing guidelines" is a random word document hosted by a Telehealth clinic. Not exactly the most reliable source. Edit: Jeesh, what’s with the downvotes?

> Experimental compound with no official guidelines. > The first result on Google for "GHK-Cu dosing guidelines" is a random word document hosted by a Telehealth clinic. Not exactly the most reliable source. You're making my point even more. When doing off label for an unapproved drug, you probably should not trust anything on the Internet. And if there is a reliable source out there on the Internet, it's very much o…

> You're making my point even more

What exactly is your point?

Is your point that I should be smarter and shouldn’t have asked ChatGPT the question?

If that’s your point, understood, but I don’t think you can assume the average ChatGPT user will have such a discerning ability to determine when and when not using a LLM is appropriate.

FWIW I agree with you. But the “you shouldn’t ask ChatGPT that question” is a weak argument if you care about contextualizing and broadening your point beyond me and my specific anecdote.

Re: An LLM is a lossy encyclopedia

#303

Post by lurking_swe »

el_benhameen wrote: post #188
dncornholio wrote: post #143

Earlier quoted context omitted.

Using a LLM for medical research is just as dangerous as Googling it. Always ask your doctors!

I don’t disagree that you should use your doctor as your primary source for medical decision making, but I also think this is kind of an unrealistic take. I should also say that I’m not an AI hype bro. I think we’re a long ways off from true functional AGI and robot doctors. I have good insurance and have a primary care doctor with whom I have good rapport. But I can’t talk to her every time I have a medical question…

I live in the U.S. and my doctor is very responsive on MyChart. A few times a year i’ll send a message and I almost always get a reply within a day! From my PCP directly, or from her assistant.

I’d encourage you to find another doctor.

Re: An LLM is a lossy encyclopedia

#304

Post by lurking_swe »

shrx wrote: post #268
el_benhameen wrote: post #188

Earlier quoted context omitted.

I don’t disagree that you should use your doctor as your primary source for medical decision making, but I also think this is kind of an unrealistic take. I should also say that I’m not an AI hype bro. I think we’re a long ways off from true functional AGI and robot doctors. I have good insurance and have a primary care doctor with whom I have good rapport. But I can’t talk to her every time I have a medical question…

> it can take weeks to just get a phone call > If I manage to get an appointment, it’s a 15 minute slot I'm sorry that this is what "good insurance" gets you.

no, that’s what happens when you pick a busy doctor or a practice that’s overbooked in general. All too common these days! :(

This probably varies by locale. For example my doctor responds within 1 day on MyChart for quick questions. I can set up an in person or video appointment with her within a week, easily booked on MyChart as well.

Re: An LLM is a lossy encyclopedia

#305

Post by parpfish »

rkomorn wrote: post #266
parpfish wrote: post #265

Earlier quoted context omitted.

I always thought “ask your doctor” was included for liability reasons and not a thing that people actually could do. I also have good insurance and a PCP. The idea that I could call them up just to ask “should I start doing this new exercise” or “how much aspirin for this sprained ankle?” is completely divorced from reality.

I think "ask your doctor" is for prescription meds since only said doctor can write prescriptions. And "your doctor" is actually "any doctor that is willing to write you a prescription for our medicine".

"ask your doctor" is more widespread than tthat. if you look up any diet or exercise advice, there's always an "ask your doctor before starting any new exercise program".

i'm not going to call my doctor to ask "is it okay if I try doing kettlebell squats?"

Re: An LLM is a lossy encyclopedia

#306

Post by tough »

SV_BubbleTime wrote: post #291
cantor_S_drug wrote: post #263

Earlier quoted context omitted.

I gave LLM a list of python packages and asked it to give me their respective licenses. Obviously it got some of them wrong. I had to manually check with the package's pypi page.

Ok? Yes. That is the problem; that sometimes it works. See the topic. Adding RAG or web search capability limits the loss and hallucinations. Yes. You always need to check the results. Your task by the way is better for an agentic AI system that can web search, get, and double check results.

his task is probably best done with a script, heck you could thell chatgpt to download all packages with a script and check LICENSE files and report back with a csv/table

Re: An LLM is a lossy encyclopedia

#308

Post by cush »

What benefit to we get from the reductionism of "An LLM is a lossy encyclopedia"? It completely negates people's lived experiences with them - "An LLM is my doctor", "An LLM is my wedding planner", "An LLM is my coding partner". It's all of those things and more. They're an entirely new paradigm with an entirely new UI

Re: An LLM is a lossy encyclopedia

#309

Post by rkomorn »

parpfish wrote: post #305
rkomorn wrote: post #266

Earlier quoted context omitted.

I think "ask your doctor" is for prescription meds since only said doctor can write prescriptions. And "your doctor" is actually "any doctor that is willing to write you a prescription for our medicine".

"ask your doctor" is more widespread than tthat. if you look up any diet or exercise advice, there's always an "ask your doctor before starting any new exercise program". i'm not going to call my doctor to ask "is it okay if I try doing kettlebell squats?"

Yes, I totally got out of context and said something a bit senseless.

But also, maybe calling your doctor would be wise (eg if you have back problems) before you start doing kettlebell squats.

I'd say that the audience for a lot of health related content skews towards people who should probably be seeing a doctor anyway.

The cynic in me also thinks some of the "ask your doctor" statements are just slapped on to artificially give credence to whatever the article is talking about (eg "this is serious exercise/diet/etc).

Edit: I guess what I meant is: I don't think it's just "liability", but genuine advice/best practice/wisdom for a sizable chunk of audiences.

Re: An LLM is a lossy encyclopedia

#310

Post by cush »

alanfranz wrote: post #50

I think it’s an old analogy, and a good one. LLMs are for knowledge what mp3s were for audio. This was widely discussed in the past years as well.

It's a terrible analogy because LLMs are already for audio what LLMs are for audio. You can use LLMs to create new songs and sounds. Encyclopedias don't create new songs.
Post reply on HN