Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overcoming the limits of current LLMs

seanpedersen.github.io

Post reply on HN
Print view View original

91–100 of 111 posts

Re: Overcoming the limits of current LLMs

#91

Post by pennomi »

motohagiography wrote: post #89
threeseed wrote: post #46

Earlier quoted context omitted.

> properly-licensed and verified datasets is immensely more difficult, expensive Arguably the bigger problem is that many of those datasets e.g. WSJ articles are proprietary and can be exclusively licensed like we've seen recently with OpenAI. So we end up with in a situation where competition is simply not possible.

the irony is that if large media providers aren't represented in the training sets, my comments on internet forums over the decades will be over-represented, which is kind of great, really.

Right? I always ask people - Hypothetically if someone created a superintelligent AI that took over the world, wouldn’t you WANT it to share your opinions and morals?

Every tiny bit of text you write is a vote in the election of our future AI overlords.

Re: Overcoming the limits of current LLMs

#92

Post by empath75 »

devjab wrote: post #28
exmadscientist wrote: post #18

Earlier quoted context omitted.

"Hallucinations" implies that someone isn't of sound mental state. We can argue forever about what that means for a LLM and whether that's appropriate, but I think it's absolutely the right attitude and approach to be taking toward these things. They simply do not behave like humans of sound minds, and "hallucinations" conveys that in a way that "confabulations" or even "bullshit" does not. (Though "bullshit" isn't b…

I disagree with this take because LLMs are, always, hallucinating. When they get things right it’s because they are lucky. Yes, yes, it’s more complicated than that, but the essence of LLMs is that they are very good at being lucky. So good that they will often give you better results than random search engine clicks, but not good enough to be useful for anything important. I think what calling the times they get thi…

> When they get things right it’s because they are lucky.

This is transparently wrong. It gets so many things right in a response that the few things it gets wrong are tremendously frustrating. I think people underestimate how much correct "knowledge about the world" is expressed in a typical chat gpt response and focus only on the parts that are incorrect.

If it were wrong about _everything_ at rates no better than chance, we wouldn't even be having this conversation because nobody would be using them.

Re: Overcoming the limits of current LLMs

#93

Post by skdotdan »

darby_nine wrote: post #4

Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is with perception rather than structural incompetence. Why not just say "incoherent output"? It's actually descriptive and doesn't require bastardizing a word we already find meaningful to mean something completely different.

It’s used incorrectly. Hallucination has (or used to have) a very specific meaning in machine learning. All hallucinations are errors but not all errors are hallucinations.

Re: Overcoming the limits of current LLMs

#94

Post by skdotdan »

Slow_Hand wrote: post #79
darby_nine wrote: post #4

Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is with perception rather than structural incompetence. Why not just say "incoherent output"? It's actually descriptive and doesn't require bastardizing a word we already find meaningful to mean something completely different.

I prefer “confabulate” to describe this phenomena. : to fill in gaps in memory by fabrication > In psychology, confabulation is a memory error consisting of the production of fabricated, distorted, or misinterpreted memories about oneself or the world. It’s more about coming up with a plausible explanation in the absence of a readily-available one.

This is not what hallucination means in the pre-LLM machine learning literature.

Re: Overcoming the limits of current LLMs

#95

Post by lopatin »

darby_nine wrote: post #4

Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is with perception rather than structural incompetence. Why not just say "incoherent output"? It's actually descriptive and doesn't require bastardizing a word we already find meaningful to mean something completely different.

Why is "incoherent output" better? When an LLM hallucinates, it coherently and confidently lies to you. I think "hallucination" is the perfect word for this.

Re: Overcoming the limits of current LLMs

#96

Post by bosch_mind »

mitthrowaway2 wrote: post #62

LLMs don't only hallucinate because of mistaken statements in their training data. It just comes hand-in-hand with the model's ability to remix, interpolate, and extrapolate answers to other questions that aren't directly answered in the dataset. For example if I ask ChatGPT a legal question, it might cite as precedent a case that doesn't exist at all (but which seems plausible, being interpolated from cases that do…

AI noob, but instead of training and fine tuning the LLM itself, don’t more specific and targeted embeddings paired with the model help alleviate the hallucination where you incorporate semantic search context with the question?

Re: Overcoming the limits of current LLMs

#97

Post by bosch_mind »

mitthrowaway2 wrote: post #73
CamperBob2 wrote: post #63

Earlier quoted context omitted.

These problems are solvable, but are we really making a better world by solving them? When you ask yourself that question -- and you do ask yourself that, right? -- what's your answer?

I do, all the time! My answer is "most likely not". (I assumed that answer was implied by my expressing sadness about all the work being invested in them.) This is why, although I try to keep up-to-date with and understand these technologies, I am not being paid to develop them.

I mean, comon man, _most_ tech people work on in general is also useless. FAANG has a retainer on talent whether they are spending their time wisely or not.

LLM stuff is not in a different echelon

Re: Overcoming the limits of current LLMs

#99

Post by bbor »

sean_pedersen wrote: post #78
dweinus wrote: post #75

Earlier quoted context omitted.

100% I think the author is really misunderstanding the issue here. "Hallucination" is a fundamental aspect of the design of Large Language Models. Narrowing the distribution of the training data will reduce the LLM's ability to generalize, but it won't stop hallucinations.

I agree in that a perfectly consistent dataset won't completely stop statistical language models from hallucinating but it will reduce it. I think it is established that data quality is more important than quantity. Bullshit in -> bullshit out, so a focus on data quality is good and needed IMO. I am also saying LMs output should cite sources and give confidence scores (which reflects how much the output is in or out…

What’s a non-statistical language model?

And I think looking to the training data for sources is a little silly - that’s the training data for intuitive language use, not true statements about the world. If you haven’t checked it out yet, two terms you’d love are “RAG” and “Manuel De Landa”

Re: Overcoming the limits of current LLMs

#100

Post by antihipocrat »

sean_pedersen wrote: post #78
dweinus wrote: post #75

Earlier quoted context omitted.

100% I think the author is really misunderstanding the issue here. "Hallucination" is a fundamental aspect of the design of Large Language Models. Narrowing the distribution of the training data will reduce the LLM's ability to generalize, but it won't stop hallucinations.

I agree in that a perfectly consistent dataset won't completely stop statistical language models from hallucinating but it will reduce it. I think it is established that data quality is more important than quantity. Bullshit in -> bullshit out, so a focus on data quality is good and needed IMO. I am also saying LMs output should cite sources and give confidence scores (which reflects how much the output is in or out…

How would confidence scores work? Multiple passthroughs and a % attached to each statement according to how often it appeared in the generated result?

If so, building this could be quite complex depending on the domain. In the legal field even one simple word that is changed can have large consequences.

Post reply on HN