Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teach your LLM to answer with facts, not fiction

blog.myscale.com

Post reply on HN
Print view View original

71–80 of 149 posts

Re: Teach your LLM to answer with facts, not fiction

#71

Post by ben_w »

RyEgswuCsn wrote: post #68

I think LLMs need to be taught to say "I don't know"/"I am not sure" or something to that effect. Another approach might be to introduce an adversarial "censor" model to guard against hallucination (or inappropriate answers).

Isn't the fundamental issue that it doesn't have any way to tell if what it thinks it knows is or isn't true?

This article sounds like an idea I had independent not too long ago, but with a different goal:

LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" capability from the "knowledge" capability and have the former just look things up in a database?

My question was more about reducing the size of the network rather than reducing hallucinations, but it's still a separately updatable knowledge resource.

(The answer may actually be "no"; I don't study this professionally, but technical jargon is kinda both factual domain knowledge and also linguistic comprehension, which is why Oracle isn't competing with Starbucks for Java beans).

Re: Teach your LLM to answer with facts, not fiction

#72

Post by kfrzcode »

quickthrower2 wrote: post #67
lqhl wrote: post #6

Earlier quoted context omitted.

This article suggests that LLMs should use a database as a reference for factual information. Rather than asking LLMs to provide their own answers, it is recommended that they summarize based on the facts extracted from the database. This approach reduces the likelihood of hallucinations among LLMs.

This are assuming LLMs are intelligent and can think "hey I am dumb, I'll look that up". What they are literally doing is guessing the next word, a word a time but doing it really really well and making statistically average output over a very large number of inputs. There is no distinction between understanding "the" vs "a" and telling me 1+1=3. It is all token generation.

Isn't this a serious simplification? Tokens are just the medium

Re: Teach your LLM to answer with facts, not fiction

#73

Post by yreg »

taneq wrote: post #56
jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

Mellon. The rest are left as an exercise to the reader. It does always amaze me that we trained LLMs on a dump of the internet and then people are shocked that they're about as trustworthy as a random web page.

People are not shocked and poor training data is not the main reason LLMs are not trustworthy.

Re: Teach your LLM to answer with facts, not fiction

#74

Post by mdp2021 »

austinjp wrote: post #54
mdp2021 wrote: post #50

Earlier quoted context omitted.

> These are all 'factual questions' Because of elision. "[Homer wrote] that Achilles died of an arrow in the heel" This is why the Wiener Kreis taught to use protocolar statements: " and witnessed ".

Tangential: I was going to suggest "protocoli(s|z)ed" instead of protocollar, but I Googled "protocollar statements" just to check and found 2 things. First, this page was the top result! Second, "protocolar" (one ell) and "protocolary" are apparently real words. New to me, thanks.

(By the way:

> "protocoli(s|z)ed"

the use of '-ize' is (a graecism) indicated by the OED as International English, as opposed to British, American etc. In fact, some call International English "British spelling with -ize" - it is not exactly that but close. One exception is 'analyse', but that is because linguists compromised on the "difficult" original 'analysize'.)

Re: Teach your LLM to answer with facts, not fiction

#75

Post by p-e-w »

flanked-evergl wrote: post #69

I think the author of that title could well do with a refresher course in epistemology and physics, as it is just not possible to do what they suggest. But even more unfortunate is how many people fall for deceptive marketing that really should not even fool the average 16-year-old.

It's weird to see how in such articles (which concern topics that are deeply philosophical by nature), philosophical terms like "facts", "consciousness", "knowledge" etc. are just thrown around as if there was any consensus on what those words even mean.

The whole debate is revolving around hot air, because nobody knows whether the other person is talking about the same thing as themselves.

Re: Teach your LLM to answer with facts, not fiction

#76

Post by bryanrasmussen »

austinjp wrote: post #54
mdp2021 wrote: post #50

Earlier quoted context omitted.

> These are all 'factual questions' Because of elision. "[Homer wrote] that Achilles died of an arrow in the heel" This is why the Wiener Kreis taught to use protocolar statements: " and witnessed ".

Tangential: I was going to suggest "protocoli(s|z)ed" instead of protocollar, but I Googled "protocollar statements" just to check and found 2 things. First, this page was the top result! Second, "protocolar" (one ell) and "protocolary" are apparently real words. New to me, thanks.

I think protocollar is in this context a misspelling of protocolar - hence its high placement for protocollar statements, if I google protocolar statements this is the highest result (for me)

https://www.britannica.com/topic/protocol-sentence

Re: Teach your LLM to answer with facts, not fiction

#77

Post by RyEgswuCsn »

ben_w wrote: post #71
RyEgswuCsn wrote: post #68

I think LLMs need to be taught to say "I don't know"/"I am not sure" or something to that effect. Another approach might be to introduce an adversarial "censor" model to guard against hallucination (or inappropriate answers).

Isn't the fundamental issue that it doesn't have any way to tell if what it thinks it knows is or isn't true? This article sounds like an idea I had independent not too long ago, but with a different goal: LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" capability from the "knowledge" capa…

Many classic statistical modelling techniques have ways to produce some measure of confidence for their predictions; perhaps LLMs could incorporate that as well, e.g. assign probabilities(/perplexity?) to each of the tokens they generate.

> LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" capability from the "knowledge" capability and have the former just look things up in a database?

I think the beauty of LLMs is exactly that all we need to do is to feed them raw text --- the hope, I guess, had been that the models will be able to develop human-like insights by learning to understand and "speak" languages on its own. Introducing "feature engineering" (e.g. the distinction as you suggested) would defeat that goal.

Re: Teach your LLM to answer with facts, not fiction

#78

Post by codetrotter »

mdp2021 wrote: post #63
balnaphone wrote: post #49

Earlier quoted context omitted.

"Confabulation" is the correct and precise term that comports with the English language, rather than being jargon requiring a neologism.

> "Confabulation" And why would that be? "Hallucination" means "erratic wandering", implying one is lost - similarly to "delirium" (maetaphor using the plough) and "error". Part of the idea is that of "instead of witnessing the correct, reporting the false" - a very ancient, traditional idea, and akin to the concept of "intelligence" ( intus-legere ). "Confabulation" means locutor and interlocutor are talking, exchan…

> In psychology, confabulation is a memory error defined as the production of fabricated, distorted, or misinterpreted memories about oneself or the world. It is generally associated with certain types of brain damage (especially aneurysm in the anterior communicating artery) or a specific subset of dementias.

https://en.wikipedia.org/wiki/Confabulation

Re: Teach your LLM to answer with facts, not fiction

#79

Post by mdp2021 »

bryanrasmussen wrote: post #76
austinjp wrote: post #54

Earlier quoted context omitted.

Tangential: I was going to suggest "protocoli(s|z)ed" instead of protocollar, but I Googled "protocollar statements" just to check and found 2 things. First, this page was the top result! Second, "protocolar" (one ell) and "protocolary" are apparently real words. New to me, thanks.

I think protocollar is in this context a misspelling of protocolar - hence its high placement for protocollar statements, if I google protocolar statements this is the highest result (for me) https://www.britannica.com/topic/protocol-sentence

> a misspelling of protocolar

It could be. I cannot bring to mind the rules for doubling right now. They both occur, 'protocolar' much more often. I will correct my original post.

Re: Teach your LLM to answer with facts, not fiction

#80

Post by stavros »

mdp2021 wrote: post #74
austinjp wrote: post #54

Earlier quoted context omitted.

Tangential: I was going to suggest "protocoli(s|z)ed" instead of protocollar, but I Googled "protocollar statements" just to check and found 2 things. First, this page was the top result! Second, "protocolar" (one ell) and "protocolary" are apparently real words. New to me, thanks.

(By the way: > "protocoli(s|z)ed" the use of '-ize' is (a graecism) indicated by the OED as International English, as opposed to British, American etc. In fact, some call International English "British spelling with -ize" - it is not exactly that but close. One exception is 'analyse', but that is because linguists compromised on the "difficult" original 'analysize'.)

What's "analysize"? That's not a Greek word.
Post reply on HN