Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teach your LLM to answer with facts, not fiction

blog.myscale.com

Post reply on HN
Print view View original

101–110 of 149 posts

Re: Teach your LLM to answer with facts, not fiction

#101

Post by ImHereToVote »

delecti wrote: post #33
s1mon wrote: post #30

Earlier quoted context omitted.

"Hallucination" makes it sound like ChatGPT drank some of the punch without realizing it was laced with LSD. "Bullshit" sounds more like what comes out of an overconfident ass who should or could know better with some better education.

Hallucination is a better descriptor for what an LLM is doing though. A bullshitter knows they don't know, an LLM just strings words together in ways that fit what it "saw" from training data. IMO the main problem with calling them hallucinations is the implication that the true things they say are true on purpose. It's hallucinating the true things too.

In Swedish the term for perception and misapprehension is the same word. Because it is the same thing.

Re: Teach your LLM to answer with facts, not fiction

#102

Post by flanked-evergl »

senectus1 wrote: post #10

would be nice if they could show a gradient score on results that show how certain it is of its answers... It should be fairly trivial for it to tell you how often its straight up lied about something.

> It should be fairly trivial for it to tell you how often its straight up lied about something.

By what mechanism would it "lie" to you? How is it even capable of lying in this sense of the word:

https://en.wiktionary.org/wiki/lie#Verb_2

> To give false information intentionally with intent to deceive.

I guess it could "lie" in other senses, but calling that lying is not really adding clarity to the situation.

Re: Teach your LLM to answer with facts, not fiction

#104

Post by Renaud »

Does anyone know if there is a “wikipedia-only” or way to constrain the knowledge of a LLM (trained on a more limited set of sources or constrained by limited sources of facts)?

I think a LLM interface to Wikipedia could be useful, at least I imagine it would.

Re: Teach your LLM to answer with facts, not fiction

#105

Post by sweezyjeezy »

jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

> it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic. So you get a bit of all that.

This is true of base LLM models that are just trained on missing-word prediction on the training corpus, but one of the main points of RLHF[1] is to tune this model to make these kind of inferences the way a human would expect. For example if you asked an untuned model to write a poem in the style of ... etc., a valid internet response might be "hmm no thanks, you go first", you need to steer the model away from replying like this.

I'm not saying it's perfect, but it's wrong to say e.g. GPT-4 has had no information about the difference between a good and bad response and is just generating internet-like text at random, the big players have made progress on this already.

[1] https://en.wikipedia.org/wiki/Reinforcement_learning_from_hu...

Re: Teach your LLM to answer with facts, not fiction

#107

Post by msp26 »

senectus1 wrote: post #10

would be nice if they could show a gradient score on results that show how certain it is of its answers... It should be fairly trivial for it to tell you how often its straight up lied about something.

It isn't trivial. That's the whole problem.

Luckily my use cases have a manual check built in but even a proxy for confidence would be amazing.

Re: Teach your LLM to answer with facts, not fiction

#108

Post by FishInTheWater »

astrange wrote: post #92
quickthrower2 wrote: post #67

Earlier quoted context omitted.

This are assuming LLMs are intelligent and can think "hey I am dumb, I'll look that up". What they are literally doing is guessing the next word, a word a time but doing it really really well and making statistically average output over a very large number of inputs. There is no distinction between understanding "the" vs "a" and telling me 1+1=3. It is all token generation.

What they are doing depends entirely on what decoding algorithm you use. An LLM is mostly a token probability function, but it's not just that - a transformer model is capable of learning anything. Tokens are the interface, not necessarily the implementation.

A transformer can only memorize, it doesn't learn to do.

For what that concerns us here: LLMs will never learn to fact-check anything. They'll blindly regurgitate the facts they have been "taught", but never consider or evaluate "the paper cited for this fact on wikipedia is a bunch of bullshit".

Any attempt to use them to produce "facts" is ultimately just folly, in the same way Google's attempt to do so with it's search engine index is.

Re: Teach your LLM to answer with facts, not fiction

#109

Post by masukomi »

I really REALLY wish people would stop assigning sentience to LLMs.

> In other words, a hallucination is an error in (or a false) perception of something real or concrete.

an llm has no "perception" it doesn't "believe" or "think" that the answers it provides are "correct" or "true" or even "false". It's just autocompleting strings with the most probably next words.

If we keep treating these things as if they're sentient entities that "want" to provide "correct" answers we're going to keep tripping over our false assumptions about their answers.

Re: Teach your LLM to answer with facts, not fiction

#110

Post by jojobas »

jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

I prefer to think that most humans actually distinguish the fictional context, and so should LLM. As such, if it is to be of any use, it'd better figure out it's fiction if someone's flying on a winged horse, levitating trolls or (obviously harder) running around a forest with a bow.

And when answering a question, unambiguously specify this fictional context, or at least indicate that it might be fiction if unsure.

Post reply on HN