Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teach your LLM to answer with facts, not fiction

blog.myscale.com

Post reply on HN
Print view View original

111–120 of 149 posts

Re: Teach your LLM to answer with facts, not fiction

#111

Post by JacobAldridge »

hertzrut wrote: post #95

For the life of me I can't understand why so many are obsessed with LLMs as search engines and knowledge databases when it seems like it's impossible for them to be that.

I’m making some youtube videos with my 4 year old (mostly as a bonding activity). They’re fairly generic fare for that age group and genre - a bit of plot, I set a puzzle for her to solve, it reveals some clues or saves the day.

For the first puzzle she had to put the cities Tokyo, Paris, LA, and Brisbane (where we live) in order. Using colours, since she can’t read yet!

Then we had to discuss why those cities were in that order (answer below).

For the video, I figured I would doctor a Google search to land on a page explaining the answer. The actual article I found that mostly worked was about 12 results down.

Then I tried Bing’s Chat instead. It spat out the correct answer in 2 sentences. Deus ex machina indeed! (The four cities are the Summer Olympic hosts, 2020-2032).

So I disagree that it’s impossible. And I can absolutely see the value - asking “why are these cities in this order?” is a real question, like “what might be causing the squeaking sound when my car brakes?” or “what was the movie Audrey Hepburn made with the photographer?”

Search Engines aren’t great for those kind of questions - just google “What’s a good brownie recipe?”. LLMs can give the user exactly what they want, or even prompt for the additional extra context.

Not that they will always be correct; hence the flaw.

Re: Teach your LLM to answer with facts, not fiction

#112

Post by rsynnott »

delecti wrote: post #33
s1mon wrote: post #30

Earlier quoted context omitted.

"Hallucination" makes it sound like ChatGPT drank some of the punch without realizing it was laced with LSD. "Bullshit" sounds more like what comes out of an overconfident ass who should or could know better with some better education.

Hallucination is a better descriptor for what an LLM is doing though. A bullshitter knows they don't know, an LLM just strings words together in ways that fit what it "saw" from training data. IMO the main problem with calling them hallucinations is the implication that the true things they say are true on purpose. It's hallucinating the true things too.

_Both_ are an inappropriate degree of anthropomorphism IMO.

Re: Teach your LLM to answer with facts, not fiction

#113

Post by IanCal »

astrange wrote: post #94
immibis wrote: post #58

Earlier quoted context omitted.

This is correct. Current LLMs work by predicting the next word based on a bunch of preceding words. In other words, they are autocomplete. You can often form a valid sentence on your phone if you click on any text field and then press the automatic suggestions several times. Transformer-based LLMs are interesting because they are such good version of autocomplete that they can, for example, complete a news article ab…

LLM chat models are not autocomplete. They can recognize and respond to user text, which is not the same thing as completing it. If you prompted GPT-2 with a question you'd get another question, not an answer.

They are autocomplete, they're just completing something in the form

Assistant: ...

User: ...

Assistant: ...

And the output is stopped when they start generating the equivalent of "User: " and the reins are handed back to you.

This isn't a problem, autocomplete at the level of "what would a person say next" is outrageously powerful, but it is how they're working afaik.

Re: Teach your LLM to answer with facts, not fiction

#114

Post by IanCal »

RyEgswuCsn wrote: post #77
ben_w wrote: post #71

Earlier quoted context omitted.

Isn't the fundamental issue that it doesn't have any way to tell if what it thinks it knows is or isn't true? This article sounds like an idea I had independent not too long ago, but with a different goal: LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" capability from the "knowledge" capa…

Many classic statistical modelling techniques have ways to produce some measure of confidence for their predictions; perhaps LLMs could incorporate that as well, e.g. assign probabilities(/perplexity?) to each of the tokens they generate. > LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" c…

> perhaps LLMs could incorporate that as well, e.g. assign probabilities(/perplexity?) to each of the tokens they generate.

They do, that's how tokens are selected. Locally run models or non-chat ones from openai can return the probabilities and you can do things like modify or filter them.

Re: Teach your LLM to answer with facts, not fiction

#115

Post by RyEgswuCsn »

IanCal wrote: post #114
RyEgswuCsn wrote: post #77

Earlier quoted context omitted.

Many classic statistical modelling techniques have ways to produce some measure of confidence for their predictions; perhaps LLMs could incorporate that as well, e.g. assign probabilities(/perplexity?) to each of the tokens they generate. > LLMs are great at natural language comprehension, but also have a lot of neurons dedicated to factoids. Using neurons that way is really inefficient, can we split the "language" c…

> perhaps LLMs could incorporate that as well, e.g. assign probabilities(/perplexity?) to each of the tokens they generate. They do, that's how tokens are selected. Locally run models or non-chat ones from openai can return the probabilities and you can do things like modify or filter them.

I was thinking more of the entropy of the distribution over the distribution sort of thing --- one may say a model is uncertain about its statements if all its potential variants are equally probable and vice versa.

Re: Teach your LLM to answer with facts, not fiction

#116

Post by prometheus76 »

BugsJustFindMe wrote: post #43
jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

> When an LLM is suggesting what might come next in a piece of text... it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic. The obvious start seems to be having separate fiction and nonfiction LLMs and not training the nonfiction ones on Archive Of Our Own. People also end up confused about the truth when…

Which of these is more true: a newspaper article about a battle in the War of 1812, or the Star-Spangled Banner, which was written by someone witnessing a battle in the War of 1812.

Hint: how many stadiums are filled with people standing up to recite a newspaper article about a battle in the War of 1812?

Re: Teach your LLM to answer with facts, not fiction

#117

Post by prometheus76 »

jojobas wrote: post #110
jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

I prefer to think that most humans actually distinguish the fictional context, and so should LLM. As such, if it is to be of any use, it'd better figure out it's fiction if someone's flying on a winged horse, levitating trolls or (obviously harder) running around a forest with a bow. And when answering a question, unambiguously specify this fictional context, or at least indicate that it might be fiction if unsure.

How do we handle historical fiction, or even more perilous, how do we handle stories that are "based on historical events"?

Re: Teach your LLM to answer with facts, not fiction

#118

Post by prometheus76 »

3cats-in-a-coat wrote: post #32
lqhl wrote: post #6

Earlier quoted context omitted.

This article suggests that LLMs should use a database as a reference for factual information. Rather than asking LLMs to provide their own answers, it is recommended that they summarize based on the facts extracted from the database. This approach reduces the likelihood of hallucinations among LLMs.

We already had databases of facts, like Wolfram Alpha, decades before LLM, and we largely ignored them. It's ironic that when trying to solve AI problems we keep reverting to these old patterns we've tried since the 80s and they kept failing. Habits die hard, I guess. There's a categorical difference between knowing a fact, and looking up a fact. When you know a fact you can recognize it in a situation where you woul…

Larger models are not always more accurate. Overbuilding a model often leads to "overfitting" the dataset. A good example: the iphone text prediction model. It now has so much data that the suggested completed words are often useless and irrelevant in context.

Re: Teach your LLM to answer with facts, not fiction

#119

Post by mjburgess »

circuit10 wrote: post #98
jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

“ When an LLM is suggesting what might come next in a piece of text... it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic” LLMs are very good at inferring context, so that only really applies if you’re using an un-RLHFed base model with no context given

Here, "supposed to guess" means "having the goal of..."

So no LLM knows what it's supposed to do. If you prefer, you could say it only ever has one goal: to generate a sequence of tokens which are jointly the most probable to occur along with the prompt tokens, given such probabilities in a historical corpus.

This imitates knowledge, goal-directness, "inferring context" etc. without doing any of those things. Consider what the aim of knowing, goal-directness, inferring , etc. is --- it is never "consistency with a historical text corpus".

For knowing: that beliefs correspond to the way the world is; for goal-directness that one's acts+desires can realise changes; for 'inferring context': that one is sensitive to reasons to speak outside of what is literally spoken.

LLMs are never sensitive to reasons to speak outside of what has been spoken.

Re: Teach your LLM to answer with facts, not fiction

#120

Post by prometheus76 »

quickthrower2 wrote: post #67
lqhl wrote: post #6

Earlier quoted context omitted.

This article suggests that LLMs should use a database as a reference for factual information. Rather than asking LLMs to provide their own answers, it is recommended that they summarize based on the facts extracted from the database. This approach reduces the likelihood of hallucinations among LLMs.

This are assuming LLMs are intelligent and can think "hey I am dumb, I'll look that up". What they are literally doing is guessing the next word, a word a time but doing it really really well and making statistically average output over a very large number of inputs. There is no distinction between understanding "the" vs "a" and telling me 1+1=3. It is all token generation.

It seems to me that LLMs are basically an algorithmic encoding of Occam's Razor. The issue seems to be that what is most probable does not always correspond to what happens, or what makes the most sense to an embodied person.

What is most probable is not always what is most correct or most accurate.

Post reply on HN