Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teach your LLM to answer with facts, not fiction

blog.myscale.com

Post reply on HN
Print view View original

51–60 of 149 posts

Re: Teach your LLM to answer with facts, not fiction

#51

Post by mdp2021 »

NoZebra120vClip wrote: post #39

You know, aside from this being a blatant feature-length advertisement for what they're selling, I almost thought this was a clever idea. I thought it involved prompting the LLM to write SQL code to query a knowledge base of documents, and index into them, so that you'd know where to look in the original documents for your authoritative answer. So it would be a meta-search agent. But apparently, they intend the queri…

> so that you'd know where to look in the original documents

Oh, we have something similar: perplexity.ai

It provides a number of sources after prompting its textual result.

Re: Teach your LLM to answer with facts, not fiction

#52

Post by mdp2021 »

hax0ron3 wrote: post #35

I am mostly a novice to the field of LLMs, but as a layman who has a basic but admittedly very rough understanding of how they work algorithmically, I have a hunch that the same thing that makes these LLMs powerful AIs that have interesting emergent behaviors is also what makes them occasionally get things wildly wrong and claim to know things that they do not know. They are supposed to be AIs, not carefully vetted e…

> They are supposed to be AIs

I.e. synthetic professionals. (Reliable things. Problem solvers.)

Re: Teach your LLM to answer with facts, not fiction

#53

Post by mdp2021 »

inopinatus wrote: post #44

Please don’t dump untreated content marketing in the reading fountain.

And how do you solve the problem of discrimination?

(Oh, what a matter: * all the epistemological debate - hardly a deterministic solution; * the fact that we cannot train a function approximator through supervised learning; * the challenge of unsupervised learning; * the scientific and teleological problem that, if we have an ANN find a solution, what we may want is to go "Ok black box, now teach us how you do it to expand our knowledge (not just our dumb capabilities)...")

Re: Teach your LLM to answer with facts, not fiction

#54

Post by austinjp »

mdp2021 wrote: post #50
jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

> These are all 'factual questions' Because of elision. "[Homer wrote] that Achilles died of an arrow in the heel" This is why the Wiener Kreis taught to use protocolar statements: " and witnessed ".

Tangential: I was going to suggest "protocoli(s|z)ed" instead of protocollar, but I Googled "protocollar statements" just to check and found 2 things. First, this page was the top result! Second, "protocolar" (one ell) and "protocolary" are apparently real words. New to me, thanks.

Re: Teach your LLM to answer with facts, not fiction

#55

Post by immibis »

xyzzy123 wrote: post #45
BugsJustFindMe wrote: post #43

Earlier quoted context omitted.

> When an LLM is suggesting what might come next in a piece of text... it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic. The obvious start seems to be having separate fiction and nonfiction LLMs and not training the nonfiction ones on Archive Of Our Own. People also end up confused about the truth when…

I kinda like this but e.g are research papers fact or fiction? How about an economics textbook, or an article in the economist? "A history of the english speaking peoples" by Winston Churchill? If we restrict to "ground truth we feel very sure about" it feels like available training data might be quite small.

and what if the economics textbook contains "much like Charmander evolves into Charizard, free markets evolve into monopolies"?

Re: Teach your LLM to answer with facts, not fiction

#56

Post by taneq »

jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

Mellon. The rest are left as an exercise to the reader.

It does always amaze me that we trained LLMs on a dump of the internet and then people are shocked that they're about as trustworthy as a random web page.

Re: Teach your LLM to answer with facts, not fiction

#57

Post by anon25783 »

ddmma wrote: post #4
Lerc wrote: post #3

It is not a good start that they begin with a dictionary definition of Hallucinations. While the similarities to what a LLM does are apparent enough for the term to be used, LLMs are under no obligation to behave similar to the dictionary definition of Hallucinations. In general facts are not the answer to Hallucinations. You can't possibly have every fact for every situation. The true solution to Hallucinations is f…

You cannot fix what you cannot measure, here is an attempt to do just that with HallMeter https://why.network/ Still have to figure a measurement unit.

How about "falsehood quotient"? Count the number of counterfactual assertions in a given text, then divide by the number of sentences. Of course, the question of what is a falsehood is an exercise for the reader, but this would at least give a unit of measurement, flawed as it is.

Re: Teach your LLM to answer with facts, not fiction

#58

Post by immibis »

hax0ron3 wrote: post #35

I am mostly a novice to the field of LLMs, but as a layman who has a basic but admittedly very rough understanding of how they work algorithmically, I have a hunch that the same thing that makes these LLMs powerful AIs that have interesting emergent behaviors is also what makes them occasionally get things wildly wrong and claim to know things that they do not know. They are supposed to be AIs, not carefully vetted e…

This is correct. Current LLMs work by predicting the next word based on a bunch of preceding words. In other words, they are autocomplete. You can often form a valid sentence on your phone if you click on any text field and then press the automatic suggestions several times.

Transformer-based LLMs are interesting because they are such good version of autocomplete that they can, for example, complete a news article about scientists discovering unicorns, using just the first sentence (this was one of the first public demonstrations of GPT-2). But fundamentally they are still just auto-complete.

Re: Teach your LLM to answer with facts, not fiction

#59

Post by somenameforme »

BugsJustFindMe wrote: post #43
jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

> When an LLM is suggesting what might come next in a piece of text... it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic. The obvious start seems to be having separate fiction and nonfiction LLMs and not training the nonfiction ones on Archive Of Our Own. People also end up confused about the truth when…

But there's a fundamental issue here. The real strength of LLMs is not just information retrieval, but being able to dynamically recombine that information. Of course that's also their weakness. The reason GPT will regularly produce code with nonexistent API calls is not because it's been trained on 'fictional APIs', but because it's combining various real calls to make new fictional ones.

The obvious answer then is to tell it to make sure that what it's finally outputting is really part of the "real" API, but I think it's safe to say there's some technical hitch there, as it's safe to say OpenAI probably spent quite a lot of energy trying to solve the code hallucinations, and ultimately was unable to do so. I'd guess that the more you restrict its recombination ability, the more you end up with it inappropriately (and incorrectly) just regurgitating large chunks of its training input verbatim. Basically it becomes more like a keyword hunting search engine, and less like a generative LLM.

Re: Teach your LLM to answer with facts, not fiction

#60

Post by mdp2021 »

austinjp wrote: post #54
mdp2021 wrote: post #50

Earlier quoted context omitted.

> These are all 'factual questions' Because of elision. "[Homer wrote] that Achilles died of an arrow in the heel" This is why the Wiener Kreis taught to use protocolar statements: " and witnessed ".

Tangential: I was going to suggest "protocoli(s|z)ed" instead of protocollar, but I Googled "protocollar statements" just to check and found 2 things. First, this page was the top result! Second, "protocolar" (one ell) and "protocolary" are apparently real words. New to me, thanks.

You had me check a few sources for found expressions in use for the concept: you can find simply "protocols" (intending that), "protocol statements", the "protocol-sentence debate", "protocollar propositions"...

Edit: oh, by the way, in case of interest: https://plato.stanford.edu/entries/vienna-circle/

Post reply on HN