Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teach your LLM to answer with facts, not fiction

blog.myscale.com

Post reply on HN
Print view View original

91–100 of 149 posts

Re: Teach your LLM to answer with facts, not fiction

#91

Post by astrange »

BugsJustFindMe wrote: post #43
jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

> When an LLM is suggesting what might come next in a piece of text... it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic. The obvious start seems to be having separate fiction and nonfiction LLMs and not training the nonfiction ones on Archive Of Our Own. People also end up confused about the truth when…

Rather than leaving some text out, it'd be better to label it with its source when training.

Re: Teach your LLM to answer with facts, not fiction

#92

Post by astrange »

quickthrower2 wrote: post #67
lqhl wrote: post #6

Earlier quoted context omitted.

This article suggests that LLMs should use a database as a reference for factual information. Rather than asking LLMs to provide their own answers, it is recommended that they summarize based on the facts extracted from the database. This approach reduces the likelihood of hallucinations among LLMs.

This are assuming LLMs are intelligent and can think "hey I am dumb, I'll look that up". What they are literally doing is guessing the next word, a word a time but doing it really really well and making statistically average output over a very large number of inputs. There is no distinction between understanding "the" vs "a" and telling me 1+1=3. It is all token generation.

What they are doing depends entirely on what decoding algorithm you use. An LLM is mostly a token probability function, but it's not just that - a transformer model is capable of learning anything. Tokens are the interface, not necessarily the implementation.

Re: Teach your LLM to answer with facts, not fiction

#93

Post by astrange »

flanked-evergl wrote: post #69

I think the author of that title could well do with a refresher course in epistemology and physics, as it is just not possible to do what they suggest. But even more unfortunate is how many people fall for deceptive marketing that really should not even fool the average 16-year-old.

> I think the author of that title could well do with a refresher course in epistemology and physics, as it is just not possible to do what they suggest.

If you define "facts" as "things actually stored in the LLM's weights", then research shows it is possible to determine if an output is a "fact" or not.

Although looking on arxiv I found a paper saying it doesn't work (https://arxiv.org/pdf/2307.00175.pdf) so maybe not.

Re: Teach your LLM to answer with facts, not fiction

#94

Post by astrange »

immibis wrote: post #58
hax0ron3 wrote: post #35

I am mostly a novice to the field of LLMs, but as a layman who has a basic but admittedly very rough understanding of how they work algorithmically, I have a hunch that the same thing that makes these LLMs powerful AIs that have interesting emergent behaviors is also what makes them occasionally get things wildly wrong and claim to know things that they do not know. They are supposed to be AIs, not carefully vetted e…

This is correct. Current LLMs work by predicting the next word based on a bunch of preceding words. In other words, they are autocomplete. You can often form a valid sentence on your phone if you click on any text field and then press the automatic suggestions several times. Transformer-based LLMs are interesting because they are such good version of autocomplete that they can, for example, complete a news article ab…

LLM chat models are not autocomplete. They can recognize and respond to user text, which is not the same thing as completing it. If you prompted GPT-2 with a question you'd get another question, not an answer.

Re: Teach your LLM to answer with facts, not fiction

#96

Post by d1sxeyes »

"The Hitch Hiker’s Guide to the Galaxy; is an indispensable companion to all those who are keen to make sense of life in an infinitely complex and confusing universe. For though it cannot hope to be useful or informative on all matters, it does make the reassuring claim that where it is inaccurate, it is at least definitively inaccurate. In cases of major discrepancy it is always reality that’s got it wrong. So, for instance, when the Guide was sued by the families of those who had died as a result of taking the entry on the planet Traal literally - it said “Ravenous Bugblatter Beasts often make a very good meal for visiting tourists” instead of “Ravenous Bugblatter Beasts often make a very good meal of visiting tourists” - the editors claimed that the first version of the sentence was the more aesthetically pleasing; summoned a qualified poet to testify under oath that beauty was truth, truth beauty, and hoped thereby to prove that the guilty party in this case was life itself for failing to be either beautiful or true."

Re: Teach your LLM to answer with facts, not fiction

#97

Post by ddmma »

janalsncm wrote: post #62
anon25783 wrote: post #57

Earlier quoted context omitted.

How about "falsehood quotient"? Count the number of counterfactual assertions in a given text, then divide by the number of sentences. Of course, the question of what is a falsehood is an exercise for the reader, but this would at least give a unit of measurement, flawed as it is.

The “exercise for the reader” in this case is the entire point of the metric. If it were possible to do at scale it could be incorporated into existing models right now. Second, it isn’t even necessarily better to have fewer lies if those few lies are more subtle. Plenty of propaganda works by twisting facts and using misleading statements. Perhaps the worst offenders won’t even have any outright falsehoods at all.

that is very interesting input, thanks! Also split into sentences, similar to what sentiment analytics does would raise the level of complexity into the output metric.

Re: Teach your LLM to answer with facts, not fiction

#98

Post by circuit10 »

jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

“ When an LLM is suggesting what might come next in a piece of text... it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic”

LLMs are very good at inferring context, so that only really applies if you’re using an un-RLHFed base model with no context given

Re: Teach your LLM to answer with facts, not fiction

#99

Post by zirgs »

wmf wrote: post #25
peterbonney wrote: post #21

If “allowing the execution of arbitrary database queries written by an LLM inside a SaaS application” is the answer, I’d love to know what the question is.

The question is how to make money from LLM hype.

You do that the same way as making money from the blockchain hype. You don't actually use the blockchain - just say that you do.

Re: Teach your LLM to answer with facts, not fiction

#100

Post by zirgs »

wizzwizz4 wrote: post #2

Situation: people try to use these predictive text chatbots as search engines. Problem: LLMs are not search engines. They extrapolate, interpolate, and approximate (so-called “hallucinations”) so they can always produce somewhat-plausible text completions. Solution: Create a search engine so good at returning relevant results that even an LLM can make use of it… then go to significant lengths to plug that search engi…

Why not show the "raw" search results as well?
Post reply on HN