Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teach your LLM to answer with facts, not fiction

blog.myscale.com

Post reply on HN
Print view View original

31–40 of 149 posts

Re: Teach your LLM to answer with facts, not fiction

#31

Post by npsomaratna »

lqhl wrote: post #15
teej wrote: post #8

Earlier quoted context omitted.

> This approach reduces the likelihood of hallucinations among LLMs. This has not been my experience. Did you create any benchmarks as a part of this project?

This idea is a simplified version of Retrieval-Augmented Generation (RAG), and RAG has been studied in various research papers, such as the one available at https://arxiv.org/abs/2005.11401

My experience with RAG is that while it reduces the incidence of hallucinations* significantly (especially if you reduce the LLM temperature to zero at the same time), it doesn't eliminate them.

My startup has a product for lawyers that uses RAG to answer legal queries (https://lawlight.ai/). We have a disclaimer that "... (we) do not guarantee the accuracy of answers. You are responsible for reviewing the cited case law and drawing your own independent conclusions."

(This works within the specific context—lawyers are domain experts; and they are supposed to read through all cases they cite in court anyway.)

* I dislike the term "hallucinations." By definition LLMs hallucinate. It's just that much (or most) of the time, the hallucinations reflect reality.

Re: Teach your LLM to answer with facts, not fiction

#32

Post by 3cats-in-a-coat »

lqhl wrote: post #6
Lerc wrote: post #3

It is not a good start that they begin with a dictionary definition of Hallucinations. While the similarities to what a LLM does are apparent enough for the term to be used, LLMs are under no obligation to behave similar to the dictionary definition of Hallucinations. In general facts are not the answer to Hallucinations. You can't possibly have every fact for every situation. The true solution to Hallucinations is f…

This article suggests that LLMs should use a database as a reference for factual information. Rather than asking LLMs to provide their own answers, it is recommended that they summarize based on the facts extracted from the database. This approach reduces the likelihood of hallucinations among LLMs.

We already had databases of facts, like Wolfram Alpha, decades before LLM, and we largely ignored them. It's ironic that when trying to solve AI problems we keep reverting to these old patterns we've tried since the 80s and they kept failing. Habits die hard, I guess.

There's a categorical difference between knowing a fact, and looking up a fact. When you know a fact you can recognize it in a situation where you wouldn't know to look it up, and you'd know to utilize it in a larger solution rather than simply parrot it when specifically asked about it.

Databases of facts have and will still have their place, but that is absolutely not the solution to LLM telling apart fact from truth. They have to innately have this in their model. I don't believe the nature of LLM is to hallucinate. It's instead a side effect of how we train them. We train them to guess, to be close, but not to be correct necessarily. And why is it a surprise that's precisely what they do?

Also LLM are too small in order to be accurate. They're tiny. GPT4 is roughly 40 times smaller than a human brain. And GPT4 is very large compared to GPT-3, and GPT-3 is very large compared to LLaMA 2.

We'll need for hardware to catch up so we can scale things up pragmatically and see what happens to their ability to grasp facts. But also architectural changes, of course.

Re: Teach your LLM to answer with facts, not fiction

#33

Post by delecti »

s1mon wrote: post #30
Lerc wrote: post #3

It is not a good start that they begin with a dictionary definition of Hallucinations. While the similarities to what a LLM does are apparent enough for the term to be used, LLMs are under no obligation to behave similar to the dictionary definition of Hallucinations. In general facts are not the answer to Hallucinations. You can't possibly have every fact for every situation. The true solution to Hallucinations is f…

"Hallucination" makes it sound like ChatGPT drank some of the punch without realizing it was laced with LSD. "Bullshit" sounds more like what comes out of an overconfident ass who should or could know better with some better education.

Hallucination is a better descriptor for what an LLM is doing though. A bullshitter knows they don't know, an LLM just strings words together in ways that fit what it "saw" from training data. IMO the main problem with calling them hallucinations is the implication that the true things they say are true on purpose. It's hallucinating the true things too.

Re: Teach your LLM to answer with facts, not fiction

#34

Post by skybrian »

That will help a bit, but it's not going to fix it.

Using GPT4 and Code Interpreter, I have asked it to write a function and test it, given some inputs and expected outputs. The function returned different values when it tested it, but it lied and said it worked as expected.

You need to read the code and the test outputs yourself. Or maybe have it write an automated test?

Despite this, it seems quite promising. I expect that in a year or two, some IDE's will come with a useful pair programming feature.

Re: Teach your LLM to answer with facts, not fiction

#35

Post by hax0ron3 »

I am mostly a novice to the field of LLMs, but as a layman who has a basic but admittedly very rough understanding of how they work algorithmically, I have a hunch that the same thing that makes these LLMs powerful AIs that have interesting emergent behaviors is also what makes them occasionally get things wildly wrong and claim to know things that they do not know. They are supposed to be AIs, not carefully vetted encyclopedias. Sure, I get that people want to eventually use AI to do life-critical stuff like surgery, and at that point "hallucinations" become a real problem. But we are nowhere close to that point yet, I think, so I feel that the focus on "hallucinations" may be misleading. It is one thing to try to get a 30 year old doctor to not make nonsense up on the fly while at work, that makes sense. But if you try to prevent a 3 year old kid from making up nonsense, that will actually probably hurt his development into a more powerful intelligence. Note: I know that the current popular LLMs do not actually learn past the scope of a single session, but I am sure that they soon will.

Re: Teach your LLM to answer with facts, not fiction

#36

Post by ec109685 »

skybrian wrote: post #34

That will help a bit, but it's not going to fix it. Using GPT4 and Code Interpreter, I have asked it to write a function and test it, given some inputs and expected outputs. The function returned different values when it tested it, but it lied and said it worked as expected. You need to read the code and the test outputs yourself. Or maybe have it write an automated test? Despite this, it seems quite promising. I exp…

With code interpreter, it actually runs the code, so how did failing tests get interpreted as being correct?

Re: Teach your LLM to answer with facts, not fiction

#37

Post by CSMastermind »

omeze wrote: post #7
wizzwizz4 wrote: post #2

Situation: people try to use these predictive text chatbots as search engines. Problem: LLMs are not search engines. They extrapolate, interpolate, and approximate (so-called “hallucinations”) so they can always produce somewhat-plausible text completions. Solution: Create a search engine so good at returning relevant results that even an LLM can make use of it… then go to significant lengths to plug that search engi…

Yea… my opinion on this is that startups are attempting to force a market for chat bots, instead of accepting that LLM embeddings are best utilized as a search feature, not new product surface area

IMO the most obvious place LLMs will be used is for interfaces.

They enable voice-based interfaces to be practical for normal users for the first time since you really can talk to them in a convincing way.

Translating user input into a set of well-defined commands seems like a better use than searching data to me.

Re: Teach your LLM to answer with facts, not fiction

#38

Post by worrycue »

jwells89 wrote: post #26
worrycue wrote: post #24

Earlier quoted context omitted.

> Half the power of LLMs as they currently exist is that they can often extract the intention of the user's question in a way that search engines usually can't I don’t know about that. Google is pretty good at including “similar” questions that others have asked to what I have queried and often that’s exactly what I needed.

In my experience, Google's ability to suggest "similar" queries is often limited if I don't know the terminology associated with the subject in question. It's decent if you're already in the ballpark, but to extend the analogy if you're stuck trying to figure out where the park is in the first place it's much more hit or miss.

I “fish” with google when I don’t know the terminology is what I’m saying. With some luck Google will have correct question on the first page of the search results.

Re: Teach your LLM to answer with facts, not fiction

#39

Post by NoZebra120vClip »

You know, aside from this being a blatant feature-length advertisement for what they're selling, I almost thought this was a clever idea.

I thought it involved prompting the LLM to write SQL code to query a knowledge base of documents, and index into them, so that you'd know where to look in the original documents for your authoritative answer. So it would be a meta-search agent.

But apparently, they intend the queried documents to feed back into training the LLM? That's just gasoline on a dumpster fire.

Post reply on HN