If “allowing the execution of arbitrary database queries written by an LLM inside a SaaS application” is the answer, I’d love to know what the question is.
Teach your LLM to answer with facts, not fiction
21–30 of 149 posts
Re: Teach your LLM to answer with facts, not fiction
#22'Facts' aren't as black and white as people think.
"What does Charmander evolve into?"
"What does the spell 'avada kedavra' do?"
"What is the Sindarin word for 'friend'?"
"What are the names of Santa's reindeer?"
"Where did Robin Hood live?"
"Where did Achilles die?"
It's no surprise that LLMs trained on human writings produce text which claims things that aren't true are facts. Humans do that all the time.
There are well attested reputable sources that will tell you Abraham Lincoln was a vampire hunter, others that say he was a Lego Master Builder, and others still will tell you that among his notable quotes is "Party on dudes - be excellent to each other". So what's an LLM to do when it's trying to extend a paragraph of information about Abraham Lincoln?
When an LLM is suggesting what might come next in a piece of text... it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic. So you get a bit of all that.
Re: Teach your LLM to answer with facts, not fiction
#23Situation: people try to use these predictive text chatbots as search engines. Problem: LLMs are not search engines. They extrapolate, interpolate, and approximate (so-called “hallucinations”) so they can always produce somewhat-plausible text completions. Solution: Create a search engine so good at returning relevant results that even an LLM can make use of it… then go to significant lengths to plug that search engi…
I’m sure you’ve heard of Google-fu. Someone is better at searching than others. I believe they propose that LLM can be superior at producing search queries. Whether they succeed, that remains to be seen, but the idea isn’t that baffling.
Re: Teach your LLM to answer with facts, not fiction
#24Earlier quoted context omitted.
Well, search engines and my ability to query them aren't good enough for highly specific or poorly worded questions yet. LLMs are sometimes better in this space.
Indeed, a frequent issue with search engines is knowing the right terms plug into them, particularly when researching a topic beyond one's scope of knowledge. Half the power of LLMs as they currently exist is that they can often extract the intention of the user's question in a way that search engines usually can't, allowing them to provide a more useful answer or at least point the user in the right direction. Perha…
> Half the power of LLMs as they currently exist is that they can often extract the intention of the user's question in a way that search engines usually can't
I don’t know about that. Google is pretty good at including “similar” questions that others have asked to what I have queried and often that’s exactly what I needed.
Re: Teach your LLM to answer with facts, not fiction
#25If “allowing the execution of arbitrary database queries written by an LLM inside a SaaS application” is the answer, I’d love to know what the question is.
The question is how to make money from LLM hype.
Re: Teach your LLM to answer with facts, not fiction
#26Earlier quoted context omitted.
Indeed, a frequent issue with search engines is knowing the right terms plug into them, particularly when researching a topic beyond one's scope of knowledge. Half the power of LLMs as they currently exist is that they can often extract the intention of the user's question in a way that search engines usually can't, allowing them to provide a more useful answer or at least point the user in the right direction. Perha…
> Half the power of LLMs as they currently exist is that they can often extract the intention of the user's question in a way that search engines usually can't I don’t know about that. Google is pretty good at including “similar” questions that others have asked to what I have queried and often that’s exactly what I needed.
In my experience, Google's ability to suggest "similar" queries is often limited if I don't know the terminology associated with the subject in question. It's decent if you're already in the ballpark, but to extend the analogy if you're stuck trying to figure out where the park is in the first place it's much more hit or miss.
Re: Teach your LLM to answer with facts, not fiction
#27One of the fun things I like to do with these examples is follow along at home. I don't have access to GPT-4 because I refuse to give money to OpenAI, but with GPT-3.5, "What is an LLM hallucination" actually gives: - The usual knowledge cutoff warning - An explanation of what a hallucination is If (in a separate conversation) I give GPT-3.5 the exact prompt that explains what an LLM is, I get gaslighted instead. GPT…
LLM, confusingly, does stand for legal master of laws, despite the acronym not quite fitting. It's an advanced law degree some lawyers get. ChatGPT isn't gaslighting you! That's a true fact.
https://law.pepperdine.edu/blog/posts/llm-versus-jd-degree.h...
Re: Teach your LLM to answer with facts, not fiction
#28'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…
[deleted]
Re: Teach your LLM to answer with facts, not fiction
#29One of the fun things I like to do with these examples is follow along at home. I don't have access to GPT-4 because I refuse to give money to OpenAI, but with GPT-3.5, "What is an LLM hallucination" actually gives: - The usual knowledge cutoff warning - An explanation of what a hallucination is If (in a separate conversation) I give GPT-3.5 the exact prompt that explains what an LLM is, I get gaslighted instead. GPT…
As jwells89 mentioned, LLMs can extract the intention from questions and generate better queries for a search engine or database.
I don't need LLM to assume my intention, I need exact matches for keywords.
Re: Teach your LLM to answer with facts, not fiction
#30It is not a good start that they begin with a dictionary definition of Hallucinations. While the similarities to what a LLM does are apparent enough for the term to be used, LLMs are under no obligation to behave similar to the dictionary definition of Hallucinations. In general facts are not the answer to Hallucinations. You can't possibly have every fact for every situation. The true solution to Hallucinations is f…
"Hallucination" makes it sound like ChatGPT drank some of the punch without realizing it was laced with LSD. "Bullshit" sounds more like what comes out of an overconfident ass who should or could know better with some better education.