Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teach your LLM to answer with facts, not fiction

blog.myscale.com

Post reply on HN
Print view View original

141–149 of 149 posts

Re: Teach your LLM to answer with facts, not fiction

#141

Post by Ajedi32 »

mjburgess wrote: post #135
Ajedi32 wrote: post #129

Earlier quoted context omitted.

What does RLHF do then? I feel like you completely ignored the central point of GP's comment. RLHF is the difference between GPT-3.5 and ChatGPT, and it's the whole reason why LLMs are suddenly such a big deal. ChatGPT demonstrated that it's possible to give language models a goal beyond just "complete most likely next word" and that they can actually be somewhat competent at achieving those goals despite not being e…

> competent at achieving those goals despite not being explicitly trained for them. Well (1) it doesn't achieve goals, since a "goal" is observer-relative. We have goals, the LLM has a formal optimisation objective which gives it the appearence of goal-directed behaviour (in a similar way, eg., that it appears pens want to fall when dropped). And (2), reading your "goal" here even in observer-relative ways, I don't t…

(1) This is a tired, pointless semantic argument. "It doesn't have a goal, it just acts like it has a goal for all intents and purposes. But, you see, it's actually a machine and not a human and therefore it can't really have goals according to my narrow definition of the term." Either point to an actually relevant difference in the resulting behavior or stop objecting when people use human behavioral terms to describe the behavior of machine learning systems. We're all well aware it's a program; that's not the point. (Sorry, just a frustration I have with the larger discussion around this topic.)

(2) "I don't see any evidence of much generalisation" Seriously? So when I tell ChatGPT to rewrite a paragraph in the style of Shakespeare and it does it, despite never being trained to do that, never seeing the source or target paragraph before, and having no information other than my text prompt and its past training, that's not evidence of generalization? And that's only one of millions of different possible tasks that the same model excels at, despite being trained on nothing but a bunch of unstructured text and a few examples indicating its goal should be to follow instructions given in the prompt text. Up until a couple years ago this level of flexibility in a machine learning model would have been considered science fiction by nearly everyone, and now it's "[not] evidence of much generalization". Okay.

Re: Teach your LLM to answer with facts, not fiction

#142

Post by mjburgess »

Ajedi32 wrote: post #141
mjburgess wrote: post #135

Earlier quoted context omitted.

> competent at achieving those goals despite not being explicitly trained for them. Well (1) it doesn't achieve goals, since a "goal" is observer-relative. We have goals, the LLM has a formal optimisation objective which gives it the appearence of goal-directed behaviour (in a similar way, eg., that it appears pens want to fall when dropped). And (2), reading your "goal" here even in observer-relative ways, I don't t…

(1) This is a tired, pointless semantic argument. "It doesn't have a goal, it just acts like it has a goal for all intents and purposes. But, you see, it's actually a machine and not a human and therefore it can't really have goals according to my narrow definition of the term." Either point to an actually relevant difference in the resulting behavior or stop objecting when people use human behavioral terms to descri…

Well (1), the reason this distinction is relevant is so we can separate out whether the system has developed a capacity or an apparent capacity.

Is the child a genius or are they just reading out of a textbook? Can the toddler really compose a sonata or did they just press play on the piano keyboard?

(2) This is indeed the power of interpolating between the data points of "everything ever written in human history" as digitised and compressed by ChatGPT.

If you have 1 billion circles of radii 0 to 1, it isn't generalisation for the machine to produce one with a radii 0.0000100003000001, ie., one not in the set but a mere interpolation of points within it.

It would be expensive, but imagining "reversing" ChatGPT from it's output to the sources which made a non-trivial difference to generating that output.

So the function there is: response -> verbatim text in the training corpus.

Then, maybe, "bolded" by how much each paragraph would "make a difference" to its output.

What you'd find is thousands of pages: all Shakespeare ever written, all papers about Shakespeare, all books about Shakespeare; and so on.

Then when it applied the bolding, and summarising it a little, the trick would be revealed: it would be apparent how a naive statistical interpolation between sequences of characters could produce the effect.

ChatGPT exists because of ebooks and social media: without it, it could do almost nothing. That is, the appearance of these capacities is strictly derivative of the work of a billion people who had them.

Without vast, unimaginable, amounts of work produced on Shakespeare this system wouldnt work. It's just a copyright laundering system. All the school essays on reddit, all the forum posts; all of usenet. All pdfs, all digitised works. All academic papers.

Is this generalisation? Is this a system which starts with little and makes a lot?

Or is it a system which is more like a child reading from a textbook? Ie., making a haphazard ability to repeat what's already written.

The size of the weights of a modern LLM are sufficient to compress everything ever written in human history: and that's exactly what they do.

Re: Teach your LLM to answer with facts, not fiction

#143

Post by mdp2021 »

mdp2021 wrote: post #53
inopinatus wrote: post #44

Please don’t dump untreated content marketing in the reading fountain.

And how do you solve the problem of discrimination? (Oh, what a matter: * all the epistemological debate - hardly a deterministic solution; * the fact that we cannot train a function approximator through supervised learning; * the challenge of unsupervised learning; * the scientific and teleological problem that, if we have an ANN find a solution, what we may want is to go "Ok black box, now teach us how you do it to…

Update: I will clarify what I have written above as I realize it is unclear.

You cannot solve the problem of discrimination (in non trivial cases of true and false, of good and bad) through a deterministic solution, as the epistemological debate did not solve the general problem. You cannot train a function approximator (e.g. an ANN) as a Discriminator through supervised learning, because the problem remains such for human judgement. Creating a Discriminator through unsupervised learning, I'd like to see how one would frame a proposal; and anyway, if we could create reliable filters, the main question - as usual for a progressive approach to AI - would be to have the oracle in the system teach us instead that knowledge that what could not achieve with good old thinking.

Re: Teach your LLM to answer with facts, not fiction

#144

Post by Ajedi32 »

mjburgess wrote: post #142
Ajedi32 wrote: post #141

Earlier quoted context omitted.

(1) This is a tired, pointless semantic argument. "It doesn't have a goal, it just acts like it has a goal for all intents and purposes. But, you see, it's actually a machine and not a human and therefore it can't really have goals according to my narrow definition of the term." Either point to an actually relevant difference in the resulting behavior or stop objecting when people use human behavioral terms to descri…

Well (1), the reason this distinction is relevant is so we can separate out whether the system has developed a capacity or an apparent capacity. Is the child a genius or are they just reading out of a textbook? Can the toddler really compose a sonata or did they just press play on the piano keyboard? (2) This is indeed the power of interpolating between the data points of "everything ever written in human history" as…

It isn't apparent that anything you've just described is relevant. You've described how it works (in a highly simplified way), but that doesn't discredit the end result.

If there's truly a difference between "a capacity [and] an apparent capacity" then you should be able to point out what that difference actually is in practice. A child pressing play on a piano can only play one song. A LLM composing poems can compose billions upon billions of unique, never-before-seen poems about every conceivable topic. Whether under the hood it does that by "interpolating numbers in n-dimensional spaces" or "some incomprehensible arrangement of neurons linked together" or some other, yet to be invented process doesn't matter if the result is the same. The fact that you can explain how something works doesn't make it less real.

Re: Teach your LLM to answer with facts, not fiction

#145

Post by QuantumGood »

jameshart wrote: post #22

'Facts' aren't as black and white as people think. "What does Charmander evolve into?" "What does the spell 'avada kedavra' do?" "What is the Sindarin word for 'friend'?" "What are the names of Santa's reindeer?" "Where did Robin Hood live?" "Where did Achilles die?" These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfr…

I'm not sure a higher level "intelligence" (which some folks think AI is moving towards) should be overly-reliant on human "intelligence", lest it inherit flaws which may outnumber benefits. (Humans believe a variety of outlandish things, such as "Q-Anon has the real facts", etc.)

Re: Teach your LLM to answer with facts, not fiction

#146

Post by Grimblewald »

masukomi wrote: post #109

I really REALLY wish people would stop assigning sentience to LLMs. > In other words, a hallucination is an error in (or a false) perception of something real or concrete. an llm has no "perception" it doesn't "believe" or "think" that the answers it provides are "correct" or "true" or even "false". It's just autocompleting strings with the most probably next words. If we keep treating these things as if they're sent…

Not arguing the sentience bit, we don't know what that is, have a definition for it, or can even agree on what that might generally be.

That being said, what you've described is different to how a human first learns and many never grow beyond in what way?

Re: Teach your LLM to answer with facts, not fiction

#147

Post by vouaobrasil »

mkl wrote: post #124
vouaobrasil wrote: post #5

I believe that LLMs should be banned, but if they have to exist, we should teach them ethics first before anything else.

Do you think your phone keyboard's predictive text should be taught ethics? How? LLMs are just predictive text scaled way up: they don't know or think anything, they just predict the next word repeatedly. They can't learn ethics, but can learn to string words together into sentences about ethics, again just by predicting the next word.

I don't use predictive text on my phone.

Re: Teach your LLM to answer with facts, not fiction

#148

Post by BugsJustFindMe »

immibis wrote: post #55
xyzzy123 wrote: post #45

Earlier quoted context omitted.

I kinda like this but e.g are research papers fact or fiction? How about an economics textbook, or an article in the economist? "A history of the english speaking peoples" by Winston Churchill? If we restrict to "ground truth we feel very sure about" it feels like available training data might be quite small.

and what if the economics textbook contains "much like Charmander evolves into Charizard, free markets evolve into monopolies"?

Hopefully such statements are sufficiently rare that they don't get reinforced, I guess. I don't know. A very real problem occurs with people too when fictional things are repeated often enough without direct mention of their fictional nature.

Re: Teach your LLM to answer with facts, not fiction

#149

Post by CharlieDigital »

yifanl wrote: post #136
tyre wrote: post #125

Earlier quoted context omitted.

> What does Charmander evolve into? wait why is this implied to not be black and white? Charmeleon is the only correct answer.

Charmanders don't evolve into anything, it doesn't exist in the natural world.

By this logic, "Is Moby Dick a sperm whale" also can't be answered factually because Moby Dick is a fictional creation and doesn't exist in the natural world?
Post reply on HN