Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overcoming the limits of current LLMs

seanpedersen.github.io

Post reply on HN
Print view View original

51–60 of 111 posts

Re: Overcoming the limits of current LLMs

#51

Post by RamblingCTO »

coldtea wrote: post #45
RamblingCTO wrote: post #38

My biggest problem with them is that I can't quite get it to behave like I want it to. I built myself a "therapy/coaching" telegram bot (I'm healthy, but like to reflect a lot, no worries). I even built a self-reflecting memory component that generates insights (sometimes spot on, sometimes random af). But the more I use it, the more I notice that neither the memory nor the prompt matters much. I just can't get it to…

> I just can't get it to behave like a therapist would import time import random SESSION_DURATION = 50 * 60 start_time = time.time() while True: current_time = time.time() elapsed_time = current_time - start_time if elapsed_time >= SESSION_DURATION: print("Our time is up. That will be $150. See you next week!") break _ = input("") print(random.choice(["Mmm hmm", "Tell me more", "How does that make you feel?"])) time.…

haha, good one! although I'm German and it was free for me when I did it. I just had the best therapist. $150 a session is insane!

Re: Overcoming the limits of current LLMs

#52

Post by Terr_ »

idle_zealot wrote: post #49
Terr_ wrote: post #36

Earlier quoted context omitted.

Yeah, you can train an LLM to recognize the vocabulary and grammatical features of logical fallacies... Except the nature of fallacies is that they look real on that same linguistic level, so those features aren't distinctive for that purpose. Heck, I think detecting sarcasm would be an easier goal, and still tricky.

> Except the nature of fallacies is that they look real on that same linguistic level, so those features aren't distinctive for that purpose Well that's actually good news. With a large enough labelled dataset of actually-sound and fallacious text with similar grammatical features you should be able to train a discriminator to distinguish between them using some other metric. Good luck with getting that data set thou…

> you should be able to train a discriminator to distinguish between them using some other metric

Not when the better metrics are likely alien/incompatible to the discriminator's core algorithm!

Then it's rather inconvenient news, because it means you have to develop something separate and novel.

As the other poster already mentioned, if we can't even get them to reliably count how many objects are being referred to, how do you expect them to also handle logical syllogisms?

Re: Overcoming the limits of current LLMs

#53

Post by tidenly »

wokwokwok wrote: post #17

Does anyone really believe that having a good corpus will remove hallucinations? Is this article even written by a person? Hard to know; they have a real blog with real article, but stuff like this reads strangely. Maybe it's just not a native english speaker? > Hallucinations are certainly the toughest nut to crack and their negative impact is basically only slightly lessened by good confidence estimates and reliabl…

I wonder to what extent is hallucination a result of a "must answer" bias?

When sampling data all over the internet, your data set only represents people who did write, did respond to questions - with no representation of what they didn't. Add into that confidently wrong people - people who respond to questions on, say, StackOverflow, even if they're wrong, and suddenly you have a data set that prefers replying bullshit, because there's no data for the people who didnt know the answer and wrote nothing.

Inherently there's no representation in the datasets of "I don't know" null values.

LLMs are forced to reply, in contrast, so they "bullshit" a response that sounds right even though not answering or saying you don't know would be more appropriate - because no-one does that on the internet.

I always assumed this was a big factor, but am I completely off the mark?

Re: Overcoming the limits of current LLMs

#54

Post by sean_pedersen »

wokwokwok wrote: post #17

Does anyone really believe that having a good corpus will remove hallucinations? Is this article even written by a person? Hard to know; they have a real blog with real article, but stuff like this reads strangely. Maybe it's just not a native english speaker? > Hallucinations are certainly the toughest nut to crack and their negative impact is basically only slightly lessened by good confidence estimates and reliabl…

I wrote up this blog post in 30 mins, that's why it reads a little rough. I could not find explicit research on the impact of contradicting training data, only on the general need for high-quality training data.

May be it is a pipe dream to drastically improve on hallucinations by curating a self-consistent data set but I am still interested in how much it actually impacts the quality of the final model.

I described one possible way to create such a self-consistent data set in this very blog post.

Re: Overcoming the limits of current LLMs

#55

Post by MattPalmer1086 »

linguistbreaker wrote: post #9
jwuphysics wrote: post #5

Earlier quoted context omitted.

> Why not just say "incoherent output"? Because the biggest problem with hallucinations is that the output is usually coherent but factually incorrect. I agree that "hallucination" isn't the best word for it... perhaps something like "confabulation" is better.

I appreciated a post on here recently that likened AI hallucination to 'bullshitting'. It's coherent, even plausible output without any regard for the truth.

More true to say that all output is bullshitting, not just the ones we call hallucinations. Some of it is true, some isn't. The model doesn't know or care.

Re: Overcoming the limits of current LLMs

#56

Post by 343rwerfd »

ainoobler wrote: post #6

The article suggests a useful line of research. Train an LLM to detect logical fallacies and then see if that can be bootstrapped into something useful because it's pretty clear that all the issues with LLMs is the lack of logical capabilities. If an LLM was capable of logical reasoning then it would be obvious when it was generating made-up nonsense instead of referencing existing sources of consistent information.

> If an LLM was capable of logical reasoning

the prompt interfaces + smartphone apps were (from the beginning), and are ongoing training for the next iteration, they provide massive RLHF for further improvements in already quite RLHFed advanced models.

Whatever tokens they're extracting from all the interactions, the most valuable are those from metadata, like "correct answer in one shot", or "correct answer in three shots".

The inputs and potentially the outputs can be gibberish, but the metadata can be mostly accurate given some implicit/explicit (the tumbs up, the "thanks" answers from users, maybe), human feedback.

The RLHF refinement extracted from getting the models face the entire human population for to be continuously, 24x7x365, prompted in all languages, about all the topics interesting for the human society, must be incredible. If you just can extract a single percentage of definitely "correct answers" from the total prompts answered, it should be massive compared to just a few thousands of QA dedicated RLHF people working on the models in the initial iterations of training.

That was GPT2,3,4, initial iterations of the training. Having the models been evolved to more powerful (mathematical) entities, you can use them to train the next models. Like is almost certainly happening.

My bet is that one of two

- The scaling thing is working spectacularly, they've seen linear improvement in blue/green deployments across the world + realtime RLHF, and maybe it is going a bit slow, but the improvements justify just a bit more waiting to get trained a more powerful,refined model, incredible more better answers from even the previous datasets used (now more deeply inquired by the new models and the new massive RLHF data), if in a year they have a 20x GPT4, Claude, Gemini, whatever, they could be "jumping" to the next 40x GPT4, Claude, Gemini, a lot faster, if they have the most popular, prompted model in the market (in the world).

- The scaling stuff already sunk, they have seen the numbers and it doesn't add by now, or they've seen disminished returns coming. This is being firmly denied by anyone on the record or off the record.

Re: Overcoming the limits of current LLMs

#57

Post by linguistbreaker »

exmadscientist wrote: post #18
darby_nine wrote: post #4

Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is with perception rather than structural incompetence. Why not just say "incoherent output"? It's actually descriptive and doesn't require bastardizing a word we already find meaningful to mean something completely different.

"Hallucinations" implies that someone isn't of sound mental state. We can argue forever about what that means for a LLM and whether that's appropriate, but I think it's absolutely the right attitude and approach to be taking toward these things. They simply do not behave like humans of sound minds, and "hallucinations" conveys that in a way that "confabulations" or even "bullshit" does not. (Though "bullshit" isn't b…

How about "dream-reality confusion (DRC)" ?

Re: Overcoming the limits of current LLMs

#59

Post by nyrikki »

idle_zealot wrote: post #49
Terr_ wrote: post #36

Earlier quoted context omitted.

Yeah, you can train an LLM to recognize the vocabulary and grammatical features of logical fallacies... Except the nature of fallacies is that they look real on that same linguistic level, so those features aren't distinctive for that purpose. Heck, I think detecting sarcasm would be an easier goal, and still tricky.

> Except the nature of fallacies is that they look real on that same linguistic level, so those features aren't distinctive for that purpose Well that's actually good news. With a large enough labelled dataset of actually-sound and fallacious text with similar grammatical features you should be able to train a discriminator to distinguish between them using some other metric. Good luck with getting that data set thou…

The Entscheidungsproblem tends to rear its ugly problem

Remember NP is equivalent to second order logic with existential quantified. E.g. for any X there exists a Y

And that only gets you to truthy Trues, co-NP is another problem.

ATP is hard, and while we get lucky with some constrained problems like type inference, which is pathological in its runtime, but decidable, Pressburger arithmetic is the highest form we know is decidable.

It is a large reason CS uses science and falsification vs proofs.

Gödel and the difference between Symantec and syntactic completeness is another rat hole.

Re: Overcoming the limits of current LLMs

#60

Post by nyrikki »

> ...manually curate a high-quality (consistent) text corpus based on undisputed, well curated wikipedia articles and battle tested scientific literature.

This assumption is based on the mistaken assumption that science is about objective truth.

It is confusing the map for the territory. Scientific models are intended to be useful, not perfect.

Statistical learning, vs symbolic learning is about existential quantification vs universal quantification respectively.

All models are wrong some are useful, this applies to even the most unreasonably accurate versions like QFT and GR.

Spherical cows, no matter how useful are hotly debated outside of the didactic half truths of low level courses.

The corpus that the above seeks doesn't exist in academic circles, only in popular science where people don't see that practical, useful models are far more important that 'correct' ones.

Post reply on HN