Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overcoming the limits of current LLMs

seanpedersen.github.io

Post reply on HN
Print view View original

41–50 of 111 posts

Re: Overcoming the limits of current LLMs

#41

Post by js8 »

lsy wrote: post #31

We can't develop a universally coherent data set because what we understand as "truth" is so intensely contextual that we can't hope to cover the amount of context needed to make the things work how we want, not to mention the numerous social situations where writing factual statements would be awkward or disastrous. Here are a few examples of statements that are not "factual" in the sense of being derivable from a u…

You're right. We don't really know how to handle uncertainty and fuzziness in logic properly (to avoid logical contradictions). There has been many mathematical attempts to model uncertainty (just to name a few - probability, Dempster-Shafer theory, fuzzy logic, non-monotone logics, etc.), but they all suffer from some kind of paradox.

At the end of the day, none of these theoretical techniques prevailed in the field of AI, and we ended up with, empirically successful, neural networks (and LLMs specifically). We know they model uncertainty but we have no clue how they do it conceptually, or whether they even have a coherent conception of uncertainty.

So I would pose that the problem isn't that we don't have the technology, but it's rather we don't understand what we want from it. I am yet to see a coherent theory of how humans manipulate the human language to express uncertainty that would encompass broad (if not all) range of how people use language. Without having that, you can't define what is a hallucination of an LLM. Maybe it's making a joke (some believe that point of the joke is to highlight a subtle logical error of some sort), because, you know, it read a lot of them and it concluded that's what humans do.

So AI eventually prevailed (over humans) in fields where we were able to precisely define the goal. But what is our goal vis-a-vis human language? What do we want from AI to answer to our prompts? I think we are stuck at the lack of definition of that.

Re: Overcoming the limits of current LLMs

#42

Post by jhanschoo »

fsndz wrote: post #29
knowaveragejoe wrote: post #13

Earlier quoted context omitted.

This is almost a year old, thoughts on it today?

LLMs still do not reason or plan. And nothing in their architecture, training, post-training points toward real reasoning as scaling continues. Thinking does not happen one token at a time.

As the other reply has said, the article points to limitations of LLMs, but that doesn't preclude synthesizing a system of multiple components that uses LLMs. To the extent that I'm bearish on AI capabilities, I'll note that program synthesis / compression / general inductive reasoning which we expect intelligent agents to do is a computationally very hard problem.

Re: Overcoming the limits of current LLMs

#43

Post by bugglebeetle »

smokel wrote: post #39
lsy wrote: post #31

We can't develop a universally coherent data set because what we understand as "truth" is so intensely contextual that we can't hope to cover the amount of context needed to make the things work how we want, not to mention the numerous social situations where writing factual statements would be awkward or disastrous. Here are a few examples of statements that are not "factual" in the sense of being derivable from a u…

> We can't develop a universally coherent data set because Yet every child seems to manage, when raised by a small village, over a period of about 18 years. I guess we just need to give these LLMs a little more love and attention.

Or maybe hundreds of millions of years of evolutionary pressure to build unbelievably efficient function approximation.

Re: Overcoming the limits of current LLMs

#44

Post by threeseed »

isaacfung wrote: post #34
fsndz wrote: post #29

Earlier quoted context omitted.

LLMs still do not reason or plan. And nothing in their architecture, training, post-training points toward real reasoning as scaling continues. Thinking does not happen one token at a time.

I don't get why some people seem to think the only way to use a LLM is for next token prediction or AGI has to be bult using LLM alone. You want planning, you can do monte carlo tree search and use LLM to evaluate which node to explore next. You want verifiable reasoning, you can ask it to generate code(an approach used by recent AI olympiad winner and many previous papers). What is even "planning", finding desirable…

The problem is that every time you ask the LLM to evaluate what to do next it will return a wrong answer X% of the time. Multiple that X across the number of steps and you have a system that is effectively useless. X today is ~5%.

I do think LLMs could be used to assist in building a world model that could be a foundation for an AGI/agent system. But it won't be the major part.

Re: Overcoming the limits of current LLMs

#45

Post by coldtea »

RamblingCTO wrote: post #38

My biggest problem with them is that I can't quite get it to behave like I want it to. I built myself a "therapy/coaching" telegram bot (I'm healthy, but like to reflect a lot, no worries). I even built a self-reflecting memory component that generates insights (sometimes spot on, sometimes random af). But the more I use it, the more I notice that neither the memory nor the prompt matters much. I just can't get it to…

>I just can't get it to behave like a therapist would

  import time
  import random

  SESSION_DURATION = 50 * 60
  start_time = time.time()
    
  while True:
    current_time = time.time()
    elapsed_time = current_time - start_time
    
    if elapsed_time >= SESSION_DURATION:
        print("Our time is up. That will be $150. See you next week!")
        break
    
    _ = input("")
    print(random.choice(["Mmm hmm", "Tell me more", "How does that make you feel?"]))
    
    time.sleep(1)
Thank me later!

Re: Overcoming the limits of current LLMs

#46

Post by threeseed »

RodgerTheGreat wrote: post #3

One of the main factors that makes LLMs popular today is that scaling up the models is a simple and (relatively) inexpensive matter of buying compute capacity and scraping together more raw text to train them. Without large and highly diverse training datasets to construct base models, LLMs cannot produce even the superficial appearance of good results. Manually curating "tidy", properly-licensed and verified dataset…

> properly-licensed and verified datasets is immensely more difficult, expensive

Arguably the bigger problem is that many of those datasets e.g. WSJ articles are proprietary and can be exclusively licensed like we've seen recently with OpenAI.

So we end up with in a situation where competition is simply not possible.

Re: Overcoming the limits of current LLMs

#47

Post by piva00 »

threeseed wrote: post #46
RodgerTheGreat wrote: post #3

One of the main factors that makes LLMs popular today is that scaling up the models is a simple and (relatively) inexpensive matter of buying compute capacity and scraping together more raw text to train them. Without large and highly diverse training datasets to construct base models, LLMs cannot produce even the superficial appearance of good results. Manually curating "tidy", properly-licensed and verified dataset…

> properly-licensed and verified datasets is immensely more difficult, expensive Arguably the bigger problem is that many of those datasets e.g. WSJ articles are proprietary and can be exclusively licensed like we've seen recently with OpenAI. So we end up with in a situation where competition is simply not possible.

> Arguably the bigger problem is that many of those datasets e.g. WSJ articles are proprietary and can be exclusively licensed like we've seen recently with OpenAI.

> So we end up with in a situation where competition is simply not possible.

Exactly, and Technofeudalism advances a little more into a new feud.

OpenAI is trying to create its moat by shoring up training data, probably attempting to not allow competitors to train on the same datasets they've been licencing, at least for a while. Training data is the only possible moat for LLMs, models seem to be advancing quite well between different companies but as mentioned here a tidy training dataset is the actual gold.

Re: Overcoming the limits of current LLMs

#48

Post by freilanzer »

darby_nine wrote: post #4

Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is with perception rather than structural incompetence. Why not just say "incoherent output"? It's actually descriptive and doesn't require bastardizing a word we already find meaningful to mean something completely different.

Hallucination perfectly describes the phenomenon.

Re: Overcoming the limits of current LLMs

#49

Post by idle_zealot »

Terr_ wrote: post #36
dtx1 wrote: post #16

Earlier quoted context omitted.

I think we should start smaller and make them able to count first.

Yeah, you can train an LLM to recognize the vocabulary and grammatical features of logical fallacies... Except the nature of fallacies is that they look real on that same linguistic level, so those features aren't distinctive for that purpose. Heck, I think detecting sarcasm would be an easier goal, and still tricky.

> Except the nature of fallacies is that they look real on that same linguistic level, so those features aren't distinctive for that purpose

Well that's actually good news. With a large enough labelled dataset of actually-sound and fallacious text with similar grammatical features you should be able to train a discriminator to distinguish between them using some other metric. Good luck with getting that data set though.

Re: Overcoming the limits of current LLMs

#50

Post by JohnVideogames »

wokwokwok wrote: post #17

Does anyone really believe that having a good corpus will remove hallucinations? Is this article even written by a person? Hard to know; they have a real blog with real article, but stuff like this reads strangely. Maybe it's just not a native english speaker? > Hallucinations are certainly the toughest nut to crack and their negative impact is basically only slightly lessened by good confidence estimates and reliabl…

It's obvious that you can't solve hallucinations by curating the dataset when you think about arithmetic.

It's trivial to create a corpus of True Maths Facts and verify that they're correct. But an LLM (as they're currently structured) will never generalise to new mathematical problems with 100% success rate because they do not fundamentally work like that.

Post reply on HN