Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM Daydreaming

gwern.net

Post reply on HN
Print view View original

111–120 of 156 posts

Re: LLM Daydreaming

#111

Post by HarHarVeryFunny »

dingnuts wrote: post #100
awongh wrote: post #89

Earlier quoted context omitted.

By volume how much of human speech / writing is pattern matching and how much of it is truly original cognition that would pass your bar of creativity? It is probably 90% rote pattern matching. I don't think LLMs are AGI, but in most senses I don't think people give enough credit to their capabilities. It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just b…

My intuition is opposite yours; due to the insane complexity of the real world nearly 90% of situations are novel and require creativity OK now we're at an impasse until someone can measure this

I think it comes down to how we define creativity for the purpose of this conversation. I would say that 100% of situations and problems are novel to some degree - the real world does not exactly repeat, and your brain at T+10 is not exactly the same as it is as T+20.

That said, I think most everyday situations are similar enough to things we've experienced before that shallow pattern matching is all it takes. The curve in the road we're driving on may not be 100% the same as any curve we've experienced before, but turning the car wheel to the left the way we've learnt do do it will let us successfully navigate it all the same.

Most everyday situations/problems we're faced with are familiar enough that shallow "reactive" behavior is good enough - we rarely have to stop to develop a plan, figure things out, or reason in any complex kind of a way, and very rarely face situations so challenging that any real creativity is needed.

Re: LLM Daydreaming

#112

Post by leptons »

awongh wrote: post #89
HarHarVeryFunny wrote: post #76

Earlier quoted context omitted.

It depends on what you mean by "creative" - they can recombine fragments of training data (i.e. apply generative rules) in any order - generate the deductive closure of the training set, but that is it. Without moving beyond LLMs to a more brain-like cognitive architecture, all you can do is squeeze the juice out of the training data, by using RL/etc to bias the generative process (according to reasoning data, good t…

By volume how much of human speech / writing is pattern matching and how much of it is truly original cognition that would pass your bar of creativity? It is probably 90% rote pattern matching. I don't think LLMs are AGI, but in most senses I don't think people give enough credit to their capabilities. It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just b…

>It is probably 90% rote pattern matching.

So what. 90% (or more) of humans aren't making any sort of breakthrough in any discipline, either. 99.9999999999% of human speech/writing isn't producing "breakthroughs" either, it's just a way to communicate.

>It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just because they are plausible from a pattern matching POV)

The LLM is not "hallucinating". It's just operating as it was designed to do, which often produces results that do not make any sense. I have actually hallucinated, and some of those experiences were profoundly insightful, quite the opposite of what an LLM does when it "hallucinates".

You can call anything a "breakthrough" if you aren't aware of prior art. And LLMs are "trained" on nothing but prior art. If an LLM does make a "breakthrough", then it's because the "breakthrough" was already in the training data. I have no doubt many of these "breakthroughs" will be followed years later by someone finding the actual human-based research that the LLM consumed in its training data, rendering the "breakthrough" not quite as exciting.

Re: LLM Daydreaming

#113

Post by CaptainFever »

ramoz wrote: post #109

Ive walked 10k steps everyday the past week and produced more code in that period than most would over months. Using Claude Code (and vibetunnel over tailscale to my phone- that I speak instructions into). There is a breakthrough happening. in real time.

Can we see an example please?

Re: LLM Daydreaming

#114

Post by zyklonix »

This idea of a “daydreaming loop” hits on a key LLM gap, the lack of background, self-driven insight. A pragmatic step in this direction is https://github.com/DivergentAI/dreamGPT , which explores divergent thinking by generating and scoring hallucinations. It shows how we might start pushing LLMs beyond prompt-response into continuous, creative cognition.

Re: LLM Daydreaming

#115

Post by zyklonix »

johnfn wrote: post #3

It's an interesting premise, but how many people - are capable of evaluating the LLM's output to the degree that they can identify truly unique insights - are prompting the LLM in such a way that it could produce truly unique insights I've prompted an LLM upwards of 1,000 times in the last month, but I doubt more than 10 of my prompts were sophisticated enough to even allow for a unique insight. (I spend a lot of tim…

Totally agree, most prompts (especially for code) aren’t designed to surface novel insights, and even when they are, it’s hard to recognize them. That’s why the daydreaming loop is so compelling: it offloads both the prompting and the novelty detection to the system itself. Projects like https://github.com/DivergentAI/dreamGPT are early steps in that direction, generating weird idea combos autonomously and scoring them for divergence, without user prompting at all.

Re: LLM Daydreaming

#116

Post by HarHarVeryFunny »

js8 wrote: post #103
HarHarVeryFunny wrote: post #94

Earlier quoted context omitted.

OK, so maybe we're talking somewhat at cross purposes. I was talking about the process/mechanism of reasoning - how do our brains appear to implement the capability that we refer to as "reasoning", and by extension how could an AI do the same by implementing the same mechanisms. If we accept prediction (i.e use of past experience) as the mechanistic basis of reasoning, then choice of logic doesn't really come into it…

But prediction as the basis for reasoning (in epistemological sense) requires the goal to be given from the outside, in the form of the system that is to be predicted. And I would even say that this problem (giving predictions) has been solved by RL. Yet, the consensus seems to be we don't quite have AGI; so what gives? Clearly just making good predictions is not enough. (I would say current models are empiricist to…

> But prediction as the basis for reasoning (in epistemological sense) requires the goal to be given from the outside, in the form of the system that is to be predicted.

Prediction doesn't have goals - it just has inputs (past and present) and outputs (expected inputs). Something that is on your mind (perhaps a "goal") is just a predictive input that will cause you to predict what happens next.

> And I would even say that this problem (giving predictions) has been solved by RL.

Making predictions is of limited use if you don't have the feedback loop of when your predictions are right or wrong (so update prediction for next time), and having the feedback (as our brain does) of when your prediction is wrong is the basis of curiosity - causing us to explore new things and learn about them.

> Yet, the consensus seems to be we don't quite have AGI; so what gives? Clearly just making good predictions is not enough.

Prediction is important, but there are lots of things missing from LLMs such as ability to learn, working memory, innate drives (curiosity, boredom), etc.

Re: LLM Daydreaming

#117

Post by tmaly »

suddenlybananas wrote: post #105
starlust2 wrote: post #101

Earlier quoted context omitted.

> through brute force The same is true of humanity in aggregate. We attribute discoveries to an individual or group of researchers but to claim humans are efficient at novel research is a form of survivorship bias. We ignore the numerous researchers who failed to achieve the same discoveries.

The fact some people don't succeed doesn't show that humans operate by brute force. To claim humans reason and invent by brute force is patently absurd.

What about Dyson and Alexander Graham Bell ?

Re: LLM Daydreaming

#118

Post by zby »

The novelty part is a hard one - but maybe in many cases we could substitute something else for it? If an idea promises to beat state of the art in some field - and it is not yet actively researched - then it is novel.

But most promising would be to use the Dessalles theories.

Here is 4.1o expanding this: https://chatgpt.com/s/t_6877de9faa40819194f95184979b5b44

By the way - this could be a classic example of this day dreaming - you take two texts: one by Gwern and some article by Dessalles (I read "Why we talk" - a great book! - but maybe there is some more concise article?) and ask LLM to generate ideas connecting these two. In this particular case it was my intuition that connected them - but I imagine that there could be an algorithm that could find this connection in a reasonable time - some kind of semantic search maybe.

Re: LLM Daydreaming

#119

Post by throwaway328 »

The fact that LLMs haven't come up with anything "novel" would be a serious puzzle - as the article claims - only if they were thinking, reasoning, being creative, etc. If they aren't doing anything of the sort, it'd be the only thing you'd expect.

So it's a bit of an anti-climactic solution to the puzzle but: maybe the naysayers were right and they're not thinking at all, or doing any of the other anthropomorphic words being marketed to users, and we've simply all been dragged along by a narrative that's very seductive to tech types (the computer gods will rise!).

It'd be a boring outcome, after the countless gallons of digital ink spilled on the topic the last years, but maybe they'll come to be accepted as "normal software", and not god-like, in the end. A medium to large improvement in some areas, and anywhere from minimal to pointless to harmful in others. And all for the very high cost of all the funding and training and data-hoovering that goes in to them, not to mention the opportunity cost of all the things we humans could have been putting money into and didn't.

Re: LLM Daydreaming

#120

Post by tmaly »

HarHarVeryFunny wrote: post #67

> Despite impressive capabilities, large language models have yet to produce a genuine breakthrough. The puzzle is why. I don't see why this is remotely surprising. Despite all the hoopla, LLMs are not AGI or artifical brains - they are predict-next-word language models. By design they are not built for creativity, but rather quite the opposite, they are designed to continue the input in the way best suggested by the…

I think we will see more breakthroughs with an AI/Human hybrid approach.

Tobias Rees had some interesting thoughts https://www.noemamag.com/why-ai-is-a-philosophical-rupture/ where he poses this idea that AI and humans together can think new types of thoughts that humans alone cannot think.

Post reply on HN