Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM Daydreaming

gwern.net

Post reply on HN
Print view View original

131–140 of 156 posts

Re: LLM Daydreaming

#131

Post by vonneumannstan »

HarHarVeryFunny wrote: post #85
vonneumannstan wrote: post #77

Earlier quoted context omitted.

> Despite all the hoopla, LLMs are not AGI or artifical brains - they are predict-next-word language models. By design they are not built for creativity, but rather quite the opposite, they are designed to continue the input in the way best suggested by the training data - they are essentially built for recall, not creativity. This is just a completely base level of understanding of LLMs. How do you predict the next…

> This is just a completely base level of understanding of LLMs. How do you predict the next token with superhuman accuracy? Really think about how that is possible. If you think it's just stochastic parroting you are ngmi. Yes, thank-you, I do understand how LLMs work. They learn a lot of generative rules from the training data, and will apply them in flexible fashion according to the context patterns they have lear…

>However, we're not discussing whether LLMs can be superhuman at tasks where they had the necessary training - we're discussing whether they are capable of creativity

"Even if all the model is capable of is memorizing text then just the sheer volume it has memorized should yield unique insights, no human can ever hope to hold this much text in their memory and then make connections between it."

Unless you think humans have magic meat then all we are really doing with "creativity" is connecting previously unconnected facts.

Re: LLM Daydreaming

#132

Post by catigula »

nico wrote: post #57
ashdksnndck wrote: post #8

I’m not sure we can accept the premise that LLMs haven’t made any breakthroughs. What if people aren’t giving the LLM credit when they get a breakthrough from it? First time I got good code out of a model, I told my friends and coworkers about it. Not anymore. The way I see it, the model is a service I (or my employer) pays for. Everyone knows it’s a tool that I can use, and nobody expects me to apportion credit for…

> but I don’t commit a comment saying “wow, this clever bit came from the model!” The other day, Claude Code started adding a small signature to the commit messages it was preparing for me. It said something like “This commit was co-written with Claude Code” and a little robot emoji I wonder if that just happened by accident or if Anthropic is trying to do something like Apple with the “sent from my iPhone”

letting claude pen commits is wild.

Re: LLM Daydreaming

#133

Post by HarHarVeryFunny »

awongh wrote: post #126
HarHarVeryFunny wrote: post #107

Earlier quoted context omitted.

> It's just ironic how human-like the flaws of the system are. (Hallucinations that are asserting untrue facts, just because they are plausible from a pattern matching POV) I think most human mistakes are different - not applying a lot of complex logic to come to an incorrect deduction/guess (= LLM hallucination), but rather just shallow recall/guess. e.g. An LLM would guess/hallucinate a capital city by using rules…

The reason the LLMs are of any use to anyone right now (and real people are using them for real things right now- see the millions of ChatGPT users) is because the qualitative difference between text created by a real human using a guessing heuristic vs. an LLM using statistics is qualitatively the same. Even for things that some subjectively deem "creative". The entropy of communication also makes it so that we most…

LLMs are just trying to mimic (predict) human output, and can obviously do a great job, which is why they are useful.

I was just referring to when LLMs fail, which can be in non-human ways, not only the way in which they hallucinate, but also when they generate output that has the "shape" of something in the training set, but is nonsense.

Re: LLM Daydreaming

#134

Post by therealpygon »

suddenlybananas wrote: post #105
starlust2 wrote: post #101

Earlier quoted context omitted.

> through brute force The same is true of humanity in aggregate. We attribute discoveries to an individual or group of researchers but to claim humans are efficient at novel research is a form of survivorship bias. We ignore the numerous researchers who failed to achieve the same discoveries.

The fact some people don't succeed doesn't show that humans operate by brute force. To claim humans reason and invent by brute force is patently absurd.

You don’t consider thousands of scientists developing competing, and often incorrect, solutions for a single domain as a “brute force” attempt by humanity, but do when the same occurs with disparate solutions from parallel LLM attempts? That’s certainly an…opinion.

Re: LLM Daydreaming

#135

Post by HarHarVeryFunny »

vonneumannstan wrote: post #131
HarHarVeryFunny wrote: post #85

Earlier quoted context omitted.

> This is just a completely base level of understanding of LLMs. How do you predict the next token with superhuman accuracy? Really think about how that is possible. If you think it's just stochastic parroting you are ngmi. Yes, thank-you, I do understand how LLMs work. They learn a lot of generative rules from the training data, and will apply them in flexible fashion according to the context patterns they have lear…

>However, we're not discussing whether LLMs can be superhuman at tasks where they had the necessary training - we're discussing whether they are capable of creativity "Even if all the model is capable of is memorizing text then just the sheer volume it has memorized should yield unique insights, no human can ever hope to hold this much text in their memory and then make connections between it." Unless you think human…

> Unless you think humans have magic meat then all we are really doing with "creativity" is connecting previously unconnected facts.

In the case of discovery and invention the "facts" being connected may not be things that were known before. An LLM is bound by it's training set. A human is not limited by what is currently known - they can explore (in directed or undirected fashion), learn, build hierarchies of new knowledge and understanding, etc.

Re: LLM Daydreaming

#136

Post by HarHarVeryFunny »

vonneumannstan wrote: post #131
HarHarVeryFunny wrote: post #85

Earlier quoted context omitted.

> This is just a completely base level of understanding of LLMs. How do you predict the next token with superhuman accuracy? Really think about how that is possible. If you think it's just stochastic parroting you are ngmi. Yes, thank-you, I do understand how LLMs work. They learn a lot of generative rules from the training data, and will apply them in flexible fashion according to the context patterns they have lear…

>However, we're not discussing whether LLMs can be superhuman at tasks where they had the necessary training - we're discussing whether they are capable of creativity "Even if all the model is capable of is memorizing text then just the sheer volume it has memorized should yield unique insights, no human can ever hope to hold this much text in their memory and then make connections between it." Unless you think human…

> "Even if all the model is capable of is memorizing text then just the sheer volume it has memorized should yield unique insights, no human can ever hope to hold this much text in their memory and then make connections between it."

Yes, potentially, but the model has no curiosity or drive to do this (or anything else) by itself. All an LLM is built to do is predict. The only way to control the output and goad it into using the vast amount of knowledge that it has is by highly specific prompting.

Basically it's only going to connect the dots if you tell what dots to connect, in which case it's the human being inventive, not the model. The model is trying to predict, so essentially if you want it to do something outside of the training set you're going to have to prompt it to do that.

A human has curiosity (e.g. "what happens if I connect these dots .."), based on prediction failure and associated focus/etc - the innate desire to explore the unknown and therefore potentially learn. The model has none of that - it can't learn and has no curiosity. If the model's predictions are bad it will just hallucinate and generate garbage, perhaps "backtrack" and try again, and likely lead to context rot.

Re: LLM Daydreaming

#137

Post by SheinhardtWigCo »

catigula wrote: post #132
nico wrote: post #57

Earlier quoted context omitted.

> but I don’t commit a comment saying “wow, this clever bit came from the model!” The other day, Claude Code started adding a small signature to the commit messages it was preparing for me. It said something like “This commit was co-written with Claude Code” and a little robot emoji I wonder if that just happened by accident or if Anthropic is trying to do something like Apple with the “sent from my iPhone”

letting claude pen commits is wild.

It’s great. It’s become my preferred workflow for vibe coding because it writes great commit messages, gives you a record of authorship, and rollbacks use far fewer tokens. You don’t have to (and probably shouldn’t) let it push to the remote branch.

Re: LLM Daydreaming

#138

Post by Nevermark »

preciousoo wrote: post #110
suddenlybananas wrote: post #105

Earlier quoted context omitted.

The fact some people don't succeed doesn't show that humans operate by brute force. To claim humans reason and invent by brute force is patently absurd.

It’s an absurd statement because you are human and are aware of how research works on an individual level. Take yourself outside of that, and imagine you invented earth, added an ecosystem, and some humans. Wheels were invented ~6k years ago, and “humans” have existed for ~40-300k years. We can do the same for other technologies. As a group, we are incredibly inefficient, and an outside observer would see our efforts…

I consider humans an "intelligent" species in the sense that a critical mass of us can organize to sustainably learn.

As individuals, without mentors, we would each die off very quickly. Even if we were fed and whatever until we were physically able to take care of ourselves, we wouldn't be able to keep ourselves out of trouble if we had to learn everything ourselves.

Contrast this with the octopus which develops from an egg without any mentorship, and within a year or so has a fantastically knowledgable and creative mind over its respective environment. And they thrive in every oceanic environment in the wet salty world, including coast lines, under permanent Arctic ice, to the deep sea.

To whatever degree they are "intelligent", it's an amazingly accelerated, fully independent, self-taught intelligence. Our species just can't compare on that dimension.

Fortunately, octopus only live a couple years and in an environment where technology is difficult (very hard to isolate and control conditions of all kinds in the ocean). Otherwise, the land octopus would have eaten all of us long ago.

Re: LLM Daydreaming

#139

Post by nwienert »

therealpygon wrote: post #134
suddenlybananas wrote: post #105

Earlier quoted context omitted.

The fact some people don't succeed doesn't show that humans operate by brute force. To claim humans reason and invent by brute force is patently absurd.

You don’t consider thousands of scientists developing competing, and often incorrect, solutions for a single domain as a “brute force” attempt by humanity, but do when the same occurs with disparate solutions from parallel LLM attempts? That’s certainly an…opinion.

My least favorite type of argument on this site is when someone takes a word with a specific meaning and warps it well beyond the reasonable interpretation just so they can claim they’re making a good analogy. It seems to happen every day here with AI.

Brute force and typical scientific research are such dramatically different things, I have to wonder if bots are getting into HN I almost can’t believe someone would try and argue that’s weird to see the difference.

“Oh you’re using your limbs to move through water? How are you not a dolphin?” lol

Re: LLM Daydreaming

#140

Post by daxfohl »

LourensT wrote: post #27

Regardless of accusations of anthropomorphizing, continual thinking seems to be a precursor to any sense of agency, simply because agency requires something to be running. Eventually LLM output degrades when most of the context is its own output. So should there also be an input stream of experience? The proverbial "staring out the window", fed into the model to keep it grounded and give hooks to go off?

Though it kind of reminds me of The Shining (too much time just thinking drives one to insanity). It seems like we need to evolve intelligence, perception, and agency very closely in tandem, or an imbalance in any will send it off the rails.
Post reply on HN