Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

Post reply on HN
Print view View original

251–260 of 376 posts

Re: Claude's Cycles [pdf]

#251

Post by qsera »

red75prime wrote: post #245
qsera wrote: post #244

Earlier quoted context omitted.

>you can be intelligent only if you fully imitate biological processes It is not that. It is about having an understanding of how it is trained. For example, if it was trained on ideas, instead of words, then it would be closer to intelligent behavior. Someone will say that during training it builds ideas and concepts, but that is just a name that we give for the internal representation that results from training and…

The latest models are mostly LMMs (large multimodal models). If a model builds an internal representation that integrates all the modalities we are dealing with (robotics even provides tactile inputs), it becomes harder and harder to imagine why those representations should be qualitatively different.

It can't, simply because the textual description of a concept is different from the concept itself.

Re: Claude's Cycles [pdf]

#252

Post by jdub »

Wowfunhappy wrote: post #170
Filligree wrote: post #138

Earlier quoted context omitted.

It’s an extremely difficult problem, and if you know how to do that you could be a billionaire. It’s not impossible, obviously—humans do it—but it’s not yet certain that it’s possible with an LLM-sized architecture.

> It’s not impossible, obviously—humans do it It's still not at all obvious to me that LLMs work in the same way as the human brain, beyond a surface level. Obviously the "neurons" in neural nets resemble our brains in a sense, but is the resemblance metaphorical or literal?

Digital neural networks and "neurons" were already vastly simpler than biological neural networks and neurons... and getting to transformers involved optimisations that took us even further away from biomimicry.

Re: Claude's Cycles [pdf]

#253

Post by jdub »

rishabhaiover wrote: post #140
konne88 wrote: post #120

I didn't expect such a misleading intro from Knuth. It reads like Claude solved Knuth's math problem. In reality, Claude generated various example solution, and Knuth then manually generalized that to a formal proof. What Claude did is certainly useful, but it would have been nice to be clear about the scope of the contribution in the intro.

That's true but the capability to go back to an older iteration, reflect and find the correct solution (for odd numbers) is, in my book, a sign of undeniable intelligence.

Or, the ability to construct additional sentences influenced by prior ones.

Re: Claude's Cycles [pdf]

#254

Post by red75prime »

qsera wrote: post #251
red75prime wrote: post #245

Earlier quoted context omitted.

The latest models are mostly LMMs (large multimodal models). If a model builds an internal representation that integrates all the modalities we are dealing with (robotics even provides tactile inputs), it becomes harder and harder to imagine why those representations should be qualitatively different.

It can't, simply because the textual description of a concept is different from the concept itself.

Obviously, a concept (which is an abstraction in more ways than one) is different from a textual representation. But LLMs don't operate on the textual description of a concept when they are doing their thing. A textual description (which is associated with other modalities in the training data) serves as an input format. LLMs perform non-linear transformations of points in their latent space. These transformations and representations are useful not only for generating text but also for controlling robots, for example (see VLAs in robotics).

Re: Claude's Cycles [pdf]

#255

Post by kqr »

gravypod wrote: post #221
Aerroon wrote: post #3

Earlier quoted context omitted.

A bit related: open weights models are basically time capsules. These models have a knowledge cut off point and essentially forever live in that time.

This is very interesting. I wonder if someone could create a future-sight benchmark for these models? Like, if given a set of newspaper articles for the past N months can it predict if certain world events would happen? We could backtest against results that have happened since the training cutoff.

These days computers compete along with humans in forecasting tournaments on Metaculus. They don't quite beat the top humans yet, but they're up there. https://www.metaculus.com/futureeval/

Re: Claude's Cycles [pdf]

#256

Post by astrange »

qsera wrote: post #237
astrange wrote: post #233

Earlier quoted context omitted.

LLMs also don't work by generating probability distributions of the next word. Your explanation isn't able to explain why they can generate words, let alone sentences.

That is exactly how they work.

No, a token is not a word.

Re: Claude's Cycles [pdf]

#257

Post by kqr »

atomicnature wrote: post #248
Chinjut wrote: post #203

Earlier quoted context omitted.

It does not seem fair to say that frequentists do not update their beliefs based on new evidence. This does not seem to accurately capture what the difference between Bayesians and frequentists (or anyone else) is.

What's the difference as you see it?

Everyone updates their belief in hypotheses based on the perceived strength of evidence they observe. That's just science.

Frequentists and Bayesians differ in which sets of statistical tools they prefer for measuring the strength of evidence.

Re: Claude's Cycles [pdf]

#258

Post by kqr »

kami23 wrote: post #197
brcmthrowaway wrote: post #165

Earlier quoted context omitted.

What is dumb zone?

When the LLMs start compacting they summarize the conversation up to that point using various techniques. Overall a lot of maybe finer points of the work goes missing and can only be retrieved by the LLM being told to search for it explicitly in old logs. Once you compact, you've thrown away a lot of relevant tokens from your problem solving and they do become significantly dumber as a result. If I see a compaction c…

What prompt do you use for the letter-to-self? I've been trying that technique myself to manually reset context without losing the important parts (e.g. when it has barked up the wrong tree and I'm sensing that misstep might influence its current generation in a pathological way), but I've not had much success.

Re: Claude's Cycles [pdf]

#259

Post by kqr »

computerex wrote: post #236

It's incredible to see work like this from him, at a ripe old age of eighty-six.

I agree. I met Knuth briefly after a guest lecture at my university a few years ago and although you could tell his body was getting old, his mind was incredibly fresh.

Although I'm not as bright as him, I can only hope to be as intellectually curious as him at that age.

Post reply on HN