Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

Post reply on HN
Print view View original

131–140 of 376 posts

Re: Claude's Cycles [pdf]

#131

Post by empath75 »

qsera wrote: post #60
throw310822 wrote: post #42

Earlier quoted context omitted.

> just find the most probable word that follows next Well, if in all situations you can predict which word Einstein would probably say next, then I think you're in a good spot. This "most probable" stuff is just absurd handwaving. Every prompt of even a few words is unique, there simply is no trivially "most probable" continuation. Probable given what? What these machines learn to do is predicting what intelligence w…

>Probable given what? The training data.. >predicting what intelligence would do No, it just predict what the next word would be if an intelligent entity translated its thoughts to words. Because it is trained on the text that are written by intelligent entities. If it was trained on text written by someone who loves to rhyme, you would be getting all rhyming responses. It imitates the behavior -- in text -- of what…

It is impossible to accurately imitate the action of intelligent beings without being intelligent. To believe otherwise is to believe that intelligence is a vacuous property.

Re: Claude's Cycles [pdf]

#132

Post by simianwords »

faxmeyourcode wrote: post #94

> Filip also told me that he asked Claude to continue on the even case after the odd case had been resolved. “But there after a while it seemed to get stuck. In the end, it was not even able to write and run explore programs correctly anymore, very weird. So I stopped the search.” Interesting snippet towards the end. I wonder if they were using claude.ai or claude code. Sounds like they ran out of context and entered…

They mentioned plan document

Re: Claude's Cycles [pdf]

#133

Post by empath75 »

gpm wrote: post #50
vidarh wrote: post #38

Earlier quoted context omitted.

Put a loop around an LLM and, it can be trivially made Turing complete, so it boils down to whether thinking requires exceeding the Turing computable, and we have no evidence to suggest that is even possible.

What are you doing in your loop? As typically deployed [1] LLMs are not turing complete. They're closer to linear bounded automaton, but because transformers have a strict maximum input size they're actually a subset of the weaker class of deterministic finite automaton. These aren't like python programs or something that can work on as much memory as you supply them, their architecture works on a fixed maximum amoun…

No physically realizable machine is technically turing complete.

But it is trivially possible to give systems-including-LLMs external storage that is accessible on demand.

Re: Claude's Cycles [pdf]

#134

Post by Symmetry »

bitexploder wrote: post #21
Aerroon wrote: post #3

Earlier quoted context omitted.

A bit related: open weights models are basically time capsules. These models have a knowledge cut off point and essentially forever live in that time.

This is the most fundamental argument that they are not, directly, an intelligence. They are not ever storing new information on a meaningful timescale. However, if you viewed them on some really large macro time scale where now LLMs are injecting information into the universe and the re-ingesting that maybe in some very philosophical way they are a /very/ slow oscillating intelligence right now. And as we narrow tha…

That means they're not conscious in the Global Workspace[1] sense but I think it would be going too far to say that that means they're not intelligent.

[1]https://en.wikipedia.org/wiki/Global_workspace_theory

Re: Claude's Cycles [pdf]

#135

Post by mccoyb »

sosodev wrote: post #121
mccoyb wrote: post #2

It's fascinating to think about the space of problems which are amenable to RL scaling of these probability distributions. Before, we didn't have a fast (we had to rely on human cognition) way to try problems - even if the techniques and workflows were known by someone. Now, we've baked these patterns into probability distributions - anyone can access them with the correct "summoning spell". Experts will naturally us…

My understanding, from listening/reading what top researchers are saying, is that model architectures in the near future are going to attempt to scale the context window dramatically. There's a generalized belief that in-context learning is quite powerful and that scaling the window might yield massive benefits for continual learning. It doesn't seem that hard because recent open weight models have shown that the mem…

I don't disagree with this, but I don't think the memory cost is the only issue right? I remember using Sonnet 4.5 (or 4, I can't remember the first of Anthropic's offerings with a million context) and how slow the model would get, how much it wanted to end the session early as tokens accrued (this latter point, of course, is just an artifact of bad training).

Less worried about memory, more worried about compute speed? Are they obviously related and is it straightforward to see?

Re: Claude's Cycles [pdf]

#136

Post by gpm »

roywiggins wrote: post #96
gpm wrote: post #50

Earlier quoted context omitted.

What are you doing in your loop? As typically deployed [1] LLMs are not turing complete. They're closer to linear bounded automaton, but because transformers have a strict maximum input size they're actually a subset of the weaker class of deterministic finite automaton. These aren't like python programs or something that can work on as much memory as you supply them, their architecture works on a fixed maximum amoun…

Turing Machines don't need access to the entire tape all at once, it's sufficient for it to see one cell at a time. You could certainly equip an LLM with a "read cell", "write cell", and "move left/right" tool and now you have a Turing machine. It doesn't need to keep any of its previous writes or reads in context. A sliding context window is more than capacious enough for this.

You're right of course, but at the point where you're saying "well we can make a turing machine with the LLM as the transition function by defining some tool calls for the LLM to interact with the tape" it feels like a stretch to call the LLM itself turing complete.

Also people definitely talk about them as "thinking" in contexts where they haven't put a harness capable of this around them. And in the common contexts where people do put harness theoretically capable of this around the LLM (e.g. giving the LLM access to bash), the LLM basically never uses that theoretical capability as the extra memory it would need to actually emulate a turing machine.

And meanwhile I can use external memory myself in a similar way (e.g. writing things down), but I think I'm perfectly capable of thinking without doing so.

So I persist in my stance that turing complete is not the relevant property, and isn't really there.

Re: Claude's Cycles [pdf]

#137

Post by pfdietz »

jdnier wrote: post #89

> I think Claude Shannon’s spirit is probably proud to know that his name is now being associated with such advances. Hats off to Claude! I didn't realize Claude was named after Claude Shannon! https://en.wikipedia.org/wiki/Claude_Shannon

It's like the diesel engine, which is named after Rudolf Engine.

Re: Claude's Cycles [pdf]

#138

Post by Filligree »

rcarr wrote: post #54
Aerroon wrote: post #3

Earlier quoted context omitted.

A bit related: open weights models are basically time capsules. These models have a knowledge cut off point and essentially forever live in that time.

Not an expert but surely it's only a matter of time until there's a way to update with the latest information without having to retrain on the entire corpus?

It’s an extremely difficult problem, and if you know how to do that you could be a billionaire.

It’s not impossible, obviously—humans do it—but it’s not yet certain that it’s possible with an LLM-sized architecture.

Re: Claude's Cycles [pdf]

#139

Post by bachmeier »

konne88 wrote: post #120

I didn't expect such a misleading intro from Knuth. It reads like Claude solved Knuth's math problem. In reality, Claude generated various example solution, and Knuth then manually generalized that to a formal proof. What Claude did is certainly useful, but it would have been nice to be clear about the scope of the contribution in the intro.

My interpretation is that Claude did what Knuth considers to be the "solution". Doing the remaining work and polishing up the proof are not necessary to have a solution from this perspective.

Re: Claude's Cycles [pdf]

#140

Post by rishabhaiover »

konne88 wrote: post #120

I didn't expect such a misleading intro from Knuth. It reads like Claude solved Knuth's math problem. In reality, Claude generated various example solution, and Knuth then manually generalized that to a formal proof. What Claude did is certainly useful, but it would have been nice to be clear about the scope of the contribution in the intro.

That's true but the capability to go back to an older iteration, reflect and find the correct solution (for odd numbers) is, in my book, a sign of undeniable intelligence.
Post reply on HN