Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

Post reply on HN
Print view View original

101–110 of 376 posts

Re: Claude's Cycles [pdf]

#101

Post by cjcole »

rustyhancock wrote: post #29
worldsavior wrote: post #24

Earlier quoted context omitted.

If AGI will ever come, then. Currently, AI is only a statistical machines, and solutions like this are purely based on distribution and no logic/actual intelligence.

I don't even think that's the issue. The issue to my mind is a lack of data at the meeting of QFT/GR. Afterall few humans historically have been capable of the initial true leap between ontologies. But humans are pretty smart so we can't say that is a requirement for AGI.

Maybe.

“The laws of nature should be expressed in beautiful equations.”

- Paul Dirac

“It is, indeed, an incredible fact that what the human mind, at its deepest and most profound, perceives as beautiful finds its realisation in external nature. What is intelligible is also beautiful. We may well ask: how does it happen that beauty in the exact sciences becomes recognizable even before it is understood in detail and before it can be rationally demonstrated? In what does this power of illumination consist?”

- Subrahmanyan Chandrasekhar

“I often follow Plato’s strategy, proposing objects of mathematical beauty as models for Nature.”

“It was beauty and symmetry that guided Maxwell and his followers.”

- Frank Wilczek

“Beauty, is bound up with symmetry.”

- Herman Weyl

"Still twice in the history of exact natural science has this shining-up of the great interconnection become the decisive signal for significant progress. I am thinking here of two events in the physics of our century: the rise of the theory of relativity and that of the quantum theory. In both cases, after yearlong unsuccessful striving for understanding, a bewildering abundance of details was almost suddenly ordered. This took place when an interconnection emerged which, thought largely unvisualizable, was finally simple in its substance. It convinced through its compactness and abstract beauty – it convinced all those who can understand and speak such an abstract language."

- Werner Heisenberg

Maybe (just maybe) these things (whatever you want to call them) will (somehow) gain access to some "compact", beautiful, "largely unvisualizable" "interconnection" which will be the self-evident solution. And if they do, many will be sure to label it a statistical accident from a stochastic parrot. And they'll right, for some definitions of "statistical", "accident", "stochastic", and "parrot".

Re: Claude's Cycles [pdf]

#102

Post by bread-wood »

jdnier wrote: post #89

> I think Claude Shannon’s spirit is probably proud to know that his name is now being associated with such advances. Hats off to Claude! I didn't realize Claude was named after Claude Shannon! https://en.wikipedia.org/wiki/Claude_Shannon

Here I was assuming it was named after https://en.wikipedia.org/wiki/Claude_(alligator)

Re: Claude's Cycles [pdf]

#103

Post by fleischhauf »

eru wrote: post #71
rustyhancock wrote: post #26

Earlier quoted context omitted.

I think a very long time because part of our limit is experiment. We need enough experimental results to explain to solve these theoretical mismatches and we don't and at present can't explore that frontier. Once we have more results at that frontier we'd build a theory out from there that has two nearly independent limits for QFT and GR. What we'd be asking if the AI is something that we can't expect a human to solv…

> I think a very long time because part of our limit is experiment. Yes, maybe. But if you are smarter, you can think up better experiments that you can actually do. Or re-use data from earlier experiments in novel and clever ways.

this. could already be useful to narrow down the search space

Re: Claude's Cycles [pdf]

#104

Post by emp17344 »

cjcole wrote: post #91
bobbylarrybobby wrote: post #52

Earlier quoted context omitted.

Did you read the linked paper? Claude out-reasoned humans on a challenging (or at least, unsolved) math problem.

"humans" Donald Knuth is an extremal outlier human and the problem is squarely in his field of expertise. Claude, guided by Filip Stappers, a friend of Knuth, solved a problem that Knuth and Stappers had been working on for several weeks. Unfortunately, it doesn't seem (from my quick scan) to have been stated how long (or how many tokens or $) it took for Claude + Stappers to complete the proof. In response, Knuth sa…

What goalposts do you think are being moved? I constantly see AI enthusiasts use this phrase, but it’s not clear what goalposts they have in mind. Specifically, what is it that you want opponents to recognize that you believe they aren’t currently?

We now have a tool that can be useful in some narrow domains in some narrow cases. It’s pretty neat that our tools have new capabilities, but it’s also pretty far from AGI.

Re: Claude's Cycles [pdf]

#106

Post by vidarh »

gpm wrote: post #50
vidarh wrote: post #38

Earlier quoted context omitted.

Put a loop around an LLM and, it can be trivially made Turing complete, so it boils down to whether thinking requires exceeding the Turing computable, and we have no evidence to suggest that is even possible.

What are you doing in your loop? As typically deployed [1] LLMs are not turing complete. They're closer to linear bounded automaton, but because transformers have a strict maximum input size they're actually a subset of the weaker class of deterministic finite automaton. These aren't like python programs or something that can work on as much memory as you supply them, their architecture works on a fixed maximum amoun…

Context effectively provifes an IO port, and so all the loop needs to do is to simulate the tape head, and provide a single token of state.

You can not be convinced Turing complete is relevant all you want - we don't know of any more expansive category of computable functions, and so given that an LLM in the setup described is Turing complete no matter that they aren't typically deployed that way is irrelevant.

They trivially can be, and that is enough to make the shallow dismissal of pointing out they're "just" predicting the next token meaningless.

Re: Claude's Cycles [pdf]

#107

Post by dtj1123 »

wang_li wrote: post #68
morleytj wrote: post #51

Earlier quoted context omitted.

A very good point. For anyone not familiar with anterograde amnesia, the classical case is patient H.M. ( https://en.wikipedia.org/wiki/Henry_Molaison ), whose condition was researched by Brenda Milner.

Or you could have just said "they can't form new memories."

I actually wasn't aware of this story. The steady stream of unexpected and enriching information like this is exactly why I love hackernews.

Re: Claude's Cycles [pdf]

#108

Post by emp17344 »

ecshafer wrote: post #16

I wonder how long we have until we start solving some truly hard problems with AI. How long until we throw AI at "connect general relativity and quantum physics", give the AI 6 months and a few data centers, and have it pop out a solution?

Hold your horses, that’s a long way off. The best math AI tool we currently have, Aletheia, was only able to solve 13 out of 700 attempted open Erdos problems, only 4 of which were solved autonomously: https://arxiv.org/html/2601.22401v3

Clearly, these models still struggle with novel problems.

Re: Claude's Cycles [pdf]

#109

Post by ramses0 »

Jweb_Guru wrote: post #43
anematode wrote: post #23

Earlier quoted context omitted.

But they're not "slow"! Unlike biological thinking, which has a speed limit, you can accelerate these chains of thought by orders of magnitude.

I assure you that LLM thinking also has a speed limit.

But imagine a beowulf cluster of them... /s

...but seriously... there was the "up until 1850" LLM or whatever... can we make an "up until 1920 => 1990 [pre-internet] => present day" and then keep prodding the "older ones" until they "invent their way" to the newer years?

We knew more in 1920 than we did in 1850, but can a "thinking machine" of 1850-knowledge invent 1860's knowledge via infinite monkeys theorem/practice?

The same way that in 2025/2026, Knuth has just invented his way to 2027-knowledge with this paper/observation/finding? If I only had a beowulf cluster of these things... ;-)

Re: Claude's Cycles [pdf]

#110

Post by afspear »

faxmeyourcode wrote: post #94

> Filip also told me that he asked Claude to continue on the even case after the odd case had been resolved. “But there after a while it seemed to get stuck. In the end, it was not even able to write and run explore programs correctly anymore, very weird. So I stopped the search.” Interesting snippet towards the end. I wonder if they were using claude.ai or claude code. Sounds like they ran out of context and entered…

What would be super cool is if this dumb zone could be quantified and surfaced to the user. I've noticed that copilot now has a little circle graph that indicates context use percentage and it changes color based on percentage. I'll bet these are very naive metrics on used tokens vs context availability. I wonder if there could be meta data streamed or sent along with the tokens that could show that you've entered the dumb zone.
Post reply on HN