Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

Post reply on HN
Print view View original

121–130 of 376 posts

Re: Claude's Cycles [pdf]

#121

Post by sosodev »

mccoyb wrote: post #2

It's fascinating to think about the space of problems which are amenable to RL scaling of these probability distributions. Before, we didn't have a fast (we had to rely on human cognition) way to try problems - even if the techniques and workflows were known by someone. Now, we've baked these patterns into probability distributions - anyone can access them with the correct "summoning spell". Experts will naturally us…

My understanding, from listening/reading what top researchers are saying, is that model architectures in the near future are going to attempt to scale the context window dramatically. There's a generalized belief that in-context learning is quite powerful and that scaling the window might yield massive benefits for continual learning.

It doesn't seem that hard because recent open weight models have shown that the memory cost of the context window can be dramatically reduced via hybrid attention architectures. Qwen3-next, Qwen3.5, and Nemotron 3 Nano are all great examples. Nemotron 3 Nano can be run with a million token context window on consumer hardware.

Re: Claude's Cycles [pdf]

#122

Post by dilap »

gpm wrote: post #32
dilap wrote: post #18

Earlier quoted context omitted.

That description is really only fair for base models†. Something like Opus 4.6 has all kinds of other training on top of that which teach it behaviors beyond "predict most probable token," like problem-solving and being a good chatbot. (†And even then is kind of overly-dismissive and underspecified. The "most probable word" is defined over some training data set. So imagine if you train on e.g. mathematicians solving…

I'd disagree, the other training on top doesn't alter the fundamental nature of the model that it's predicting the probabilities of the next token (and then there's a sampling step which can roughly be described as picking the most probable one). It just changes the probability distribution that it is approximating. To the extent that thinking is making a series of deductions from prior facts, it seems to me that thi…

The fundamental nature of the model is that it consumes tokens as input and produces token probabilities as output, but there's nothing inherently "predictive" about it -- that's just perspective hangover from the historical development of how LLMs were trained. It is, fundamentally, I think, a general-purpose thinking machine, operating over the inputs and outputs of tokens.

(With this perspective, I can feel my own brain subtly oferring up a panoply of possible responses in a similar way. I can even turn up the temperature on my own brain, making it more likely to decide to say the less-obvious words in response, by having a drink or two.)

(Similarly, mimicry is in humans too a very good learning technique to get started -- kids learning to speak are little parrots, artists just starting out will often copy existing works, etc. Before going on to develop further into their own style.)

Re: Claude's Cycles [pdf]

#123

Post by ezst »

crocowhile wrote: post #27
ainiriand wrote: post #4

Are not LLMs supposed to just find the most probable word that follows next like many people here have touted? How this can be explained under that pretense? Is this way of problem solving 'thinking'?

Those people still exist? I only know one guy who is still fighting those windmills

[flagged]

Re: Claude's Cycles [pdf]

#124

Post by cjcole »

emp17344 wrote: post #104
cjcole wrote: post #91

Earlier quoted context omitted.

"humans" Donald Knuth is an extremal outlier human and the problem is squarely in his field of expertise. Claude, guided by Filip Stappers, a friend of Knuth, solved a problem that Knuth and Stappers had been working on for several weeks. Unfortunately, it doesn't seem (from my quick scan) to have been stated how long (or how many tokens or $) it took for Claude + Stappers to complete the proof. In response, Knuth sa…

What goalposts do you think are being moved? I constantly see AI enthusiasts use this phrase, but it’s not clear what goalposts they have in mind. Specifically, what is it that you want opponents to recognize that you believe they aren’t currently? We now have a tool that can be useful in some narrow domains in some narrow cases. It’s pretty neat that our tools have new capabilities, but it’s also pretty far from AGI…

I'm not an enthusiast. I'm a Butlerian.

Imagine hearing pre-attention-is-all-you-need that "AI" could do something that Donald Knuth could not (quickly solve the stated problem in collaboration with his friend).

The idea that this (Putnam perfect, IMO gold, etc) is all just "statistical parrot" stuff is wearing a little thin.

Re: Claude's Cycles [pdf]

#125

Post by rnxrx »

charcircuit wrote: post #41
mlyle wrote: post #34

Earlier quoted context omitted.

There's nothing to say that you can't build something intelligent out of them by bolting a memory on it, though. Sure, it's not how we work, but I can imagine a system where the LLM does a lot of heavy lifting and allows more expensive, smaller networks that train during inference and RAG systems to learn how to do new things and keep persistent state and plan.

Memory is not just bolted on top of the latest models. They under go training on how and when to effectively use memory and how to use compaction to avoid running out of context when working on problems.

Maybe there's an analogy to our long and short term memory - immediate stimuli is processed in the context deep patterns that have accreted over a lifetime. The effect of new information can absolutely challenge a lot of those patterns but to have that information reshape how we basically think takes a lot longer - more processing, more practice, etc.

In the case of the LLM that longer-term learning / fundamental structure is a proxy for the static weights produced by a finite training process, and that the ability to use tools and store new insights and facts is analogous to shorter-term memory and "shallow" learning.

Perhaps periodic fine-tuning has an analogy in sleep or even our time spent in contemplation or practice (..or even repetition) to truly "master" a new idea and incorporate it into our broader cognitive processing. We do an amazing job of doing this kind of thing on a continuous basis while the machines (at least at this point) perform this process in discrete steps.

If our own learning process is a curve then the LLM's is a step function trying to model it. Digital vs analog.

Re: Claude's Cycles [pdf]

#126

Post by sosodev »

wang_li wrote: post #78
zarzavat wrote: post #45

Earlier quoted context omitted.

I swear that AI could independently develop a cure for cancer and people would still say that it's not actually intelligent, just matrix multiplications giving a statistically probable answer! LLMs are at least designed to be intelligent. Our monkey brains have much less reason to be intelligent, since we only evolved to survive nature, not to understand it. We are at this moment extremely deep into what most people…

Last week I put "was val kilmer in heat" into the search box on my browser. The AI answer came back with "No, Val Kilmer was not in heat. Val Kilmer played Chris Shiherlis in the movie Heat but the film did not indicate that he was pregnant or in heat. His performance was nuanced and skilled and represents a high point of the film." I was not curious about whether he was pregnant. We are not only not close to human l…

The model that processes search results is tiny and dumb. You shouldn't compare it to the frontier models that are solving complex math problems.

Re: Claude's Cycles [pdf]

#127

Post by theblazehen »

Aerroon wrote: post #3
mccoyb wrote: post #2

It's fascinating to think about the space of problems which are amenable to RL scaling of these probability distributions. Before, we didn't have a fast (we had to rely on human cognition) way to try problems - even if the techniques and workflows were known by someone. Now, we've baked these patterns into probability distributions - anyone can access them with the correct "summoning spell". Experts will naturally us…

A bit related: open weights models are basically time capsules. These models have a knowledge cut off point and essentially forever live in that time.

I enjoyed chatting to Opus 3 recently around recent world events, as well as more recent agentic development patterns etc

Re: Claude's Cycles [pdf]

#128

Post by tzumaoli »

jdnier wrote: post #89

> I think Claude Shannon’s spirit is probably proud to know that his name is now being associated with such advances. Hats off to Claude! I didn't realize Claude was named after Claude Shannon! https://en.wikipedia.org/wiki/Claude_Shannon

Trivia: Claude Shannon proposed the idea of predicting the next token (letter) using statistics/probabilities in the training data corpus in 1950: "Prediction and Entropy of Printed English" https://languagelog.ldc.upenn.edu/myl/Shannon1950.pdf

Re: Claude's Cycles [pdf]

#129

Post by slibhb »

emp17344 wrote: post #108
ecshafer wrote: post #16

I wonder how long we have until we start solving some truly hard problems with AI. How long until we throw AI at "connect general relativity and quantum physics", give the AI 6 months and a few data centers, and have it pop out a solution?

Hold your horses, that’s a long way off. The best math AI tool we currently have, Aletheia, was only able to solve 13 out of 700 attempted open Erdos problems, only 4 of which were solved autonomously: https://arxiv.org/html/2601.22401v3 Clearly, these models still struggle with novel problems.

> Clearly, these models still struggle with novel problems.

Do they struggle with novel problems more or less than humans?

Re: Claude's Cycles [pdf]

#130

Post by naughtyrabisu »

beej71 wrote: post #77

From my naive standpoint, LLMs like this seem to have some big strengths. One: possession of a superhuman expanse of knowledge. Two: making connections. Three: tireless trial and error. If you put those three things together, you end up with some cool stuff from time to time. Perhaps the proof of P!=NP is tied to an obscure connection that humans don't easily see due to individual lack of knowledge or predisposition…

Three: tireless trial and error. Cannot agree more. I figured this probably be the biggest advantage of LLM considering for other variables humans hold the same-level competency.
Post reply on HN