Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

Post reply on HN
Print view View original

351–360 of 376 posts

Re: Claude's Cycles [pdf]

#351

Post by dilap »

earthscienceman wrote: post #323
dilap wrote: post #122

Earlier quoted context omitted.

The fundamental nature of the model is that it consumes tokens as input and produces token probabilities as output, but there's nothing inherently "predictive" about it -- that's just perspective hangover from the historical development of how LLMs were trained. It is, fundamentally, I think, a general-purpose thinking machine, operating over the inputs and outputs of tokens. (With this perspective, I can feel my own…

Non-sequitor: "perspective hangover" might be my favorite phrase I've ever read. So much of what we deal with is trying to correct-the-record on how we used to think about things. But the inertia that old ideas or modes have is monumental to overcome. If you just came up with that, kudos.

Ha, thanks!

Re: Claude's Cycles [pdf]

#352

Post by ajam1507 »

rustyhancock wrote: post #26
ecshafer wrote: post #16

I wonder how long we have until we start solving some truly hard problems with AI. How long until we throw AI at "connect general relativity and quantum physics", give the AI 6 months and a few data centers, and have it pop out a solution?

I think a very long time because part of our limit is experiment. We need enough experimental results to explain to solve these theoretical mismatches and we don't and at present can't explore that frontier. Once we have more results at that frontier we'd build a theory out from there that has two nearly independent limits for QFT and GR. What we'd be asking if the AI is something that we can't expect a human to solv…

This assumes that what's holding back solving hard problems is designing experiments to get novel data. Einstein's though experiments were very productive despite not taking place in a lab.

Re: Claude's Cycles [pdf]

#353

Post by dilap »

gpm wrote: post #336
dilap wrote: post #122

Earlier quoted context omitted.

The fundamental nature of the model is that it consumes tokens as input and produces token probabilities as output, but there's nothing inherently "predictive" about it -- that's just perspective hangover from the historical development of how LLMs were trained. It is, fundamentally, I think, a general-purpose thinking machine, operating over the inputs and outputs of tokens. (With this perspective, I can feel my own…

We could argue about whether fine tuning is still about predicting a distribution or not, but really I feel like whether or not that word is accurate misses the point of why the description is useful. I like the phrasing because it distinguishes it from other things the generative model might be doing including: - Creating and then refining the whole response simultaneously, like diffusion models do. - Having hidden…

Welllll I'm not so sure that phrase is well-suited for your intended meaning, then. (Also, tangentially, I think could argue thinking models w/ the elided thought prelude satisfy "having hidden state where it first forms an opinion.")

Re: Claude's Cycles [pdf]

#354

Post by bitexploder »

supern0va wrote: post #322
bitexploder wrote: post #92

Earlier quoted context omitted.

That is a good area to explore. Their map of the past is fixed. They are frozen at some point in their psychological time. What has stopped working? Their hippocampus and medial temporal lobe. These are like the write-head that move data from the hippocampus to the neo cortex. Their "I" can no longer update itself. Their DMN is frozen in time. So if intelligence is purely the "I" telling a continuous coherent story a…

>But the human still has the capacity to rewire at least some of their brain in real time even with amnesia. Sure, but just because LLMs don't have what we'd describe as human intelligence, doesn't mean they don't have intelligence. I think we're witnessing the creation and growth a weird new type of intelligence right now.

Anyone who dismisses your assertion is not very curious. What I am more interested in is what are its limits and can it perform novel reasoning. It probably needs efficient enough novel reasoning to update itself with new information to become a general reasoning intelligence capable of solving unknown problems. Right now they operate purely in the domain of words. They solve problems with words. They don’t seem to have very complex semantic maps. They approximate semantic maps with statistical brute force by generating words. They have a model of the past to generate the words. When something matches the word map is easy. When something is not reducible or did not have a good word match the only thing it can do is experimentally generate words until it seems to match the problem. But it is brute force. It is good they can solve known problems that fit known problem shapes. But their language dependency makes this very fragile. Without semantic meaning it has no way to evaluate if it is hallucinating easiy.

Re: Claude's Cycles [pdf]

#355

Post by rustyhancock »

ajam1507 wrote: post #352
rustyhancock wrote: post #26

Earlier quoted context omitted.

I think a very long time because part of our limit is experiment. We need enough experimental results to explain to solve these theoretical mismatches and we don't and at present can't explore that frontier. Once we have more results at that frontier we'd build a theory out from there that has two nearly independent limits for QFT and GR. What we'd be asking if the AI is something that we can't expect a human to solv…

This assumes that what's holding back solving hard problems is designing experiments to get novel data. Einstein's though experiments were very productive despite not taking place in a lab.

Einstein and SR is an interesting case.

Nearly the entirety of the theory had already been laid out before Einstein.

Lorenz transforms contain the length contraction and local time, Poincaré had already written about E=mc^2 for radiation, he'd also set out the idea of relativity. All this before 1905.

Einstein's revolution was in turning that patchwork into a self contained theory with a couple postulates.

He had all the data he could want most of it from decades earlier.

We have approximately 0 experimental evidence at the GR/QFT boundary.

The best we have is Hawkings radiation something we currently can't possibly observe experimentally.

If we wanted to study the GR/QFT with a particle accelerator it would need to be the size of the Milky Way.

Re: Claude's Cycles [pdf]

#356

Post by ajam1507 »

rustyhancock wrote: post #355
ajam1507 wrote: post #352

Earlier quoted context omitted.

This assumes that what's holding back solving hard problems is designing experiments to get novel data. Einstein's though experiments were very productive despite not taking place in a lab.

Einstein and SR is an interesting case. Nearly the entirety of the theory had already been laid out before Einstein. Lorenz transforms contain the length contraction and local time, Poincaré had already written about E=mc^2 for radiation, he'd also set out the idea of relativity. All this before 1905. Einstein's revolution was in turning that patchwork into a self contained theory with a couple postulates. He had all…

Again, this is all assuming that we have formulated the problem correctly. So much of the value of experiments in solving hard problems is not so much the results, but in how those results steer the formulation of a theory. It's hard to know how much evidence we might already have for resolving GR/QFT without the benefit of hindsight.

Re: Claude's Cycles [pdf]

#357

Post by zackmorris »

zackmorris wrote: post #148

Amazing paper. The simulated annealing portion reminds me of genetic algorithms (GAs). A good intro to that are the Genetic Programming series of books by John Koza, I read III in the early 2000s: https://www.amazon.com/Genetic-Programming-III-Darwinian-Inv... https://www.genetic-programming.com/ Note that the Python solution in the pdf is extremely short, so could have been found by simply trying permutations of mat…

*Project Hail Mary

Re: Claude's Cycles [pdf]

#358

Post by throw310822 »

empath75 wrote: post #346
qsera wrote: post #327

Earlier quoted context omitted.

People don't hallucinate. That is they can pretty reliably assess if they know or don't know something.

Your comment is a perfect example of a human hallucinating something and not knowing they are wrong about it. People are confidently wrong about things _all the time_.

You say this confidently, but you're wrong... ah no wait... ;)

Re: Claude's Cycles [pdf]

#360

Post by hmmmmmmmmmmmmmm »

red75prime wrote: post #188
hmmmmmmmmmmmmmm wrote: post #118

Earlier quoted context omitted.

Hamiltonian paths and previous work by Donald Knuth is more than likely in the training data.

The specific sequence of tokens that comprise the Knuth's problem with an answer to it is not in the training data. A naive probability distribution based on counting token sequences that are present in the training data would assign 0 probability to it. The trained network represents extremely non-naive approach to estimating the ground-truth distribution (the distribution that corresponds to what a human brain migh…

Obviously there is some level of memorisation involved. That's why you can even get LLMs to write parts of Harry Potter from scratch with perfect precision.
Post reply on HN