Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

Post reply on HN
Print view View original

51–60 of 376 posts

Re: Claude's Cycles [pdf]

#51

Post by morleytj »

dtj1123 wrote: post #39
bitexploder wrote: post #21

Earlier quoted context omitted.

This is the most fundamental argument that they are not, directly, an intelligence. They are not ever storing new information on a meaningful timescale. However, if you viewed them on some really large macro time scale where now LLMs are injecting information into the universe and the re-ingesting that maybe in some very philosophical way they are a /very/ slow oscillating intelligence right now. And as we narrow tha…

Would you consider someone with anterograde amnesia not to be intelligent?

A very good point. For anyone not familiar with anterograde amnesia, the classical case is patient H.M. (https://en.wikipedia.org/wiki/Henry_Molaison), whose condition was researched by Brenda Milner.

Re: Claude's Cycles [pdf]

#52

Post by bobbylarrybobby »

worldsavior wrote: post #24
ecshafer wrote: post #16

I wonder how long we have until we start solving some truly hard problems with AI. How long until we throw AI at "connect general relativity and quantum physics", give the AI 6 months and a few data centers, and have it pop out a solution?

If AGI will ever come, then. Currently, AI is only a statistical machines, and solutions like this are purely based on distribution and no logic/actual intelligence.

Did you read the linked paper? Claude out-reasoned humans on a challenging (or at least, unsolved) math problem.

Re: Claude's Cycles [pdf]

#53

Post by adampunk »

ainiriand wrote: post #4

Are not LLMs supposed to just find the most probable word that follows next like many people here have touted? How this can be explained under that pretense? Is this way of problem solving 'thinking'?

Thinking is a big word that sweeps up a lot of different human behavior, so I don't know if it's right to jump to that; HOWEVER, explanations of LLMs that depend heavily on next-token prediction are defunct. They stopped being fundamentally accurate with the rise of massive reinforcement learning and w/ 'reasoning' models the analogy falls apart when you try to do work with it.

Be on the lookout for folks who tell you these machines are limited because they are "just predicting the next word." They may not know what they're talking about.

Re: Claude's Cycles [pdf]

#54

Post by rcarr »

Aerroon wrote: post #3
mccoyb wrote: post #2

It's fascinating to think about the space of problems which are amenable to RL scaling of these probability distributions. Before, we didn't have a fast (we had to rely on human cognition) way to try problems - even if the techniques and workflows were known by someone. Now, we've baked these patterns into probability distributions - anyone can access them with the correct "summoning spell". Experts will naturally us…

A bit related: open weights models are basically time capsules. These models have a knowledge cut off point and essentially forever live in that time.

Not an expert but surely it's only a matter of time until there's a way to update with the latest information without having to retrain on the entire corpus?

Re: Claude's Cycles [pdf]

#55

Post by andsoitis »

mccoyb wrote: post #2

It's fascinating to think about the space of problems which are amenable to RL scaling of these probability distributions. Before, we didn't have a fast (we had to rely on human cognition) way to try problems - even if the techniques and workflows were known by someone. Now, we've baked these patterns into probability distributions - anyone can access them with the correct "summoning spell". Experts will naturally us…

> Experts will naturally use these systems more productively, because they know how to coerce models into the correct conditional distributions which light up the right techniques.

Part of it comes down to “knowing” what questions to ask.

Re: Claude's Cycles [pdf]

#56

Post by ontouchstart »

ontouchstart wrote: post #36

Fascinating report by DEK himself. Time to sit down, read, digest and understand it without the help of LLM.

I don't have time to do that myself yet so I just dug a quick TL;DR rabbit hole for fun:

https://ontouchstart.github.io/rabbit-holes/llm_rabbit_hole_...

Re: Claude's Cycles [pdf]

#57

Post by worldsavior »

zarzavat wrote: post #45
worldsavior wrote: post #24

Earlier quoted context omitted.

If AGI will ever come, then. Currently, AI is only a statistical machines, and solutions like this are purely based on distribution and no logic/actual intelligence.

I swear that AI could independently develop a cure for cancer and people would still say that it's not actually intelligent, just matrix multiplications giving a statistically probable answer! LLMs are at least designed to be intelligent. Our monkey brains have much less reason to be intelligent, since we only evolved to survive nature, not to understand it. We are at this moment extremely deep into what most people…

That's wrong. Humans were evolved to have big brains so they can better understand the env and use it to their advantage.

I still see AI making stupid silly mistakes. I rather think and not waste time on something that only remembers data, and doesn't even understand it.

Reasoning in AI is only about finding contradictions between his "thoughts", not actually understand it.

Re: Claude's Cycles [pdf]

#58

Post by worldsavior »

bobbylarrybobby wrote: post #52
worldsavior wrote: post #24

Earlier quoted context omitted.

If AGI will ever come, then. Currently, AI is only a statistical machines, and solutions like this are purely based on distribution and no logic/actual intelligence.

Did you read the linked paper? Claude out-reasoned humans on a challenging (or at least, unsolved) math problem.

Merely luck in my opinion. There could be also multiple times where it didn't solve it.

Re: Claude's Cycles [pdf]

#59

Post by esafak »

andsoitis wrote: post #55
mccoyb wrote: post #2

It's fascinating to think about the space of problems which are amenable to RL scaling of these probability distributions. Before, we didn't have a fast (we had to rely on human cognition) way to try problems - even if the techniques and workflows were known by someone. Now, we've baked these patterns into probability distributions - anyone can access them with the correct "summoning spell". Experts will naturally us…

> Experts will naturally use these systems more productively, because they know how to coerce models into the correct conditional distributions which light up the right techniques. Part of it comes down to “knowing” what questions to ask.

I see it like the relationship between a student and research advisor. The advisor will ideally know the terrain and suggest a fruitful line of attack (what to ask), and the student will follow through, learning along the way.

Re: Claude's Cycles [pdf]

#60

Post by qsera »

throw310822 wrote: post #42
ainiriand wrote: post #4

Are not LLMs supposed to just find the most probable word that follows next like many people here have touted? How this can be explained under that pretense? Is this way of problem solving 'thinking'?

> just find the most probable word that follows next Well, if in all situations you can predict which word Einstein would probably say next, then I think you're in a good spot. This "most probable" stuff is just absurd handwaving. Every prompt of even a few words is unique, there simply is no trivially "most probable" continuation. Probable given what? What these machines learn to do is predicting what intelligence w…

>Probable given what?

The training data..

>predicting what intelligence would do

No, it just predict what the next word would be if an intelligent entity translated its thoughts to words. Because it is trained on the text that are written by intelligent entities.

If it was trained on text written by someone who loves to rhyme, you would be getting all rhyming responses.

It imitates the behavior -- in text -- of what ever entity that generated the training data. Here the training data was made by intelligent humans, so we get an imitation of the same.

It is a clever party trick that works often enough.

Post reply on HN