Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

Post reply on HN
Print view View original

61–70 of 376 posts

Re: Claude's Cycles [pdf]

#61

Post by someplaceguy »

worldsavior wrote: post #57
zarzavat wrote: post #45

Earlier quoted context omitted.

I swear that AI could independently develop a cure for cancer and people would still say that it's not actually intelligent, just matrix multiplications giving a statistically probable answer! LLMs are at least designed to be intelligent. Our monkey brains have much less reason to be intelligent, since we only evolved to survive nature, not to understand it. We are at this moment extremely deep into what most people…

That's wrong. Humans were evolved to have big brains so they can better understand the env and use it to their advantage. I still see AI making stupid silly mistakes. I rather think and not waste time on something that only remembers data, and doesn't even understand it. Reasoning in AI is only about finding contradictions between his "thoughts", not actually understand it.

> I still see AI making stupid silly mistakes.

In contrast with humans, who are famously known for never making stupid silly mistakes...

Re: Claude's Cycles [pdf]

#62

Post by adamtaylor_13 »

ainiriand wrote: post #4

Are not LLMs supposed to just find the most probable word that follows next like many people here have touted? How this can be explained under that pretense? Is this way of problem solving 'thinking'?

That's the way many people reduce it, and mathematically, I think that's true. I think what we fail to realize is just far that will actually take you.

"just the most probable word" is a pretty powerful mechanism when you have all of human knowledge at your fingertips.

I say that people "reduce it" that way because it neatly packs in the assumption that general intelligence is something other than next token prediction. I'm not saying we've arrived at AGI, in fact, I do not believe we have. But, it feels like people who use that framing are snarkily writing off something that they themselves to do not fully comprehend behind the guise of being "technically correct."

I'm not saying all people do this. But I've noticed many do.

Re: Claude's Cycles [pdf]

#63

Post by _fizz_buzz_ »

worldsavior wrote: post #57
zarzavat wrote: post #45

Earlier quoted context omitted.

I swear that AI could independently develop a cure for cancer and people would still say that it's not actually intelligent, just matrix multiplications giving a statistically probable answer! LLMs are at least designed to be intelligent. Our monkey brains have much less reason to be intelligent, since we only evolved to survive nature, not to understand it. We are at this moment extremely deep into what most people…

That's wrong. Humans were evolved to have big brains so they can better understand the env and use it to their advantage. I still see AI making stupid silly mistakes. I rather think and not waste time on something that only remembers data, and doesn't even understand it. Reasoning in AI is only about finding contradictions between his "thoughts", not actually understand it.

> I still see AI making stupid silly mistakes.

Humans also make silly mistakes.

Re: Claude's Cycles [pdf]

#64

Post by visarga »

mccoyb wrote: post #2

It's fascinating to think about the space of problems which are amenable to RL scaling of these probability distributions. Before, we didn't have a fast (we had to rely on human cognition) way to try problems - even if the techniques and workflows were known by someone. Now, we've baked these patterns into probability distributions - anyone can access them with the correct "summoning spell". Experts will naturally us…

> In 2030, how is Anthropic going to keep Claude "up-to-date"

I think the majority of research, design and learning goes through LLMs and coding agents today, considering the large user base and usage it must be trillions of tokens per day. You can take a long research session or a series of them and apply hindsight - what idea above can be validated below? This creates a dense learning signal based on validation in real world with human in the loop and other tools, code & search.

Re: Claude's Cycles [pdf]

#66

Post by allreduce »

miroljub wrote: post #7

Solves? It's a part of the training set. Nothing more, nothing less.

I encourage you to look at what the current models with a bit of harnessing are capable of, e.g. Opus 4.6 and Claude Code. Try to make it solve some mathematics-heavy problem you come up with. If only to get a more accurate picture of whats going on.

Unfortunately, these tools generalize way beyond regurgitating the training set. I would not assume they stay below human capabilities in the next few years.

Why any moral person would continue building these at this point I don't know. I guess in the best case the future will have a small privileged class of humans having total power, without need for human workers or soldiers. Picture a mechanical boot stomping on a human face forever.

Re: Claude's Cycles [pdf]

#67

Post by rustyhancock »

fragmede wrote: post #47
rustyhancock wrote: post #26

Earlier quoted context omitted.

I think a very long time because part of our limit is experiment. We need enough experimental results to explain to solve these theoretical mismatches and we don't and at present can't explore that frontier. Once we have more results at that frontier we'd build a theory out from there that has two nearly independent limits for QFT and GR. What we'd be asking if the AI is something that we can't expect a human to solv…

The question is, if you trained an LLM on everything up until 1904, could it come up with E=MC² or not?

In 1900 Henri Poincaré wrote that radiation (light) has an effective mass given by E/c^2.

So it really isn't far fetched. What intrigues me more is if it was capable of it would our Victorian conservative minded scientists have RLHF it out of that kind of thing?

Re: Claude's Cycles [pdf]

#68

Post by wang_li »

morleytj wrote: post #51
dtj1123 wrote: post #39

Earlier quoted context omitted.

Would you consider someone with anterograde amnesia not to be intelligent?

A very good point. For anyone not familiar with anterograde amnesia, the classical case is patient H.M. ( https://en.wikipedia.org/wiki/Henry_Molaison ), whose condition was researched by Brenda Milner.

Or you could have just said "they can't form new memories."

Re: Claude's Cycles [pdf]

#69

Post by throw310822 »

qsera wrote: post #60
throw310822 wrote: post #42

Earlier quoted context omitted.

> just find the most probable word that follows next Well, if in all situations you can predict which word Einstein would probably say next, then I think you're in a good spot. This "most probable" stuff is just absurd handwaving. Every prompt of even a few words is unique, there simply is no trivially "most probable" continuation. Probable given what? What these machines learn to do is predicting what intelligence w…

>Probable given what? The training data.. >predicting what intelligence would do No, it just predict what the next word would be if an intelligent entity translated its thoughts to words. Because it is trained on the text that are written by intelligent entities. If it was trained on text written by someone who loves to rhyme, you would be getting all rhyming responses. It imitates the behavior -- in text -- of what…

> The training data

If the prompt is unique, it is not in the training data. True for basically every prompt. So how is this probability calculated?

Re: Claude's Cycles [pdf]

#70

Post by fazkan »

time to use claude code to understand DEKs paper, in plain English. As someone who did a bit of formal verification in grad school. I feel like, there are a long tail of problems that can be solved by human-model collab like this one. The problems may not mean much but hopefully it can stack up understanding of intelligence.
Post reply on HN