Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

Post reply on HN
Print view View original

191–200 of 376 posts

Re: Claude's Cycles [pdf]

#191

Post by emp17344 »

whimsicalism wrote: post #174
emp17344 wrote: post #104

Earlier quoted context omitted.

What goalposts do you think are being moved? I constantly see AI enthusiasts use this phrase, but it’s not clear what goalposts they have in mind. Specifically, what is it that you want opponents to recognize that you believe they aren’t currently? We now have a tool that can be useful in some narrow domains in some narrow cases. It’s pretty neat that our tools have new capabilities, but it’s also pretty far from AGI…

You must have forgotten the /s at the end of your comment?

Uh, no? You think LLMs are AGI?

Re: Claude's Cycles [pdf]

#192

Post by wang_li »

StilesCrisis wrote: post #143
sosodev wrote: post #126

Earlier quoted context omitted.

The model that processes search results is tiny and dumb. You shouldn't compare it to the frontier models that are solving complex math problems.

On Google, just clicking "AI Mode" gives you a substantially smarter model, and it's still pretty weak. But I assume the OP wasn't talking about Google because it doesn't seem to make this mistake even in a search.

It was bing as that is the default for Edge as supplied on my work laptop. It doesn't do this now, but it does do something else quite weird:

search: was val kilmer pregnant or in heat

answer: Not pregnant Val Kilmer was not pregnant or in heat during the events of "Heat." His character, Chris Shiherlis, is involved in a shootout and is shot, which indicates he is not in a reproductive or mating state at that time.

And then cites wikipedia as the source of information.

In terms of cognition the answer is meaningless. Nothing in the question implies or suggests that the question has to do with a movie. Additionally, "involved in a shootout and is shot, which indicates he is not in a reproductive or mating state" makes no sense at all.

AI as deployed shows no intelligence.

Re: Claude's Cycles [pdf]

#193

Post by Barbing »

beej71 wrote: post #77

From my naive standpoint, LLMs like this seem to have some big strengths. One: possession of a superhuman expanse of knowledge. Two: making connections. Three: tireless trial and error. If you put those three things together, you end up with some cool stuff from time to time. Perhaps the proof of P!=NP is tied to an obscure connection that humans don't easily see due to individual lack of knowledge or predisposition…

Well put.

>If you put [possession of a superhuman expanse of knowledge, making connections, tireless trial and error] together, you end up with some cool stuff from time to time.

Hard to argue.

Re: Claude's Cycles [pdf]

#194

Post by pvillano »

ainiriand wrote: post #4

Are not LLMs supposed to just find the most probable word that follows next like many people here have touted? How this can be explained under that pretense? Is this way of problem solving 'thinking'?

Does water flowing through a maze solve it by 'thinking'? No. The rules of physics eventually result in the water flowing out the exit. Water also hits every dead end along the way.

The power of LLMs is that by only selecting sequences of words that fit a statistical model, they avoid a lot of dead ends.[^1]

I would not call that, by itself, thinking. However, if you start with an extrapolation engine and add the ability to try multiple times and build on previous results, you get something that's kind of like thinking.

[1]: Like, a lot of dead ends. There are an unfathomable number of dead ends in generating 500 characters of code, and it is a miracle of technology that Claude only hit 30.

Re: Claude's Cycles [pdf]

#195

Post by UltraSane »

slopinthebag wrote: post #187
UltraSane wrote: post #180

Earlier quoted context omitted.

How can you distinguish intelligence form a sufficiently accurate imitation of intelligence?

By "sufficiently accurate" do you mean identical? Because if so, it's not an imitation of intelligence at all, and the question is thus nonsensical.

"it's not an imitation of intelligence at all"

But that is the key insight, how can you tell when an imitation of intelligence becomes the real thing?

Re: Claude's Cycles [pdf]

#196

Post by skinner_ »

mwigdahl wrote: post #20
bluGill wrote: post #12

Earlier quoted context omitted.

Was it? It was an open problem to Knuth - who generally knows how to search literature. However there is enough literature to search that it wouldn't be a surprise at all to discover it was already solved but he just used slightly different terms and so didn't find it. Or maybe it was sovled because this is a specialization of something that looks unrelated and so he wouldn't have realized it when he read it. Or... O…

Agreed with all of that, but with the added point that Knuth has done a lot of work in this exact area in The Art of Computer Programming Volume 4. If he considers this conjecture open given his particular knowledge of the field, it likely is (although agreed, it's not guaranteed).

Also, if Claude had regurgitated a known solution, it would have come up with it in the first exploration round, not the 31st, as it actually did.

Re: Claude's Cycles [pdf]

#197

Post by kami23 »

brcmthrowaway wrote: post #165
faxmeyourcode wrote: post #94

> Filip also told me that he asked Claude to continue on the even case after the odd case had been resolved. “But there after a while it seemed to get stuck. In the end, it was not even able to write and run explore programs correctly anymore, very weird. So I stopped the search.” Interesting snippet towards the end. I wonder if they were using claude.ai or claude code. Sounds like they ran out of context and entered…

What is dumb zone?

When the LLMs start compacting they summarize the conversation up to that point using various techniques. Overall a lot of maybe finer points of the work goes missing and can only be retrieved by the LLM being told to search for it explicitly in old logs.

Once you compact, you've thrown away a lot of relevant tokens from your problem solving and they do become significantly dumber as a result. If I see a compaction coming soon I ask it to write a letter to its future self, and then start a new session by having it read the letter.

There are some days where I let the same session compact 4-5 times and just use the letter to future self method to keep it going with enough context because resetting context also resets my brain :)

If you're ever curious in Claude once you compact you can read the new initial prompt after compaction and see how severe it gets cut down. It's very informative of what it forgets and deems not important. For example I have some internal CLIs that are horribly documented so Claude has to try a few flags a few times to figure out specifics and those corrections always get thrown away and it has to relearn them next time it wants to use the CLI. If you notice things like that happening constantly, my move is to codify those things into my CLAUDE.md or lately I've been making a small script or MCP server to run very specific flags of stuff.

Re: Claude's Cycles [pdf]

#198

Post by dotancohen »

bitexploder wrote: post #21
Aerroon wrote: post #3

Earlier quoted context omitted.

A bit related: open weights models are basically time capsules. These models have a knowledge cut off point and essentially forever live in that time.

This is the most fundamental argument that they are not, directly, an intelligence. They are not ever storing new information on a meaningful timescale. However, if you viewed them on some really large macro time scale where now LLMs are injecting information into the universe and the re-ingesting that maybe in some very philosophical way they are a /very/ slow oscillating intelligence right now. And as we narrow tha…

  > This is the most fundamental argument that they are not, directly, an intelligence. They are not ever storing new information on a meaningful timescale.
All major LLMs today have a nontrivial context window. Whether or not this constitutes "a meaningful timescale" is application dependant - for me it has been more than adequate.

I also disagree that this has any bearing on whether or not "the machine is intelligent" or whether or not "submarines can swim".

Re: Claude's Cycles [pdf]

#199

Post by [deleted] »

kami23 wrote: post #197
brcmthrowaway wrote: post #165

Earlier quoted context omitted.

What is dumb zone?

When the LLMs start compacting they summarize the conversation up to that point using various techniques. Overall a lot of maybe finer points of the work goes missing and can only be retrieved by the LLM being told to search for it explicitly in old logs. Once you compact, you've thrown away a lot of relevant tokens from your problem solving and they do become significantly dumber as a result. If I see a compaction c…

[deleted]

Re: Claude's Cycles [pdf]

#200

Post by lmf4lol »

charcircuit wrote: post #41
mlyle wrote: post #34

Earlier quoted context omitted.

There's nothing to say that you can't build something intelligent out of them by bolting a memory on it, though. Sure, it's not how we work, but I can imagine a system where the LLM does a lot of heavy lifting and allows more expensive, smaller networks that train during inference and RAG systems to learn how to do new things and keep persistent state and plan.

Memory is not just bolted on top of the latest models. They under go training on how and when to effectively use memory and how to use compaction to avoid running out of context when working on problems.

do you have some reading material to share on this matter?

thanks already

Post reply on HN