Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's Cycles [pdf]

www-cs-faculty.stanford.edu

Post reply on HN
Print view View original

301–310 of 376 posts

Re: Claude's Cycles [pdf]

#301

Post by mccoyb »

wvlia5 wrote: post #282
mccoyb wrote: post #2

It's fascinating to think about the space of problems which are amenable to RL scaling of these probability distributions. Before, we didn't have a fast (we had to rely on human cognition) way to try problems - even if the techniques and workflows were known by someone. Now, we've baked these patterns into probability distributions - anyone can access them with the correct "summoning spell". Experts will naturally us…

This seems to be a bot comment. HN will lose its value if these bots are not purged.

Tune your bot detector, I'm a real person and I think about my comments before posting them.

Re: Claude's Cycles [pdf]

#302

Post by quinndupont »

Interesting to see the mathematical solution space get optimized away. On account of “there’s no accounting for taste” this actually makes me hopeful that creative workers have durable skills that can’t be optimized, which I can’t say about mathematics and computer science.

Re: Claude's Cycles [pdf]

#303

Post by wvlia5 »

wvlia5 wrote: post #282
mccoyb wrote: post #2

It's fascinating to think about the space of problems which are amenable to RL scaling of these probability distributions. Before, we didn't have a fast (we had to rely on human cognition) way to try problems - even if the techniques and workflows were known by someone. Now, we've baked these patterns into probability distributions - anyone can access them with the correct "summoning spell". Experts will naturally us…

This seems to be a bot comment. HN will lose its value if these bots are not purged.

Moderators: banning all accounts since 2025 from posting would be better than doing nothing. Not the solution we want, but what we have for now.

Re: Claude's Cycles [pdf]

#304

Post by beej71 »

IAmGraydon wrote: post #178
beej71 wrote: post #77

From my naive standpoint, LLMs like this seem to have some big strengths. One: possession of a superhuman expanse of knowledge. Two: making connections. Three: tireless trial and error. If you put those three things together, you end up with some cool stuff from time to time. Perhaps the proof of P!=NP is tied to an obscure connection that humans don't easily see due to individual lack of knowledge or predisposition…

>One: possession of a superhuman expanse of knowledge. Two: making connections. Three: tireless trial and error. One and three I believe are correct. The second point, making connections, is something LLMs seem to be incapable of truly doing unless the connection is already known and in its training data.

I agree partially, but I think there might be a ton of connections in the training data that aren't obvious to humans. And being a word prediction engine is all about making those connections.

Re: Claude's Cycles [pdf]

#305

Post by lhl »

I was a bit interested to do a replication and see if better harness could avoid some of the problems they ran w/ context management, poor instruction following, etc and it looks like yes, it's definitely possible.

Here's my repo: https://github.com/lhl/claudecycles-revisited

I used Codex w/ 5.2 xhigh and a relatively simple AGENTS.md - I have some session-analysis as well. The original replication was 47 minutes, then another 30 minutes of gap filling, and finally about 30 minutes of writing an extension to take the work a bit further, with Claude Code Opus 4.6 doing some documentation cleanup and verification.

Re: Claude's Cycles [pdf]

#306

Post by carterschonwald »

lhl wrote: post #305

I was a bit interested to do a replication and see if better harness could avoid some of the problems they ran w/ context management, poor instruction following, etc and it looks like yes, it's definitely possible. Here's my repo: https://github.com/lhl/claudecycles-revisited I used Codex w/ 5.2 xhigh and a relatively simple AGENTS.md - I have some session-analysis as well. The original replication was 47 minutes, th…

omg this is so cool. because im writing my own harness and i need some cognitive benchmarks. i have a bunch of harness level infra around llm interactions that seems to help with reasoning, but i dont have a structured way evaluate things

thx for sharing your test setup, i really appreciate the time you took. this will help me so much

Re: Claude's Cycles [pdf]

#307

Post by pushedx »

lhl wrote: post #305

I was a bit interested to do a replication and see if better harness could avoid some of the problems they ran w/ context management, poor instruction following, etc and it looks like yes, it's definitely possible. Here's my repo: https://github.com/lhl/claudecycles-revisited I used Codex w/ 5.2 xhigh and a relatively simple AGENTS.md - I have some session-analysis as well. The original replication was 47 minutes, th…

As described in the readme of your repo (did you read it?) your agent found the Knuth paper located one directory level above its working directory.

So, you didn't produce a replication in 47 minutes, it just took around 30 minutes for your agent to find that you had the answer in a PDF in a nearby directory.

Re: Claude's Cycles [pdf]

#308

Post by igravious »

buffalobuffalo wrote: post #182
konne88 wrote: post #120

I didn't expect such a misleading intro from Knuth. It reads like Claude solved Knuth's math problem. In reality, Claude generated various example solution, and Knuth then manually generalized that to a formal proof. What Claude did is certainly useful, but it would have been nice to be clear about the scope of the contribution in the intro.

While not on the same level as these guys, I've done some similar stuff using Claude. This is a classic synergy example, where the output of human + LLM is far greater than just the human or just the LLM working on a problem. My experience has been that the LLM lacks fine grained judgement when it comes to allocating resources, or choosing a direction to work in. But once a direction is pointed out, it can do a deep…

> But with someone holding the leash

i've been thinking about why we call them agent harnesses

i know all analogies suck in different ways but here goes:

coding agents are like horses. without a harness and bridle they'll the horse will do as it pleases -- a human can't travel very far and fast by foot but put a bridle and a harness on a horse, give it a bit of coaxing with carrot and stick, add in a bit a pointing the thing in the right direction and bingo you're off to the races!

Re: Claude's Cycles [pdf]

#309

Post by roywiggins »

vidarh wrote: post #270
gpm wrote: post #136

Earlier quoted context omitted.

You're right of course, but at the point where you're saying "well we can make a turing machine with the LLM as the transition function by defining some tool calls for the LLM to interact with the tape" it feels like a stretch to call the LLM itself turing complete. Also people definitely talk about them as "thinking" in contexts where they haven't put a harness capable of this around them. And in the common contexts…

That's why I specifically didn't call the LLM itself Turing complete, but stated that if you put a loop around a Turing machine you can trivially make it Turing complete. Maybe I should have been clearer and write "the combined system" instead of it. But the point is that this is irrelevant , because it is proof that unlesss human brains exceed the Turing computable, LLM's can at least theoretically be made to think.…

I am not sure it is proof, at least not in an interesting way. It's also proof that Magic: The Gathering could theoretically be made to think. Which is true but doesn't tell you anything much about MtG other than that it is a slightly complicated ruleset that has a couple of properties that are pretty common.

I think both sides of this end up proving "too much" in their respective directions.

Re: Claude's Cycles [pdf]

#310

Post by morleytj »

goodmythical wrote: post #214
pdntspa wrote: post #210

Earlier quoted context omitted.

Sure, if you want to speak with the precision of a sledgehammer instead of a scalpel

lol, as if pointing at a wikipedia article (without any relevant discussion of the contents therein) is some kind of conversational excellence. Or perhaps you were referring to the impact of the two in that the "sledgehammer" of "they can't make new memories" is a lot more effective than the tiny scalpel of "if you do a wikipedia search this is a single one of the relevant articles"

The extra information is that he is the canonical case which defined our clinical understanding of the condition. Not just a "single relevant article."

I pulled it up because I was familiar with this fact.

Post reply on HN