Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude is good at assembling blocks, but still falls apart at creating them

approachwithalacrity.com

Post reply on HN
Print view View original

31–40 of 249 posts

Re: Claude is good at assembling blocks, but still falls apart at creating them

#31

Post by maxilevi »

johnisgood wrote: post #11
maxilevi wrote: post #8

LLMs are just really good search. Ask it to create something and it's searching within the pretrained weights. Ask it to find something and it's semantically searching within your codebase. Ask it to modify something and it will do both. Once you understand its just search, you can get really good results.

Calling it "just search" is like calling a compiler "just string manipulation". Not false, but aggressively missing the point.

I don't mean search in the reductionist way but rather that its much better at translating, finding and mapping concepts if everything is provided vs creating from scratch. If it could truly think it would be able to bootstrap creations from basic principles like we do, but it really can't. Doesn't mean its not a great powerful tool.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#32

Post by jvanderbot »

soulofmischief wrote: post #26
jvanderbot wrote: post #18

Earlier quoted context omitted.

What would you add? To me it's "search" like a missile does "flight". It's got a target and a closed loop guidance, and is mostly fire and forget (for search). At that, it excels. I think the closed loop+great summary is the key to all the magic.

It's a prediction algorithm that walks a high-dimensional manifold, in that sense all application of knowledge it just "search", so yes, you're fundamentally correct but still fundamentally wrong since you think this foundational truth is the end and beginning of what LLMs do, and thus your mental model does not adequately describe what these tools are capable of.

Me? My mental model? I gave an analogy for Claude not a explanation for LLMs.

But you know what? I was mentally thinking of both deep think / research and Claude code, both of which are literally closed loop. I see this is slightly off topic b/c others are talking about the LLM only.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#33

Post by bblcla »

simonw wrote: post #30

I'm not entirely convinced by the anecdote here where Claude wrote "bad" React code: > But in context, this was obviously insane. I knew that key and id came from the same upstream source. So the correct solution was to have the upstream source also pass id to the code that had key, to let it do a fast lookup. I've seen Claude make mistakes like that too, but then the moment you say "you can modify the calling code a…

(Author here)

> I'm not entirely convinced by the anecdote here where Claude wrote "bad" React code

Yeah, that's fair - a friend of mine also called this out on Twitter (https://x.com/konstiwohlwend/status/2010799158261936281) and I went into more technical detail about the specific problem there.

> I've seen Claude make mistakes like that too, but then the moment you say "you can modify the calling code as well" or even ask "any way we could do this better?" it suggests the optimal solution.

I agree, but I think I'm less optimistic than you that Claude will be able to catch its own mistakes in the future. On the other hand, I can definitely see how a ~more intelligent model might be able to catch mistakes on a larger and larger scale.

> I expect that adding a CLAUDE.md rule saying "always look for more efficient implementations that might involve larger changes and propose those to the user for their confirmation if appropriate" might solve the author's complaint here.

I'm not sure about this! There are a few things Claude does that seem unfixable even by updating CLAUDE.md.

Some other footguns I keep seeing in Python and constantly have to fix despite CLAUDE.md instructions are:

- writing lots of nested if clauses instead of writing simple functions by returning early

- putting imports in functions instead of at the top-level

- swallowing exceptions instead of raising (constantly a huge problem)

These are small, but I think it's informative of what the models can do that even Opus 4.5 still fails at these simple tasks.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#34

Post by joshribakoff »

simonw wrote: post #30

I'm not entirely convinced by the anecdote here where Claude wrote "bad" React code: > But in context, this was obviously insane. I knew that key and id came from the same upstream source. So the correct solution was to have the upstream source also pass id to the code that had key, to let it do a fast lookup. I've seen Claude make mistakes like that too, but then the moment you say "you can modify the calling code a…

I expect that adding instructions that attempt to undo training produces worse results than not including the overbroad generalization in the training in the first place. I think the author isn’t making a complaint they’re documenting a tradeoff.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#35

Post by AIorNot »

simonw wrote: post #30

I'm not entirely convinced by the anecdote here where Claude wrote "bad" React code: > But in context, this was obviously insane. I knew that key and id came from the same upstream source. So the correct solution was to have the upstream source also pass id to the code that had key, to let it do a fast lookup. I've seen Claude make mistakes like that too, but then the moment you say "you can modify the calling code a…

Well yes but the wider point is that it takes new Human skills to manage them - like a pair of horses so to speak under your bridle

When it comes down to it these AI tools are like going to power tools or machines from the artisanal era

- like going from surgical knife to a machine gun- so they operate at a faster pace without comprehending like humans - and without allowing humans time to comprehend all side effects and massive assumptions they make on every run in their context window

humans have to adapt to managing them correctly and at the right scale to be effective and that becomes something you learn

Re: Claude is good at assembling blocks, but still falls apart at creating them

#36

Post by Kuinox »

simonw wrote: post #30

I'm not entirely convinced by the anecdote here where Claude wrote "bad" React code: > But in context, this was obviously insane. I knew that key and id came from the same upstream source. So the correct solution was to have the upstream source also pass id to the code that had key, to let it do a fast lookup. I've seen Claude make mistakes like that too, but then the moment you say "you can modify the calling code a…

> My guess is that Claude is trained to bias towards making minimal edits to solve problems.

I don't have the same feeling. I find that claude tends to produce wayyyyy too much code to solve a problem, compared to other LLMs.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#37

Post by arcanemachiner »

soulofmischief wrote: post #25
andrei_says_ wrote: post #17

Earlier quoted context omitted.

“Novel” to the person who has not consumed the training data. Otherwise, just training data combined in highly probable ways. Not quite autocomplete but not intelligence either.

Citation needed that grokked capabilities in a sufficiently advanced model cannot combinatorially lead to contextually novel output distributions, especially with a skilled guiding hand.

Pretty sure burden of proof is on you, here.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#38

Post by soulofmischief »

jvanderbot wrote: post #32
soulofmischief wrote: post #26

Earlier quoted context omitted.

It's a prediction algorithm that walks a high-dimensional manifold, in that sense all application of knowledge it just "search", so yes, you're fundamentally correct but still fundamentally wrong since you think this foundational truth is the end and beginning of what LLMs do, and thus your mental model does not adequately describe what these tools are capable of.

Me? My mental model? I gave an analogy for Claude not a explanation for LLMs. But you know what? I was mentally thinking of both deep think / research and Claude code, both of which are literally closed loop. I see this is slightly off topic b/c others are talking about the LLM only.

Sorry, I should have said "analogy" and not "mental model", that was presumptuous. Maybe I also should have replied to the GP comment instead.

Anyway, since we're here, I personally think giving LLMs agency helps unlock this latent knowledge, as it provides the agent more mobility when walking the manifold. It has a better chance at avoiding or leaving local minima/maxima, among other things. So I don't know if agentic loops are entirely off-topic when discussing the latent power of LLMs.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#39

Post by soulofmischief »

arcanemachiner wrote: post #37
soulofmischief wrote: post #25

Earlier quoted context omitted.

Citation needed that grokked capabilities in a sufficiently advanced model cannot combinatorially lead to contextually novel output distributions, especially with a skilled guiding hand.

Pretty sure burden of proof is on you, here.

It's not, because I haven't ruled out the possibility. I could share anecdata about how my discussions with LLMs have led to novel insights, but it's not necessary. I'm keeping my mind open, but you're asserting an unproven claim that is currently not community consensus. Therefore, the burden of proof is on you.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#40

Post by pc86 »

andrei_says_ wrote: post #17
bhadass wrote: post #16

Earlier quoted context omitted.

better mental model: it's a lossy compression of human knowledge that can decompress and recombine in novel (sometimes useful, sometimes sloppy) ways. classical search simply retrieves, llms can synthesize as well.

“Novel” to the person who has not consumed the training data. Otherwise, just training data combined in highly probable ways. Not quite autocomplete but not intelligence either.

What is the difference between "novel" and "novel to someone who hasn't consumed the entire corpus of training data, which is several orders of magnitude greater than any human being could consume?"
Post reply on HN