Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude is good at assembling blocks, but still falls apart at creating them

approachwithalacrity.com

Post reply on HN
Print view View original

131–140 of 249 posts

Re: Claude is good at assembling blocks, but still falls apart at creating them

#131

Post by nextaccountic »

emp17344 wrote: post #121
johnfn wrote: post #119

Earlier quoted context omitted.

Hallucinations are more or less a solved problem for me ever since I made a simple harness to have Codex/Claude check its work by using static typechecking.

But there aren’t very many domains where this type of verification is even possible.

Then you apply LLMs in domains where things can be checked

Indeed I expect to see a huge push into formally verified software just because sound mathematical proofs provide an excellent verifier to put into a LLM hardness. Just see how Aristotle has been successful at math, and it could be applied to coding too

Maybe Lean will become the new Python

https://harmonic.fun/news#blog-post-verina-bench-sota

Re: Claude is good at assembling blocks, but still falls apart at creating them

#132

Post by zeroonetwothree »

Uehreka wrote: post #102
disconcision wrote: post #94

I've yet to be convinced by any article, including this one, that attempts to draw boxes around what coding agents are and aren't good at in a way that is robust on a 6 to 12 month horizon. I agree that the examples listed here are relatable, and I've seen similar in my uses of various coding harnesses, including, to some degree, ones driven by opus 4.5. But my general experience with using LLMs for development over…

It feels like a lot of people keep falling into the trap of thinking we’ve hit a plateau, and that they can shift from “aggressively explore and learn the thing” mode to “teach people solid facts” mode. A week ago Scott Hanselman went on the Stack Overflow podcast to talk about AI-assisted coding. I generally respect that guy a lot, so I tuned in and… well it was kind of jarring. The dude kept saying things in this r…

Why is it every time anyone has a critique someone has to say “oh but you aren’t using model X, which clearly never has this problem and is far better”?

Yet the data doesn’t show all that much difference between SOTA models. So I have a hard time believing it.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#133

Post by woeirua »

It's just amazing to me how fast the goal posts are moving. Four years ago, if you had told someone that a LLM would be able to one-shot either of those first two tasks they would've said you're crazy. The tech is moving so fast. I slept on Opus 4.5 because GPT 5 was kind of an air ball, and just started using it in the past few weeks. It's so good. Way better than almost anything that's come before it. It can one-shot tasks that we never would've considered possible before.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#134

Post by [deleted] »

zeroonetwothree wrote: post #132
Uehreka wrote: post #102

Earlier quoted context omitted.

It feels like a lot of people keep falling into the trap of thinking we’ve hit a plateau, and that they can shift from “aggressively explore and learn the thing” mode to “teach people solid facts” mode. A week ago Scott Hanselman went on the Stack Overflow podcast to talk about AI-assisted coding. I generally respect that guy a lot, so I tuned in and… well it was kind of jarring. The dude kept saying things in this r…

Why is it every time anyone has a critique someone has to say “oh but you aren’t using model X, which clearly never has this problem and is far better”? Yet the data doesn’t show all that much difference between SOTA models. So I have a hard time believing it.

[deleted]

Re: Claude is good at assembling blocks, but still falls apart at creating them

#136

Post by TeMPOraL »

atty wrote: post #118
TeMPOraL wrote: post #112

Earlier quoted context omitted.

> The technology cannot create new concepts/abstractions, and so fails at abstraction. Reliably. That statement is way too strong, as it implies either that humans cannot create new concepts/abstractions, or that magic exists.

I think both your statement and their statement are too strong. There is no reason to think LLMs can do everything a human can do, which seems to be your implication. On the other hand, the technology is still improving, so maybe it’ll get there.

My take is that:

1) LLMs cannot do everything humans can, but

2) There's no fundamental reason preventing some future technology to do everything humans can, and

3) LLMs are explicitly designed and trained to mimic human capabilities in fully general sense.

Point 2) is the "or else magic exists" bit; point 3) says you need a more specific reason to justify assertion that LLMs can't create new concepts/abstractions, given that they're trained in order to achieve just that.

Note: I read OP as saying they fundamentally can't and thus never will. If they meant just that the current breed can't, I'm not going to dispute it.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#137

Post by nextaccountic »

bhadass wrote: post #16
maxilevi wrote: post #8

LLMs are just really good search. Ask it to create something and it's searching within the pretrained weights. Ask it to find something and it's semantically searching within your codebase. Ask it to modify something and it will do both. Once you understand its just search, you can get really good results.

better mental model: it's a lossy compression of human knowledge that can decompress and recombine in novel (sometimes useful, sometimes sloppy) ways. classical search simply retrieves, llms can synthesize as well.

Maybe the base model is just a compression of the training data?

There is also a RLHF training step on top of that

Re: Claude is good at assembling blocks, but still falls apart at creating them

#138

Post by jihadjihad »

zeroonetwothree wrote: post #132
Uehreka wrote: post #102

Earlier quoted context omitted.

It feels like a lot of people keep falling into the trap of thinking we’ve hit a plateau, and that they can shift from “aggressively explore and learn the thing” mode to “teach people solid facts” mode. A week ago Scott Hanselman went on the Stack Overflow podcast to talk about AI-assisted coding. I generally respect that guy a lot, so I tuned in and… well it was kind of jarring. The dude kept saying things in this r…

Why is it every time anyone has a critique someone has to say “oh but you aren’t using model X, which clearly never has this problem and is far better”? Yet the data doesn’t show all that much difference between SOTA models. So I have a hard time believing it.

Because the answer to the question, “Does this model work for my use case?” is subjective.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#139

Post by Uehreka »

zeroonetwothree wrote: post #132
Uehreka wrote: post #102

Earlier quoted context omitted.

It feels like a lot of people keep falling into the trap of thinking we’ve hit a plateau, and that they can shift from “aggressively explore and learn the thing” mode to “teach people solid facts” mode. A week ago Scott Hanselman went on the Stack Overflow podcast to talk about AI-assisted coding. I generally respect that guy a lot, so I tuned in and… well it was kind of jarring. The dude kept saying things in this r…

Why is it every time anyone has a critique someone has to say “oh but you aren’t using model X, which clearly never has this problem and is far better”? Yet the data doesn’t show all that much difference between SOTA models. So I have a hard time believing it.

GP here: My problem with a lot of studies and data is that they seem to measure how good LLMs are at a particular task, but often don't account for "how good the LLM is to work with". The latter feels extremely difficult to quantify, but matters a lot when you're having a couple dozen turns of conversation with an LLM over the course of a project.

Like, I think there's definitely value in prompting a dozen LLMs with a detailed description of a CMS you want built with 12 specific features, a unit testing suite and mobile support, and then timing them to see how long they take and grading their results. But that's not how most developers use an LLM in practice.

Until LLMs become reliable one-shot machines, the thing I care most about is how well they augment my problem solving process as I work through a problem with them. I have no earthly idea of how to measure that, and I'm highly skeptical of anyone who claims they do. In the absence of empirical evidence we have to fall back on intuition.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#140

Post by __MatrixMan__ »

maxilevi wrote: post #8

LLMs are just really good search. Ask it to create something and it's searching within the pretrained weights. Ask it to find something and it's semantically searching within your codebase. Ask it to modify something and it will do both. Once you understand its just search, you can get really good results.

Its a very useful model but not a complete one. You just gotta acknowledge that if you're making something new its gonna take all day and require a lot of guard rails, but then you can search for that concept later (add the repo to the workspace and prompt at it) and the agent will apply it elsewhere as if it was a pattern in widespread use. "Just search" doesn't quite fit. I've never wondered how best to use a search engine to make something in a way that will be easily searchable later.
Post reply on HN