Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude is good at assembling blocks, but still falls apart at creating them

approachwithalacrity.com

Post reply on HN
Print view View original

151–160 of 249 posts

Re: Claude is good at assembling blocks, but still falls apart at creating them

#151

Post by CJefferson »

Uehreka wrote: post #139
zeroonetwothree wrote: post #132

Earlier quoted context omitted.

Why is it every time anyone has a critique someone has to say “oh but you aren’t using model X, which clearly never has this problem and is far better”? Yet the data doesn’t show all that much difference between SOTA models. So I have a hard time believing it.

GP here: My problem with a lot of studies and data is that they seem to measure how good LLMs are at a particular task, but often don't account for "how good the LLM is to work with". The latter feels extremely difficult to quantify, but matters a lot when you're having a couple dozen turns of conversation with an LLM over the course of a project. Like, I think there's definitely value in prompting a dozen LLMs with…

A friend recommended to me having a D&D style roleplay with some different engines, to see which you vibe with. I thought this sounded crazy but I took their advice.

I found this worked suprisingly well, I was certain 'claude' was best, while they like grok and someone else liked ChatGPT. Some AIs just end up fitting best with how you like to chat I think. I do definately also find claude best for coding with as well.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#152

Post by measurablefunc »

danpalmer wrote: post #127
disconcision wrote: post #94

I've yet to be convinced by any article, including this one, that attempts to draw boxes around what coding agents are and aren't good at in a way that is robust on a 6 to 12 month horizon. I agree that the examples listed here are relatable, and I've seen similar in my uses of various coding harnesses, including, to some degree, ones driven by opus 4.5. But my general experience with using LLMs for development over…

I used to get made up APIs in functions, now I get them in modules. I used to get confidently incorrect assertions in files now I get them across codebases. Hell, I get poorly defined APIs across files and still get them between functions. LLMs aren't good at writing well defined APIs at any level of the stack. They can attempt it at levels of the stack they couldn't a year ago, but they're still terrible at it unles…

This is the right answer. Unless there is some equivalent of it on the open internet which their search engine can find you should not expect a good outcome.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#153

Post by danpalmer »

measurablefunc wrote: post #152
danpalmer wrote: post #127

Earlier quoted context omitted.

I used to get made up APIs in functions, now I get them in modules. I used to get confidently incorrect assertions in files now I get them across codebases. Hell, I get poorly defined APIs across files and still get them between functions. LLMs aren't good at writing well defined APIs at any level of the stack. They can attempt it at levels of the stack they couldn't a year ago, but they're still terrible at it unles…

This is the right answer. Unless there is some equivalent of it on the open internet which their search engine can find you should not expect a good outcome.

"good outcome" is pretty subjective, I do get useful productivity gains from some LLM work, but the issues are the same as they always have been.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#154

Post by gubicle »

emp17344 wrote: post #13
johnisgood wrote: post #11

Earlier quoted context omitted.

Calling it "just search" is like calling a compiler "just string manipulation". Not false, but aggressively missing the point.

No, “just search” is correct. Boosters desperately want it to be something more, but it really is just a tool.

[dead]

Re: Claude is good at assembling blocks, but still falls apart at creating them

#155

Post by bhadass »

nextaccountic wrote: post #137
bhadass wrote: post #16

Earlier quoted context omitted.

better mental model: it's a lossy compression of human knowledge that can decompress and recombine in novel (sometimes useful, sometimes sloppy) ways. classical search simply retrieves, llms can synthesize as well.

Maybe the base model is just a compression of the training data? There is also a RLHF training step on top of that

yep the base model is the compression, but RLHF (and other types of post training) doesn't really change this picture, it's still working within that same compressed knowledge.

nathan lambert (who wrote the RLHF book @ https://rlhfbook.com/ ) describes this as the "elicitation theory of post training", the idea is that RLHF is extracting and reshaping what's already latent in the base model, not adding new knowledge. as he puts it: when you use preferences to change model behavior "it doesn't mean that the model believes these things. it's just trained to prioritize these things."

so like when you RLHF a model to not give virus production info, you're not necessarily erasing those weights, the theory is that you're just making it harder for that information to surface. the knowledge is still in the compression, RLHF just changes what gets prioritized during decompression.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#156

Post by andoando »

maxilevi wrote: post #8

LLMs are just really good search. Ask it to create something and it's searching within the pretrained weights. Ask it to find something and it's semantically searching within your codebase. Ask it to modify something and it will do both. Once you understand its just search, you can get really good results.

Im not sure how anyone can say this. It is really good search, but its also able to combine ideas and reason about and do fairly complex logic on tasks surely absolutely no one has asked before.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#157

Post by bhadass »

XenophileJKO wrote: post #50
RhythmFox wrote: post #21

Earlier quoted context omitted.

This isn't strictly better to me. It captures some intuitions about how a neural network ends up encoding its inputs over time in a 'lossy' way (doesn't store previous input states in an explicit form). Maybe saying 'probabilistic compression/decompression' makes it a bit more accurate? I do not really think it connects to your 'synthesize' claim at the very end to call it compression/decompression, but I am curious…

It's really way more interesting that that. The act of compression builds up behaviors/concepts of greater and greater abstraction. Another way you could think about it is that the model learns to extract commonality, hence the compression. What this means is because it is learning higher level abstractions AND the relationships between these higher level abstractions, it can ABSOLUTELY learn to infer or apply things…

ya, exactly... i'd also say that when you compress large amounts of content into weights and then decompress via a novel prompt, you're also forcing interpolation between learned abstractions that may never have cooccurred in training.

that interpolation is where synthesis happens. whether it is coherent or not depends.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#158

Post by joduplessis »

Recently I've put Claude/others to use in some agentic workflows with easy menial/repetitive tasks. I just don't understand how people are using these agents in production. The automation is absolutely great, but it requires an insane amount of hand-holding and cleanup.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#159

Post by [deleted] »

joduplessis wrote: post #158

Recently I've put Claude/others to use in some agentic workflows with easy menial/repetitive tasks. I just don't understand how people are using these agents in production. The automation is absolutely great, but it requires an insane amount of hand-holding and cleanup.

[deleted]

Re: Claude is good at assembling blocks, but still falls apart at creating them

#160

Post by baq »

refactor_master wrote: post #128
danpalmer wrote: post #127

Earlier quoted context omitted.

I used to get made up APIs in functions, now I get them in modules. I used to get confidently incorrect assertions in files now I get them across codebases. Hell, I get poorly defined APIs across files and still get them between functions. LLMs aren't good at writing well defined APIs at any level of the stack. They can attempt it at levels of the stack they couldn't a year ago, but they're still terrible at it unles…

I still get made-up Python types all the time with Gemini. Really quite distracting when your codebase is massive and triggers a type error, and Gemini says "To solve it you just need to use WrongType[ThisCannotBeUsedHere[Object]]" and then I spend 15 minutes running in circles, because everything from there on is just a downward spiral, until I shut off the AI noise and just read the docs.

Gemini unfortunately sucks at calling tools, including ‘read the docs’ tool… it’s a great model otherwise. I’m sure Hassabis’ team is on it since it’s how the model can ground itself in non-coding contexts, too.
Post reply on HN