Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude is good at assembling blocks, but still falls apart at creating them

approachwithalacrity.com

Post reply on HN
Print view View original

21–30 of 249 posts

Re: Claude is good at assembling blocks, but still falls apart at creating them

#21

Post by RhythmFox »

bhadass wrote: post #16
maxilevi wrote: post #8

LLMs are just really good search. Ask it to create something and it's searching within the pretrained weights. Ask it to find something and it's semantically searching within your codebase. Ask it to modify something and it will do both. Once you understand its just search, you can get really good results.

better mental model: it's a lossy compression of human knowledge that can decompress and recombine in novel (sometimes useful, sometimes sloppy) ways. classical search simply retrieves, llms can synthesize as well.

This isn't strictly better to me. It captures some intuitions about how a neural network ends up encoding its inputs over time in a 'lossy' way (doesn't store previous input states in an explicit form). Maybe saying 'probabilistic compression/decompression' makes it a bit more accurate? I do not really think it connects to your 'synthesize' claim at the very end to call it compression/decompression, but I am curious if you had a specific reason to use the term.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#22

Post by MrDunham »

arjie wrote: post #5
alfalfasprout wrote: post #3

Earlier quoted context omitted.

> I don't think we'll get to the point where all you have is a CEO and a massive Claude account but it's not completely science fiction the more I think about it. At that point, why do you even need the CEO?

Reminds me of an old joke[0]: > The factory of the future will have only two employees, a man and a dog. The man will be there to feed the dog. The dog will be there to keep the man from touching the equipment. But really, the reason is that people like Pieter Levels do exist: masters at product vision and marketing. He also happens to be a proficient programmer, but there are probably other versions of him which are…

My technical cofounder reminds me of this story on a weekly basis.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#23

Post by pixelready »

alfalfasprout wrote: post #3
mikece wrote: post #2

In my experience Claude is like a "good junior developer" -- can do some things really well, FUBARS other things, but on the whole something to which tasks can be delegated if things are well explained. If/when it gets to the ability level of a mid-level engineer it will be revolutionary. Typically a mid-level engineer can be relied upon to do the right thing with no/minimal oversight, can figure out incomplete instr…

> I don't think we'll get to the point where all you have is a CEO and a massive Claude account but it's not completely science fiction the more I think about it. At that point, why do you even need the CEO?

The board (in theory) represents the interests of investors, and even with all of the other duties of a CEO stripped away, they will want a ringable neck / PR mouthpiece / fall guy for strategic missteps or publicly unpopular moves by the company. The managerial equivalent of having your hands on the driving wheel of a self-driving car.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#24

Post by bitwize »

jvanderbot wrote: post #18
johnisgood wrote: post #14

Earlier quoted context omitted.

Yes, it is a tool. No, it is not "just search". Is your CPU running arbitrary code "just search over transistor states"? Calling LLMs "just search" is the kind of reductive take that sounds clever while explaining nothing. By that logic, your brain is "just electrochemical gradients".

What would you add? To me it's "search" like a missile does "flight". It's got a target and a closed loop guidance, and is mostly fire and forget (for search). At that, it excels. I think the closed loop+great summary is the key to all the magic.

Which is kind of funny because my standard quip is that AI research, beginning in the 1950s/1960s, and indeed much of late 20th century computer tech especially along the Boston/SV axis, was funded by the government so that "the missile could know where it is". The DoD wanted smarter ICBMs that could autonomously identify and steer toward enemy targets, and smarter defense networks that could discern a genuine missile strike from, say, 99 red balloons going by.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#25

Post by soulofmischief »

andrei_says_ wrote: post #17
bhadass wrote: post #16

Earlier quoted context omitted.

better mental model: it's a lossy compression of human knowledge that can decompress and recombine in novel (sometimes useful, sometimes sloppy) ways. classical search simply retrieves, llms can synthesize as well.

“Novel” to the person who has not consumed the training data. Otherwise, just training data combined in highly probable ways. Not quite autocomplete but not intelligence either.

Citation needed that grokked capabilities in a sufficiently advanced model cannot combinatorially lead to contextually novel output distributions, especially with a skilled guiding hand.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#26

Post by soulofmischief »

jvanderbot wrote: post #18
johnisgood wrote: post #14

Earlier quoted context omitted.

Yes, it is a tool. No, it is not "just search". Is your CPU running arbitrary code "just search over transistor states"? Calling LLMs "just search" is the kind of reductive take that sounds clever while explaining nothing. By that logic, your brain is "just electrochemical gradients".

What would you add? To me it's "search" like a missile does "flight". It's got a target and a closed loop guidance, and is mostly fire and forget (for search). At that, it excels. I think the closed loop+great summary is the key to all the magic.

It's a prediction algorithm that walks a high-dimensional manifold, in that sense all application of knowledge it just "search", so yes, you're fundamentally correct but still fundamentally wrong since you think this foundational truth is the end and beginning of what LLMs do, and thus your mental model does not adequately describe what these tools are capable of.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#28

Post by [deleted] »

johnisgood wrote: post #11
maxilevi wrote: post #8

LLMs are just really good search. Ask it to create something and it's searching within the pretrained weights. Ask it to find something and it's semantically searching within your codebase. Ask it to modify something and it will do both. Once you understand its just search, you can get really good results.

Calling it "just search" is like calling a compiler "just string manipulation". Not false, but aggressively missing the point.

[deleted]

Re: Claude is good at assembling blocks, but still falls apart at creating them

#29

Post by jerf »

alfalfasprout wrote: post #3
mikece wrote: post #2

In my experience Claude is like a "good junior developer" -- can do some things really well, FUBARS other things, but on the whole something to which tasks can be delegated if things are well explained. If/when it gets to the ability level of a mid-level engineer it will be revolutionary. Typically a mid-level engineer can be relied upon to do the right thing with no/minimal oversight, can figure out incomplete instr…

> I don't think we'll get to the point where all you have is a CEO and a massive Claude account but it's not completely science fiction the more I think about it. At that point, why do you even need the CEO?

You will need the CEO to watch over the AI and ensure that the interests of the company are being pursued and not the interests of the owners of the AI.

That's probably the biggest threat to the long-term success of the AI industry; the inevitable pull towards encroaching more and more of their own interests into the AI themselves, driven by that Harvard Business School mentality we're all so familiar with, trying to "capture" more and more of the value being generated and leaving less and less for their customers, until their customer's full time job is ensuring the AIs are actually generating some value for them and not just the AI owner.

Re: Claude is good at assembling blocks, but still falls apart at creating them

#30

Post by simonw »

I'm not entirely convinced by the anecdote here where Claude wrote "bad" React code:

> But in context, this was obviously insane. I knew that key and id came from the same upstream source. So the correct solution was to have the upstream source also pass id to the code that had key, to let it do a fast lookup.

I've seen Claude make mistakes like that too, but then the moment you say "you can modify the calling code as well" or even ask "any way we could do this better?" it suggests the optimal solution.

My guess is that Claude is trained to bias towards making minimal edits to solve problems. This is a desirable property, because six months ago a common complaint about LLMs is that you'd ask for a small change and they would rewrite dozens of additional lines of code.

I expect that adding a CLAUDE.md rule saying "always look for more efficient implementations that might involve larger changes and propose those to the user for their confirmation if appropriate" might solve the author's complaint here.

Post reply on HN