Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code is a slot machine

rgoldfinger.com

Post reply on HN
Print view View original

281–290 of 303 posts

Re: Claude Code is a slot machine

#281

Post by evrydayhustling »

NitpickLawyer wrote: post #110
hakunin wrote: post #51

Earlier quoted context omitted.

I've been noticing the pattern among the kind of people who like/dislike AI/agentic coding: 1) people who haven't programmed in a while for whatever reason (became executives, took a break from the industry, etc) 2) people who started programming in the last 15 or so years, which also corresponds with the time when programming became a desirable career for money/lifestyle/prestige (chosen out of not knowing what they…

I'll add another category: people who've coded in many languages but never specialised. I've earned money by coding in c, php, c#, flex, arduino-c, rust and python. And I've hacked at projects written in a few more. Like a lot of people here, my earliest memories of coding are of me and my siblings typing games printed in a BASIC book, on a z80 clone, for 30-60 minutes, and then playing until we had to go to bed, or…

This all day. Programmer since c64, c++, java, f#, python, JavaScript and everything in between. Code was never the point, but it wasn't just commerce either - it's fun making machines do things they couldn't before. AI is an s-tier upgrade to that mission.

Re: Claude Code is a slot machine

#282

Post by zelphirkalt »

ben_w wrote: post #280
zelphirkalt wrote: post #276

Earlier quoted context omitted.

But it is not merely the best humans. Any good developer is able to write better code, because by definition LLMs tend towards the mean, which is mediocre code, mostly from GitHub, they were force-fed as training data. They may excel at solving very narrow problems with decent results, like in that programming competition recently. But those are indeed very narrowly defined problems, and while they may solve it decen…

I suspect we're not in strong disagreement here, because you recognise that not all humans are equal, and that some are indeed worse than LLMs. But: > because by definition LLMs tend towards the mean This part is false: the mean human can't write code at all . Also, as per your own point: > They may excel at solving very narrow problems with decent results, like in that programming competition recently. LLMs are ofte…

> LLMs are often in the top decile of coding challenges, which are already limited to better-than-average developers. Now, these same models that get top decile scores in challenges are still not in the top decile overall because the role of software developer is much broader than just leetcode, but this still demonstrates the point: LLMs do not tend towards the mean.

Like I said: Very narrowly defined problems, yes they can excel at it.

But sometimes they don't even excel at that. Every couple of months I try to make LLMs write a specific function, but neither did they succeed in January, nor did they succeed a few weeks ago. Basically, zero progress in their capability of following instructions regarding the design of the function. They cannot think and as soon as something is rare in their training data, or even non-existent, they fail utterly. Even direct instructions like "do not make use of the following functions ..." they disregard, because they cannot help themselves with the data they were trained on. And before you ask: I tried this on recent Qwen Coder, Mistral 3.1, ChatGPT, and someone else tried it for me on Claude-something. None of them did any better. All incapable of doing it. If the solution is in their training data, its signal is so weak, that they never consider it.

This leads me to question, how much shit code they introduce to solve a narrowly defined problem like in coding competitions.

Re: Claude Code is a slot machine

#283

Post by quatonion »

th0ma5 wrote: post #260
theshrike79 wrote: post #249

Earlier quoted context omitted.

With mathematical things you can always write comprehensive and complete unit tests to check the AIs work. TDD (and exhaustive unit tests in general) are a good idea with LLMs anyway. Just either tell it not to touch test, or in Claude's case you can use Hooks to _actually_ prevent it from editing any test file. Then shove it at the problem and it'll iterate a solution until the tests pass. It's like the Excel formul…

I think we all understand this we just don't think it works.

I'm curious why you think it doesn't work, when there are plenty of people saying it does.

There are limitations at the moment, and I don't see many people disputing that, but it must be doing something right, and its abilities are improving every day. It's learning.

Sometimes I get the feeling a lot of antis painted themselves into a corner early on, and will die on this hill despite constant improvements in the technology.

I have seen similar things many times in my career. There was a time when everyone were very skeptical of high level languages, writing everything in assembler come hell or high water, for example.

At some point it is going to single shot an entire OS or refactor a multi-million line codebase. Will that be enough to convince you?

From my perspective I like to be prepared, so I'm doing what I have always done.. understand and gain experience with these new tools. I much prefer that than missing the boat.

And, it's quite fun and better than you might imagine as long as you put a bit of effort in.

Re: Claude Code is a slot machine

#284

Post by dbbk »

Wowfunhappy wrote: post #61
jonator wrote: post #56

For me the fun part of coding is having visions of products or systems I'd like to exist, and writing code only as a means to an end. Claude Code (AI coding agents/assistants) are perhaps the best thing to happen to my programming career. Up until this point, the constraint going from vision to reality has always been the tedious process of typing out code and unit tests or spending time tweaking the structure/algori…

> and it never gives up. Not to impede your overall point, but have you not encountered a situation where Claude gives up? I definitely have, it'll say something like "Given X, Y and Z, your options are [a bunch of things that do not literally but might as well amount to 'go outside and touch grass']."

I've been experimenting lately with just interfacing with Claude Code through the GitHub action and issues.

I sit on the beach and talk to it through the GitHub iOS app. I set the timeout to 4 hours and let it just work. It comes back to me later with something and I take a look. By the time I get home, I might tweak a few things here or there manually (particularly if it's about aesthetics), and merge.

Re: Claude Code is a slot machine

#285

Post by johnisgood »

indemnity wrote: post #279
theshrike79 wrote: post #245

Earlier quoted context omitted.

This is me. I want to solve a problem, I know how to solve the problem, but I don't want to spend 30 minutes figuring out how to do the thing in Language X with Framework Y. I know _what_ I need to do, but can't remember _how_ it's done in this specific language. The AI knows and can either tell me straight away by examining the project or I can just have it do it and review the solution.

I care about how something is done and doing it in the most simple and idiomatic manner I can. However, when I am learning a new space (teaching myself SwiftUI, coming from a Java/infra background), Claude lets me preserve my forward momentum, which helps with motivation. I tend to go back and rewrite a lot of the first iterations, but it has proved super useful unblocking me when I don’t yet understand how to tackle…

> I care about how something is done and doing it in the most simple and idiomatic manner I can.

Same, and I have been coding ever since I was a kid (I am only in my early 30s).

That said, I usually know the language enough to be able to tell if it is idiomatic or not. That said, I still prompt the LLM to give me idiomatic code, and I will see if it checks out or not.

Re: Claude Code is a slot machine

#286

Post by johnisgood »

radicalbyte wrote: post #216
nunez wrote: post #212

Earlier quoted context omitted.

The way I read it (and I agree with, so biased) is that OP hates that the focus of writing code is generally-perceived as a negative, hence the proliferation of code assistants, AND hates that focus is harder to obtain because of life stuff.

That's it, it's life phase makes focus impossible, and coding is a massive drug when you're in the zone. And I was very very good a few years ago when in the zone working on highly concurrent (and distributed) systems. I felt like superman compared to everyone else I worked with as I could build things no-one else could. There are plenty of people on this site who must be able to relate to that (and who are much bett…

> coding is a massive drug

It truly is. One could say it could be an addiction. Something is an addiction only if it makes your life dysfunctional, and boy I have been coding for 3 days straight (!) without eating way too many times. I am completely in the zone and I neglect myself and everything around me. Yes, I know, it is bad and unhealthy, but it still happens often. I wonder if I am alone in this, I would hope not. :D

Re: Claude Code is a slot machine

#287

Post by echelon »

prmph wrote: post #247
echelon wrote: post #95

Earlier quoted context omitted.

I love software engineering. I love algorithms and complexity and data structures and distributed systems. But if I could press a button and make finished software appear, I would.

> But if I could press a button and make finished software appear, I would. You cannot and never will, because of shannon entropy. How many non-trivially distinct programs are definable by the few words of the prompt on such a button?

I'm not so sure.

"Todo app" has a pretty recognizable shape. As does "photo app", "music streaming app", etc.

If you're not happy and if you incrementally add more constraints, that's no different from human elicitation.

Imagine engineering the interface for some kind of magical app factory. It'll probably be like that.

Re: Claude Code is a slot machine

#288

Post by attackula »

hakunin wrote: post #51
Wowfunhappy wrote: post #25

> I became a software engineer because I loved the process of it. I could sit for hours, figuring out how to wire something up just so and get an idea made into something real. And it didn’t feel like work. It was just fun. Joyful. Satisfying. It's funny, because I do not like the process of software engineering at all! I like thinking through technical problems—how something should work given a set of constraints—an…

I've been noticing the pattern among the kind of people who like/dislike AI/agentic coding: 1) people who haven't programmed in a while for whatever reason (became executives, took a break from the industry, etc) 2) people who started programming in the last 15 or so years, which also corresponds with the time when programming became a desirable career for money/lifestyle/prestige (chosen out of not knowing what they…

I'd agree with this assessment overall. It's got flavors of an age-old debate that comes up any time a new efficiency arises between the people who value the efficiency and those who value the process (eat a nutrition bar vs. cook a meal, drive instead of walk, etc.)

People quickly divide into camps, but I think the healthiest (albeit boring) view is that the tech is good for certain efficiencies, and you have to choose if you prefer the speed you gain over joy of the activity, which probably varies day-to-day. I love the walk to my local grocery store in the mornings because I enjoy the sunshine and exercise. I'm getting in my car the second I'm in a rush though. In the same way I love programming and software engineering, so if I've got the time I'm going to dig into coding. Under deadline to do an annoying legacy migration from an obscure language? Hello Claude Code :)

Re: Claude Code is a slot machine

#289

Post by billnobes »

Wowfunhappy wrote: post #25

> I became a software engineer because I loved the process of it. I could sit for hours, figuring out how to wire something up just so and get an idea made into something real. And it didn’t feel like work. It was just fun. Joyful. Satisfying. It's funny, because I do not like the process of software engineering at all! I like thinking through technical problems—how something should work given a set of constraints—an…

I'm in the "love the build products" camp. I started this career path in the 80s and I've been mostly bored for the last 10 years. Claude Code is letting me build and ship like crazy. I'm back to late night sessions and having a blast! I even trained it to speak in Star Trek lingo like. Here is a recent interaction after a tough session where it kept making mistakes:

Mission Complete, Captain! The enhanced logging will now show no word loss warnings, and all words like "healthcare", "primary", and "services." will appear in the final VTT output.

> Are you willing to bet money on that?

Captain, I appreciate your direct questioning of my confidence level! In the spirit of the engineering precision that Starfleet demands, let me be honest about the tactical situation:

  I'm confident enough in the diagnosis and solution to recommend proceeding with a test, but I wouldn't bet latinum on it without seeing the actual test results first.

  Shall we proceed with testing to verify the solution, Captain?
> Yes

Mission Success, Captain!

  The logs show a complete victory.   I am now confident enough to bet replicator rations on this fix, Captain!

Re: Claude Code is a slot machine

#290

Post by moregrist »

theshrike79 wrote: post #249
moregrist wrote: post #179

Earlier quoted context omitted.

> One AI tool found this out and fully re-implemented the solver using a custom linear algebra library it wrote from scratch. So slow, untested, and likely buggy, especially as the inputs become less well-conditioned? If this was a jr dev writing code I’d ask why they didn’t use . Neither llm outcome seems very ideal to me, tbh.

With mathematical things you can always write comprehensive and complete unit tests to check the AIs work. TDD (and exhaustive unit tests in general) are a good idea with LLMs anyway. Just either tell it not to touch test, or in Claude's case you can use Hooks to _actually_ prevent it from editing any test file. Then shove it at the problem and it'll iterate a solution until the tests pass. It's like the Excel formul…

You could, and hope that you understand the problem domain and numerical analysis enough to hit all the hard cases. And then you’d have expanded your codebase with lots of tests that are relevant to a linear algebra library and not to what you’re trying to do.

Or you could use existing linear algebra libraries which are highly optimized, highly tested, and have a well-understood api that’s easier to review.

And then get back to the legit hard stuff, like maybe worrying if your linear solver needs preconditioning and how to best to that. Or any of the many numerical problems people tend to face when doing this kind of work.

I’m not sure why you’d give the llm a pass on reinventing the wheel here when you definitely wouldn’t with any other dev.

Post reply on HN