Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: AI productivity gains – do you fire devs or build better products?

news.ycombinator.com

Post reply on HN
Print view View original

131–140 of 244 posts

Re: Ask HN: AI productivity gains – do you fire devs or build better products?

#131

Post by Kiro »

maccard wrote: post #97
wrs wrote: post #75

Earlier quoted context omitted.

I’m honestly baffled by this. I don’t want to tell you “you’re holding it wrong” but if this is your normal experience there’s something weird happening. Friday afternoon I made a new directory and told Claude Code I wanted to make a Go proxy so I could have a request/callback HTTP API for a 3rd party service whose official API is only persistent websocket connections. I had it read the service’s API docs, engage in…

> I have noticed some people I work with have more trouble, and my vague intuition is it happens when they give Claude too much autonomy What’s giving too much autonomy about “Please load settings.toml using a library and print out the name key from the application table”? Even if it’s under specified, surely it should at least leave it _compiling_? I’ve been posting comments like this monthly here, my experience has…

Not even the worst possible prompt would explain your unusual experience, so I don't think that's it either.

Re: Ask HN: AI productivity gains – do you fire devs or build better products?

#132

Post by alecbz »

jmalicki wrote: post #125
alecbz wrote: post #121

Earlier quoted context omitted.

Stuff like this works for things that can be verified programmatically (though I find LLMs still do occasionally ignore instructions like this), but ensuring correct functionality and sensible code organization are bigger challenges. There are techniques that can help deal with this but none of them work perfectly, and most of the time some direct oversight from me is required. And this really clips the potential pro…

It depends - there are some very very difficult things that can still be easily verifiable! For instance, if you are working on a compiler and have a huge test database of code to compile that all has tests itself, "all sample code must compile and pass tests, ensuring your new optimizer code gets adequate branch coverage in the process" - the underlying task can be very difficult, but you have large amounts of test…

Yeah performance optimization is potentially another good area for LLMs to shine, if you already have a sufficiently comprehensive test suite, because no functionality is changing. But if functionality is changing, you need to be in the loop to, at the very least, review the tests that the LLM outputs. Sometimes that's easier than reviewing the code itself, but other times I think it requires similar levels of context.

But honestly I think sane code organization is the bigger hurdle, which is a lot harder to get right without manual oversight. Which of course leads to the temptation to give up on reviewing the code and just trusting whatever the LLM outputs. But I'm skeptical this is a viable approach. LLMs, like human devs, seem to need reasonably well-organized code to be able to work in a codebase, but I think the code they output often falls short of this standard.

(But yes agree that getting the LLM to iterate until CI passes is table-stakes.)

Re: Ask HN: AI productivity gains – do you fire devs or build better products?

#133

Post by senordevnyc »

foobarchu wrote: post #96
maccard wrote: post #61

> try it, you might actually be amazed. I keep being told this and the tools keep falling at the first hurdle. This morning I asked Claude to use a library to load a toml file in .net and print a value. It immediately explained how it was an easy file format to parse and didn’t need a library. I undid, went back to plan mode and it picked a library, added it and claimed it was done. Except the code didn’t compile. Th…

The best experiences I have are those where I can describe what I want done with details. Rather than asking it add toml parsing, I would tell it to exactly which library to use ahead of time and reduce the number of decisions available to the model to make. Some of the most effective use-cases are when you have a reference to give it, e.g. "add x feature the same way as in this other project that is also in the work…

I basically never have agents do work without using plan mode first (I use Cursor). So I'll start with a high-level like: "I want to plan out the architecture for feature X, I think there's a linear project already that we should update once our plan is done, but if not we'll need to create one. Here's a picture of my whiteboard, I think I also wrote some notes in Obsidian a few weeks ago that you should look up. Two things off the top of my head to think about are: should we do X or does it make more sense to do Y? And also, I'm worried about how this might interfere with the blahblah system, so can you do some research around that? "

Then it crawls around for awhile, does some web searches, fetches docs from here and there, whatever. Sometimes it'll ask me some questions. And then it'll finally spit out a plan. I'll read through it and just give it a massive dump of issues, big and small, more questions I have, whatever. (I'll also often be spinning off new planning sessions for pre-work or ancillary tasks that I thought of while reviewing that plan). No structure or anything, just brain dump. Maybe two rounds of that, but usually just one. And then I'll either have it start building, or I'll have it stash in the linear agent so I can kick it off later.

Re: Ask HN: AI productivity gains – do you fire devs or build better products?

#134

Post by maccard »

ball_of_lint wrote: post #118
maccard wrote: post #61

> try it, you might actually be amazed. I keep being told this and the tools keep falling at the first hurdle. This morning I asked Claude to use a library to load a toml file in .net and print a value. It immediately explained how it was an easy file format to parse and didn’t need a library. I undid, went back to plan mode and it picked a library, added it and claimed it was done. Except the code didn’t compile. Th…

Did you give claude access to run the compile step? I remember having to write code on paper for my CS exams, and they expected it to compile! It was hard but I mostly got there. definitely made a few small mistakes though

Yes.

Re: Ask HN: AI productivity gains – do you fire devs or build better products?

#135

Post by maccard »

linsomniac wrote: post #123
maccard wrote: post #97

Earlier quoted context omitted.

> I have noticed some people I work with have more trouble, and my vague intuition is it happens when they give Claude too much autonomy What’s giving too much autonomy about “Please load settings.toml using a library and print out the name key from the application table”? Even if it’s under specified, surely it should at least leave it _compiling_? I’ve been posting comments like this monthly here, my experience has…

> Even if it’s under specified, surely it should at least leave it _compiling_? Are you using Claude Code? Do yo have it configured so that you are not allowing it to run the build? Because I've observed that Claude Code is extremely good at making sure the code compiles, because it'll run a compile and address any compile errors as part of the work. I just asked it to build a TOML example program in DotNet using Tom…

I am using Claude code. I didn’t explicitly tell it what the build command was (it’s dotnet build), and it didn’t ask. Thats not my fault.

> I’ve observed Claude code is extremely good at making sure the code compiles

My observation is that it’s fine until it’s absolutely not, and the agentic loop fails.

Re: Ask HN: AI productivity gains – do you fire devs or build better products?

#136

Post by maccard »

wrs wrote: post #119
maccard wrote: post #97

Earlier quoted context omitted.

> I have noticed some people I work with have more trouble, and my vague intuition is it happens when they give Claude too much autonomy What’s giving too much autonomy about “Please load settings.toml using a library and print out the name key from the application table”? Even if it’s under specified, surely it should at least leave it _compiling_? I’ve been posting comments like this monthly here, my experience has…

There’s nothing wrong with it that I can see. Like I said, I’m a bit baffled at your experience. I will say, it’s not unusual for the initial output not to compile, but usually one short iteration later that’s fixed. Claude Code will usually even do that iteration by itself.

> I will say, it’s not unusual for the initial output not to compile,

We’ve gone from “I’m baffled at your experience” to well yeah it often fails” in two sentences here…

Re: Ask HN: AI productivity gains – do you fire devs or build better products?

#137

Post by jmalicki »

alecbz wrote: post #132
jmalicki wrote: post #125

Earlier quoted context omitted.

It depends - there are some very very difficult things that can still be easily verifiable! For instance, if you are working on a compiler and have a huge test database of code to compile that all has tests itself, "all sample code must compile and pass tests, ensuring your new optimizer code gets adequate branch coverage in the process" - the underlying task can be very difficult, but you have large amounts of test…

Yeah performance optimization is potentially another good area for LLMs to shine, if you already have a sufficiently comprehensive test suite, because no functionality is changing. But if functionality is changing, you need to be in the loop to, at the very least, review the tests that the LLM outputs. Sometimes that's easier than reviewing the code itself, but other times I think it requires similar levels of contex…

Strongly agreed!

I think getting good code organization out of an LLM is one of the subtler things - I've learned quite a bit about what sort of things need to be specified, realizing that the LLM isn't actively learning my preferences particularly well, so there are some things about code organization I just have to be explicit about.

Which is more work, but less work than just writing the code myself to begin with.

Re: Ask HN: AI productivity gains – do you fire devs or build better products?

#138

Post by maccard »

ffsm8 wrote: post #110
maccard wrote: post #61

> try it, you might actually be amazed. I keep being told this and the tools keep falling at the first hurdle. This morning I asked Claude to use a library to load a toml file in .net and print a value. It immediately explained how it was an easy file format to parse and didn’t need a library. I undid, went back to plan mode and it picked a library, added it and claimed it was done. Except the code didn’t compile. Th…

> This morning I asked Claude to use a library to load a toml file in .net and print a value. Legit this morning Claude was essentially unusable for me I could explicitly state things it should adjust and it wouldn't do it. Not even after specifying again, reverting everything eventually and reprompt from the beginning etc. Even super trivial frontend things like "extract [code] into a separate component" After 30 mi…

> Legit this morning Claude was essentially unusable for me I could explicitly state things it should adjust and it wouldn't do it.

Honestly, this is my experience. Every now and again it just completely self implodes and gives up, and I’m left to pick up the pieces. Look at the other replies who are making sure I’m using the agrntic loop/correct model/specific enough prompt - I don’t know what they’re doing but I would love to try the tools they’re using.

Re: Ask HN: AI productivity gains – do you fire devs or build better products?

#139

Post by maccard »

jmalicki wrote: post #107
fxtentacle wrote: post #91

Earlier quoted context omitted.

Same here. While LLMs sometimes work surprisingly well, I also encounter edge cases where they fail surprisingly badly multiple times per day. My guess is that other people maybe just don't bother to check what the AI says which would cause them to not notify omission errors. Like when I was trying to find a physical store again with ChatGPT Pro 5.4 and asked it to prepare a list of candidates, but the shop just wasn…

"I also encounter edge cases where they fail surprisingly badly multiple times per day. " If 80% of the time they 10x my output, and the other 20% I can say "well they failed, I guess this one I have to do manually" - that's still an absolutely massive productivity boost.

But they don’t 10x my output - they write some code for a problem I/you have already thought about. The hard part isn’t writing the code, it never has been. It’s always been solving and breaking down the problem.

Re: Ask HN: AI productivity gains – do you fire devs or build better products?

#140

Post by jmalicki »

maccard wrote: post #139
jmalicki wrote: post #107

Earlier quoted context omitted.

"I also encounter edge cases where they fail surprisingly badly multiple times per day. " If 80% of the time they 10x my output, and the other 20% I can say "well they failed, I guess this one I have to do manually" - that's still an absolutely massive productivity boost.

But they don’t 10x my output - they write some code for a problem I/you have already thought about. The hard part isn’t writing the code, it never has been. It’s always been solving and breaking down the problem.

It's not the hard part, just the tedious part that takes the most time.
Post reply on HN