What would happen if you gave the same task to 200 human contractors?
I suspect SLOC growth wouldn't be quite as dramatic but things like converting everything to Rust's error handling approach could easily happen.
The highest quality codebase
141–150 of 409 posts
Re: The highest quality codebase
#142This is an interesting experiment that we can summarize as "I gave a smart model a bad objective", with the key result at the end
"...oh and the app still works, there's no new features, and just a few new bugs."
Nobody thinks that doing 200 improvement passes on functioning code base is a good idea. The prompt tells the model that it is a principal engineer, then contradicts that role the imperative "We need to improve the quality of this codebase". Determining when code needs to be improved is a responsibility for the principal engineer but the prompt doesn't tell the model that it can decide the code is good enough. I think we would see a different behavior if the prompt was changed to "Inspect the codebase, determine if we can do anything to improve code quality, then immediately implement it." If the model is smart enough, this will increasingly result in passes where the agent decides there is nothing left to do.
In my experience with CC I get great results where I make an open ended question about a large module and instruct it to come back to me with suggestions. Claude generates 5-10 suggestions and ranks them by impact. It's very low-effort from the developer's perspective and it can generate some good ideas.
Re: The highest quality codebase
#143Earlier quoted context omitted.
That's why you treat it like a junior dev. You do the fun stuff of supervising the product, overseeing design and implementation, breaking up the work, and reviewing the outputs. It does the boring stuff of actually writing the code. I am phenomenally productive this way, I am happier at my job, and its quality of work is extremely high as long as I occasionally have it stop and self-review it's progress against the…
Maybe I'm weird but I enjoy "actually writing the code."
I sometimes think of it as a sculptor analogy.
Some famous sculptors had an atelier full of students that helped them with mundane tasks, like carving out a basic shape from a block of stone.
When the basic shape was done, the master came and did the rest. You may want to have the physical exercise of doing the work yourself, but maybe someone sometimes likes to do the fine work and leave the crude one to the AI.
Re: The highest quality codebase
#144Earlier quoted context omitted.
Maybe I'm weird but I enjoy "actually writing the code."
Me writing code is me spending 3/4 of my time wading through documentation and google searches. It's absolutely hell on my ADD. My ability to memorize is absolutely garbage. Throughout my career I've worked in like 10 different languages, and in any given project I'm usually working in at least 3 or 4. There's a lot of "now what is a map operation in this stupid fucking language called again?!" Claude writing code ge…
I would say notetaking would be a much bigger help than Claude at this point. There's a lot of methods to organize information that I believe would help you, better than an hallucination machine.
Re: The highest quality codebase
#145Earlier quoted context omitted.
Me writing code is me spending 3/4 of my time wading through documentation and google searches. It's absolutely hell on my ADD. My ability to memorize is absolutely garbage. Throughout my career I've worked in like 10 different languages, and in any given project I'm usually working in at least 3 or 4. There's a lot of "now what is a map operation in this stupid fucking language called again?!" Claude writing code ge…
How do you end up with 3 to 4 languages in one project?
Recently I've been experimenting with using multiple languages in some projects where certain components have a far better ecosystem in one language but the majority of the project is easier to write in a different one.
For example, I often find Python has very mature and comprehensive packages for a specific need I have, but it is a poor language for the larger project (I also just hate writing Python). So I'll often put the component behind a http server and communicate that way. Or in other cases I've used Rust for working with WASAPI and win32 which has some good crates for it, but the ecosystem is a lot less mature elsewhere.
I used to prefer reinventing the wheel in the primary project language, but I wasted so much time doing that. The tradeoff is the project structure gets a lot more complicated, but it's also a lot faster to iterate.
Plus your usual html/css/js on the frontend and something else on the backend, plus SQL.
Re: The highest quality codebase
#146Earlier quoted context omitted.
It's necessary to allow Claude Code to be fully autonomous, otherwise it will stop and ask you to run commands.
and just letting it to do whatever it thinks it should do, without a human intervening, is a good plan?
the "best practice" suggestion would be to do this in a sandboxed container
Re: The highest quality codebase
#147Earlier quoted context omitted.
That's why you treat it like a junior dev. You do the fun stuff of supervising the product, overseeing design and implementation, breaking up the work, and reviewing the outputs. It does the boring stuff of actually writing the code. I am phenomenally productive this way, I am happier at my job, and its quality of work is extremely high as long as I occasionally have it stop and self-review it's progress against the…
Cool cool cool. So if you use LLMs as junior devs, let me ask you how future awesome senior devs like you will come around? From WHAT job experience? From what coding struggle?
My last job there was effectively a gun held to the back of my head, ordering me to use this stuff. And this started about a year ago, when the tooling for agentic dev was absolutely atrocious, because we had a CTO who had the biggest most raging boner for anything that offered even a whiff of "AI".
Unfortunately the bar is being raised on us. If you can't hang with the new order you are out of a job. I promise I was one of the holdouts who resisted this the most. It's probably why I got laid off last spring.
Thankfully, as of this last summer, agentic dev started to really get good, and my opinion made a complete 180. I used the off time to knock out a personal project in a month or two's worth of time, that would have taken me a year+ the old way. I leveraged that experience to get me where I am now.
Re: The highest quality codebase
#148Earlier quoted context omitted.
“I want my AI to do laundry and dishes so I can code, not for my AI to code so I can do laundry and dishes”
Claude is very good at unfun-but-necessary coding tasks such as writing docstrings and type hints, which is a prominent instance of "laundry and dishes" for a dev.
“Sorry, the autogenerated api documentation was wrong because the ai hallucinated the docstring”
Re: The highest quality codebase
#149Interesting experiment. Looking at this I immediately thought similar experiment run by Google: AlphaEvolve. Throwing LLM compute at problems might work if the problem is well defined and the result can be objectively measured.
As for this experiment: What does quality even mean? Most human devs will have different opinions on it. If you would ask 200 different devs (Claude starts from 0 after each iteration) to do the same, I have doubts the code would look much better.
I am also wondering what would happen if Claude would have an option to just walk away from the code if its "good enough". For each problem most human devs run cost->benefit equation in their head, only worthy ideas are realized. Claude does not do it, the code writing cost is very low on his site and the prompt does not allow any graceful exit :)
Re: The highest quality codebase
#150About a year ago I wrote a blog post (HN discussion: https://news.ycombinator.com/item?id=42584400) experimenting if asking Claude to "write code better" repeatedly would indeed cause it to write better code, determined by speed as better code implies more efficient algorithms. I found that it did indeed work (at n=5 iterations), but additionally providing a system prompt also explicitly improved it.
Given with what I've seen from Claude 4.5 Opus, I suspect the following test would be interesting: attempt to have Claude Code + Haiku/Sonnet/Opus implement and benchmark an algorithm with:
- no CLAUDE.md file
- a basic CLAUDE.md file
- an overly nuanced CLAUDE.md file
And then both test the algorithm speed and number of turns it takes to hit that algorithm speed.