71–80 of 87 posts
Just imagine when we have quantum computing. Unlimited context windows, instant answers. What would you make if you could make anything? Does it all just lose meaning?
I have used custom code generators at work for 25+ years. The generators typically generates about 90% of the code I need to write a biz app. Leaving the most important code to me: the biz logic. No AI. Just code that takes a (simple) declarative spec file and generates Typescript/C++/Java/... code. I am also using AI's daily. However the code generators are still generating more productivity for me than AI's ever ha…
How come such a large portion of the code is generated?
Earlier quoted context omitted.
As one often finds with “effective LLM usage” advice, all of those things would help humans on the team as well! As would other advice like keeping the architecture docs up to date, writing down important design decisions with rationale, breaking big features down into steps, etc. Maybe one should just search for advice from the last 20 years on how to make a human development team more effective, and do that stuff.…
So depressing that this got downvoted
If I know what I want to code and it's a purely mechanical exercise to code it, I'll just tell Claude what to do and it does it. Pretty neat. When I don't know what I want to do, I read existing code, think about it, and figure it out. Sometimes I'll sketch out ideas by writing code, then when I have something I like I'll get Claude to take my sketch as an example and having it go forward. The big mistake I see peopl…
Earlier quoted context omitted.
But the model doesn't need to read the node_modules to write a React app, it just needs to write the React code (which it is heavily post-trained to be able to use). So the fair counter example is like: function Hello() { return Hello }
Fair challenge to the idea. But what i am saying is that every line of boilerplate, every import statement, every configuration file consumes precious tokens. The more code, the more surface area the LLM needs to cover before understanding or implementing correctly. Right now the solution to expensive token limits is the most token-efficient technology. let's reframe it better. Was react made to help humans organize…
At current prices you can pretty much get away with murder even for the most expensive models out there. You know, $14/million output tokens. 10k output tokens is 14 cents. Which is ~40k words, or whatever.
The way to use LLM's for development is to use the API.
True I spent a year making a platform for using the API.. but the results... are stupendous!! Very cheap and unlimited access and custom tooling, etc... to get large amounts done of anything you want to do with an LLM!
Looks like this: https://mastodon.social/@rcarmo/115937685095982965
SyncThing syncs their workspaces to my desktop/laptop, I make small adjustments (I don’t do stupid wasteful things like the Ralph approach, I prefer clear SPEC documents and TODO checklists, plus extensive testing and switching models for doing code audits on each other), my changes sync back, etc.
I’m considering calling it “million monkeys”, really.
The real issue is that I can't trust the AI generated code, or trust the AI to code-review for me. Some repeated issues I see:
- In my experience the AI doesn't integrate well with the code that there is already there: it often rewrites functionality and tend not to adhere to the project's conventions, but rather use what it is trained on.
- The AI often lacks depth into more complex issues. And because it doesn't see the broader implication of changes, it often doesn't write the tests that would cover them. Developers that wrote the PRs accept the AI tests without much investigation into the code-base. Since the changes passes the (also insufficient) tests, they send the PR to code-review.
- With AI I think (?) I'm more often the one careful deep diving into the project and re-designing the generated code in the code-review. In a way it's an indirect re-prompting.
I'm very happy with the increased PRs: they push the project forward, with great ideas of what to implement, and I'm very happy about AI increased productivity. Also, with AI developers are bolder in their contributions.
But this doesn't scale -- or I'll spend all my time code-reviewing :) I hope the AIs get better quickly.
Testing is no yet the main focus, so we haven't looked into automating that. But we will in the future. We have automated most of our documentation updates though. After releases or big merges we use askmanu to automatically update/create the docs. These are internal docs, but super useful for us and Claude.
Disclaimer: I'm the founder of askmanu