I believe they can make ANY software project the largest it can be! This solution should look for more problems.
The benchmark does not reward producing more code.
What's the largest software project AI can complete on its own?
31–40 of 110 posts
Re: What's the largest software project AI can complete on its own?
#32I have been using Claude to build a clone of Bash in Rust ( https://github.com/jdstanhope/huck ). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 te…
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that if I, human overseer, don't course correct it many times a day, it goes off into Neverland.
Yet I see comments and articles like above every week, where someone says "Yeah, I just left my agent run for a few nights, and then it made this awesome tool."
There is a huge gap here, but I can't pinpoint it yet. Either I (and everyone I talk to) is Doing It Wrong, or your code is actually flawed at its core and bugs will take exponentially longer to fix and never converge, or... I just need to let my agent run for 3 months??
Re: What's the largest software project AI can complete on its own?
#33my biggest so far was https://apps.apple.com/at/app/aifcc-ai-first-computer/id6782... a whole linux distro in a mac app. claude worked for about 2 weeks.
Re: What's the largest software project AI can complete on its own?
#34I have been using Claude to build a clone of Bash in Rust ( https://github.com/jdstanhope/huck ). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 te…
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…
The only way I've found to make it work on long term AI-code-only projects is to have it constantly refactor, with the axes of refactoring usually provided by me.
Re: What's the largest software project AI can complete on its own?
#35I'm curious about that too. I'm building a language with AI, and when I leave the AI running on its own, it keeps generating code in strange directions. I wonder what kind of structure AI-only code would have.
Is it like self driving cars, where 100% non-human code would be arguably safer? But then it always comes down to having some human who is accountable for verifying the behavior. I think ultimately we just need a better set of languages and frameworks to minimally encode the types of applications we want to build today, so that human comprehension and verification of the source code becomes easier.
Category error. Self-driving cars are still meticulously coded by humans.
Re: What's the largest software project AI can complete on its own?
#36I have been using Claude to build a clone of Bash in Rust ( https://github.com/jdstanhope/huck ). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 te…
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…
> tons of duplication that lead to whack-a-mole
The future is duplication. Code is no longer meant to be read by humans but by machines.
Re: What's the largest software project AI can complete on its own?
#37I have been using Claude to build a clone of Bash in Rust ( https://github.com/jdstanhope/huck ). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 te…
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…
Please check out the code and let me know what you think. I have driven a few large refactorings that I don't think Claude would have done on its own. The two big ones were related to handling tokenization and parsing (Claude's original code had circular dependencies and would scan forward in the lever instead of using the parser to determine structure) and the second was for using subshells for command substitution instead of inline and having dual FD and buffer handling.
Re: What's the largest software project AI can complete on its own?
#38I have been using Claude to build a clone of Bash in Rust ( https://github.com/jdstanhope/huck ). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 te…
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…
The current project I am developing with AI is so bloated that I've spent more time and tokens cleaning up the repo (to the tune of tens of thousands of lines removed) than actually creating it in the first place. AI is very very happy to create a mess.
Re: What's the largest software project AI can complete on its own?
#39We (Team of 3) built a retrieval engine from scratch in .NET for investigative search, with AI doing most of the backend implementation while humans handled the design and architecture and front end. It was developed using a task based, incremental approach. Includes more than 3,000 tests and extensive documentation, and is optimized for asynchronous ingestion using NATS and a disk backed WAL.
It includes a custom WAL, structure aware semantic chunking, ingestion and indexing pipelines, hybrid (rrf), text and vector retrieval, ACL aware filtering at query level, query classifier, citation tracking, and a CrateDB backend that performs fused retrieval over a single table. total LOC: 180 K. Status: evaluation by State agencies for investigative search on legacy documents. tool used - Codex, Luna, and earlier codex 5.3, and gpt 4 mini.
Re: What's the largest software project AI can complete on its own?
#40"Reproduce an existing piece of software" doesn't seem like a problem that generalizes well to "develop a new piece of software," for a fair number of reasons.
I agree...if there is a manual that documents well specified software including edge/corner cases i would kinda consider it a waste of tokens unless the aim is to build a better version of the software