Live data from Hacker News

What's the largest software project AI can complete on its own?

epoch.ai

51–60 of 110 posts

Re: What's the largest software project AI can complete on its own?

#51
post #38

Earlier quoted context omitted.

The current project I am developing with AI is so bloated that I've spent more time and tokens cleaning up the repo (to the tune of tens of thousands of lines removed) than actually creating it in the first place. AI is very very happy to create a mess.

As few people have mentioned you have to guide the AI to clean up stuff, it doesn't seem to choose that on its own. If prompted it can come up with a lot of things to clean up.

Yeah, it always wants to add another layer or append another fix. It almost never goes back and fixes the original thing in place.

Re: What's the largest software project AI can complete on its own?

#52

I have been using Claude to build a clone of Bash in Rust ( https://github.com/jdstanhope/huck ). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 te…

Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…

[dead]

Re: What's the largest software project AI can complete on its own?

#53

I have been using Claude to build a clone of Bash in Rust ( https://github.com/jdstanhope/huck ). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 te…

Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…

IME you can get good results if:

1. You have very good tests. The agent can write them, but you have to prompt it to do that, and some human input may be needed here.

2. You have a good, modular architecture. Again, this is something that needs to be prompted and/or developed up front. Agent work can scale if you divide and conquer, so that any given task has a well-defined scope, limiting the amount of context needed to handle it well. If you let an agent just build a big pile of code, you’ll have issues.

Once you have the above, you’re more likely to be able to just let the agent run.

This is why you hear about things like SQLite being ported to Rust (by multiple groups!) It has a comprehensive unit test suite and a good existing architecture. Such porting projects are benefitting from already having done the upfront work that I mentioned, plus they have the original source code as a reference implementation.

Because of that, most of what I do involves a lot of interaction with the agent. That’s because I’m typically working on complex existing systems, that don’t necessarily have the above properties, and agents aren’t at the point where they can handle that autonomously.

Re: What's the largest software project AI can complete on its own?

#54
What size project an AI can complete on its own is IMHO not an interesting question. The AIs and harnesses are getting good enough that quite large things can be built, primarily as a function of cost.

The interesting question in my view is what is the largest system an AI can maintain over a long time horizon -- while keeping a coherent architecture that adapts and responds well to change. All the while remaining performant, scalable, and understandable to humans. I am unaware of what is the currently longest-lived vibe coded project but it cannot be more than a few years old.

So to my mind, we don't have very much real data yet.

Re: What's the largest software project AI can complete on its own?

#55
post #36

Earlier quoted context omitted.

Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…

> tons of duplication that lead to whack-a-mole The future is duplication. Code is no longer meant to be read by humans but by machines.

> Code is no longer meant to be read by humans but by machines.

That isn't true. Everyone knows it and you know it. So why would you make a statement like this? Does it make you feel like a "thought leader"? This isn't X or LinkedIn. Bulshitting here just makes you look like a fool.

Re: What's the largest software project AI can complete on its own?

#56
post #53

Earlier quoted context omitted.

Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…

IME you can get good results if: 1. You have very good tests. The agent can write them, but you have to prompt it to do that, and some human input may be needed here. 2. You have a good, modular architecture. Again, this is something that needs to be prompted and/or developed up front. Agent work can scale if you divide and conquer, so that any given task has a well-defined scope, limiting the amount of context neede…

This is exactly the case for "porting" Bash to Rust. There is a huge amount of existing tests that can be leverage and when in doubt you can run the same code in bash and Huck and compare the result and then fix the issue.

Re: What's the largest software project AI can complete on its own?

#57
i've been working on several rather large projects these past few months, and i'm trying to write as little code as possible.

I don't think i wrote more than 10 lines of code in the largest project i'm working on. Lines of code: Java: 900_635, typescript: 725_418, C++: 180_445, Dart: 96_181.

It's been obvious from the start that no model, as good as it is, can do large(-ish) amounts of work by its own without supervision, control, criticism, etc ... If left unsupervised, models usually do half the work, leaving stubs and todos everywhere.

Quality comes from applying software engineering principles as much as possible, just like you would do with teams of junior devs: planning sessions and implementation sessions with adversarial critiques, specifying as much as possible upfront, planning unit/smoke/integration tests, etc ...

Many systems rely on swarm of agents to build software but i've found it very difficult to get good results without lots of overhead/token waste because of inter agent communications mostly.

So instead i built what is mostly a workflow engine to structure / organize processes into workflows with different agents assigned different roles. I've setup a basic landing page here https://kodfactory.com if anyone wants to follow along.

Re: What's the largest software project AI can complete on its own?

#58

I believe they can make ANY software project the largest it can be! This solution should look for more problems.

You say that, but I asked Codex to build a Asteroids (but FPS) as a web app, and it wrote it in under 4K LOC, and a Tempest implementation in 6.5K LOC. Those seem like really reasonable line counts to me. I also built a caching proxy server for apt packages, that's a bigger project, and that is 30.5K lines of golang, which doesn't seem bad compared to apt-cacher-ng which is 19.3K of C++ (my project does significantly…

[flagged]

Re: What's the largest software project AI can complete on its own?

#59

Earlier quoted context omitted.

You say that, but I asked Codex to build a Asteroids (but FPS) as a web app, and it wrote it in under 4K LOC, and a Tempest implementation in 6.5K LOC. Those seem like really reasonable line counts to me. I also built a caching proxy server for apt packages, that's a bigger project, and that is 30.5K lines of golang, which doesn't seem bad compared to apt-cacher-ng which is 19.3K of C++ (my project does significantly…

[flagged]

I'm going to take it that you were being sarcastic.

However, in the case of the apt cacher, the DDoS of Ubuntu infrastructure showed that the existing apt cachers left a *LOT* of room for improvement, not to mention that apt-cacher-ng has regularly (over the last 13 years of use) gotten wedged and needed intervention.

This cacher understands Deb package repo structure, and when it finds new meta-data it grabs and checks the meta-data, downloads new version of any package you regularly download (configurable), and then makes a snapshot of that, which it then promotes for cache users to see.

I can shut down my Internet connection and then install a fresh Ubuntu from ISO, then "apt update; apt dist-upgrade" successfully via the cache.

If you are saying that AI isn't solving novel problems, you are waaaaaay off.

But you're also missing the point: This branch of the conversation is about AI tools writing wildly inflated code. Which I proposed a few data-points refuting.

Re: What's the largest software project AI can complete on its own?

#60
post #36

Earlier quoted context omitted.

Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…

> tons of duplication that lead to whack-a-mole The future is duplication. Code is no longer meant to be read by humans but by machines.

If that's true (I very much doubt it), then have the LLMs generate something like WASM binary and run that everywhere instead. No need for human readable languages. Or if LLMs are not that good at binary format, then some textual representation optimized for LLMs instead of humans that compiles to WASM, the JVM or Rust binary.

No need for multiple programming languages anymore. But again, doubt that happens.

Post reply on HN