Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What's the largest software project AI can complete on its own?

epoch.ai

Post reply on HN
Print view View original

61–70 of 110 posts

Re: What's the largest software project AI can complete on its own?

#61

Post by htrp »

_doctor_love wrote: post #54

What size project an AI can complete on its own is IMHO not an interesting question. The AIs and harnesses are getting good enough that quite large things can be built, primarily as a function of cost. The interesting question in my view is what is the largest system an AI can maintain over a long time horizon -- while keeping a coherent architecture that adapts and responds well to change. All the while remaining pe…

isn't this a harness engineering problem at its core (and the human equivalent being a product org problem)?

Re: What's the largest software project AI can complete on its own?

#62

Post by khazhoux »

deaton wrote: post #38
khazhoux wrote: post #32

Earlier quoted context omitted.

Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…

The current project I am developing with AI is so bloated that I've spent more time and tokens cleaning up the repo (to the tune of tens of thousands of lines removed) than actually creating it in the first place. AI is very very happy to create a mess.

My experience exactly. I think we’re doing great, but about 3 weeks in I realized it’s an unworkable mess that keeps breaking, and I spend the next two weeks cleaning it up or rebuilding it

Re: What's the largest software project AI can complete on its own?

#63

Post by khazhoux »

peder wrote: post #36
khazhoux wrote: post #32

Earlier quoted context omitted.

Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…

> tons of duplication that lead to whack-a-mole The future is duplication. Code is no longer meant to be read by humans but by machines.

It’s not human readability I care about. The problem with duplicate code is that the agent will fix one instance, but leave the others buggy. Hilarity ensues.

Re: What's the largest software project AI can complete on its own?

#64

Post by _doctor_love »

htrp wrote: post #61
_doctor_love wrote: post #54

What size project an AI can complete on its own is IMHO not an interesting question. The AIs and harnesses are getting good enough that quite large things can be built, primarily as a function of cost. The interesting question in my view is what is the largest system an AI can maintain over a long time horizon -- while keeping a coherent architecture that adapts and responds well to change. All the while remaining pe…

isn't this a harness engineering problem at its core (and the human equivalent being a product org problem)?

I don't believe it is. If we think of AI-maintained software as automated gardening / automated agriculture, the harness is all the machinery that can plow, sow, water, monitor, harvest, etc. There still has to be a human stewarding the overall ecosystem and tending the garden. The human is 'on top' of loop rather than 'in it' but I don't think a harness by itself can be a good steward of architecture.

Product org problems also is very context-dependent. Product orgs can work significantly different, in some orgs humans desire to be deep in the details, in other orgs the product folks are happy to let engineering drive the boat beyond a certain level of complexity. So one size harness can't necessarily fit all.

Re: What's the largest software project AI can complete on its own?

#65

Post by francoisdevlin »

Jdstanhope wrote: post #24
rco8786 wrote: post #21

Earlier quoted context omitted.

> it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 tests. you ever worked with human engineers before? :D

I was going to say that it's pretty much exactly like working with an experienced software engineer ... except it never seems to take a step back and look at the bigger picture. It will happily fix the same class of bugs over and over without realizing a bigger change is required.

Wait, are you talking about llms or people?

Re: What's the largest software project AI can complete on its own?

#66

Post by eithed »

I'm looking at these reports and then at my sessions with Claude. In latest one, it got stuck in a loop fixing one thing and breaking another for 10 iterations, until I've noticed and gave it the insight. Or where it started continuously writing "echo", I don't know for what purpose. And I ask how, how can you not be the driver; how can you let these agents roam free and actually accomplish things.

Re: What's the largest software project AI can complete on its own?

#67

Post by boothby »

francoisdevlin wrote: post #65
Jdstanhope wrote: post #24

Earlier quoted context omitted.

I was going to say that it's pretty much exactly like working with an experienced software engineer ... except it never seems to take a step back and look at the bigger picture. It will happily fix the same class of bugs over and over without realizing a bigger change is required.

Wait, are you talking about llms or people?

It's sad that LLMs are learning to devalue human life from people like you.

Re: What's the largest software project AI can complete on its own?

#68

Post by cyanydeez »

GrinningFool wrote: post #34
khazhoux wrote: post #32

Earlier quoted context omitted.

Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…

The only way I've found to make it work on long term AI-code-only projects is to have it constantly refactor, with the axes of refactoring usually provided by me.

Yeah, i basically guide it by writing docs, tests, then code. Then try to get it looping that way. It occasionally happens reliably. If it can reliable prune context itd be pretty close to autonomous. If it achievef that, id sit down and plan out much larger scopes but for now, its just feat->docs->tests|code->loop till done.

Re: What's the largest software project AI can complete on its own?

#70

Post by andai »

boothby wrote: post #67
francoisdevlin wrote: post #65

Earlier quoted context omitted.

Wait, are you talking about llms or people?

It's sad that LLMs are learning to devalue human life from people like you.

Step 1: Develop human-like minds

Step 2: Dehumanize them constantly

Post reply on HN