Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling long-running autonomous coding

simonwillison.net

Post reply on HN
Print view View original

81–90 of 112 posts

Re: Scaling long-running autonomous coding

#81

Post by polyglotfacto »

daxfohl wrote: post #79
polyglotfacto wrote: post #75

I'm a maintainer of Servo which is another web engine project. Although I dissented on the decision, we banned the use of AI. Outside of the project I've been enjoying agentic coding and I do think it can be used already today to build production-grade software of browser-like complexity. But this project shows that autonomous agents without human oversight is not the way forward. Why? Because the generated code make…

Though the fact that the code is so incoherent and inconsistent plausibly makes it more impressive that they still managed to make something that works at all, and weakens the argument that "all they did was copy/translate some existing other things to Rust." That said, it's possible that none of that code even gets executed at run time, and the only code that is actually run is some translated glue code, with the ot…

I don't think it's all copy/pasted; it is quite an original byzantine architecture.

You're right that lots of code appears only used in unit-tests, of which there is an enormous amount(making it hard to tell whether what is being tested makes sense). In Servo we don't have a single line of unit-tests in the script component, because all of it is covered by the WPT integration test suite shared with all other engines...

Re: Scaling long-running autonomous coding

#82

Post by simonw »

polyglotfacto wrote: post #75

I'm a maintainer of Servo which is another web engine project. Although I dissented on the decision, we banned the use of AI. Outside of the project I've been enjoying agentic coding and I do think it can be used already today to build production-grade software of browser-like complexity. But this project shows that autonomous agents without human oversight is not the way forward. Why? Because the generated code make…

Thanks for this, that was a really informative comment.

Re: Scaling long-running autonomous coding

#83

Post by polyglotfacto »

simonw wrote: post #82
polyglotfacto wrote: post #75

I'm a maintainer of Servo which is another web engine project. Although I dissented on the decision, we banned the use of AI. Outside of the project I've been enjoying agentic coding and I do think it can be used already today to build production-grade software of browser-like complexity. But this project shows that autonomous agents without human oversight is not the way forward. Why? Because the generated code make…

Thanks for this, that was a really informative comment.

You're welcome; big fan of your blog and a former Django dev myself.

Just made some last edits above so not sure which version you saw. I toned it down a bit and clarified some stuff...

Re: Scaling long-running autonomous coding

#84

Post by krashidov »

utopiah wrote: post #72
andrewchambers wrote: post #43

Test suites just increased in value by a lot and code decreased in value.

Doubt it, code will be generated to pass tests, not the intent behind the tests.

if you can steer an LLM to write an application based on what you want, you can steer an LLM to write the tests you want. Some people will be better at getting the LLM to write tests, but it's only going to get easier and easier

Re: Scaling long-running autonomous coding

#85

Post by daxfohl »

daxfohl wrote: post #76

So we've graduated from unmaintainable slop code to unusable slop products. Sorry, this just doesn't feel like progress toward any meaningful future. But I'm sure it will unburden lots of investors of their money.

The whole industry is like one of those projects that claims "90% finished" from the time of the first demo, then for the next N years, all the way up until the project is eventually canceled. Except this project already has trillions of dollars at stake.

Re: Scaling long-running autonomous coding

#86

Post by andrewchambers »

utopiah wrote: post #72
andrewchambers wrote: post #43

Test suites just increased in value by a lot and code decreased in value.

Doubt it, code will be generated to pass tests, not the intent behind the tests.

I think we agree - getting the llms to understand your intent is the hard part, at the very least you need well specified tests.

Perhaps more advanced llms + specifications + better tests.

Re: Scaling long-running autonomous coding

#87

Post by fulafel »

shubhamjain wrote: post #15
simonw wrote: post #3

One of the big open questions for me right now concerns how library dependencies are used. Most of the big ones are things like skia, harfbuzz, wgpu - all totally reasonable IMO. The two that stand out for me as more notable are html5ever for parsing HTML and taffy for handling CSS grids and flexbox - that's vendored with an explanation of some minor changes here: https://github.com/wilsonzlin/fastrender/blob/19bf103…

Why attempt something that has abundant number of libraries to pick and choose? To me, however impressive it is, 'browser build from scratch' simply overstates it. Why not attempt something like a 3D game where it's hard to find open source code to use?

There's many open source ones around.

Also graphics acceleration makes it hard to do from scratch rather than using using the 3D APIs but I guess you could in principle go bare iron on hardware that has published specs such as AMD, or just do software only rendering.

Re: Scaling long-running autonomous coding

#88

Post by fulafel »

cheevly wrote: post #29
shubhamjain wrote: post #15

Earlier quoted context omitted.

Why attempt something that has abundant number of libraries to pick and choose? To me, however impressive it is, 'browser build from scratch' simply overstates it. Why not attempt something like a 3D game where it's hard to find open source code to use?

Assets are very hard to produce and largely unsolved by AI at the moment.

There's AI based 3d asset generation tools around. For example https://www.meshy.ai/ https://hyper3d.ai/ https://www.sloyd.ai/

Re: Scaling long-running autonomous coding

#89

Post by well_ackshually »

andrewchambers wrote: post #43

Test suites just increased in value by a lot and code decreased in value.

No, OP is merely an AI deepthroater that will blindly swallow whatever drivel is put out by AI companies and then "benchmark" it by having it generate a pelican (oh and he got early access to the model), then call whatever he puts out "AI optimism"

The reality of things is, AI still can't handle long running tasks without blowing $500k worth of tokens for an end result that doesn't work, and further work is another $100k worth to get nothing novel.

Re: Scaling long-running autonomous coding

#90

Post by Xmd5a »

well_ackshually wrote: post #89
andrewchambers wrote: post #43

Test suites just increased in value by a lot and code decreased in value.

No, OP is merely an AI deepthroater that will blindly swallow whatever drivel is put out by AI companies and then "benchmark" it by having it generate a pelican (oh and he got early access to the model), then call whatever he puts out "AI optimism" The reality of things is, AI still can't handle long running tasks without blowing $500k worth of tokens for an end result that doesn't work, and further work is another $…

Where are you pulling these numbers from? I'm genuinely interested. Is it the kind of budget you need to spend in order to have Claude build a Word clone?
Post reply on HN