Test suites just increased in value by a lot and code decreased in value.
Doubt it, code will be generated to pass tests, not the intent behind the tests.
Scaling long-running autonomous coding
91–100 of 112 posts
Re: Scaling long-running autonomous coding
#92After reading that post it feels so basic to sit here, watching my single humble claude code agent go along with its work... confident, but brittle and so easily distracted.
Re: Scaling long-running autonomous coding
#93> When I made my 2029 prediction this is more-or-less the quality of result I had in mind.
There seems to be a lot of compensation and leniency made by the author here.
So, it is seemingly impressive that someone was able to use agents to build a browser.
But they used trillions of tokens? This equates to millions of dollars of spend. Are we really happy with this?
The browser itself is not fully complete. There's rendering glitches stated in the article. So millions of dollars for something that has obvious bugs.
This is also pure agent code. Can a code base like this ever be maintained by a team of humans? Are you vendor locked into a specific model if you want to build more features? How will support work? How will releases work? The lack of reflection over the rest of the software lifecycle except building is shocking.
So I'm not sure after reflecting, whether any of this is impressive outside of "someone with unlimited tokens built a browser using ai agents". It's the same class of problem being solved over and over again. Nothing new is really being done here.
Maybe it's just me but there's much more to software than just building.
Re: Scaling long-running autonomous coding
#94Earlier quoted context omitted.
Yeah, Yann LeCun is just some luddite lol
You don't understand Yann's argument. It's similar to Richard Sutton's, in that these things aren't thinking, they're emulating thinking, and the weak implicit world models that get built in the weights are insufficient for true "AGI." This is orthogonal to the issue of whether all ideas are essentially "remixes." For the record I agree that they are.
Re: Scaling long-running autonomous coding
#95Earlier quoted context omitted.
The complex thing is that you would need to take into account the energy used to feed the programmers, the energy used for their education or simply them growing up to the age they are working. For the LLMs it would have to take into account energy used for the GPU, the machine building the GPUs, datacenters, engineers maintaining it, their education etc etc. It’s so complex to really estimate these things from botto…
It is well known that a programmer that stops programming stops requiring food
Re: Scaling long-running autonomous coding
#96> I think somebody will have built a full web browser mostly using AI assistance, and it won’t even be surprising > When I made my 2029 prediction this is more-or-less the quality of result I had in mind. There seems to be a lot of compensation and leniency made by the author here. So, it is seemingly impressive that someone was able to use agents to build a browser. But they used trillions of tokens? This equates to…
Re: Scaling long-running autonomous coding
#97> I think somebody will have built a full web browser mostly using AI assistance, and it won’t even be surprising > When I made my 2029 prediction this is more-or-less the quality of result I had in mind. There seems to be a lot of compensation and leniency made by the author here. So, it is seemingly impressive that someone was able to use agents to build a browser. But they used trillions of tokens? This equates to…
Yes, arguably 5 million is a fair price and cheaper than what it would take to pay humans.
Re: Scaling long-running autonomous coding
#98Browsers are pretty much the best case scenario for autonomous coding agents. A totally unique situation that mostly doesn't occur in the real world. At a minimum: 1. You've got an incredibly clearly defined problem at the high level. 2. Extremely thorough tests for every part that build up in complexity. 3. Libraries, APIs, and tooling that are all compatible with one another because all of these technologies are bu…
Re: Scaling long-running autonomous coding
#99 $ time target/release/fetch_and_render "https://www.lauf-goethe-lauf.de/"
real 0m0,685s
user 0m0,548s
sys 0m0,070s
$ time chromium --headless --disable-gpu --screenshot=out.png --window-size=1200,800 https://www.lauf-goethe-lauf.de/
real 0m1,099s
user 0m0,927s
sys 0m0,692s
# edit: with a hot-standby chrome and a running node instance a can reach 0,369s seconds hereRe: Scaling long-running autonomous coding
#100> I think somebody will have built a full web browser mostly using AI assistance, and it won’t even be surprising > When I made my 2029 prediction this is more-or-less the quality of result I had in mind. There seems to be a lot of compensation and leniency made by the author here. So, it is seemingly impressive that someone was able to use agents to build a browser. But they used trillions of tokens? This equates to…
If an AI system autonomously built a rocket and went to the moon, would you call it unimpressive because it's already been done? The moving of goalposts is shocking.
This isn't a POC web engine; it's throw-away code that can never scale to a full web engine.
So instead of wasting millions on this autonomous run, they should have put together a small team of people with some ideas on how to improve on existing web engines, and then give that team a large token development budget. You could get a nice POC after a couple of weeks, and after a year or two of further iterations you might have something really interesting.
So this is a great example of how AI fails when left unsupervised; a more interesting experiment would be about how a small team can leverage AI to leapfrog Chromium; not in one week but in a year or two.