Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling long-running autonomous coding

simonwillison.net

Post reply on HN
Print view View original

41–50 of 112 posts

Re: Scaling long-running autonomous coding

#41

Post by azan_ »

Gazoche wrote: post #39
Chipshuffle wrote: post #37

The more I think about LLMs the stranger it feels trying to grasp what they are. To me, when I'm working with them, they don't feel intelligence but rather an attempt at mimicking it. You can never trust, that the AI actually did something smart or dump. The judge always has to be you. It's ability to pattern match it's way through a code base is impressive until it's not and you always have to pull it back to realit…

> they don't feel intelligence but rather an attempt at mimicking it Because that's exactly what they are. An LLM is just a big optimization function with the objective "return the most probabilistically plausible sequence of words in a given context". There is no higher thinking. They were literally built as a mimicry of intelligence.

> Because that's exactly what they are. An LLM is just a big optimization function with the objective "return the most probabilistically plausible sequence of words in a given context". > There is no higher thinking. They were literally built as a mimicry of intelligence.

Maybe real intelligence also is a big optimization function? Brain isn't magical, there are rules that govern our intelligence and I wouldn't be terribly surprised if our intelligence in fact turned out to be kind of returning the most plausible thoughs. Might as well be something else of course - my point is that "it's not intelligence, it's just predicting next token" doesn't make sense to me - it could be both!

Re: Scaling long-running autonomous coding

#42

Post by bauerd »

xnx wrote: post #40
ramon156 wrote: post #33

I would also love to see the statistics regarding token cost, electricity cost, environmental damage etc. Not saying that this only happens with LLMs, in fact it should be compared against e.g. a dev team of 4-5

Generally, if something costs less it has less environmental impact.

Generally wrong. It may cost less because its externalities aren't priced in.

Re: Scaling long-running autonomous coding

#44

Post by cocoto »

ramon156 wrote: post #33

I would also love to see the statistics regarding token cost, electricity cost, environmental damage etc. Not saying that this only happens with LLMs, in fact it should be compared against e.g. a dev team of 4-5

The complex thing is that you would need to take into account the energy used to feed the programmers, the energy used for their education or simply them growing up to the age they are working. For the LLMs it would have to take into account energy used for the GPU, the machine building the GPUs, datacenters, engineers maintaining it, their education etc etc. It’s so complex to really estimate these things from bottom up if you are not only looking locally, it feels impossible…

Re: Scaling long-running autonomous coding

#45

Post by visarga »

simonw wrote: post #9
janoelze wrote: post #7

Earlier quoted context omitted.

Any views on the nature of "maintainability" shifting now? If a fleet of agents demonstrated the ability to bootstrap a project like that, would that be enough indication to you that orchestration would be able to carry the code base forward? I've seen fully llm'd codebases hit a certain critical weight where agents struggled to maintain coherent feature development, keeping patterns aligned, as well as spiralling in…

Almost no idea at all. Coding agents are messing with all 25+ years of my existing intuitions about what features cost to build and maintain. Features that I'd normally never have considered building because they weren't worth the added time and complexity are now just a few well-structured prompts away. But how much will it cost to maintain those features in the future? So far the answer appears to be a whole lot le…

> But how much will it cost to maintain those features in the future?

Very little if they have good specs and tests.

Re: Scaling long-running autonomous coding

#46

Post by visarga »

Chipshuffle wrote: post #37

The more I think about LLMs the stranger it feels trying to grasp what they are. To me, when I'm working with them, they don't feel intelligence but rather an attempt at mimicking it. You can never trust, that the AI actually did something smart or dump. The judge always has to be you. It's ability to pattern match it's way through a code base is impressive until it's not and you always have to pull it back to realit…

> The judge always has to be you.

But you can automate much of that work by having good tests. Why vibe-test AI code when you can code-test it? Spend your extra time thinking how to make testing even better.

Re: Scaling long-running autonomous coding

#47

Post by oefrha »

xnx wrote: post #40
ramon156 wrote: post #33

I would also love to see the statistics regarding token cost, electricity cost, environmental damage etc. Not saying that this only happens with LLMs, in fact it should be compared against e.g. a dev team of 4-5

Generally, if something costs less it has less environmental impact.

If you exterminate the replaced human coders, sure.

Re: Scaling long-running autonomous coding

#48

Post by marisen »

cocoto wrote: post #44
ramon156 wrote: post #33

I would also love to see the statistics regarding token cost, electricity cost, environmental damage etc. Not saying that this only happens with LLMs, in fact it should be compared against e.g. a dev team of 4-5

The complex thing is that you would need to take into account the energy used to feed the programmers, the energy used for their education or simply them growing up to the age they are working. For the LLMs it would have to take into account energy used for the GPU, the machine building the GPUs, datacenters, engineers maintaining it, their education etc etc. It’s so complex to really estimate these things from botto…

It is well known that a programmer that stops programming stops requiring food

Re: Scaling long-running autonomous coding

#49

Post by encyclopedism »

Gazoche wrote: post #39
Chipshuffle wrote: post #37

The more I think about LLMs the stranger it feels trying to grasp what they are. To me, when I'm working with them, they don't feel intelligence but rather an attempt at mimicking it. You can never trust, that the AI actually did something smart or dump. The judge always has to be you. It's ability to pattern match it's way through a code base is impressive until it's not and you always have to pull it back to realit…

> they don't feel intelligence but rather an attempt at mimicking it Because that's exactly what they are. An LLM is just a big optimization function with the objective "return the most probabilistically plausible sequence of words in a given context". There is no higher thinking. They were literally built as a mimicry of intelligence.

I don't understand why this point is NOT getting across to so many on HN.

LLM's do not think, understand, reason, reflect, comprehend and they never shall. I have commented elsewhere but this bears repeating

If you had enough paper and ink and the patience to go through it, you could take all the training data and manually step through and train the same model. Then once you have trained the model you could use even more pen and paper to step through the correct prompts to arrive at the answer. All of this would be a completely mechanical process. This really does bear thinking about. It's amazing the results that LLM's are able to acheive. But let's not kid ourselves and start throwing about terms like AGI or emergence just yet. It makes a mechanical process seem magical (as do computers in general).

I should add it also makes sense as to why it would, just look at the volume of human knowledge (the training data). It's the training data with the mass quite literally of mankind's knowledge, genius, logic, inferences, language and intellect that does the heavy lifting.

Re: Scaling long-running autonomous coding

#50

Post by simonw »

teaearlgraycold wrote: post #19
simonw wrote: post #3

One of the big open questions for me right now concerns how library dependencies are used. Most of the big ones are things like skia, harfbuzz, wgpu - all totally reasonable IMO. The two that stand out for me as more notable are html5ever for parsing HTML and taffy for handling CSS grids and flexbox - that's vendored with an explanation of some minor changes here: https://github.com/wilsonzlin/fastrender/blob/19bf103…

It looks like JS execution is outsourced to QuickJS?

No, it has its own JS implementation: https://github.com/wilsonzlin/fastrender/tree/main/vendor/ec...

See also: https://news.ycombinator.com/item?id=46650998

Post reply on HN