Scaling long-running autonomous coding • Hacker News Board

Re: Scaling long-running autonomous coding

#61

Post by omnicognate » Tue, Jan 20, 2026, 2:09 PM UTC

ramraj07 wrote: post #16

halfcat wrote: post #10

So AI makes it cheaper to remix anything already-seen, or anything with a stable pattern, if you’re willing to throw enough resources at it. AI makes it cheap (eventually almost free) to traverse the already-discovered and reach the edge of uncharted territory. If we think of a sphere, where we start at the center, and the surface is the edge of uncharted territory, then AI lets you move instantly to the surface. If…

The fundamental idea that modern LLMs can only ever remix, even if its technically true (doubt), in my opinion only says to me that all knowledge is only ever a remix, perhaps even mathematically so. Anyone who still keeps implying these are statistical parrots or whatever is just going to regret these decisions in the future.

Why doubt? Transformers are a form of kernel smoothing [1]. It's literally interpolation [2]. That doesn't mean it can only echo the exact items in its training data - generating new data items is the entire point of interpolation - but it does mean it's "remixing" (literally forming a weighted sum of) those items and we would expect it to lose fidelity when moving outside the area covered by those points - i.e. where it attempts to extrapolate. And indeed we do see that, and for some reason we call it "hallucinating".

The subsequent argument that "LLMs only remix" => "all knowledge is a remix" seems absurd, and I'm surprised to have seen it now more than once here. Humanity didn't get from discovering fire to launching the JWST solely by remixing existing knowledge.

[1] http://bactra.org/notebooks/nn-attention-and-transformers.ht...

[2] Well, smoothing/estimation but the difference doesn't matter for my point.

Re: Scaling long-running autonomous coding

#62

Post by Zababa » Tue, Jan 20, 2026, 2:11 PM UTC

encyclopedism wrote: post #59

Zababa wrote: post #56

Earlier quoted context omitted.

Can you give examples of how that "LLM's do not think, understand, reason, reflect, comprehend and they never shall" or that "completely mechanical process" helps you understand better when LLM works and when they don't? Many people are throwing around that they don't "think", that they aren't "conscious", that they don't "reason", but I don't see those people sharing interesting heuristics to use LLMs well. The "the…

We know what an LLM is in fact you can build one from scratch if you like. e.g. https://www.manning.com/books/build-a-large-language-model-f... It's an algorithm and a completely mechanical process which you can quite literally copy time and time again. Unless of course you think 'physical' computers have magical powers that a pen and paper Turing machine doesn't? > Many people are throwing around that they don't "th…

>My digital thermometer doesn't think. Imbibing LLM's with thought will start leading to some absurd conclusions.

What kind of absurd conclusions? And what kind of non absurd conclusions can you make when you follow your let's call it "mechanistic" view?

>It's an algorithm and a completely mechanical process which you can quite literally copy time and time again. Unless of course you think 'physical' computers have magical powers that a pen and paper Turing machine doesn't?

I don't, just like I don't think a human or animal brain has any magical power that imbues it with "intelligence" and "reasoning".

>A cursory read of basic philosophy would help elucidate why casually saying LLM's think, reason etc is not good enough.

I'm not saying they do or they don't, I'm saying that from what I've seen having a strong opinion about whether they think or they don't seem to lead people to weird places.

>What is thinking? What is intelligence? What is consciousness? These questions are difficult to answer. There is NO clear definition.

You see pretty certain that whatever those three things are a LLM isn't doing it, a paper and pencil aren't doing it even when manipulated by a human, the system of a human manipulating a paper and pencil isn't doing it.

Re: Scaling long-running autonomous coding

#63

Post by myrmidon » Tue, Jan 20, 2026, 2:22 PM UTC

encyclopedism wrote: post #58

myrmidon wrote: post #55

Earlier quoted context omitted.

> If you had enough paper and ink and the patience to go through it, you could take all the training data and manually step through and train the same model. But you could make the exact same argument for a human mind? (could just simulate all those neural interactions with pen and paper) The only way to get out of it is to basically admit magic (or some other metaphysical construct with a different name).

> But you could make the exact same argument for a human mind? It would be an argument and you are free to make it. What the human mind is, is an open scientific and philosophical problem many are working on. The point is that LLM's are NOT the same because we DO know that LLM's are. Please see the myriad of tutorials 'write an LLM from scratch'

We do know that they are different, and that there are some systematic shortcomings in LLMs for now (e.g. no mechanism for online learning).

But we have no idea how many "essential" differences there are (if any!).

Dismissing LLMs as avenues toward intelligence just because they are simpler and easier to understand than our minds is a bit like looking at a modern phone from a 19th century point of view and dismissing the notion that it could be "just a Turing machine": Sure, the phone is infinitely more complex, but at its core those things are the same regardless.

Re: Scaling long-running autonomous coding

#64

Post by __mharrison__ » Tue, Jan 20, 2026, 2:43 PM UTC

andrewchambers wrote: post #43

Test suites just increased in value by a lot and code decreased in value.

This is one of the reasons why I just wrote a testing book (beta reviews giving feedback now). Testing is one of those boring subjects that many programmers ignore. But it just got very relevant. Especially TDD.

Re: Scaling long-running autonomous coding

#65

Post by Chipshuffle » Tue, Jan 20, 2026, 2:53 PM UTC

NiloCK wrote: post #53

Chipshuffle wrote: post #37

The more I think about LLMs the stranger it feels trying to grasp what they are. To me, when I'm working with them, they don't feel intelligence but rather an attempt at mimicking it. You can never trust, that the AI actually did something smart or dump. The judge always has to be you. It's ability to pattern match it's way through a code base is impressive until it's not and you always have to pull it back to realit…

If you find yourself 50-first-dating your LLMs, it may be worth it to invest some energy into building up some better context indexing of both the codebase itself and of your roadmap.

Yeah, I admit I'm probably not doing that quite optimally. I'm still just letting the LLM generate ephemeral .md files that I delete after a certain task is done.

The other day I found [beads](https://github.com/steveyegge/beads) and thought maybe that could be a good improvement over my current state.

But I'm quite hesitant because I also have seen these AGENTS.md files become stale and then there is also the question of how much information is too much especially with the limited context windows.

Probably all things that could again just be solved by leveraging AI more and I'm just an LLM noob. :D

Re: Scaling long-running autonomous coding

#66

Post by ramraj07 » Tue, Jan 20, 2026, 3:19 PM UTC

omnicognate wrote: post #61

ramraj07 wrote: post #16

Earlier quoted context omitted.

The fundamental idea that modern LLMs can only ever remix, even if its technically true (doubt), in my opinion only says to me that all knowledge is only ever a remix, perhaps even mathematically so. Anyone who still keeps implying these are statistical parrots or whatever is just going to regret these decisions in the future.

Why doubt? Transformers are a form of kernel smoothing [1]. It's literally interpolation [2]. That doesn't mean it can only echo the exact items in its training data - generating new data items is the entire point of interpolation - but it does mean it's "remixing" (literally forming a weighted sum of) those items and we would expect it to lose fidelity when moving outside the area covered by those points - i.e. wher…

Its not clear to me that LLMs hallucinating is because of they are extrapolating beyond their training data. Is that proven? Or are you extrapolating?

Even acknowledging it is interpolation, models can extrapolate slightly without making things up, within the range where the model still applies. Whos to say what this range is for an LLM operating in thousand dimensional space? As far as I can tell the main limiters to LLM creativity are guardrails we put in place for safety and usefulness.

And what exactly is your proof that human ingenuity is not just pattern matching. Im sure a hypothesis can be put that fire was discovered by just adding up all known facts the people of those times knew and stumbling on something that put it all together. Sounds like knowledge remix + slight extrapolating to me.

Re: Scaling long-running autonomous coding

#67

Post by ramraj07 » Tue, Jan 20, 2026, 3:20 PM UTC

mrbungie wrote: post #21

ramraj07 wrote: post #16

Earlier quoted context omitted.

The fundamental idea that modern LLMs can only ever remix, even if its technically true (doubt), in my opinion only says to me that all knowledge is only ever a remix, perhaps even mathematically so. Anyone who still keeps implying these are statistical parrots or whatever is just going to regret these decisions in the future.

> Anyone who still keeps implying these are statistical parrots or whatever is just going to regret these decisions in the future. You know this is a false dichotomy right? You can treat and consider LLMs statistical parrots and at the same time take advantage of them.

Yes, but the immediate equivalent scenario to me is how people treated other people as slaves merely using them like machines. Sure you got use out of them, but was that the best use?

Re: Scaling long-running autonomous coding

#68

Post by cousinbryce » Tue, Jan 20, 2026, 4:00 PM UTC

simonw wrote: post #5

sealeck wrote: post #4

Earlier quoted context omitted.

I think the other question is how far away this is from a "working" browser. It isn't impossible to render a meaningful subset of HTML (especially when you use external libraries to handle a lot of this). The real difficulty is doing this (a) quickly, (b) correctly and (c) securely. All of those are very hard problems, and also quite tricky to verify. I think this kind of approach is interesting, but it's a bit sad t…

Yeah, I'm hoping they publish a lot more about this project! It deserves way more then the few sentences they've shared about it so far.

I’m interested to see how much more they know about the project

Re: Scaling long-running autonomous coding

#69

Post by AstroBen » Tue, Jan 20, 2026, 4:05 PM UTC

cocoto wrote: post #44

ramon156 wrote: post #33

I would also love to see the statistics regarding token cost, electricity cost, environmental damage etc. Not saying that this only happens with LLMs, in fact it should be compared against e.g. a dev team of 4-5

The complex thing is that you would need to take into account the energy used to feed the programmers, the energy used for their education or simply them growing up to the age they are working. For the LLMs it would have to take into account energy used for the GPU, the machine building the GPUs, datacenters, engineers maintaining it, their education etc etc. It’s so complex to really estimate these things from botto…

Yes. I absolutely agree. To fully optimize this system we must enact breeding restrictions to reduce the energy demands from the humans.

Re: Scaling long-running autonomous coding

#70

Post by omnicognate » Tue, Jan 20, 2026, 4:12 PM UTC

ramraj07 wrote: post #66

omnicognate wrote: post #61

Earlier quoted context omitted.

Why doubt? Transformers are a form of kernel smoothing [1]. It's literally interpolation [2]. That doesn't mean it can only echo the exact items in its training data - generating new data items is the entire point of interpolation - but it does mean it's "remixing" (literally forming a weighted sum of) those items and we would expect it to lose fidelity when moving outside the area covered by those points - i.e. wher…

Its not clear to me that LLMs hallucinating is because of they are extrapolating beyond their training data. Is that proven? Or are you extrapolating? Even acknowledging it is interpolation, models can extrapolate slightly without making things up, within the range where the model still applies. Whos to say what this range is for an LLM operating in thousand dimensional space? As far as I can tell the main limiters t…

> Its not clear to me that LLMs hallucinating is because of they are extrapolating beyond their training data. Is that proven? Or are you extrapolating?

It's a hypothesis at this stage, but I'm going to have a go at making it more quantitative. It seems the obvious explanation for "hallucinations", and it seems like it should also be rather straightforward to attribute particular inference results to the training data that influenced them. I'm expecting to encounter difficulties, though, since the idea seems so obvious it's vanishingly unlikely it hasn't been tried.

> And what exactly is your proof that human ingenuity is not just pattern matching.

Firstly, I'm not the one making a strong claim that needs to "proved". Secondly, "pattern matching" is ill-defined and not what I'm saying human intelligence isn't. I'm saying human intelligence isn't a kernel smoothing algorithm run over a corpus of text. This seems rather obvious. What's your proof that it is that?