They’re made out of weights

> should read philosophy

subject/object dichotomy is a springboard to many schools of thought.

1) that subject emerges from objects - ie, anything has a material explanation, and everything is a machine.

2) that objects emerge out of a subject as a world model (platonic, descartes)

3) the subject and objects are one and the same representation of nature (spinoza)

4) subjects and objects emerge and disappear together (buddhist)

Anything reduced to computation is a fixed function from input to output, and is "dead" in the sense that it is unadaptable to its environment. Weights therefore is a dead machine.

Another view of this is that any closed system has unanswerable questions within it. Therefore, there is no system that can encompass everything. Hence weights being a closed system doesnt encompass everything.

>A language is a set of sentences. A sentence is a finite sequence of symbols drawn from an alphabet.

A language is a structured system of communication used to express arbitrary ideas between multiple parties. Math operation tables do not, and cannot, do that on their own.

That distinction matters here because we are talking about what properties the model is expected to learn. English and operation tables are fundamentally different objects, so it is not surprising that a model learns different kinds of structure from them.

The LLM is still a "normal algorithm", just one with a fairly large dataset to use. Assigning the algorithm part of LLMs magical properties hinders understanding. The work needed to pick the next output token is very much a classical algorithm.

Algorithms can be based on training and/or use data just fine, too. https://arxiv.org/abs/1712.01208

(Now, the weights used, those we kinda really don't understand the same way we understand the processing, and the approach to looking for structures in weights sometimes looks more like archeology or anthropology than computer science.)

It sounds like you're trying to express some kind of "but LLMs are so much more" thought. Yes, very much, they are. It's because of the size of the data, there's interesting emergence there. They're still a normal algorithm. (And our brains aren't quite like that; biological things are much more random/chaotic and generally non-reproducible. And the data and algorithm aren't separate.)

Ahh thanks. I didn't follow the article links.

Not really. The history of "big gains" of machine learning is: put together a simple architecture that makes few assumptions but scales well. Then up the data and compute by 2 OOMs. By itself, the new architecture underperforms. Paired with the bitter lesson, however?

Don't make assumptions. Make a setup where the gradient descent can make them for you.

Empirically? LLMs are nowhere near "the wall". We've been hearing "the wall is nigh" since 2020. Six years in, we're still scaling LLMs, and the graveyards are full of "LLM killers". The system that kills the LLM is always a bigger, badder LLM, and never a new revolutionary architecture. The scaling doesn't just keep working - it works so well that it's seen as the only viable path forward at the frontier of reasoning and agentic work. Or even outside it. ChatGPT Images 2.0 is an image model with an agentic LLM at its core - generational gains in compositional capability.

For just about every "failure mode that confirms they're not thinking", you see one of two things. The first is that a new LLM releases a few months after and the "fundamental" issue abruptly goes away. The second is that we take a good, long look at a human, and find that the human also fails like this - and thus, "not thinking". Often both! Always funny when it's both.

One thing that's very biologically distinct is: local connectivity. In a GPU, global connectivity is cheap. In a brain, it's prohibitively expensive. The brain has no true backpropagation because it has no true global connectivity, and has to make do with local rules. A GPU is a strictly more expressive substrate connectivity-wise. So any point in the design of a computational substrate where you could remove complexity or increase performance by adding more connectivity? Silicon advantage. The brain isn't a "strictly better computational substrate" - it makes different tradeoffs. Which tradeoffs are better for attaining intelligence is an open question.

And, sure. Having a substrate with a capacity for intelligence doesn't mean having intelligence. No elephant has ever learned to code. The problem is: LLMs already did! LLMs already compete with humans on just about every task that was once thought to "require human intelligence". They don't always win - but they perform significantly above chance, and often above an average non-expert human.

So, my bet is on "LLM but even bigger". If there's a point where LLMs begin to lag behind and novel architectures get a sharp advantage, we are yet to hit it.