Ilya Sutskever's SSI Inc raises $1B

Worse than that, the dystopian stories are in the training data...

You seem to repeatedly insist that hidden computation is a distinction of any relevance whatsoever.

First of all, your understanding of the architecture itself is mistaken. A transformer can iterate endlessly because each token it produces allows it a forward pass, and each of these tokens is postpended to its input in the next inference. That's the autoregressive in autoregressive transformer, and the entire reason why it was proposed for arbitrary seq2seq transduction.

This means you get layers * tokens iterations, where tokens is up to two million, and is in practice unlimited due to the LLM being able to summarize and select from that. Parallelism is irrelevant, since the transformer is sequential in the output of tokens. A transformer can iterate endlessly, it simply has to output enough tokens.

And no, the throughput isn't limited either, since each token gets translated into a high-dimensional internal representation, that in turn is influenced by each other token in the model input. Models can encode whatever they want not just by choosing a token, but by choosing an arbitrary pattern of tokens encoding arbitrary latent-space interactions.

Secondly, internal thoughts are irrelevant, because something being "internal" is an arbitrary distinction without impact. If I trained an LLM to prepend and postpend to some part of its output, and then simply didn't show that part, then the LLM wouldn't magically become human. This is something many models do even today, in fact.

Similarly, if I were to take a human and modify their brain to only be able to iterate using pen and paper, or by speaking out loud, then I wouldn't magically make them into something non-human. And I would definitely not reduce their capacity for reasoning in any way whatsoever. There are people with aphantasia working in the arts, there are people without an internal monologue working as authors - how "internal" something is can be trivially changed with no influence on either the architecture or the capabilities of that architecture.

Reasoning itself isn't some unified process, neither is it infinite iteration. It requires specific understanding about the domain being reasoned over, especially understanding of which transformation rules are applicable to produce desired states, where the judgement about which states are desirable has to be learned itself. LLMs can reason today, they're just not as good at it than humans are in some domains.

Looking back at the history of artificial cognition machines, how many of them hit a ceiling at human-level capabilities?

- A simple calculator can beat any human at arithmetic

- A word processor with a hard disk is orders of magnitude better than any human at memorizing text

- No human can approach the ELO of a chess program running on a cheap laptop

- No human polymath has journeyman-level knowledge of a tiny fraction of the fields that LLMs can recite textbook answers from

Why should we expect AGI to be the first cognitive capabilities that does not quickly shoot past human-level ability?

> Who would have thought that vectorized linear algebra will be at the center of so much financial speculation?

"vectorized linear algebra" is at the root of most of modern Physics.

Specifically, the laws of Physics are represented by the Lie groups U(1), SU(2), SU(3) and SO(3,1).

While the manifolds that Physics act on are curved, they're "locally flat". That is why local operations are tensor operations. Or linear algebra, if you prefer.

It's not all that surprising to me that "intelligence" is represented by similar math.

In fact, there is active work being done on making sense of deep learning using Lie algebra [1] (and Algebraic Topology, which generalizes the Lie algebra).

This math can be a bit hard, though, so the learning curve can be steep. However, when we're creating AI models to be ML scientists, I suspect that this kind of math may be a source of "unhobbling", as meant in Situational Awareness [2].

Because if we can understand the symmetries at play in a problem domain, it's generally a lot easier to find a mathematical architecture (like in the algebras above) that effectively describe the domain, which allows us to potentially reduce the degrees of freedom by many OOM.

> Heck, maybe we will even eventually see the famously absent productivity gains from digital tech?

I think it's a mistake to think of AI as "digital tech", especially so to assume that the development of the Internet, Social Media or crypto that we've seen over the last generation.

AI fundamentally comes with the potential to do anything a human can do in the economy (provided robotic tech keeps up). If so, the word "productivity" as currently used (economic value produced per hour of human work) becomes meaningless, since it would go to infinite (because of division by zero).

[1] https://arxiv.org/pdf/2402.08871 [2] https://situational-awareness.ai

Any intelligence on the level of average human and for sure on the level above it will be able to learn. And learning means it will acquire new motivations, among other things. Fixed motivation thing is simply a program, not AI. A very advanced program maybe, but ultimately just a scaled up version of the stuff we already have. AI will be different, if we will create it.