Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understand how transformers work by demystifying the math behind them

osanseviero.github.io

Post reply on HN
Print view View original

121–130 of 139 posts

Re: Understand how transformers work by demystifying the math behind them

#121

Post by ActorNightly »

danielmarkbruce wrote: post #118
ActorNightly wrote: post #116

Earlier quoted context omitted.

Let me rephrase that. One level deep information compression is cool, but I want actual AI. Its true that our brains compress information, but we compress it in a much more complex manner, in the sense that we can not only recall stuff, but also execute a decision tree that often involves physical actions to find the answer we are looking for.

An LLM isn't just recalling stuff. Brand new stuff, which it never saw in it's training, can come out. The minute you take a token and turn it into an embedding, then start changing the numbers in that embedding based on other embeddings and learned weights, you are playing around with concepts. As for executing a decision tree, ReAct or Tree of Thought or Graph of Thought is doing that. It might not be doing it as w…

>Brand new stuff, which it never saw in it's training, can come out.

Sort of. You can get LLMs to produce some new things, but these are statistical averages of existing information. Its kinda like a static "knowledge tree", where it can do some interpolation, but even then, its interpolation based on statistically occurring text.

Re: Understand how transformers work by demystifying the math behind them

#123

Post by ActorNightly »

knightoffaith wrote: post #120
ActorNightly wrote: post #115

Earlier quoted context omitted.

From brief look at the paper, they are doing a gradient descent of the architecture based on validation loss, which does good for efficiency, but its not ground breaking. The problem is that you are still training towards a target of a correct answer. I don't think this is gonna be applicable in the future, in the sense that we have to train on other things (like logical consistency somehow encoded into the network),…

Your expectations are pretty high. Differentiable architecture search as you mentioned in the original comment is one thing; going beyond empirical risk minimization-based learning is another thing entirely. In fact, they seem mostly orthogonal. That aside, it seems like AI has had the most empirical success by not imposing hard constraints/structure, but letting models learn completely "organically". The computation…

They expectations high, but its not so much as orthogonal as more basic. Our brains work on add/multiply/activation this is well known. But the composition of the neural connection strengths in our brain that makes us us is definitely not trained on any sort of final loss. Or at least not completely.

Re: Understand how transformers work by demystifying the math behind them

#124

Post by nomel »

mrguyorama wrote: post #110
nethi wrote: post #73

Earlier quoted context omitted.

True, but it is the process of climbing trees that gives the insight whether taller trees help or not and if not, what to do next.

Not true. Climbing trees for millions of years taught us nothing about orbits, or rockets, or literally incomprehensible to human distances, or the vacuum of space, or any possible way to get higher than a tree. We eventually moved on to lighter than air flight, which once again did not teach us any of those things and also was a dead end from the "get to the sky/moon" perspective, so then we invented heavier than ai…

> Climbing trees for millions of years taught us nothing about

Considering the chasm in the number of neurons between the apes and most other animals, I think one could claim that climbing those trees had some contribution to the ability to understand those things. ;) Navigating trees, at weight and speed, has a minimum intelligence reqiurement.

Re: Understand how transformers work by demystifying the math behind them

#125

Post by ActorNightly »

knightoffaith wrote: post #47
ActorNightly wrote: post #6

The whole "mystery" of transformer is that instead of a linear sequence of static weights times values in each layer, you now have 3 different matrices that are obtained from the same input through multiplication of learned weights, and then you just multiply the matrices together. I.e more parallelism which works out nice, but very restrictive since the attention formula is static. We arent going to see more progres…

How can gradient descent work on compute graphs when the space of compute graphs is discrete?

I don't know if it can in the traditional sense of back propagation.

I think that Hebbian Learning is going to make a comeback at some point and time which will be used to connect static subgraphs to to other subgraphs subgraphs, which can be trained either separately or on the fly.

Re: Understand how transformers work by demystifying the math behind them

#126

Post by danielmarkbruce »

ActorNightly wrote: post #121
danielmarkbruce wrote: post #118

Earlier quoted context omitted.

An LLM isn't just recalling stuff. Brand new stuff, which it never saw in it's training, can come out. The minute you take a token and turn it into an embedding, then start changing the numbers in that embedding based on other embeddings and learned weights, you are playing around with concepts. As for executing a decision tree, ReAct or Tree of Thought or Graph of Thought is doing that. It might not be doing it as w…

>Brand new stuff, which it never saw in it's training, can come out. Sort of. You can get LLMs to produce some new things, but these are statistical averages of existing information. Its kinda like a static "knowledge tree", where it can do some interpolation, but even then, its interpolation based on statistically occurring text.

The interpolation isn't really based on statistically occurring text. It's based on statistically occurring concepts. A single token can have many meanings depending on context and many tokens can represent a concept depending on context. A (good) LLM is capturing that.

Re: Understand how transformers work by demystifying the math behind them

#127

Post by knightoffaith »

ActorNightly wrote: post #123
knightoffaith wrote: post #120

Earlier quoted context omitted.

Your expectations are pretty high. Differentiable architecture search as you mentioned in the original comment is one thing; going beyond empirical risk minimization-based learning is another thing entirely. In fact, they seem mostly orthogonal. That aside, it seems like AI has had the most empirical success by not imposing hard constraints/structure, but letting models learn completely "organically". The computation…

They expectations high, but its not so much as orthogonal as more basic. Our brains work on add/multiply/activation this is well known. But the composition of the neural connection strengths in our brain that makes us us is definitely not trained on any sort of final loss. Or at least not completely.

I'm not sure that AI has been successful recently because of its similarities to the human brain. It seems like the project of making human-like AI (in the sense of, models that function similarly to the brain) have had a lot less empirical success than the project of trying minimize loss on a dataset, whatever that takes. Like, look what happened to Hebbian learning, as you mentioned in your other comment. Completely absent from models that are seriously trying to beat SOTA on benchmarks.

Like, it really just seems like LLMs are a really good way of doing statistics rather than the closest model we have of the brain/mind, even if there are some connections we can draw post-hoc between transformers and the human brain.

Re: Understand how transformers work by demystifying the math behind them

#128

Post by bagful »

danielmarkbruce wrote: post #126
ActorNightly wrote: post #121

Earlier quoted context omitted.

>Brand new stuff, which it never saw in it's training, can come out. Sort of. You can get LLMs to produce some new things, but these are statistical averages of existing information. Its kinda like a static "knowledge tree", where it can do some interpolation, but even then, its interpolation based on statistically occurring text.

The interpolation isn't really based on statistically occurring text. It's based on statistically occurring concepts. A single token can have many meanings depending on context and many tokens can represent a concept depending on context. A (good) LLM is capturing that.

Neither just text or just concepts, but text-concepts — LLMs can only manipulate concepts as they can be conveyed via text. But I think wordlessly, in pure concepts and sense-images, and serialize my thoughts to text. That I have thoughts that I am incapable of verbalizing is what makes me different from an LLM - and, I would argue, actually capable of conceptual synthesis. I have been told some people think “in words” though.

Re: Understand how transformers work by demystifying the math behind them

#129

Post by danielmarkbruce »

bagful wrote: post #128
danielmarkbruce wrote: post #126

Earlier quoted context omitted.

The interpolation isn't really based on statistically occurring text. It's based on statistically occurring concepts. A single token can have many meanings depending on context and many tokens can represent a concept depending on context. A (good) LLM is capturing that.

Neither just text or just concepts, but text-concepts — LLMs can only manipulate concepts as they can be conveyed via text. But I think wordlessly, in pure concepts and sense-images, and serialize my thoughts to text. That I have thoughts that I am incapable of verbalizing is what makes me different from an LLM - and, I would argue, actually capable of conceptual synthesis. I have been told some people think “in word…

Nope, you could shove in an embedding that didn't represent an existing token. It would work just fine.

(if not obvious.. you'd shove it in right after the embedding layer...)

Re: Understand how transformers work by demystifying the math behind them

#130

Post by pmayrgundter »

pmayrgundter wrote: post #109
kmmlng wrote: post #105

Earlier quoted context omitted.

What do you mean, the development of the "Perception"? Do you mean the Perceptron? In that case, Backprop was invented way later than the Perceptron (see https://people.idsia.ch/~juergen/who-invented-backpropagatio... ). I don't see any information in your linked Wikipedia article that supports a bio-inspired origin. In fact, researchers have been wondering whether an equivalent to Backprop might be found in biologic…

Ah yes, Perceptron. Had a couple typos.. sorry, was on phone. The bio-inspiration was via Frank Rosenblatt, who is referred to in that article tho yeah, the history is over in his article: https://en.wikipedia.org/wiki/Frank_Rosenblatt#Perceptron "Rosenblatt was best known for the Perceptron, an electronic device which was constructed in accordance with biological principles and showed an ability to learn. He develop…

Got me reading the original. It's rad.

Link to PDF and some screens from intro here..

https://twitter.com/PMayrgundter/status/1743096776456867921

Post reply on HN