Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understand how transformers work by demystifying the math behind them

osanseviero.github.io

Post reply on HN
Print view View original

101–110 of 139 posts

Re: Understand how transformers work by demystifying the math behind them

#101

Post by pmayrgundter »

kmmlng wrote: post #88
pmayrgundter wrote: post #71

Earlier quoted context omitted.

This is all true in a neutral net, but Transformers aren't Neural Nets in the traditional sense. I was under that impression originally, but there's not a back propagation or Hebbian learning here, which were the key bits of biomimicry that earned classic NNs their name. Transformers do have coefficients that are fit, but that's more broad.. could be used for any sort of regression or optimization, and not necessaril…

> This is all true in a neutral net, but Transformers aren't Neural Nets in the traditional sense. I was under that impression originally, but there's not a back propagation or Hebbian learning here, which were the key bits of biomimicry that earned classic NNs their name. Hebbian learning has never been used with much success in training neural nets. Backpropagation is not bio-inspired, but backpropagation is certai…

Agreed Hebbian learning isn't used.. just meant it as an example of what would signal a NN.

For Backprop, I'm basing this off the development of the Perception. Wiki supports this and its bio-inslired origin[1].

As for its use in Transformers, if you mean simple regressing of errors or use of gradient descent, I'd agree, but that's not usually called Backprop and the term isn't used in the original paper. The term typically means back propagating the errors thru the entire network at a certain stage of learning, and that's not present in Transformers that I can tell.

Happy to see any support for your claims tho.

https://en.m.wikipedia.org/wiki/Backpropagation

Re: Understand how transformers work by demystifying the math behind them

#102

Post by osanseviero »

godelski wrote: post #38

I love and hate these. I love them because they do give another resource at explaining models such as transformers and I think this one is pretty well done (note: you really need to do something about the equation in 4.2...) First, the critique is coming from love. Great work, so I don't want it to be taken as I'm saying anything it isn't. Why I hate these is that they are labeled as "math behind" but I think this is…

Hey @godelski! Author of the blog post here.

I really appreciate you taking the time to provide all this feedback. This feedback + additional resources are extremely useful.

I agree that the subtitle is not as accurate as it could be. I'll revisit it! As for content updates, I've been doing some additional updates in the last days based on feedback (e.g. more info about tokenization and the token embeddings). Although diving in some of your suggestions is likely out of scope for this article, I in particular agree that expanding the attention mechanism content (e.g. the analogy with databases or explaining what is dot product) would increase the quality of the article. I will look into expanding this!

I also think a more rigorous, separate mathematical exploration into attention mechanisms and recent advancements would be a great tool for the ecosystem.

Once again, thank you for all the amazing feedback!

Re: Understand how transformers work by demystifying the math behind them

#103

Post by pmayrgundter »

theonlybutlet wrote: post #82
pmayrgundter wrote: post #71

Earlier quoted context omitted.

This is all true in a neutral net, but Transformers aren't Neural Nets in the traditional sense. I was under that impression originally, but there's not a back propagation or Hebbian learning here, which were the key bits of biomimicry that earned classic NNs their name. Transformers do have coefficients that are fit, but that's more broad.. could be used for any sort of regression or optimization, and not necessaril…

Thanks for your reply, you raise a very good point, transformer models are a lot more complex. I'd argue conceptually they're the same, just the data and process is more abstracted. Autoencoded data implies using efficient representations, basically semantically abstracted data and opting for measures like back propagation through time.

So like in my sister reply, I don't see the Backprop, but maybe I'm missing it. This article does use the word, but in a generic way

"For example, when doing the backpropagation (the technique through which the models learn), the gradients can become too large"

But I think this is more of a borrowing and it's not used again in description and may just be a misconception. There's no use of the Backprop term in the original paper nor any stage of learning where output errors are run thru the whole network in a deep regression.

What I do see in Transformers is localized uses of gradient descent, and Backprop in NNs also uses GD...but that seems the extent of it.

Is there a deep regression? Maybe I'm missing it

Re: Understand how transformers work by demystifying the math behind them

#104

Post by api »

mdp2021 wrote: post #81
api wrote: post #61

Earlier quoted context omitted.

Genetic algorithms figured out GI the first time, but it took a while.

Could you please expand?

Evolution built our brains.

Though to be fair, actual biological evolution is more complex than simple genetic algorithms. More like evolution strategies with meta-parameter-learning and adaptive rate tuning among other things.

Re: Understand how transformers work by demystifying the math behind them

#105

Post by kmmlng »

pmayrgundter wrote: post #101
kmmlng wrote: post #88

Earlier quoted context omitted.

> This is all true in a neutral net, but Transformers aren't Neural Nets in the traditional sense. I was under that impression originally, but there's not a back propagation or Hebbian learning here, which were the key bits of biomimicry that earned classic NNs their name. Hebbian learning has never been used with much success in training neural nets. Backpropagation is not bio-inspired, but backpropagation is certai…

Agreed Hebbian learning isn't used.. just meant it as an example of what would signal a NN. For Backprop, I'm basing this off the development of the Perception. Wiki supports this and its bio-inslired origin[1]. As for its use in Transformers, if you mean simple regressing of errors or use of gradient descent, I'd agree, but that's not usually called Backprop and the term isn't used in the original paper. The term ty…

What do you mean, the development of the "Perception"? Do you mean the Perceptron? In that case, Backprop was invented way later than the Perceptron (see https://people.idsia.ch/~juergen/who-invented-backpropagatio...).

I don't see any information in your linked Wikipedia article that supports a bio-inspired origin. In fact, researchers have been wondering whether an equivalent to Backprop might be found in biological brains, but Backprop is widely believed to be biologically implausible (see e.g. https://arxiv.org/pdf/1502.04156.pdf, https://www.sciencedirect.com/science/article/pii/S089360801...).

It's not surprising that the term Backprop is not mentioned in the original paper, it isn't mentioned in most neural network research, because it's simply the default method to optimize weights and additionally it's hidden away by modern autodiff frameworks, so no one actually has to give it any thought. But backprop is definitely used in transformers (see e.g. https://aclanthology.org/2020.emnlp-main.463.pdf, https://arxiv.org/pdf/2004.08249, https://proceedings.mlr.press/v202/phang23a/phang23a.pdf, https://dinkofranceschi.com/docs/bft.pdf)

Re: Understand how transformers work by demystifying the math behind them

#106

Post by stormfather »

heisenburgzero wrote: post #67
treyd wrote: post #64

Earlier quoted context omitted.

If I had to guess, single characters are able to be encoded as tokens, but there's more "bandwidth" in the model being dedicated to handling them and there's less semantic meaning encoded in them "natively" compared to tokens for concrete words. If it decides to, it can recreate unknown sequences by copying over the tokens for the single letters or create them if it makes sense.

I think some earlier NLP applications have something called "Unknown token", which they will replace any unseen word. But for recent implementations, I don't think they are being used anymore. It still baffles me why such stochastic parrot / next token predictor, will recognize these "Unseen combinations of tokens" and reuse them in response.

This helped me understand but not well enough to explain it yet: https://transformer-circuits.pub/2022/in-context-learning-an...

Re: Understand how transformers work by demystifying the math behind them

#107

Post by lambdatronics »

ActorNightly wrote: post #6

The whole "mystery" of transformer is that instead of a linear sequence of static weights times values in each layer, you now have 3 different matrices that are obtained from the same input through multiplication of learned weights, and then you just multiply the matrices together. I.e more parallelism which works out nice, but very restrictive since the attention formula is static. We arent going to see more progres…

>a way to generalize the compute graph as a learnable parameter.

Agreed. Seems analogous with how human mental processes are used to solve the kind of problems we'd like LLMs to solve (going beyond "language processing" which transformers do well, to actual reasoning which they can only mimic). Although you risk it becoming a Turing machine by giving it flow control & then training is a problem as you say. Perhaps not intractable though.

Re: Understand how transformers work by demystifying the math behind them

#108

Post by knightoffaith »

blackbear_ wrote: post #92
knightoffaith wrote: post #47

Earlier quoted context omitted.

How can gradient descent work on compute graphs when the space of compute graphs is discrete?

Perhaps in a way similar to this paper: https://arxiv.org/abs/1806.09055

I wonder why this hasn't taken off.

Re: Understand how transformers work by demystifying the math behind them

#109

Post by pmayrgundter »

kmmlng wrote: post #105
pmayrgundter wrote: post #101

Earlier quoted context omitted.

Agreed Hebbian learning isn't used.. just meant it as an example of what would signal a NN. For Backprop, I'm basing this off the development of the Perception. Wiki supports this and its bio-inslired origin[1]. As for its use in Transformers, if you mean simple regressing of errors or use of gradient descent, I'd agree, but that's not usually called Backprop and the term isn't used in the original paper. The term ty…

What do you mean, the development of the "Perception"? Do you mean the Perceptron? In that case, Backprop was invented way later than the Perceptron (see https://people.idsia.ch/~juergen/who-invented-backpropagatio... ). I don't see any information in your linked Wikipedia article that supports a bio-inspired origin. In fact, researchers have been wondering whether an equivalent to Backprop might be found in biologic…

Ah yes, Perceptron. Had a couple typos.. sorry, was on phone.

The bio-inspiration was via Frank Rosenblatt, who is referred to in that article tho yeah, the history is over in his article:

https://en.wikipedia.org/wiki/Frank_Rosenblatt#Perceptron

"Rosenblatt was best known for the Perceptron, an electronic device which was constructed in accordance with biological principles and showed an ability to learn.

He developed and extended this approach in numerous papers and a book called Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms, published by Spartan Books in 1962.[6] He received international recognition for the Perceptron.

The Mark I Perceptron, which is generally recognized as a forerunner to artificial intelligence, currently resides in the Smithsonian Institution in Washington D.C."

Your Juergen page is interesting, tho no direct comment on Rosenblatt there. He does cite the work on this page:

https://people.idsia.ch/~juergen/deep-learning-overview.html (refs R58, R61)

My reading is that a long-known idea, about multi-variate regression, was reinterpreted by Rosenblatt by 1958 via the bio-inspired Perceptron, and then that was criticized by Minksy and others and viable methods were achieved by 1965. When I was taught NNs by Mitchell at CMU in the 1990s (lectures similar to his book Machine Learning), this was the same basic story. Also reminds me of a moment in class one day when a Stats Prof who was surveying the course broke out with "but wait, isn't this all just multivariate regression??" :) Mitchell agreed to the functional similarity, but I think that helps highlight how the biomimicry was crucial to developing the idea. it had laid hidden in plain sight for a century.

Agreed, and I was aware, there has since been criticism of the biological plausibility of backprop.

Your further links with refs to backprop in transformers are interesting; I hadn't seen these. It's clear the term is being used like you say, tho I still see ambiguity of it utility here. Autodifferentiation, gradient descent, multi-variate regerssion etc. are ofc in common use and scanning these papers it's not clear to me the terms aren't simply to a point of conflation. What had stood unique for me with backprop was a coherent whole-network regression. This to me looks like a piecewise approach.

But anyways, I see your point. Thanks!

Re: Understand how transformers work by demystifying the math behind them

#110

Post by mrguyorama »

nethi wrote: post #73
uoaei wrote: post #68

Earlier quoted context omitted.

A ton of progress can be made climbing a tree, but if your goal is reaching the moon it becomes clear pretty quickly that climbing taller trees will never get you there.

True, but it is the process of climbing trees that gives the insight whether taller trees help or not and if not, what to do next.

Not true. Climbing trees for millions of years taught us nothing about orbits, or rockets, or literally incomprehensible to human distances, or the vacuum of space, or any possible way to get higher than a tree.

We eventually moved on to lighter than air flight, which once again did not teach us any of those things and also was a dead end from the "get to the sky/moon" perspective, so then we invented heavier than air flight, which once again could not teach us about orbits, rockets, distances, or the vacuum of space.

What got us to the moon was rigorous analysis of reality with math to discover Newton's laws of motion, from which you can derive rockets, orbits, the insane scale of space, etc. No amount of further progress in planes, airships, kites, birds, anything on earth would ever have taught us the techniques to get to the moon. We had to analyze the form and nature of reality itself and derive an internally consistent model of that physical reality in order to understand anything about doing space.

Post reply on HN