21–30 of 197 posts
Sigh...learning about TPUs a decade ago made me invest heavily in $GOOG for the coming AI revolution...got that one 100% wrong. +400% over 10 years isn't bad but I can't help but feel shortchanged seeing nvidia/etc
Earlier quoted context omitted.
It's branding (see: TensorFlow); also, pretty much anything (linear) you would do with an arbitrarily ranked tensor can be expressed in terms of vector ops and matmuls
"Fixed-Function Matrix Accelerator" just doesn't have the same buzzy ring to it.
Earlier quoted context omitted.
Thanks for the background! I still don’t think it’s appropriate to call a batch of matrices a tensor.
You'd hate particle physics then. "Spin" and "action" and so on are terrible names, but scientists live with them, because convention. Convention dominates most of what we do. I'm not sure there's a good way around this. Most conventions suck, but they were established back before there was a clear idea of what the best long-term convention should be.
Earlier quoted context omitted.
If nothing else, the term "tensor" is shorter than "vectors and matrices," and then has the added benefit of representing n-dimensional arrays.
How is that an added benefit if the hardware doesn’t actually support n-dimensional arrays (other the n = 1 and 2)? And, strictly speaking, a vector can be considered a 1x n (or n x1) matrix, so Matrix Processing Unit would have been fine.
Earlier quoted context omitted.
I know what a tensor is mathematically. However, as far as I can see, ML isn’t based on tensor calculus as such.
Something similar happens on Wikipedia, where topics that use math inevitably get explained in the highest level math possible. It makes topics harder to understand than they need to be.
In the virtual compendium of Wikipedia, an extensive repository of human knowledge, there is a discernible proclivity for the hermeneutics of mathematically-infused topics to be articulated through the prism of esoteric and sophisticated mathematical constructs, often employing a panoply of arcane lexemes and syntactic structures of Greek and Latin etymology. This phenomenon, redolent of an academic periphrasis, tends to transmute the exegesis of such subjects into a crucible of abstruse and high-order mathematical discourse. Consequently, this modus operandi obfuscates the intrinsic didactic intent, thereby precipitating an epistemological chasm that challenges the layperson's erudition and obviates the pedagogical utility of the exposition.
Earlier quoted context omitted.
Tensor is from mathematics and was popularized over a century ago.
I know what a tensor is mathematically. However, as far as I can see, ML isn’t based on tensor calculus as such.
Earlier quoted context omitted.
If nothing else, the term "tensor" is shorter than "vectors and matrices," and then has the added benefit of representing n-dimensional arrays.
How is that an added benefit if the hardware doesn’t actually support n-dimensional arrays (other the n = 1 and 2)? And, strictly speaking, a vector can be considered a 1x n (or n x1) matrix, so Matrix Processing Unit would have been fine.
Sigh...learning about TPUs a decade ago made me invest heavily in $GOOG for the coming AI revolution...got that one 100% wrong. +400% over 10 years isn't bad but I can't help but feel shortchanged seeing nvidia/etc
+400% over 10 years isn't bad.
They got it working, then Jeff Dean did the math and the decided to do an ASIC.
Now of course Google should spin off the TPU team as a separate company. It's the only credible competition NVidia has, and the software support is second only to NVidia.
[1] https://open.spotify.com/episode/0V9kRgNS7Ds6zh3GjdXUAQ?si=q...
Earlier quoted context omitted.
I was confused as hell for a long time when I first got into ML, until I figured out how to think about tensors in a visual way. You're right: fundamentally ML is about vector and matrix operations (1D and 2D). So then why are most ML programs 3D, 4D, and in a transformer sometimes up to 6D (?!) One reasonable guess is that the third dimension is time. Actually not. It turns out that time is pretty rare in ML, and it…
Thanks for the background! I still don’t think it’s appropriate to call a batch of matrices a tensor.