Matrix Multiplication Inches Closer To Mythic Goal
quantamagazine.org
Matrix Multiplication Inches Closer To Mythic Goal
1–10 of 53 posts
Re: Matrix Multiplication Inches Closer To Mythic Goal
#2Fun article. The approach to n^2.
> The result, by Josh Alman, a postdoctoral researcher at Harvard University, and Virginia Vassilevska Williams of the Massachusetts Institute of Technology, shaves about one-hundred-thousandth off the exponent of the previous best mark. It’s typical of the recent painstaking gains in the field.
Re: Matrix Multiplication Inches Closer To Mythic Goal
#3...and in case anyone is wondering, no, this is still not a practical speedup. https://en.wikipedia.org/wiki/Galactic_algorithm#Matrix_mult...
Re: Matrix Multiplication Inches Closer To Mythic Goal
#4Which one of these algos does Intel/Nvidia/Google use in their AVX/Cuda/TPU implementations?
Re: Matrix Multiplication Inches Closer To Mythic Goal
#5Which one of these algos does Intel/Nvidia/Google use in their AVX/Cuda/TPU implementations?
Strassen's method is the only one practical on modern hardware
Re: Matrix Multiplication Inches Closer To Mythic Goal
#6Which one of these algos does Intel/Nvidia/Google use in their AVX/Cuda/TPU implementations?
Simple block partitioning, usually 4x4, with the full ordinary algorithm per block. This is the most numerically-stable and parallel.
https://developer.nvidia.com/blog/cutlass-linear-algebra-cud...
Re: Matrix Multiplication Inches Closer To Mythic Goal
#7Re: Matrix Multiplication Inches Closer To Mythic Goal
#8Earlier quoted context omitted.
Strassen's method is the only one practical on modern hardware
And only in fixed precision.
What makes you say that? I've had good speedups with Strassen multiplication in variable precision (or with floating-point, in case you meant fixed-point arithmetic).
Re: Matrix Multiplication Inches Closer To Mythic Goal
#9Earlier quoted context omitted.
And only in fixed precision.
What makes you say that? I've had good speedups with Strassen multiplication in variable precision (or with floating-point, in case you meant fixed-point arithmetic).
Strassen method is faster than native multiplication, but it is not stable: you will get much larger rounding errors when implementing it using floating point numbers, compared to native multipllication. And fine precision is required for a lot of algorithm implemented using matrix multiplication, such as matrix inversion or gradient descent, so this is often a problem.