Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TurboQuant: Redefining AI efficiency with extreme compression

research.google

Post reply on HN
Print view View original

131–140 of 202 posts

Re: TurboQuant: Redefining AI efficiency with extreme compression

#131

Post by davesque »

benob wrote: post #5

This is the worst lay-people explanation of an AI component I have seen in a long time. It doesn't even seem AI generated.

Yeah, and some parts of the article are just bizarre:

> Instead of looking at a memory vector using standard coordinates (i.e., X, Y, Z) that indicate the distance along each axis, PolarQuant converts the vector into polar coordinates using a Cartesian coordinate system. This is comparable to replacing "Go 3 blocks East, 4 blocks North" with "Go 5 blocks total at a 37-degree angle”

Why bother explaining this? Were they targeting the high school and middle school student reader base??

Re: TurboQuant: Redefining AI efficiency with extreme compression

#132

Post by irthomasthomas »

cubefox wrote: post #101
irthomasthomas wrote: post #68

Earlier quoted context omitted.

Efficiency gains can be used to make existing models more profitable, or to make new larger and more intelligent models.

Some yes, others no. Distillation and quantization can't be used to make new base models since they require a preexisting one.

it enables models larger than was previously possible.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#134

Post by cubefox »

irthomasthomas wrote: post #132
cubefox wrote: post #101

Earlier quoted context omitted.

Some yes, others no. Distillation and quantization can't be used to make new base models since they require a preexisting one.

it enables models larger than was previously possible.

No because the base model from which the distilled or quantized models are derived is larger.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#135

Post by gavinray »

photon_lines wrote: post #100
gavinray wrote: post #80

Can someone ELI5 these two concepts please, which make no sense to me: > "TurboQuant starts by randomly rotating the data vectors. This clever step simplifies the data's geometry" I don't understand how taking a series of data and applying a random rotation could mathemetically lead every time to "simpler" geometry. If I throw a bunch of shapes on the ground, tightly packed and touching each other, then rotate all of…

The whole goal of quantisation is to put the data into 'bins' so that it can easily be 'packed' so that you can represent it using less bits (less information). You can think of it like rounding essentially (3.14159 -> 3). Now, sometimes within data, the distribution will be non-ideal for separating it out into bins (let's say that our rounding rules are simple -- we simply use a floor function so 2.45 maps to 2 and…

I had to read this over a few times to piece it together, thanks for the thorough and digestable explanation!

Re: TurboQuant: Redefining AI efficiency with extreme compression

#136

Post by kingstnap »

Bolwin wrote: post #117
kingstnap wrote: post #112

Earlier quoted context omitted.

Other people have answered here but the real answer is that deep neural networks don't learn isotropic distributions of activations. What happens is that you get very spikey activations, there are so called "outlier" activations. A easy to read paper that tells you about this is SmoothQuant [0]. Another source from Anthropic and the Mechanistic Interperability people is calling these "privileged basis" [1]. Now based…

Do you know if this also applies to the muon optimizer? It seems to be replacing adamw

My guess is that probably not for Muon. What I said about ADAM was partly based on this blogpost I read some time ago, should have cited it as well [0].

The thing about Muon is that it doesn't have this specific feature of ADAM that causes it to "move along the diagonal". Basically if you flatten weights as a huge vector of a few billion elements. SGD moves along the gradient, which isn't biased. ADAM normalizes everything elementwise, so it sort of moves along a vector of +-1.

This isn't a proof or anything, but what you can imagine might be happening is that if you move along +-1, then you find spikey solutions somehow. Not sure how to prove that. Muon doesn't really do this, but it has its own sort of funky reshaping of the update (it moves along low rank directions).

[0] https://www.lesswrong.com/posts/yrhu6MeFddnGRSLtQ/adam-optim...

Re: TurboQuant: Redefining AI efficiency with extreme compression

#137

Post by gavinray »

kingstnap wrote: post #112
gavinray wrote: post #80

Can someone ELI5 these two concepts please, which make no sense to me: > "TurboQuant starts by randomly rotating the data vectors. This clever step simplifies the data's geometry" I don't understand how taking a series of data and applying a random rotation could mathemetically lead every time to "simpler" geometry. If I throw a bunch of shapes on the ground, tightly packed and touching each other, then rotate all of…

Other people have answered here but the real answer is that deep neural networks don't learn isotropic distributions of activations. What happens is that you get very spikey activations, there are so called "outlier" activations. A easy to read paper that tells you about this is SmoothQuant [0]. Another source from Anthropic and the Mechanistic Interperability people is calling these "privileged basis" [1]. Now based…

From your second paper:

  > In particular, we can generate fixed random rotation matrices at initialization, and multiply them into the activations any time we read from or write to the residual stream.
I guess I was mistaken in assuming this part was part of the TurboQuant-specific innovations. Still an interesting concept though

Re: TurboQuant: Redefining AI efficiency with extreme compression

#138

Post by Serhii-Set »

computerbuster wrote: post #124
Serhii-Set wrote: post #106

[flagged]

JPEG XL is mainly based on unique image-specific research, but you're right to say a lot of the techniques are compatible with videos in theory (the XYB color space comes to mind). AVIF is an AV1 OBU in an image-specific container, and required a lot of image-specific engineering to make AV1's tools useful for images; see libaom's tune "iq", and the same in SVT-AV1. The compression gains translated when engineering e…

[flagged]

Re: TurboQuant: Redefining AI efficiency with extreme compression

#139

Post by Geee »

Maxious wrote: post #4
bluequbit wrote: post #2

I did not understand what polarQuant is. Is is something like pattern based compression where the algorithm finds repeating patterns and creates an index of those common symbols or numbers?

https://mesuvash.github.io/blog/2026/turboquant-interactive/ has a little visualisation

Is there an error in the visualization? It shows that every vector is rotated the same amount. My understanding was that they are randomized with different values, which results in a predictable distribution, which is easier to quantize.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#140

Post by adi_kurian »

ramon156 wrote: post #110
ilija139 wrote: post #108

Earlier quoted context omitted.

It has a lot clearer explanation of the method than Google's own post.

Well, yeah. Claude simplified it. That doesn't mean it's a better explanation.

Did it lose important detail?
Post reply on HN