Aren’t polar coordinates still n-1 + 1 for radius for n-dim vector? If so I understand that angles can be quantized better but when radius r is big the error is large for highly quantized angles right? What am I missing?
TurboQuant: Redefining AI efficiency with extreme compression
11–20 of 202 posts
Re: TurboQuant: Redefining AI efficiency with extreme compression
#12Aren’t polar coordinates still n-1 + 1 for radius for n-dim vector? If so I understand that angles can be quantized better but when radius r is big the error is large for highly quantized angles right? What am I missing?
r is a single value per vector. You don't have to quantize it, you can keep it and quantize the billion+ other coordinates of the vector.
Re: TurboQuant: Redefining AI efficiency with extreme compression
#13Earlier quoted context omitted.
https://mesuvash.github.io/blog/2026/turboquant-interactive/ has a little visualisation
I like the visualization, but I don’t understand the grid quantization. If every point is on the unit circle aren’t all the center grid cords unused?
i think grid can be a surface of the unit sphere
Re: TurboQuant: Redefining AI efficiency with extreme compression
#14I'm somewhat at a loss here other than understanding the fundamentals. Can someone tell me how the compression impact performance?
Re: TurboQuant: Redefining AI efficiency with extreme compression
#15This is a great development for KV cache compression. I did notice a missing citation in the related works regarding the core mathematical mechanism, though. The foundational technique of applying a geometric rotation prior to extreme quantization, specifically for managing the high-dimensional geometry and enabling proper bias correction, was introduced in our NeurIPS 2021 paper, "DRIVE" (https://proceedings.neurips.cc/paper/2021/hash/0397758f8990c...). We used this exact rotational approach and a similar bias correction mechanism to achieve optimal distributed mean estimation. I also presented this work and subsequent papers in a private invited talk at Google shortly after publication. Given the strong theoretical overlap with the mechanisms in TurboQuant and PolarQuant, I hope to see this prior art acknowledged in the upcoming camera-ready versions.
Re: TurboQuant: Redefining AI efficiency with extreme compression
#16[dead]
Re: TurboQuant: Redefining AI efficiency with extreme compression
#17This is the worst lay-people explanation of an AI component I have seen in a long time. It doesn't even seem AI generated.
It is AI generated. Or was written by someone a bit far from the technical advances IMHO. The Johnson-Lindenstrauss Lemma is a very specific and powerful concept, when in the article the QLJ explanation is vacuous. A knowledgeable human would not have left the reader wanting for how that relates to the Lemma.
Re: TurboQuant: Redefining AI efficiency with extreme compression
#18This is the worst lay-people explanation of an AI component I have seen in a long time. It doesn't even seem AI generated.
I think it is though- “ TurboQuant, QJL, and PolarQuant are more than just practical engineering solutions; they’re fundamental algorithmic contributions backed by strong theoretical proofs. These methods don't just work well in real-world applications; they are provably efficient and operate near theoretical lower bounds.”
I also instinctively reacted to that fragment, but at this point I think this is overreacting to a single expression. It's not just a normal thing to say in English, it's something people have been saying for a long time before LLMs existed.
Re: TurboQuant: Redefining AI efficiency with extreme compression
#19Aren’t polar coordinates still n-1 + 1 for radius for n-dim vector? If so I understand that angles can be quantized better but when radius r is big the error is large for highly quantized angles right? What am I missing?
r is a single value per vector. You don't have to quantize it, you can keep it and quantize the billion+ other coordinates of the vector.
What they're saying is that the error for a vector increases with r, which is true.
Trivially, with r=0, the error is 0, regardless of how heavily the direction is quantized. Larger r means larger absolute error in the reconstructed vector.
Re: TurboQuant: Redefining AI efficiency with extreme compression
#20I'm somewhat at a loss here other than understanding the fundamentals. Can someone tell me how the compression impact performance?
If in short, for many inference tasks the bottleneck is memory bandwidth. Suppose you have a machine with a memory bandwidth of 256 GB/s, and let's say you want to do inference for 4B model (model with 4 billion parameters). If you will load the model in BF16 format (16 bits), each forward pass (i.e. each token generated) will require roughly ~8 GB of memory bandwidth. So, 256/8 = 32 t/s, and that's the generation speed you will be strictly capped at even if your processing power is measured in exaFLOPS. But let's say now that you have decided to instead quantize the model and then run the quantized version. Suppose you have made a Q4_K_M version (4 bits + some weights will take more). Now each of your forward passes will take roughly 2-3 GB (rough approximations, reality is different) of memory bandwith (actually, it will be around 2 GB), and even in the worst case 256/3 = 85.3, while 256/2 = 128 t/s. Quants can reduce quality of the model and lower it's performance, but in most modern quantization methods those losses are usually negligible (although, of course, they're still present). So, as you can see, it can be concluded that quantization "widens" (it's not removing it fully) memory bottleneck while still preserving (not always though) acceptable quality.
(Sorry for my terrible English, it's not my native language)