Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TurboQuant: Redefining AI efficiency with extreme compression

research.google

Post reply on HN
Print view View original

151–160 of 202 posts

Re: TurboQuant: Redefining AI efficiency with extreme compression

#151

Post by sta1n »

veunes wrote: post #32
nvme0n1p1 wrote: post #23

Earlier quoted context omitted.

There are tells all over the page: > Redefining AI efficiency with extreme compression "Redefine" is a favorite word of AI. Honestly no need to read further. > the key-value cache, a high-speed "digital cheat sheet" that stores frequently used information under simple labels No competent engineer would describe a cache as a "cheat sheet". Cheat sheets are static, but caches dynamically update during execution. Studen…

Looks like Google canned all their tech writers just to pivot the budget into H100s for training these very same writers

[dead]

Re: TurboQuant: Redefining AI efficiency with extreme compression

#153

Post by sta1n »

GistNoesis wrote: post #102
akhenakh wrote: post #65

Someone implementing it on llamacpp already https://github.com/mudler/llama.cpp/commit/dee102db1bfd723c9...

He even attempts to improve on the paper by replacing the random rotation operation which is O(d^2), by a Subsampled Randomized Hadamard Transform which can be computed in O(d*log d). Hopefully Johnson–Lindenstrauss lemma applies in the same way for SRHTransformed vectors as they do for randomly rotated vectors and the independence of the distribution laws of the coordinates remains and therefore the quantization of…

[dead]

Re: TurboQuant: Redefining AI efficiency with extreme compression

#155

Post by fc417fc802 »

Geee wrote: post #139
Maxious wrote: post #4

Earlier quoted context omitted.

https://mesuvash.github.io/blog/2026/turboquant-interactive/ has a little visualisation

Is there an error in the visualization? It shows that every vector is rotated the same amount. My understanding was that they are randomized with different values, which results in a predictable distribution, which is easier to quantize.

I believe they are all rotated by the same random matrix, the purpose being (IIUC) to distribute the signal evenly across all dimensions. So effectively it drowns any structure that might be present in noise. That's essential for data efficiency in addition to avoiding bias related issues during the initial quantization step. However there are still some other issues due to bias that are addressed by a second quantization step involving the residual.

That said, I don't believe the visualization is correct. The grid for one doesn't seem to match what's described in the paper.

Also it's entirely possible I've misunderstood or neglected to notice key details.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#156

Post by mesuvash »

pstoll wrote: post #53
Maxious wrote: post #4

Earlier quoted context omitted.

https://mesuvash.github.io/blog/2026/turboquant-interactive/ has a little visualisation

Good post but link at the end is broken. “”” For the full technical explanation with equations, proofs, and PyTorch pseudocode, see the companion post: TurboQuant: Near-Optimal Vector Quantization Without Looking at Your Data.“

Author here. Sorry still working on refining the post. Will share once the post is ready.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#157

Post by mesuvash »

spencerflem wrote: post #9
Maxious wrote: post #4

Earlier quoted context omitted.

https://mesuvash.github.io/blog/2026/turboquant-interactive/ has a little visualisation

I like the visualization, but I don’t understand the grid quantization. If every point is on the unit circle aren’t all the center grid cords unused?

Yes. Great catch. I simplified the grid just for visualization purpose.

I've updated the visualization. The grid is actually not uniformly spaced. Each coordinate is quantized independently using optimal centroids for the known coordinate distribution. In 2D, unit-circle coordinates follow the arcsine distribution (concentrating near ±1), so the centroids cluster at the edges, not the center.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#158

Post by mesuvash »

Geee wrote: post #139
Maxious wrote: post #4

Earlier quoted context omitted.

https://mesuvash.github.io/blog/2026/turboquant-interactive/ has a little visualisation

Is there an error in the visualization? It shows that every vector is rotated the same amount. My understanding was that they are randomized with different values, which results in a predictable distribution, which is easier to quantize.

That's actually correct and intentional. TurboQuant applies the same rotation matrix to every vector. The key insight is that any unit vector, when multiplied by a random orthogonal matrix, produces coordinates with a known distribution (Beta/arcsine in 2D, near-Gaussian in high-d). The randomness is in the matrix itself (generated once from a seed), not per-vector. Since the distribution is the same regardless of the input vector, a single precomputed quantization grid works for everything. I've updated the description to make this clearer.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#159

Post by davesque »

parsimo2010 wrote: post #142

This blog post sucks. It does not make me want to read the papers. Look at this figure: https://storage.googleapis.com/gweb-research2023-media/image... The speedup labels on the vertical axis are 0, 2, 2, 4, 6, 8... Why is 2 repeated? Did they just have nano-banana make them some charts? Can they not be bothered to use matplotlib or bokeh and directly render a graph? I don't know, maybe there is some legitimate reaso…

Yeah, the viz for polar quantization is straight up nonsensical. Okay, so some colors are converted into clocks and then into a bigger box with a pink box inside of it. Got it. Even understanding what polar coordinates are doesn't help you make sense out of it.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#160

Post by jmalicki »

CyberDildonics wrote: post #120
jmalicki wrote: post #75

Earlier quoted context omitted.

If they didn't cite your paper that's bullshit. But if they read your paper enough that they invited you to a talk, that probably means they were far enough along to independently inventing it they were going to do so anyway, and wanted to chat with someone who was also doing the thing they were already doing. Good ideas tend to reveal themselves to anyone who is aware of the problem.

That's rationalizing like crazy. If they knew about it they should have cited it.

That's what I'm saying - not citing is total bullshit.

But if they invited a talk, and published a paper and cited it, it might be a little off, but not horrible.

Post reply on HN