Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TurboQuant: A first-principles walkthrough

arkaung.github.io

Post reply on HN
Print view View original

41–50 of 78 posts

Re: TurboQuant: A first-principles walkthrough

#41

Post by moffkalast »

adrian17 wrote: post #28
amitport wrote: post #6

TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555 . We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federa…

I wonder how often this happens in practice - by "this", I mean industry/LLM world not noticing* some research until a bigger player repeats it with louder PR. (*hopefully I didn't misunderstand the situation)

If we go only by the cases that have been publicly known it already happens all the time. Lots of patents are a race to register by multiple parties too and it's rarely done fairly.

Re: TurboQuant: A first-principles walkthrough

#42

Post by sva_ »

adrian17 wrote: post #28
amitport wrote: post #6

TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555 . We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federa…

I wonder how often this happens in practice - by "this", I mean industry/LLM world not noticing* some research until a bigger player repeats it with louder PR. (*hopefully I didn't misunderstand the situation)

Ask Jürgen Schmidhuber

Re: TurboQuant: A first-principles walkthrough

#43

Post by 0xA2kag »

scaillib wrote: post #33
0xA2kag wrote: post #20

Earlier quoted context omitted.

Thanks a lot <3

what are the tools you used (if any)?

It is just a bunch of ideas on how to make things intuitive and a bunch of hand holding Claude Code to get exactly what I want. Underlying is plain HTML CSS and JS.

Re: TurboQuant: A first-principles walkthrough

#44

Post by om8 »

amitport wrote: post #6

TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555 . We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federa…

https://docs.vllm.ai/en/v0.20.0/api/vllm/model_executor/laye...

`vllm.model_executor.layers.quantization.turboquant`

> The technique implemented here consists of the scalar case of the HIGGS quantization method (Malinovskii et al., "Pushing the Limits of Large Language Model Quantization via the Linearity Theorem", NAACL 2025; preprint arXiv:2411.17525): rotation + optimized grid + optional re-normalization, applied to KV cache compression. A first application of this approach to KV-cache compression is in "Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models" (Shutova et al., ICML 2025; preprint arXiv:2501.19392). Both these references pre-date the TurboQuant paper (Zandieh et al., ICLR 2026).

Re: TurboQuant: A first-principles walkthrough

#45

Post by gajjanag »

amitport wrote: post #6

TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555 . We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federa…

There are also more papers on similar themes.

For example, TurboQuant makes use of QJL (quantized Johnson Lindenstrauss transformations). One of the first papers to characterize the QJL and in fact the rate distortion tradeoff for quantized matrix multiplication in general is "Optimal Quantization for Matrix Multiplication" (https://arxiv.org/abs/2410.13780) by Ordentlich and Polyanskiy.

There is also a more accessible survey paper around quantized matrix multiplication called "High-Rate Quantized Matrix Multiplication: Theory and Practice" (https://arxiv.org/abs/2601.17187), by the same authors.

TurboQuant cites none of them.

Re: TurboQuant: A first-principles walkthrough

#46

Post by gcr »

On TheTom’s llama-cpp fork, TurboQuant makes inference about five to ten times slower than vanilla (M1 Max, qwen3.6-35b-a3b). Seems like the productionization is still a ways away. Excited to see what we can get it down to though.

Re: TurboQuant: A first-principles walkthrough

#47

Post by [deleted] »

om8 wrote: post #44
amitport wrote: post #6

TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555 . We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federa…

https://docs.vllm.ai/en/v0.20.0/api/vllm/model_executor/laye... `vllm.model_executor.layers.quantization.turboquant` > The technique implemented here consists of the scalar case of the HIGGS quantization method (Malinovskii et al., "Pushing the Limits of Large Language Model Quantization via the Linearity Theorem", NAACL 2025; preprint arXiv:2411.17525): rotation + optimized grid + optional re-normalization, applied…

[deleted]

Re: TurboQuant: A first-principles walkthrough

#48

Post by gcr »

KnuthIsGod wrote: post #9
amitport wrote: post #6

TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555 . We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federa…

https://arxiv.org/abs/2604.18555 "This note clarifies the relationship between the recent TurboQuant work and the earlier DRIVE (NeurIPS 2021) and EDEN (ICML 2022) schemes. DRIVE is a 1-bit quantizer that EDEN extended to any bits per coordinate; we refer to them collectively as EDEN. First, TurboQuant is a special case of EDEN obtained by fixing EDEN's scalar scale parameter to . EDEN supports both biased and unbias…

FYI your comment is missing several constants/words and is hard to read

Re: TurboQuant: A first-principles walkthrough

#49

Post by amitport »

gajjanag wrote: post #45
amitport wrote: post #6

TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555 . We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federa…

There are also more papers on similar themes. For example, TurboQuant makes use of QJL (quantized Johnson Lindenstrauss transformations). One of the first papers to characterize the QJL and in fact the rate distortion tradeoff for quantized matrix multiplication in general is "Optimal Quantization for Matrix Multiplication" ( https://arxiv.org/abs/2410.13780 ) by Ordentlich and Polyanskiy. There is also a more access…

I believe our claim at this point is more fundamental than just lack of citation.

The quantizer in TurboQuant is EDEN quantization (2021) applied to the KV-cache. It is neither a novel quantizer nor an improvement in quantization techniques.

In DRIVE/EDEN, we already introduced the version used in "TurboQuant"'s paper and suggested an optimal scale configurations which are better in both mse-minimizing and unbiased scenarios.

Re: TurboQuant: A first-principles walkthrough

#50

Post by kumarhn »

gajjanag wrote: post #45
amitport wrote: post #6

TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555 . We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federa…

There are also more papers on similar themes. For example, TurboQuant makes use of QJL (quantized Johnson Lindenstrauss transformations). One of the first papers to characterize the QJL and in fact the rate distortion tradeoff for quantized matrix multiplication in general is "Optimal Quantization for Matrix Multiplication" ( https://arxiv.org/abs/2410.13780 ) by Ordentlich and Polyanskiy. There is also a more access…

TurboQuant is starting to look like a case study in how to turn a fragile paper into a breakthrough story.

The attribution is thin, the “6x compression” headline is not clearly separated from prior KV-cache quantization baselines like KIVI, and the RaBitQ comparison is hard to take seriously: single-core CPU for the baseline, A100 GPU for TurboQuant. It is comparing apples-to-datacenter. Worse, there are also public OpenReview comments saying that even the reported accuracy results are not reproducible.

Hard to believe this is the standard for something being promoted as a breakthrough. If this came from a random startup blog, people would be much harsher about it.

Post reply on HN