Live data from Hacker News

TurboQuant: A first-principles walkthrough

arkaung.github.io

71–78 of 78 posts

Re: TurboQuant: A first-principles walkthrough

#71
post #63

Earlier quoted context omitted.

But how can these poor googlers be expected to sift through the thousands of research papers published on these topics to find relevant citations? They don’t have time for such trivialities. They have far more important work to be doing not being evil. /s

Gemini helped them build it but didn’t / couldn’t attribute it from its corpus. I think we will see a surge of “rediscovery” that’s unattributed training surfacing of prior work that wasn’t widely recognized at the time.

Gemini is perfectly capable of searching the web. Pretty good at it really. As are most agents. If such a surge happens, it’s purely because of laziness.

Re: TurboQuant: A first-principles walkthrough

#72
post #71

Earlier quoted context omitted.

Gemini helped them build it but didn’t / couldn’t attribute it from its corpus. I think we will see a surge of “rediscovery” that’s unattributed training surfacing of prior work that wasn’t widely recognized at the time.

Gemini is perfectly capable of searching the web. Pretty good at it really. As are most agents. If such a surge happens, it’s purely because of laziness.

Laziness, aka, the human condition?

Re: TurboQuant: A first-principles walkthrough

#75

Earlier quoted context omitted.

hey what's your Venmo?

The B-52s, er I mean Base64s: VGhvdWdoIEkgYXBwcmVjaWF0ZSB0aGUgZ2VzdHVyZSBhbmQga2luZCBpbnRlbnRpb25zLCBJJ2QgcmF0aGVyIGxlYXJuIHRvIGZpc2ggKG9yIGNhdGNoIHRoZSBiaWcgYmFycmFjdWRhKHMpIHRoYXQgc3RvbGUgdGhlIHNjaG9vbCBvZiBmaXNoIEkgd2FzIGdpZnRlZCwgd2hpY2ggd291bGQgaGF2ZSBrZXB0IG1lIGZlZCBmb3IgbXVsdGlwbGUgbGlmZXRpbWVzLCBhbmQgc3Bvb2tlZCBhIGZldyBteXN0ZXJ5IGZyaWVuZHMgaW4gdGhlIHByb2Nlc3MtLS1hIHRhc2sgSSBhbSBjbG9zZSB0byBjb21wbGV0aW5nKSwgYW…

WWVhaCBtYW4sIEkgaGVhciB5YS4gSSByZXNwZWN0IHRoYXQgeW91IHdhbnQgdG8gc29sdmUgdGhlIHJlYWwgcHJvYmxlbSwgbm90IGp1c3QgZ2V0IHRocm91Z2ggdG9kYXkuCgpCdXQgZXZlbiBzb21lb25lIHdobyBrbm93cyBob3cgdG8gZmlzaCBzdGlsbCBuZWVkcyBtb25leSB0byBidXkgYSBwb2xlIGluIG9yZGVyIHRvIGZpc2guIExldCB0aGlzIGJlIHRoYXQuIE5vdCBhIGhhbmRvdXQsIGp1c3Qgc3VwcG9ydCBmb3IgdGhlIHBhcnQgeW91IGFyZSBhbHJlYWR5IGRvaW5nLgo=

Re: TurboQuant: A first-principles walkthrough

#76

Neat thank you for creating this! That said I think someone who cares about TurboQuant probably already has a bit of linear algebra knowledge. While the initial review section is definitely appreciated, I don't think it's going to help much since you need a decent level of "mathematical maturity" anyway. The "Coordinates of a random unit vector are all small" had me scratching my head a bit, and the language is a bit…

Thanks a lot for this feedback. I have updated the primer which misleads with "Coordinates of a random unit vector are all small" with an updated version

Re: TurboQuant: A first-principles walkthrough

#78
post #6

TurboQuant is a restricted version of EDEN quantization (NeurIPS 21, ICML 22). It lacks the optimal scale derivations, which makes the TurboQuant variant considerably less accurate than those works. We show this thoroughly in a new note at https://arxiv.org/abs/2604.18555 . We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federa…

I recently wrote a beginner-friendly explanation of this situation in TDS:

https://towardsdatascience.com/how-a-2021-quantization-algor...

Post reply on HN