Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TurboQuant: Redefining AI efficiency with extreme compression

research.google

Post reply on HN
Print view View original

181–190 of 202 posts

Re: TurboQuant: Redefining AI efficiency with extreme compression

#181

Post by mesuvash »

fc417fc802 wrote: post #178
mesuvash wrote: post #168

Earlier quoted context omitted.

Fair point. I've updated the animation to address this. The grid now uses the correct non-uniform centroids (optimal for the arcsine distribution in 2D), so you'll see grid lines cluster near the edges where unit-circle coordinates actually concentrate, rather than being evenly spaced. The spacing does change with bit depth. On the second quantization step: the paper's inner-product variant uses (b-1) bits for the MS…

It looks nice! Fair enough about QJL - it seems to be nothing more than an unbiasing measure anyway. I'm not sure if it's my own misunderstanding or if the paper [0] has something of an error. Section 3.1 starts out to the effect "let x be on the unit hypersphere" (but I'm fairly certain it's actually not). Neither algorithm 1 nor algorithm 2 show a normalization step prior to rotating x. Algorithm 2 line 8 shows tha…

IIUC, The paper's notation S^(d-1) means the unit sphere in R^d (e.g., the familiar unit circle is S^1 living in R^2). So, i think, x in the algorithm is already a unit vector.

Reference: Section 2:Preliminaries ... We use the notation S^d−1 to denote the hypersphere in R^d of radius 1.

Section 3.1 Let x ∈ S^d−1 be a (worst-case) vector on the unit sphere in dimension d.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#183

Post by prabal97 »

ssijak wrote: post #51

For my grug brain can somebody translate this to ELIgrug terms? Does this mean I would be able to run 500b model on my 48gb macbook without loosing quality?

I wrote this more intuitive explanation. I think you might find it helpful!

https://prabal.ca/posts/google-long-context-cheaper/

Re: TurboQuant: Redefining AI efficiency with extreme compression

#184

Post by prabal97 »

maurelius2 wrote: post #14

I'm somewhat at a loss here other than understanding the fundamentals. Can someone tell me how the compression impact performance?

Reposting it here ... I wrote this more intuitive explanation. I think you might find it helpful too!

https://prabal.ca/posts/google-long-context-cheaper/

Re: TurboQuant: Redefining AI efficiency with extreme compression

#185

Post by fc417fc802 »

mesuvash wrote: post #181
fc417fc802 wrote: post #178

Earlier quoted context omitted.

It looks nice! Fair enough about QJL - it seems to be nothing more than an unbiasing measure anyway. I'm not sure if it's my own misunderstanding or if the paper [0] has something of an error. Section 3.1 starts out to the effect "let x be on the unit hypersphere" (but I'm fairly certain it's actually not). Neither algorithm 1 nor algorithm 2 show a normalization step prior to rotating x. Algorithm 2 line 8 shows tha…

IIUC, The paper's notation S^(d-1) means the unit sphere in R^d (e.g., the familiar unit circle is S^1 living in R^2). So, i think, x in the algorithm is already a unit vector. Reference: Section 2:Preliminaries ... We use the notation S^d−1 to denote the hypersphere in R^d of radius 1. Section 3.1 Let x ∈ S^d−1 be a (worst-case) vector on the unit sphere in dimension d.

Right but in reality IIUC w ∈ R^d and it's x = w / ||w|| ∈ S^(d-1) and then given r = x - Qmse^-1( Qmse( x ) ) the scalar you use is derived as ||r|| (I'm missing a couple subscript twos there I think).

I was primarily aiming to confirm my understanding given the author's omission but also the scalar is subtly different than in your linked explanation (although conceptually equivalent).

Re: TurboQuant: Redefining AI efficiency with extreme compression

#188

Post by alkenrinnstet »

This article is AI-generated slop.

> This clever step simplifies the data's geometry

No self-respecting researcher talks about their work in this way. But it is characteristic of these chatbots' tendency to over-use superlatives and sycophantic language.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#189

Post by alkenrinnstet »

parsimo2010 wrote: post #142

This blog post sucks. It does not make me want to read the papers. Look at this figure: https://storage.googleapis.com/gweb-research2023-media/image... The speedup labels on the vertical axis are 0, 2, 2, 4, 6, 8... Why is 2 repeated? Did they just have nano-banana make them some charts? Can they not be bothered to use matplotlib or bokeh and directly render a graph? I don't know, maybe there is some legitimate reaso…

It's slop. The text is also clearly generated by a chatbot with its nonsensical comparisons and bizarrely superlative language.

Re: TurboQuant: Redefining AI efficiency with extreme compression

#190

Post by thrtythreeforty »

photon_lines wrote: post #100
gavinray wrote: post #80

Can someone ELI5 these two concepts please, which make no sense to me: > "TurboQuant starts by randomly rotating the data vectors. This clever step simplifies the data's geometry" I don't understand how taking a series of data and applying a random rotation could mathemetically lead every time to "simpler" geometry. If I throw a bunch of shapes on the ground, tightly packed and touching each other, then rotate all of…

The whole goal of quantisation is to put the data into 'bins' so that it can easily be 'packed' so that you can represent it using less bits (less information). You can think of it like rounding essentially (3.14159 -> 3). Now, sometimes within data, the distribution will be non-ideal for separating it out into bins (let's say that our rounding rules are simple -- we simply use a floor function so 2.45 maps to 2 and…

This is a fantastic explanation. Thank you. The only part I am not following is how it is guaranteed that 1 bit is sufficient for the error value. Is this something the Lloyd-Max algorithm is responsible for ensuring? (Seems to me that if your quantization algorithm is crappy enough, you could need a large number of bits to store the error.)
Post reply on HN