Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The universal weight subspace hypothesis

arxiv.org

Post reply on HN
Print view View original

61–70 of 146 posts

Re: The universal weight subspace hypothesis

#61

Post by levocardia »

nothrowaways wrote: post #30

> Principal component analysis of 200 GPT2, 500 Vision Transformers, 50 LLaMA- 8B, and 8 Flan-T5 models reveals consistent sharp spectral decay - strong evidence that a small number of weight directions capture dominant variance despite vast differences in training data, objectives, and initialization. Isn't it obvious?

This general idea shows up all over the place though. If you do 3D scans on thousands of mammal skulls, you'll find that a few PCs account for the vast majority of the variance. If you do frequency domain analysis of various physiological signals...same thing. Ditto for many, many other natural phenomena in the world. Interesting (maybe not surprising?) to see it in artificial phenomena as well

Re: The universal weight subspace hypothesis

#62

Post by joquarky »

VikingCoder wrote: post #5

I find myself wanting genetic algorithms to be applied to try to develop and improve these structures... But I always want Genetic Algorithms to show up in any discussion about neural networks...

I got crazy obsessed with EvoLisa¹ back in the day and although there is nothing in common between that algorithm and those that make up training an LLM, I can't help but feel like they are similar.

¹ https://www.rogeralsing.com/2008/12/07/genetic-programming-e...

Re: The universal weight subspace hypothesis

#63

Post by moelf »

kacesensitive wrote: post #4

interesting.. this could make training much faster if there’s a universal low dimensional space that models naturally converge into, since you could initialize or constrain training inside that space instead of spending massive compute rediscovering it from scratch every time

>instead of spending massive compute rediscovering it from scratch every time

it's interesting that this paper was discovered by JHU, not some groups from OAI/Google/Apple, considering that the latter probably have spent 1000x more resource on "rediscovering"

Re: The universal weight subspace hypothesis

#64

Post by moelf »

scotty79 wrote: post #47
bigbuppo wrote: post #29

Earlier quoted context omitted.

Wouldn't this also mean that there's an inherent limit to that sort of model?

> Wouldn't this also mean that there's an inherent limit to that sort of model? If all need just 16 dimensions if we ever make one that needs 17 we know we are making progress instead of running in circles.

you can always make a new vector that's orthogonal to all the ones currently used and see if the inclusion improves performance on your tasks

Re: The universal weight subspace hypothesis

#65

Post by alyxya »

I’ve had a hard time parsing what exactly the paper is trying to explain. So far I’ve understood that their comparison seems to be models within the same family and same weight tensor dimensions, so they aren’t showing a common subspace when there isn’t a 1:1 match between weight tensors in a ViT and GPT2. The plots showing the distribution of principal component values presumably does this on every weight tensor, but this seems to be an expected result that the principal component values shows a decaying curve like a log curve where only a few principal components are the most meaningful.

What I don’t get is what is meant by a universal shared subspace, because there is some invariance regarding the specific values in weights and the directions of vectors in the model. For instance, if you were doing matrix multiplication with a weight tensor, you could swap two rows/columns (depending on the order of multiplication) and all that would do is swap two values in the resulting product, and whatever uses that output could undo the effects of the swap so the whole model has identical behavior, yet you’ve changed the direction of the principal components. There can’t be fully independently trained models that share the exact subspace directions for analogous weight tensors because of that.

Re: The universal weight subspace hypothesis

#66

Post by altairprime »

scotty79 wrote: post #51
altairprime wrote: post #19

For those trying to understand the most important parts of the paper, here's what I think is the most significant two statements, subquoted out of two (consecutive) paragraphs midway through the paper: > we selected five additional, previously unseen pretrained ViT models for which we had access to evaluation data. These models, considered out-of-domain relative to the initial set, had all their weights reconstructed…

16 seems like a suspiciously round number ... why not 17 or 13? ... is this just result of some bug in the code they used to do their science? or is it just that 16 was arbitrarily chosen by them as close enough to the actual minimal number of dimensions necessary?

There’s lots of hockey stick charts in the paper that might answer this visually, if that’s of interest.

Re: The universal weight subspace hypothesis

#67

Post by mapontosevenths »

CGMthrowaway wrote: post #2

They compressed the compression? Or identified an embedding that can "bootstrap" training with a headstart ? Not a technical person just trying to put it in other words.

To use an analogy: Imagine a spreadsheet with 500 smoothie recipes one in each row, each with a dozen ingredients as the columns.

Now imagine you discover that all 500 are really just the same 11 base ingredients plus something extra.

What they've done here is use SVD, (which is normally used for image compression and noise reduction), to find that "base recipe". Now we can reproduce those other recipes by only recording the one igredient that differs.

More interestingly it might tell us something new about smoothies in general to know that they all share a common base. Maybe we can even build a simpler base using this info.

At least in theory. The code hasn't actually been released yet.

https://toshi2k2.github.io/unisub/#key-insights

Re: The universal weight subspace hypothesis

#68

Post by westoncb »

farhanhubble wrote: post #41
westoncb wrote: post #31

Earlier quoted context omitted.

> So, they found an underlying commonality among the post-training structures in 50 LLaMA3-8B models, 177 GPT-2 models, and 8 Flan-T5 models; and, they demonstrated that the commonality could in every case be substituted for those in the original models with no loss of function; and noted that they seem to be the first to discover this. Could someone clarify what this means in practice? If there is a 'commonality' wh…

It might we worth it to use that subset to initialize the weights of future models but more importantly you could save a huge number of computational cycles by using the lower dimensional weights at the time of inference.

Ah interesting, I missed that possibility. Digging a little more though my understanding is that what's universal is a shared basis in weight space, and particular models of same architecture can express their specific weights via coefficients in a lower-dimensional subspace using that universal basis (so we get weight compression, simplified param search). But it also sounds like to what extent there will be gains during inference is in the air?

Key point being: the parameters might be picked off a lower dimensional manifold (in weight space), but this doesn't imply that lower-rank activation space operators will be found. So translation to inference-time isn't clear.

Re: The universal weight subspace hypothesis

#69

Post by tsurba »

kacesensitive wrote: post #4

interesting.. this could make training much faster if there’s a universal low dimensional space that models naturally converge into, since you could initialize or constrain training inside that space instead of spending massive compute rediscovering it from scratch every time

You can show for example that siamese encoders for time-series, with MSE loss on similarity, without a decoder, will converge to the the same latent space up to orthogonal transformations (as MSE is kinda like gaussian prior which doesn’t distinguish between different rotations).

Similarly I would expect that transformers trained on the same loss function for predicting the next word, if the data is at all similar (like human language), would converge to approx the same space. And to represent that same space probably weights are similar, too. Weights in general seem to occupy low-dimensional spaces.

All in all, I don’t think this is that surprising, and I think the theoretical angle should be (have been?) to find mathematical proofs like this paper https://openreview.net/forum?id=ONfWFluZBI

Re: The universal weight subspace hypothesis

#70

Post by tsurba »

altairprime wrote: post #19

For those trying to understand the most important parts of the paper, here's what I think is the most significant two statements, subquoted out of two (consecutive) paragraphs midway through the paper: > we selected five additional, previously unseen pretrained ViT models for which we had access to evaluation data. These models, considered out-of-domain relative to the initial set, had all their weights reconstructed…

Edit: actually this paper is the canonical reference (?): https://arxiv.org/abs/2007.00810 models converge to same space up to a linear transformation. Makes sense that a linear transformation (like PCA) would be able to undo that transformation.

You can show for example that siamese encoders for time-series, with MSE loss on similarity, without a decoder, will converge to the the same latent space up to orthogonal transformations (as MSE is kinda like gaussian prior which doesn’t distinguish between different rotations).

Similarly I would expect that transformers trained on the same loss function for predicting the next word, if the data is at all similar (like human language), would converge to approx the same space, up to some, likely linear, transformations. And to represent that same space probably weights are similar, too. Weights in general seem to occupy low-dimensional spaces.

All in all, I don’t think this is that surprising, and I think the theoretical angle should be (have been?) to find mathematical proofs like this paper https://openreview.net/forum?id=ONfWFluZBI

They also have a previous paper (”CEBRA”) published in Nature with similar results.

Post reply on HN