Live data from Hacker News

Self-Compressing Neural Networks

arxiv.org

21–30 of 59 posts

Re: Self-Compressing Neural Networks

#21
post #17
post #16

Earlier quoted context omitted.

Unfortunately dendritic compartmentalization, spike timing etc are still not present. All efforts at models of SNNs that I know of have hit problems like riddled basins so far, that is what to look for to move past the limits of perceptron based networks IMHO. As PAC learning with autograd and perceptrons is just compression, or set shattering, this paper is more of an optimization method that reduces ANN expressiven…

> reduces ANN expressiveness But does it? It’s been my hypothesis for a while that every grad-trained NN is hauling around a lot of “nascent” nodes — nodes that were on their way to being useful, but haven’t received enough input yet to actually have their outputs be distinguishable from noise / ever influence the output. Sort of the neuroplastic equivalent of an evolutionary pre-adaptation . If such nodes exist in N…

Easiest way I can figure out how to explain my claim.

Consider when you use 'partial connectivity', E.G. convolution or pooling layers for local feature extraction on say MNIST.

While useful, those partial connection layers are explicitly used because fully connected layers do not have translational invariance.

So with a fully connected network, shifting the letter 'i' a few pixels to the right wouldn't match.

We choose to discard some of those connections for local feature detection. But as the reason that the fully connected model lacks translational invariance is because it maintains that position data.

Note how that is more 'expressive', even if counterproductive for the actual use case.

Another lens is the fact that neural networks have extreme simplicity bias. In that they learn only the simplest features to solve a task at hand.

If you want to recognize an i, irrespective of the translational location, that bias is useful. But you 'throw away' (in a very loose sense) the positional data to do so.

Horses for courses, not good vs bad.

Re: Self-Compressing Neural Networks

#22

I think this might be the first step to making neural networks that actually mimic biological brains. IMO the biggest piece missing from NN architectures is a mechanism like neuroplasticity that modifies the topology of neurons. Brains reorganize themselves around the things they learn. This paper is a long way from implementing synaptic pruning/strengthening/weakening, neurogenesis, or synaptogenesis but it’s the fi…

> mechanism like neuroplasticity that modifies the topology of neurons

Isn't this already accomplished via weights?

Re: Self-Compressing Neural Networks

#23

I think this might be the first step to making neural networks that actually mimic biological brains. IMO the biggest piece missing from NN architectures is a mechanism like neuroplasticity that modifies the topology of neurons. Brains reorganize themselves around the things they learn. This paper is a long way from implementing synaptic pruning/strengthening/weakening, neurogenesis, or synaptogenesis but it’s the fi…

Stop trying to mimic brains. Do what works best for transistors.

It would be foolish not to look for inspiration in a system that had billions of years of evolution invested in it.

Re: Self-Compressing Neural Networks

#24
post #16

Earlier quoted context omitted.

Unfortunately dendritic compartmentalization, spike timing etc are still not present. All efforts at models of SNNs that I know of have hit problems like riddled basins so far, that is what to look for to move past the limits of perceptron based networks IMHO. As PAC learning with autograd and perceptrons is just compression, or set shattering, this paper is more of an optimization method that reduces ANN expressiven…

Are dendritic sub-compartments necessary to explicitly model, or does this work just imply that biological neurons are complicated and are better modeled as a multi-layered artificial network, rather than a single simple computational unit? Similarly, do you think that spiking networks are important, or just a specific mechanism used in the brain to transmit information, which dense (or sparse) vectors of floats do i…

If the goal was to create an artificial neural network that better approximated the biological human brain, yes the perceptron model is insufficient.

If your goal is to produce a useful model on real hardware and it works...no

Remember the constraints of ANNs being universal approximaters (in theory)

1) The function you are learning needs to be continuous 2) Your model is over a closed, bounded subset of R^n 3) The activation function is bounded and monodial

Obviously that is the theoretical UAT constraints. For gradient decent typically used in real ML models, the constraint of finding only smooth approximations of continuous functions can be problematic depending on your needs.

But people leveraged phlogiston theory for beer brewing with great success and obviously Newtonian Mechanics is good enough for many tasks.

SNNs in theory should be able to solve problems that are challenging for perceptron models, but as I said, features like riddled basins are problematic so far.

https://arxiv.org/abs/1711.02160

Re: Self-Compressing Neural Networks

#25
post #23

Earlier quoted context omitted.

Stop trying to mimic brains. Do what works best for transistors.

It would be foolish not to look for inspiration in a system that had billions of years of evolution invested in it.

We already found the inspiration. That’s how we invented neural networks. Now we need to focus on what works.

Re: Self-Compressing Neural Networks

#26
post #8

Earlier quoted context omitted.

Why? Do you think lossless compression is intelligence?

Compressing understanding (not just information) in a way that uses semantic links in information is a big part of intelligence, I'd say.

We're doing a double search - searching for experience outside, collecting data - and searching for understanding inside, by compressing the data. Search and learn, they define both AI and us.

Re: Self-Compressing Neural Networks

#27
Using as little computational resources (memory and/or FLOPS) as possible as an additional optimization criterion when training NNs is an interesting avenue. I think the current state of pre-trained model families is weird. Take Llama 3.1 or Segment Anything 2: you get tiny/small/medium/larger/huge models, where for each tier the model size was predefined, and they are trained somewhat (completely?) independently. This feels iffy, patchy, and like we haven't really arrived yet.

I'd want a model that scales up and down depending on the task given at inference, and a model that doesn't have a fixed size when starting the training. Shouldn't it specialize over training progress, when seeing more tokens, and grow larger where needed? Without some human fixing a size beforehand?

Self-organization is a fascinating topic to me. This last year I've been working on Self-Organizing Gaussian Splats [0]. With a lot of squinting, this lives in a similar space as the Self-Compressing Neural Networks from the link above. The idea of the Gaussians was to build on Self-Organizing Maps (lovely 90s concept, look for some GIFs if you don't know it), and use that to represent 3D scenes in a memory-efficient way. By mapping attributes into a locally smooth 2D grid. It's quite a simple algorithm, but works really well, and better than many quite complicated coding schemes. So this has me excited that we'll (re-)discover great methods in this space in the near future.

[0]: https://fraunhoferhhi.github.io/Self-Organizing-Gaussians/

Re: Self-Compressing Neural Networks

#28

This is super cool. It's surprising to me that it took so long for someone to try this. It seems like such an obvious idea (in hindsight). But I guess that's easy to say now that someone came up with it. If this turns out to work well even for much larger models, then we might see loss functions that incorporate ever more specific performance metrics, conceivably even actual execution times on specific hardware.

There was related work that happened before, as mentioned in the paper.

Re: Self-Compressing Neural Networks

#29
post #27

Using as little computational resources (memory and/or FLOPS) as possible as an additional optimization criterion when training NNs is an interesting avenue. I think the current state of pre-trained model families is weird. Take Llama 3.1 or Segment Anything 2: you get tiny/small/medium/larger/huge models, where for each tier the model size was predefined, and they are trained somewhat (completely?) independently. Th…

Afaik, they aren't really trained independently -- for most models, e.g. DINO, etc., the smaller sizes are actually distilled from larger models. It's much easier to generate performant models at smaller size via distillation.

And I'd be curious of the utility of model that scales up and down at inference - if this was the case you'd still need to have storage that is the same as the maximum model size. This would essentially be useless for embedded applications, etc., unless you have heavy quantization - but quantization in a small parameter space would probably make the smaller modes useless. I could see the benefit here in terms of optimizing latency for different applications but maybe you have other ideas.

Given all that, I think training for smaller number of parameters, as noted in OP, would kind of beat out some model that scales at inference time - especially when most people know what kind of application they are aiming to build and the required level of performance.

Re: Self-Compressing Neural Networks

#30

I think this might be the first step to making neural networks that actually mimic biological brains. IMO the biggest piece missing from NN architectures is a mechanism like neuroplasticity that modifies the topology of neurons. Brains reorganize themselves around the things they learn. This paper is a long way from implementing synaptic pruning/strengthening/weakening, neurogenesis, or synaptogenesis but it’s the fi…

Stop trying to mimic brains. Do what works best for transistors.

Or grow beyond both with optics.
Post reply on HN