Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TopoNets: High performing vision and language models with brain-like topography

arxiv.org

Post reply on HN
Print view View original

21–30 of 76 posts

Re: TopoNets: High performing vision and language models with brain-like topography

#21

Post by jaek »

whynotminot wrote: post #10
vlovich123 wrote: post #8

Earlier quoted context omitted.

Unless GPUs work markedly differently somehow or there’s been some fundamental shift in computer architecture I’m not aware of, spatial locality is still a factor in computers. Aside from HW acceleration today, designs like Cebras would benefit heavily by reducing the amount of random access from accessing the weights (and thus freeing up cross-chip memory bandwidth for other things).

This makes me remember game developers back when games could still be played directly from the physical disc. They would often duplicate data to different parts of the disc, knowing that certain data would often be streamed from disc together, so that seek times were minimized. But those game devs knew where everything was spatially on the disc, and how the data would generally be used during gameplay. It was consist…

This brings to mind The Story of Mel from programming folklore.

http://beza1e1.tuxen.de/lore/story_of_mel.html

Re: TopoNets: High performing vision and language models with brain-like topography

#22

Post by mayukhdeb »

energy123 wrote: post #12
xpl wrote: post #6

Earlier quoted context omitted.

> So what's the motivation here? Better interpretability, I suppose. Could give insights into how cognition works.

That's true, and interpretability is helpful for AI safety.

Indeed. What's cool is that we were able to localize literal "regions" in the GPTs which encoded toxic concepts related to racism, politics, etc. A similar video can be found here: https://toponets.github.io

More work is being done on this as we speak.

Re: TopoNets: High performing vision and language models with brain-like topography

#23

Post by cwillu »

mayukhdeb wrote: post #22
energy123 wrote: post #12

Earlier quoted context omitted.

That's true, and interpretability is helpful for AI safety.

Indeed. What's cool is that we were able to localize literal "regions" in the GPTs which encoded toxic concepts related to racism, politics, etc. A similar video can be found here: https://toponets.github.io More work is being done on this as we speak.

Was it toxicity though as understood by the model, or just a cluster of concepts that you've chosen to label as toxic?

I.e., is this something that could (and therefore, will) be turned towards identifying toxic concepts as understood by the chinese or us government, or to identify (say) pro-union concepts so they can be down-weighted in a released model, etc?

Re: TopoNets: High performing vision and language models with brain-like topography

#24

Post by mayukhdeb »

ziofill wrote: post #19
energy123 wrote: post #5

The main reason topography emerges in physical brains is because spatially distant connections are physically difficult and expensive in biological systems. Artificial neural nets have no such trade-off. So what's the motivation here? I can understand this might be a very good regularizer, so it could help with generalization error on small-data tasks. But hard to see why this should be on the critical path to AGI. A…

Perhaps they are more easily compressible? Once a bunch of nearby weights have similar roles one may not need all of them.

Yep. That is exactly the idea here. Our compression method is super duper naive. We literally keep every n-th weight column and discard the rest. Turns out that even after getting rid of 80% of the weight columns in this way, we were able to retain the same performance in a 125M GPT.

Re: TopoNets: High performing vision and language models with brain-like topography

#25

Post by jv22222 »

energy123 wrote: post #5

The main reason topography emerges in physical brains is because spatially distant connections are physically difficult and expensive in biological systems. Artificial neural nets have no such trade-off. So what's the motivation here? I can understand this might be a very good regularizer, so it could help with generalization error on small-data tasks. But hard to see why this should be on the critical path to AGI. A…

I had this idea the other day. Not sure if it relates but maybe?

https://twitter.com/justinvincent/status/1884357300703400274

Re: TopoNets: High performing vision and language models with brain-like topography

#26

Post by [deleted] »

vessenes wrote: post #14

I hate to dog on research papers. They’re work to write. That said, I think this paper is not likely to be of interest to AI researchers — instead it may be of interest to Neuroscience folks or other brain research types. The lede — adding topography worsens networks at similar weights — is not only buried, it’s obscured with statements claiming that topo networks show less upheaval when scaled down, e.g. they are mo…

[deleted]

Re: TopoNets: High performing vision and language models with brain-like topography

#27

Post by mayukhdeb »

cwillu wrote: post #23
mayukhdeb wrote: post #22

Earlier quoted context omitted.

Indeed. What's cool is that we were able to localize literal "regions" in the GPTs which encoded toxic concepts related to racism, politics, etc. A similar video can be found here: https://toponets.github.io More work is being done on this as we speak.

Was it toxicity though as understood by the model, or just a cluster of concepts that you've chosen to label as toxic? I.e., is this something that could (and therefore, will) be turned towards identifying toxic concepts as understood by the chinese or us government, or to identify (say) pro-union concepts so they can be down-weighted in a released model, etc?

We localized "toxic" neurons by contrasting the activations of each neuron for toxic v/s normal texts. It's a method inspired by old-school neuroscience.

Re: TopoNets: High performing vision and language models with brain-like topography

#29

Post by dang »

slama wrote: post #4

The title here doesn't seem to match. The paper is called "TopoNets: High Performing Vision and Language Models with Brain-Like Topography" Even with their new method, models with topography seem to perform worse than models without.

Submitted title was "Inducing brain-like structure in GPT's weights makes them parameter efficient". We've reverted it now in keeping with the site guidelines (https://news.ycombinator.com/newsguidelines.html).

Since the submitter appears to be one of the authors, maybe they can explain the connection between the two titles? (Or maybe they already have! I haven't read the entire thread)

Re: TopoNets: High performing vision and language models with brain-like topography

#30

Post by FrereKhan »

This paper imports an arbitrarily-chosen aspect of cortical architecture — topological maps of function — and ignores every other aspect of biological neural tissue. The resulting models show lower performance for the same number of parameters — not surprising, since they are more constrained compared with baseline. They may be slightly more robust against pruning — not surprising, since they are more regularised.

The figures show individual seeds, presumably, with no statistical analysis in the performance or pruning comparisons, so the null hypothesis is there is no difference between toponets and baseline. I would never let this paper be submitted by my team.

We haven't learned anything about the brain, or about ANNs.

Post reply on HN