Neural Network Diffusion
arxiv.org
Neural Network Diffusion
1–10 of 92 posts
Re: Neural Network Diffusion
#2Author thread: https://twitter.com/liuzhuang1234/status/1760195922502312197
Re: Neural Network Diffusion
#3[deleted]
Re: Neural Network Diffusion
#4Seems like we're getting very close to recursive self-improvement [0].
[0] https://www.lesswrong.com/tag/recursive-self-improvement
Re: Neural Network Diffusion
#5Author thread: https://twitter.com/liuzhuang1234/status/1760195922502312197
Is there any sites for viewing Twitter threads without signing up?
Re: Neural Network Diffusion
#6Author thread: https://twitter.com/liuzhuang1234/status/1760195922502312197
Is there any sites for viewing Twitter threads without signing up?
https://nitter.esmailelbob.xyz/liuzhuang1234/status/17601959...
(bit of trial and error from https://github.com/zedeus/nitter/wiki/Instances)
Re: Neural Network Diffusion
#7Seems like we're getting very close to recursive self-improvement [0]. [0] https://www.lesswrong.com/tag/recursive-self-improvement
I upvoted because this was my first thought too, but reading the abstract and skimming the paper makes me think it’s not really an advance for general recursive improvement. I think the title makes people think this is a text -> model model, when it is really a bunch of model weights -> new model weights optimizer for a specific architecture and problem. Still a potentially very useful idea for learning from a bunch of training runs and very interesting work!
Re: Neural Network Diffusion
#8Seems like we're getting very close to recursive self-improvement [0]. [0] https://www.lesswrong.com/tag/recursive-self-improvement
[deleted]
Re: Neural Network Diffusion
#9Why does Figure 7 not include a validation curve (afaict only the training curve is shown)?
Re: Neural Network Diffusion
#10Seems like we're getting very close to recursive self-improvement [0]. [0] https://www.lesswrong.com/tag/recursive-self-improvement
Doesn't look that different from what we are already doing. For example AlphaGo/AlphaZero/MuZero learn to play board games by playing repeatedly against itself, it is a self improvement loop leading to superhuman play. It was a major breakthrough for the game of Go, and it lead to advances in the field of machine learning, but we are still far from something resembling technological singularity.
GANs are another example of self-improvement. It was famous for creating "deep fakes". It works by pitting a fake generator and a fake detector against each other, resulting in a cycle of improvement. It didn't get much further than that, in fact, it is all about attention and transformers now.
This is just a way of optimizing parameters, it will not invent new techniques. It can say "put 1000 neurons there, 2000 there, etc...", but it still has to pick from what designers tell it to pick from. It may adjust these parameters better than a human can, leading to more efficient systems, I expect some improvement to existing systems, but not a breaking change.