Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Differentiable Neural Computers

deepmind.com

Post reply on HN
Print view View original

1–10 of 77 posts

Re: Differentiable Neural Computers

#5

Post by rkaplan »

This paper builds off of DeepMind's previous work on differentiable computation: Neural Turing Machines. That paper generated a lot of enthusiasm when it came out in 2014, but not many researchers use NTMs today.

The feeling among researchers I've spoken to is not that NTMs aren't useful. DeepMind is simply operating on another level. Other researchers don't understand the intuitions behind the architecture well enough to make progress with it. But it seems like DeepMind, and specifically Alex Graves (first author on NTMs and now this), can.

Re: Differentiable Neural Computers

#7

Post by wiz21c »

rkaplan wrote: post #5

This paper builds off of DeepMind's previous work on differentiable computation: Neural Turing Machines. That paper generated a lot of enthusiasm when it came out in 2014, but not many researchers use NTMs today. The feeling among researchers I've spoken to is not that NTMs aren't useful. DeepMind is simply operating on another level. Other researchers don't understand the intuitions behind the architecture well enou…

>> DeepMind is simply operating on another level.

Would you be so kind as to to explain what you mean here ?

Thanks !

Re: Differentiable Neural Computers

#8

Post by visarga »

rkaplan wrote: post #5

This paper builds off of DeepMind's previous work on differentiable computation: Neural Turing Machines. That paper generated a lot of enthusiasm when it came out in 2014, but not many researchers use NTMs today. The feeling among researchers I've spoken to is not that NTMs aren't useful. DeepMind is simply operating on another level. Other researchers don't understand the intuitions behind the architecture well enou…

Deep mind is breaking new ground in number of directions. For example, "Decoupled Neural Interfaces using Synthetic Gradients" is simply amazing - they can make training a net async and run individual layers on separate machines by approximating the gradients with a local net. It's the kind of thing that sounds crazy on paper, but they proved it works.

Another amazing thing they did was to generate audio by direct synthesis from a neural net, beating all previous benchmarks. If they can make it work in real time, it would be a huge upgrade in our TTS technology.

We're still waiting for the new and improved AlphaGo. I hope they don't bury that project.

Re: Differentiable Neural Computers

#9

Post by usgroup »

rkaplan wrote: post #5

This paper builds off of DeepMind's previous work on differentiable computation: Neural Turing Machines. That paper generated a lot of enthusiasm when it came out in 2014, but not many researchers use NTMs today. The feeling among researchers I've spoken to is not that NTMs aren't useful. DeepMind is simply operating on another level. Other researchers don't understand the intuitions behind the architecture well enou…

Any chance you could fix this statement:

Input = Data

Process = Optimisation to create an automata.

Output = Automata

Computer power means much larger variable spaces can be handled in optimisation problems. NN are a means to prune the variable space during optimisation in a domain unspecific way.

Post reply on HN