Hybrid computing using a neural network with dynamic external memory

#1

Post by idunning » Wed, Oct 12, 2016, 5:20 PM UTC

Hybrid computing using a neural network with dynamic external memory
nature.com

Re: Hybrid computing using a neural network with dynamic external memory

#2

Post by idunning » Wed, Oct 12, 2016, 5:22 PM UTC

Blog post for the paper: https://deepmind.com/blog/differentiable-neural-computers/

Re: Hybrid computing using a neural network with dynamic external memory

#3

Post by gallerdude » Wed, Oct 12, 2016, 5:36 PM UTC

Can someone explain what the full implications of this are? This seems really cool, but I can't really wrap my head around it.

From what I can tell you can give the DNC simple inputs and it can derive complex answers.

Re: Hybrid computing using a neural network with dynamic external memory

#4

Post by bra-ket » Wed, Oct 12, 2016, 5:49 PM UTC

if you're interested in this check out "Reasoning, Attention, Memory (RAM)" NIPS Workshop 2015 organized by Jason Weston (Facebook Research): http://www.thespermwhale.com/jaseweston/ram/

Re: Hybrid computing using a neural network with dynamic external memory

#5

Post by 0xdeadbeefbabe » Wed, Oct 12, 2016, 6:07 PM UTC

> a DNC can complete a moving blocks puzzle in which changing goals are specified by sequences of symbols

A neural network without memory can't do that or can't do it as well perhaps?

Re: Hybrid computing using a neural network with dynamic external memory

#6

Post by the_decider » Wed, Oct 12, 2016, 6:28 PM UTC

Some interesting ideas sadly blocked behind a pay-wall journal, all for the purpose of boosting a researcher's prestige because they now hold a "Nature" publication. Thankfully, this article is easily accessible via Sci-Hub. http://www.nature.com.sci-hub.cc/nature/journal/vaop/ncurren...

Re: Hybrid computing using a neural network with dynamic external memory

#7

Post by ktamiola » Wed, Oct 12, 2016, 6:36 PM UTC

This is remarkable!

Re: Hybrid computing using a neural network with dynamic external memory

#8

Post by kylek » Wed, Oct 12, 2016, 6:48 PM UTC

I'm probably totally off base here (neural networks/AI is not my wheelhouse), but is having "memory" in neural networks a new thing? Isn't this just a different application of a more typical 'feedback loop' in the network?

Re: Hybrid computing using a neural network with dynamic external memory

#9

Post by AlexCoventry » Wed, Oct 12, 2016, 7:00 PM UTC

kylek wrote: post #8

I'm probably totally off base here (neural networks/AI is not my wheelhouse), but is having "memory" in neural networks a new thing? Isn't this just a different application of a more typical 'feedback loop' in the network?

From the paper's "System Overview":

  An earlier form of DNC, the neural Turing machine16, had a
  similar structure, but more limited memory access methods
  (see Methods for further discussion).

Re: Hybrid computing using a neural network with dynamic external memory

#10

Post by superfx » Wed, Oct 12, 2016, 7:04 PM UTC

the_decider wrote: post #6

Some interesting ideas sadly blocked behind a pay-wall journal, all for the purpose of boosting a researcher's prestige because they now hold a "Nature" publication. Thankfully, this article is easily accessible via Sci-Hub. http://www.nature.com.sci-hub.cc/nature/journal/vaop/ncurren...

Here's an official, publicly accessible, link to the article: http://rdcu.be/kXhV