Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Deep Learning has outlived its usefulness as a buzz-phrase”

facebook.com

Post reply on HN
Print view View original

41–50 of 107 posts

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#41

Post by platz »

rememberlenny wrote: post #3

[Text from post] OK, Deep Learning has outlived its usefulness as a buzz-phrase. Deep Learning est mort. Vive Differentiable Programming! Yeah, Differentiable Programming is little more than a rebranding of the modern collection Deep Learning techniques, the same way Deep Learning was a rebranding of the modern incarnations of neural nets with more than two layers. But the important point is that people are now build…

> It's really very much like a regular progam, except it's parameterized, automatically differentiated, and trainable/optimizable.

> People are now actively working on compilers for imperative differentiable programming languages.

Do you have an example of either of these things?

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#42

Post by seanmcdirmid »

I'm seriously pondering what this means for PL research. There has been some work in probalistic programming languages, and a significant part of the community would like to avoid imperative features. However, it seems like this is a chance for some real invigoration in PL research agendas.

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#43

Post by innagadadavida »

danbmil99 wrote: post #6

"See more of Yann LeCun on Facebook" popup, no access to the page. No, I don't want to create a Facebook account to read a blog post. Perhaps links to walled-garden pages where you need an account and need to be logged in should be prohibited or at least discouraged.

He is monetizing for his employer using his 100+B neuron network...

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#44

Post by Momquist »

danbmil99 wrote: post #6

"See more of Yann LeCun on Facebook" popup, no access to the page. No, I don't want to create a Facebook account to read a blog post. Perhaps links to walled-garden pages where you need an account and need to be logged in should be prohibited or at least discouraged.

Disabling Javascript on Facebook allows you to read these public blog posts without any popups, login, or ads.

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#45

Post by sedachv »

billconan wrote: post #17

"working on compilers for imperative differentiable programming languages" what would be an example of such language?

Common Lisp without any changes: https://people.eecs.berkeley.edu/~fateman/papers/ADIL.pdf

Fortran with some changes required: http://www.ens.utulsa.edu/~diaz/cs8243/adifor.html

C with some changes required: http://www.ens.utulsa.edu/~diaz/cs8243/adiff.html

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#46

Post by sytelus »

BucketSort wrote: post #19

I believe this paper by Marcus ( https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf ) earlier this week inspired this. Edit: I don't mean Marcus inspired the term differentiable programming; he inspired LeCun to emphasize the wider scope of deep learning after Marcus attacked it. In fact, LeCun liked a post on twitter rebutting Marcus' paper that also talks about differentiable programming: https://twitter.com/td…

I don't think so. LeCun seems to oppose Marcus's views...

Related: https://twitter.com/ylecun/status/921409820825178114?lang=en

I think LeCun doesn't want a repeat of AI winter because of exponentially rising hype and expectations out of Deep Learning. There have been few examples like Selena which he seems to think that people are trying to ride the deep learning wave to generate false buzz (and cash!) for themselves.

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#47

Post by joshuamorton »

platz wrote: post #41
rememberlenny wrote: post #3

[Text from post] OK, Deep Learning has outlived its usefulness as a buzz-phrase. Deep Learning est mort. Vive Differentiable Programming! Yeah, Differentiable Programming is little more than a rebranding of the modern collection Deep Learning techniques, the same way Deep Learning was a rebranding of the modern incarnations of neural nets with more than two layers. But the important point is that people are now build…

> It's really very much like a regular progam, except it's parameterized, automatically differentiated, and trainable/optimizable. > People are now actively working on compilers for imperative differentiable programming languages. Do you have an example of either of these things?

I believe https://github.com/google/tangent counts, though I'm not 100% sure.

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#48

Post by BucketSort »

sytelus wrote: post #46
BucketSort wrote: post #19

I believe this paper by Marcus ( https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf ) earlier this week inspired this. Edit: I don't mean Marcus inspired the term differentiable programming; he inspired LeCun to emphasize the wider scope of deep learning after Marcus attacked it. In fact, LeCun liked a post on twitter rebutting Marcus' paper that also talks about differentiable programming: https://twitter.com/td…

I don't think so. LeCun seems to oppose Marcus's views... Related: https://twitter.com/ylecun/status/921409820825178114?lang=en I think LeCun doesn't want a repeat of AI winter because of exponentially rising hype and expectations out of Deep Learning. There have been few examples like Selena which he seems to think that people are trying to ride the deep learning wave to generate false buzz (and cash!) for themselve…

He does oppose Marcus' views, but he also knows neural nets are only one approach to differentiable programming. The term is confusing though. It should read like "linear programming" does, but people are not interpreting it that way.

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#49

Post by cs702 »

I wish we could come up with a catchier name, but I LOVE the idea of calling this programming, because that is precisely what we do when we compose deep neural nets.

For example, here's how you compose a neural net consisting of two "dense" layers (linear transformations), using Keras's functional API, and then apply these two layers to some tensor x to obtain a tensor y:

  f = Dense(n)
  g = Dense(n)

  y = f(g(x))
This looks, smells, and tastes like programming (in this case with a strong functional flavor), doesn't it?

Imagine how interesting things will get once we have nice facilities for composing large, complex applications made up of lots of components and subcomponents that are differentiable, both independently and end-to-end.

Andrej Karpathy has a great post about this: https://medium.com/@karpathy/software-2-0-a64152b37c35

Post reply on HN