Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Deep Learning has outlived its usefulness as a buzz-phrase”

facebook.com

Post reply on HN
Print view View original

21–30 of 107 posts

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#21

Post by pasquinelli »

tomdre wrote: post #16
bennofs wrote: post #9

Earlier quoted context omitted.

I like it, at least from the little that I read about it. The name describes the core of what it is: differentiate programs, in order to figure out how changes the the program affect the output and using that for optimization purposes. Do we really need to invent obscure, new names for everything just so that it sounds catchy?

Couldn't agree more. A technique should be judged by its usefulness. Not by its catchiness.

to be fair though, the subject is a judgement of a technique's name, not a judgement of the technique itself.

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#22

Post by jostmey »

ehsankia wrote: post #5

1. Differentiable Programming is horrible branding. It's hard to say, not catchy, and not as easily decipherable. 2. Isn't the evolution of Deep Networks more advance setups such as GANs, RNNs, and so on?

> 2. Isn't the evolution of Deep Networks more advance setups such as GANs, RNNs, and so on?

That's not what Yann LeCun is getting at I think. Most neural networks models are sort of like a non-programmable computer. They are built around assuming the data is a fixed size, fixed dimensional array. But in computer programming there is an ocean of data structures. You have sorted sets, linked lists, dictionaries, and everything else. Imagine that we knew that a data-set was arranged as a sort of "fuzzy" dictionary and we wanted the computer to do the rest. All we need to do is load up the right deep neural network (I mean differential programming something or other) and wallah.

Something like where the the value in a piece of data dictate the layers that get stacked together and how those layers connect to layers for the next value in that piece of data

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#23

Post by currymj »

ehsankia wrote: post #5

1. Differentiable Programming is horrible branding. It's hard to say, not catchy, and not as easily decipherable. 2. Isn't the evolution of Deep Networks more advance setups such as GANs, RNNs, and so on?

it seems like it's made by analogy with "probabilistic programming", i.e. defining complex probability distributions by writing familiar-looking imperative code (with loops and whatnot).

I think the idea is that thinking in terms of passing data through layers in a graph is cumbersome sometimes, and that expressing it as a "regular" program that just happens to come with gradients could be more comfortable.

I'd argue that GANs in particular are a natural fit for this style. The training procedure doesn't really fit exactly into the standard "minimize loss function of many layers using backprop".

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#24

Post by currymj »

billconan wrote: post #17

"working on compilers for imperative differentiable programming languages" what would be an example of such language?

working with dynamic neural net libraries that have autodiff (PyTorch, regular Torch, and I think also MXNet now) feels a lot like this. you just write normal functions, but after you execute them you can ask for gradients too.

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#26

Post by woodson »

username223 wrote: post #13
oh-kumudo wrote: post #7

Just a rebranding, though necessary one. Deep Learning is not really all about 'deep' anymore, many successful models don't really need a lot of layers.

So it's still just neural nets? Cool -- we've seen that before.

I’m in favor of getting rid of „neural“. That would help dispense with all the unhelpful discussions about how different DNNs work compared to the human brain.

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#28

Post by hyperpallium »

kinkrtyavimoodh wrote: post #20
ehsankia wrote: post #5

1. Differentiable Programming is horrible branding. It's hard to say, not catchy, and not as easily decipherable. 2. Isn't the evolution of Deep Networks more advance setups such as GANs, RNNs, and so on?

> Differentiable Programming is horrible branding. It's hard to say, not catchy, and not as easily decipherable Tell that to the people who deliberately popularized the term Dynamic Programming for something that was neither dynamic nor programming. ____ (From Wiki) Bellman explains the reasoning behind the term dynamic programming in his autobiography, Eye of the Hurricane: An Autobiography (1984, page 159). He expl…

Deep Learning -> "Dynamic Learning"? Post even describes it "dynamic".

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#29

Post by make3 »

danbmil99 wrote: post #6

"See more of Yann LeCun on Facebook" popup, no access to the page. No, I don't want to create a Facebook account to read a blog post. Perhaps links to walled-garden pages where you need an account and need to be logged in should be prohibited or at least discouraged.

this will get downvoted as all held, but I really think most socially apt people should have Facebook accounts these days

Re: “Deep Learning has outlived its usefulness as a buzz-phrase”

#30

Post by seanmcdirmid »

BucketSort wrote: post #19

I believe this paper by Marcus ( https://arxiv.org/ftp/arxiv/papers/1801/1801.00631.pdf ) earlier this week inspired this. Edit: I don't mean Marcus inspired the term differentiable programming; he inspired LeCun to emphasize the wider scope of deep learning after Marcus attacked it. In fact, LeCun liked a post on twitter rebutting Marcus' paper that also talks about differentiable programming: https://twitter.com/td…

Or the other way around... He has been throwing around the term for awhile now.
Post reply on HN