Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Deep Learning

udlbook.github.io

Post reply on HN
Print view View original

11–20 of 103 posts

Re: Understanding Deep Learning

#11

Post by reqo »

ldjkfkdsjnv wrote: post #4

I spent a decade working on various machine learning platforms at well known tech companies. Everything I ever worked on became obsolete pretty fast. From the ML algorithm to the compute platform, all of it was very transitory. That coupled with the fact that a few elite companies are responsible for all ML innovation, its oxymoronic to me to even learn a lot of this material.

Very few things stay the same in Technology. You should think of technology as another type of evolution! It is driven by the same type of forces as evolution IMO. I think even Linus Torvalds once stated that Linux evolved trough natural selection.

Re: Understanding Deep Learning

#13

Post by ksherlock »

nsxwolf wrote: post #9

As someone who missed the boat on this, is learning about this just for historical purposes now, or is there still relevance to future employment? I just imagine the OpenAI eats everyone's lunch in regards to anything AI related, am I way off base?

Maybe last week's drama should have been a left-pad moment. For many things you can train your own NN and be just as good without being dependent on internet access, third parties, etc. Knowing how things work should give you insight into using them better.

Re: Understanding Deep Learning

#14

Post by nabla9 »

ldjkfkdsjnv wrote: post #4

I spent a decade working on various machine learning platforms at well known tech companies. Everything I ever worked on became obsolete pretty fast. From the ML algorithm to the compute platform, all of it was very transitory. That coupled with the fact that a few elite companies are responsible for all ML innovation, its oxymoronic to me to even learn a lot of this material.

>machine learning platforms

Machine learning platforms become obsolete.

Machine learning algorithms and ideas don't. If learning SVN or Naive Bayes did not teach you things that are useful today, you didn't learn anything.

Re: Understanding Deep Learning

#15

Post by HighFreqAsuka »

ldjkfkdsjnv wrote: post #4

I spent a decade working on various machine learning platforms at well known tech companies. Everything I ever worked on became obsolete pretty fast. From the ML algorithm to the compute platform, all of it was very transitory. That coupled with the fact that a few elite companies are responsible for all ML innovation, its oxymoronic to me to even learn a lot of this material.

Quite a lot of techniques in deep learning have stood the test of time at this point. Also new techniques are developed either depending on or trying to solved deficiencies in old techniques. For example Transformers were developed to solve vanishing gradients in LSTMs over long sequences and improve GPU utilization since LSTMs were inherently sequential in the time dimension.

Re: Understanding Deep Learning

#16

Post by lamroger »

ksherlock wrote: post #13
nsxwolf wrote: post #9

As someone who missed the boat on this, is learning about this just for historical purposes now, or is there still relevance to future employment? I just imagine the OpenAI eats everyone's lunch in regards to anything AI related, am I way off base?

Maybe last week's drama should have been a left-pad moment. For many things you can train your own NN and be just as good without being dependent on internet access, third parties, etc. Knowing how things work should give you insight into using them better.

I wonder if using APIs was more of a first to market move

Re: Understanding Deep Learning

#17

Post by adamnemecek »

All machine learning is Hopf convolution, analogous to renormalization. This should come as no surprise, renormalization can be modeled via the Ising model which itself is closely related to Hopfield networks which are recurrent networks.

Re: Understanding Deep Learning

#18

Post by PeterisP »

water-your-self wrote: post #5

No chapter on RNNs, but one on transformers is interesting, having last read Deep learning by ian goodfellow in 2016

RNNs have "lost the hardware lottery" by being structurally not that efficient to train on the cost-effective hardware that's available. So they're not really used for much right now - though IMHO they are conceptually sufficiently interesting enough to cover in such a course.

Re: Understanding Deep Learning

#19

Post by mnky9800n »

ksherlock wrote: post #13
nsxwolf wrote: post #9

As someone who missed the boat on this, is learning about this just for historical purposes now, or is there still relevance to future employment? I just imagine the OpenAI eats everyone's lunch in regards to anything AI related, am I way off base?

Maybe last week's drama should have been a left-pad moment. For many things you can train your own NN and be just as good without being dependent on internet access, third parties, etc. Knowing how things work should give you insight into using them better.

Which drama of last week are you referring to? The one about the openai guy saying it's all just the data set? Or something else?
Post reply on HN