Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Deep Learning

udlbook.github.io

Post reply on HN
Print view View original

31–40 of 103 posts

Re: Understanding Deep Learning

#31

Post by deepsquirrelnet »

nsxwolf wrote: post #9

As someone who missed the boat on this, is learning about this just for historical purposes now, or is there still relevance to future employment? I just imagine the OpenAI eats everyone's lunch in regards to anything AI related, am I way off base?

This is about deep learning, of which LLMs are a subset. If you are interested in machine learning, then you should learn deep learning. It is incredibly useful for a lot of reasons.

Unlike other areas of ML, the nature of deep learning is such that its parts are interoperable. You could use a transformer with a CNN if you wish. Also, deep learning enables you to do machine learning on any type of data, text, images, video, audio. Finally, it can naturally scale computationally.

As someone pretty involved in the field, I lament that LLMs are turning people away from ML and deep learning, and following the misconceptions that there’s no reason to do it anymore. Large algorithms are expensive to run, have slow throughput and are still generally poorer performing than purpose built models. They’re not even that easy to use for a lot of tasks, in comparison to encoder networks.

I’m biased, but I think it’s one of the most fun things to learn in computing. And if you have a good idea, you can still build state of the art things with a regular gpu at your house. You just have to find a niche that isn’t getting the attention that LLMs are ;)

Re: Understanding Deep Learning

#33

Post by theogravity »

nootopian wrote: post #6
msie wrote: post #3

This book looks impressive. There's a chapter on the unreasonable effectiveness of Deep Learning which I love. Any other books I should be on the lookout for?

https://news.ycombinator.com/item?id=38425368

I wish it wasn't an X post. Can't see responses at all without an account.

Re: Understanding Deep Learning

#34

Post by nextos »

msie wrote: post #3

This book looks impressive. There's a chapter on the unreasonable effectiveness of Deep Learning which I love. Any other books I should be on the lookout for?

This presentation from Deep Mind outlines some foundational ML books: https://drive.google.com/file/d/1lPePNMGMEKoaDvxiftc8hcy-rFp...

For the impatient, look into slide #123. Essentially, the recommendations are Murphy, Gelman, Barber, and Deisenroth.

Note these slides have a Bayesian bias. In spite of that, Murphy is a great DL book. Besides, going through GLMs is a great way to get into DL.

Re: Understanding Deep Learning

#35

Post by nextos »

theogravity wrote: post #33
nootopian wrote: post #6

Earlier quoted context omitted.

https://news.ycombinator.com/item?id=38425368

I wish it wasn't an X post. Can't see responses at all without an account.

Use nitter to go around X authwalls: https://nitter.net/suhail/status/1728676402864812466

Re: Understanding Deep Learning

#36

Post by Philpax »

ldjkfkdsjnv wrote: post #29
nabla9 wrote: post #28

Earlier quoted context omitted.

>Spending your precious life learning them is a waste So you really did not learn them. There is nothing wrong with being user. You don't have to know how compilers work to use compiler. But then you should not say you understand compilers. In the same way, you probably would benefit from a book "Using deep learning", not "Understanding deep learning".

I know them and am a founder of a vc funded ai startup. Nobody is deploying naive bayes algorithms

Yes, they’re not deploying them. That doesn’t mean it doesn’t still help to know the fundamentals of the field, especially when you’re trying to innovate.

Re: Understanding Deep Learning

#37

Post by coef2 »

ldjkfkdsjnv wrote: post #22
nabla9 wrote: post #14

Earlier quoted context omitted.

>machine learning platforms Machine learning platforms become obsolete. Machine learning algorithms and ideas don't. If learning SVN or Naive Bayes did not teach you things that are useful today, you didn't learn anything.

Nobody is building real technology with either of those algorithms. Sure, they are theoretically helpful, but they arent valuable anymore. Spending your precious life learning them is a waste

I started my journey in machine learning fifteen years ago. Ironically, at that time, my professor told me that neural networks were outdated and trying them wouldn't result in publishable research. SVMs were popular and emphasized in my coursework. I concur that SVMs don't hold as much practical significance today. But the progress in AI and ML is generally unpredictable, and no one knows what theory leads to the next leap in the field.

Re: Understanding Deep Learning

#38

Post by trace_id »

ldjkfkdsjnv wrote: post #8
drBonkers wrote: post #7

Earlier quoted context omitted.

What would you recommend someone read instead?

Better to understand the bounds of whats currently possible. And then recognize when that changes. Much more economically valuable

Even better: change the bounds of whats possible ;)

Re: Understanding Deep Learning

#39

Post by nextos »

PeterisP wrote: post #18
water-your-self wrote: post #5

No chapter on RNNs, but one on transformers is interesting, having last read Deep learning by ian goodfellow in 2016

RNNs have "lost the hardware lottery" by being structurally not that efficient to train on the cost-effective hardware that's available. So they're not really used for much right now - though IMHO they are conceptually sufficiently interesting enough to cover in such a course.

That is not completely true. There are RNNs with transformer/LLM-like performance. See e.g. https://github.com/BlinkDL/RWKV-LM.

They are less popular, and less explored. But an interesting route ahead.

Re: Understanding Deep Learning

#40

Post by [deleted] »

ldjkfkdsjnv wrote: post #23
HighFreqAsuka wrote: post #15

Earlier quoted context omitted.

Quite a lot of techniques in deep learning have stood the test of time at this point. Also new techniques are developed either depending on or trying to solved deficiencies in old techniques. For example Transformers were developed to solve vanishing gradients in LSTMs over long sequences and improve GPU utilization since LSTMs were inherently sequential in the time dimension.

Sure, but if you were an expert in LSTM, thats nice, you know the lineage of algorithms. But it probably isnt valuable, companies dont care, and you cant directly use that knowledge. You would never just randomly study LSTMs now.

[deleted]
Post reply on HN