Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding Deep Learning

udlbook.github.io

Post reply on HN
Print view View original

81–90 of 103 posts

Re: Understanding Deep Learning

#81

Post by wiz21c »

ldjkfkdsjnv wrote: post #4

I spent a decade working on various machine learning platforms at well known tech companies. Everything I ever worked on became obsolete pretty fast. From the ML algorithm to the compute platform, all of it was very transitory. That coupled with the fact that a few elite companies are responsible for all ML innovation, its oxymoronic to me to even learn a lot of this material.

So, what fundamental stuff should I learn ? I understand ML has some general principles that keeps on bein valid throughout the years. No ?

Re: Understanding Deep Learning

#82

Post by GeneralMayhem »

bjacobt wrote: post #51
wilkystyle wrote: post #49

Earlier quoted context omitted.

> Sources like Karpathy's recent video on what LLMs actually do are good anti-hype for the lay audience Which video is this?

I believe op means Intro to Large Language Model https://youtu.be/zjkBMFhNj_g?si=XQQ3p92ajuQYOyqN

Yep, that's the one I meant - sorry, should have linked.

His series on making a GPT from scratch is also great for building intuition specifically about text-based generative AI, with an audience of software developers.

Re: Understanding Deep Learning

#84

Post by fastneutron »

nextos wrote: post #34
msie wrote: post #3

This book looks impressive. There's a chapter on the unreasonable effectiveness of Deep Learning which I love. Any other books I should be on the lookout for?

This presentation from Deep Mind outlines some foundational ML books: https://drive.google.com/file/d/1lPePNMGMEKoaDvxiftc8hcy-rFp... For the impatient, look into slide #123. Essentially, the recommendations are Murphy, Gelman, Barber, and Deisenroth. Note these slides have a Bayesian bias. In spite of that, Murphy is a great DL book. Besides, going through GLMs is a great way to get into DL.

Reality has a well known Bayesian bias…

Joking aside, these slides are excellent! Is there an associated video or course that they were a part of?

Re: Understanding Deep Learning

#85

Post by martingoodson »

mi_lk wrote: post #79
martingoodson wrote: post #68

Most comments here are in one of two camps: 1) you don't need to know any of this stuff, you can make AI systems without this knowledge, or 2) you need this foundational knowledge to really understand what's going on. Both perspectives are correct. The field is bifurcating into two different skill sets: ML engineer and ML scientist (or researcher). It's great to have both types on a team. The scientists will be too s…

I guess this message is delivered by an AI scientist, sure. It's almost self-exploratory that when you hit a roadblock in practice you go back to foundations, and good people should aim to do both. In that case I don't see where ML engineer/scientist bifurcation comes from except for some to feel good about themselves

Not at all. It's something I've seen in practice over many years. Neither skill set is 'better' than the other, just different.

There is a need for people who are able to build using available tools, but who don't have an interest in the theory or foundations of the field. It's a valuable mindset and nothing in my original comment suggested otherwise.

It's also pretty clear that many comments on this post divide into the two mindsets I've described.

Re: Understanding Deep Learning

#86

Post by martingoodson »

gardenhedge wrote: post #76
martingoodson wrote: post #68

Most comments here are in one of two camps: 1) you don't need to know any of this stuff, you can make AI systems without this knowledge, or 2) you need this foundational knowledge to really understand what's going on. Both perspectives are correct. The field is bifurcating into two different skill sets: ML engineer and ML scientist (or researcher). It's great to have both types on a team. The scientists will be too s…

Would you see these as analogous? The people who create the models and the people that use them. The people who create the programming languages and the people that use them.

I think because it's a relatively 'younger' field, there is a bit more need to know about the foundations in AI than in programming. You hit the perimeters a bit more often and need to do a bit of research to modify or create a model.

Whereas it's unlikely in most programming jobs you would need to do any research into programming language design.

Re: Understanding Deep Learning

#87

Post by deepsquirrelnet »

Slix wrote: post #66
dafaqueue wrote: post #45

Earlier quoted context omitted.

I started off being really excited to learn, but as time went on I actually lost interest in the field. The whole thing is essentially curve fitting. The ML field is essentially an art more than a science and it's all about tricks and intuitions on different ways of getting that best fit curve. From this angle the whole field got way less interesting. The field has nothing deeper or more insightful to offer beyond th…

Are deep learning and neural networks just curve fitting? I thought those were significantly different.

By that logic, anything that is predictive is curve fitting, including entire academic fields like physics and climatology. You could say that all automation is curve fitting. I don’t think there’s much to be gained by being that reductive.

From a technical standpoint, it’s not correct analogy either, because it assumes you have a curve to fit. What curve is language? What’s curve is images? No answer, because there isn’t one. Deep learning is about modeling complex behaviors, not curve fitting. Images and language for instance are based in social and cultural patterns and not intrinsic curves to be fit.

At best, it’s an imprecise statement. But I’d disagree entirely.

Re: Understanding Deep Learning

#88

Post by rising-sky »

3abiton wrote: post #77
martingoodson wrote: post #68

Most comments here are in one of two camps: 1) you don't need to know any of this stuff, you can make AI systems without this knowledge, or 2) you need this foundational knowledge to really understand what's going on. Both perspectives are correct. The field is bifurcating into two different skill sets: ML engineer and ML scientist (or researcher). It's great to have both types on a team. The scientists will be too s…

This sounds like a sell-pitch for an AI scientist.

This sounds like a dont-buy-pitch for an AI engineer...

The point the commenter is making is that both schools of thought in the comments are valuable and unless you perform both roles, i.e. an engineer who is familiar with the scientific foundations, both are symbiotic and not in contention.

Re: Understanding Deep Learning

#89

Post by whatever_yeah »

mnky9800n wrote: post #19
ksherlock wrote: post #13

Earlier quoted context omitted.

Maybe last week's drama should have been a left-pad moment. For many things you can train your own NN and be just as good without being dependent on internet access, third parties, etc. Knowing how things work should give you insight into using them better.

Which drama of last week are you referring to? The one about the openai guy saying it's all just the data set? Or something else?

I must have missed the "dataset" news you're referring to, could you elucidate?

Re: Understanding Deep Learning

#90

Post by strikelaserclaw »

Slix wrote: post #67

If I start now and start reading up on AI, will I become anything close to an expert? I'm worried that I'm starting a journey that requires a Master's or PhD.

The only guide post to use in this world of ever increasing information to learn is to ask yourself "do i find learning this stuff enjoyable?", questions like "can i become an expert" are vague and not good guideposts.
Post reply on HN