Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cutting Edge Deep Learning for Coders, Part 2

course.fast.ai

Post reply on HN
Print view View original

51–60 of 77 posts

Re: Cutting Edge Deep Learning for Coders, Part 2

#51

Post by jph00 »

RSchaeffer wrote: post #34
jph00 wrote: post #2

Jeremy from fast.ai here. I've posted a summary of the course materials for anyone who's interested : http://www.fast.ai/2018/05/07/part2-launch/ Let me know if you have any questions about the material or approach. There's also a discussion for the course here: http://forums.fast.ai/c/part2-v2

Hey! Thanks for these great courses and materials! How much additional math (beyond high school and introductory college courses) do these courses teach? For example, if I were to take both courses, would I be able to understand the papers published by Surya Ganguli (e.g., The Emergence of Spectral Universality in Deep Networks, Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net)?

Ganguli's papers are at the more "math-y" end of the spectrum when it comes to DL papers. So don't worry if you're finding them a bit unapproachable.

In part 2 of the course I provide quite a bit of advice about how to approach papers in general, and you'll get plenty of practice in reading and implementing papers - but we don't cover the specific math in this particular papers.

My view is the best approach to the math in papers is to generally learn what you need as you get there. It's nearly impossible to know all the math that covers every paper you'll come across, but if you learn the meta-skill of how to learn it on demand, then you'll be just fine! :)

Re: Cutting Edge Deep Learning for Coders, Part 2

#52

Post by jph00 »

bitL wrote: post #39
ghufran_syed wrote: post #30

Earlier quoted context omitted.

Personally, I find the questions helpful, and for everyone who's watching on video, you can skip whatever portion you like, can't you?

The thing is you can visibly see how it interrupts Jeremy; he is in the middle of explaining something, then has the face of a surprised person, loses context for a few seconds etc. Better IMO would be if he just finished a small section in its entirety then did Q&A, instead of allowing himself being interrupted all the time. And often those questions are missing the point, which is to be expected with newbies, so th…

Rachel's mic is always off until I turn it on. So if she's asking a question, it's only after she's visually indicated that she wishes to do so, and I've found a time in my presentation that I'm ready to take it. So I'm literally never being interrupted, and can't be surprised by the fact that she's asking a question (although I may well be surprised by the content).

I do try to limit the time I wait to take a question, since I don't want to move on with a topic where I've failed to properly explain some foundational piece.

Re: Cutting Edge Deep Learning for Coders, Part 2

#53

Post by geezerjay »

gumballhead wrote: post #50
jph00 wrote: post #2

Jeremy from fast.ai here. I've posted a summary of the course materials for anyone who's interested : http://www.fast.ai/2018/05/07/part2-launch/ Let me know if you have any questions about the material or approach. There's also a discussion for the course here: http://forums.fast.ai/c/part2-v2

Thank you for creating these courses and putting them online! I started part 1 a few weeks ago. I couldn't agree more with your teaching style. I'm someone who started college as a computer science major and couldn't finish, because it felt like I was pounding my head on the wall trying to learn abstract math when what I really wanted to do was build stuff. I finished my degree with economics, went to work in finance…

> because it felt like I was pounding my head on the wall trying to learn abstract math when what I really wanted to do was build stuff.

But you do need to know abstract math to build stuff, particularly if what you want to build is deep learning stuff, both models and implementations.

For example, how do you expect to understand how to minimize an utility function if you have no idea of what a gradient is, how you calculate it, and why you want to descend through it.

Re: Cutting Edge Deep Learning for Coders, Part 2

#54

Post by zengid »

jph00 wrote: post #2

Jeremy from fast.ai here. I've posted a summary of the course materials for anyone who's interested : http://www.fast.ai/2018/05/07/part2-launch/ Let me know if you have any questions about the material or approach. There's also a discussion for the course here: http://forums.fast.ai/c/part2-v2

Thanks Jeremy (and Rachel) for everything y'all have done; I learned a lot just from watching part 1!

Somewhat tangential question, but how do you feel about sticking with PyTorch for future lessons/research work? What are your thoughts on the new Swift integration with TensorFlow?

Re: Cutting Edge Deep Learning for Coders, Part 2

#55

Post by tedsanders »

Question: What is the job market like for folks who spend two months learning deep learning, either through a class like fast.ai or a bootcamp like Insight AI? What sorts of companies hire these people and how much do they pay? I know it varies, but I'm just trying to get a sense to calibrate my expectations of the current market. Thanks for any perspective anyone can offer.

Re: Cutting Edge Deep Learning for Coders, Part 2

#56

Post by bitL »

rashkov wrote: post #48
bitL wrote: post #22

Earlier quoted context omitted.

I am a firm believer in candid feedback; I talked to multiple people taking fast.ai and all of them took it as disturbing the flow of lectures. I really like fast.ai, it's an excellent hands-on course, so please don't get offended.

Since you appreciate candid feedback, I would like to explain why I down-voted you. Your post comes off as super rude and dismissive. You may have some valid criticism to give but, take more care in how you deliver it. It's a bad look for HN.

I appreciate your feedback! :) It's difficult to convey mental state over a single line of text, giving rise to various interpretations that weren't in the intent. Imagine I am your best friend and I say the same in a playful fashion - would that be more acceptable?

Re: Cutting Edge Deep Learning for Coders, Part 2

#57

Post by DrNuke »

tedsanders wrote: post #55

Question: What is the job market like for folks who spend two months learning deep learning, either through a class like fast.ai or a bootcamp like Insight AI? What sorts of companies hire these people and how much do they pay? I know it varies, but I'm just trying to get a sense to calibrate my expectations of the current market. Thanks for any perspective anyone can offer.

As an outsider and / or away from the SV, where such an environment thrives and opportuinities are a few dozens, your mileage may vary a lot. Personal network is always better to refer to than shots in the dark though. In general, you might need showcasing your knowledge through marketable case studies in a sub-domain you are interested in, so that interviews will come from a shared context.

Re: Cutting Edge Deep Learning for Coders, Part 2

#58

Post by jph00 »

geezerjay wrote: post #53
gumballhead wrote: post #50

Earlier quoted context omitted.

Thank you for creating these courses and putting them online! I started part 1 a few weeks ago. I couldn't agree more with your teaching style. I'm someone who started college as a computer science major and couldn't finish, because it felt like I was pounding my head on the wall trying to learn abstract math when what I really wanted to do was build stuff. I finished my degree with economics, went to work in finance…

> because it felt like I was pounding my head on the wall trying to learn abstract math when what I really wanted to do was build stuff. But you do need to know abstract math to build stuff, particularly if what you want to build is deep learning stuff, both models and implementations. For example, how do you expect to understand how to minimize an utility function if you have no idea of what a gradient is, how you c…

> For example, how do you expect to understand how to minimize an utility function if you have no idea of what a gradient is, how you calculate it, and why you want to descend through it.

The course teaches all those things - as the comment you're replying to states, you go deeper and deeper during the course to understand all the details.

There's been a lot of research into teaching strategies that shows that this is often a more effective approach for many people than the bottom up approach widely used in math and CS. It doesn't mean that you learn any less of the foundations - just that it's in a different order.

Re: Cutting Edge Deep Learning for Coders, Part 2

#59

Post by jph00 »

zengid wrote: post #54
jph00 wrote: post #2

Jeremy from fast.ai here. I've posted a summary of the course materials for anyone who's interested : http://www.fast.ai/2018/05/07/part2-launch/ Let me know if you have any questions about the material or approach. There's also a discussion for the course here: http://forums.fast.ai/c/part2-v2

Thanks Jeremy (and Rachel) for everything y'all have done; I learned a lot just from watching part 1! Somewhat tangential question, but how do you feel about sticking with PyTorch for future lessons/research work? What are your thoughts on the new Swift integration with TensorFlow?

We're so early in the software and tools cycle for DL that it's unlikely anything that exists today will remain relevant in its current form for very long. You can see that every couple of years the entire DL ecosystem is getting replaced - I don't think that will change.

I'm glad you asked about Swift. It's one of the two directions I'm most excited about for the next stage of DL software (the other being javascript!) Swift is a much better language than Python, and having autograd and tensor types built in will be quite transformative. But it has nothing like the same data science library and tooling, so it'll take time to be a real option for most people.

Javascript has the potential to make DL accessible even for those without a discrete GPU, and allows people to get started even without installing a tool-chain. It also allows more work to be pushed to the client - possibly even on mobile devices.

Re: Cutting Edge Deep Learning for Coders, Part 2

#60

Post by jph00 »

DrNuke wrote: post #57
tedsanders wrote: post #55

Question: What is the job market like for folks who spend two months learning deep learning, either through a class like fast.ai or a bootcamp like Insight AI? What sorts of companies hire these people and how much do they pay? I know it varies, but I'm just trying to get a sense to calibrate my expectations of the current market. Thanks for any perspective anyone can offer.

As an outsider and / or away from the SV, where such an environment thrives and opportuinities are a few dozens, your mileage may vary a lot. Personal network is always better to refer to than shots in the dark though. In general, you might need showcasing your knowledge through marketable case studies in a sub-domain you are interested in, so that interviews will come from a shared context.

Exactly right - it's all about your portfolio. We spend a lot of time throughout the course talking about ways to build that portfolio, and folks on the forums support each other with feedback and help on these projects.

Completing a MOOC isn't going to help you land an interview for a job, generally speaking (although if it's any good, it should certainly help you once you do get an interview!) However if you use your study time to, for example, create useful projects that you make available on github, web apps that you host on heroku, deep technical blog posts you post on medium, etc then you should definitely get plenty of interview opportunities.

I've seen many many students go through this process, so I know it works. That includes students that hadn't been getting interviews or job offers - until I convinced them to spend time building a portfolio, and then they got multiple offers from top companies.

(There are some companies - not too many thankfully - that strictly require a PhD for DL applicants. I've noticed that such companies, that focus on credentials over actual work output, seem to overlap a lot with companies that turn out to have toxic cultures. So I'm not sure you should worry too much about them...)

Post reply on HN