Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cutting Edge Deep Learning for Coders, Part 2

course.fast.ai

Post reply on HN
Print view View original

61–70 of 77 posts

Re: Cutting Edge Deep Learning for Coders, Part 2

#61

Post by jph00 »

DrNuke wrote: post #36

Hi Jeremy, Thanks again to you and Dr Rachel for these amazing courses and the support to the community. My question is the following: are you thinking of an organic fast.ai connected hub or foundry acting as a distillator, that is disseminating & implementing in / with / through the fast.ai library the most interesting research papers coming out day in day out from arxiv.org and events? It is nearly impossible to fo…

We're certainly looking for ways to help develop the community. Obviously http://forums.fast.ai is a good starting point, and is a very active and supportive community nowadays. But I'm interested in finding ways to support face-to-face communities too. Some students are already doing that - for instance AI Saturdays (https://nurture.ai/ai-saturdays) has study groups in 50 cities now, and I'm told ~5000 people have been involved in them.

I'm always open for ideas in other ways we can help people get involved and stay up to date. And I'm very interested in supporting students who are trying to build these things in their local community.

Re: Cutting Edge Deep Learning for Coders, Part 2

#62

Post by jph00 »

pixelHD wrote: post #46
YeGoblynQueenne wrote: post #35

>> Welcome to the new 2018 edition of fast.ai's second 7 week course, Cutting Edge Deep Learning For Coders, Part 2, where you'll learn the latest developments in deep learning, how to read and implement new academic papers, and how to solve challenging end-to-end problems such as natural language translation. I would really like to know how to solve natural language translation. I think everyone would. Many people h…

Well, I looked at the summary, and they're implementing a Seq2Seq model for this. It is what I think of as an archetype for machine translation and chat bot tasks. Quite a few new network architectures in this space have been updates to this model, which uses an RNN encoder and a decoder, along with attention between them and a beam search for better results. I wouldn't call this model a solution for natural language…

Yup it's multi-layer bidir seq2seq with attention, and a few tricks like teacher forcing. Same as Google Translate. Their version takes a long time to train on a lot of GPUs, so we simplify it by using less layers, and a smaller, simplified corpus (it only contains questions, and limits them to 30 words long).

By "solve end-to-end problems" I only mean that we show how to do the whole process from beginning to end - I didn't mean to imply that the final model would be human-equivalent or perfect or anything like that.

Re: Cutting Edge Deep Learning for Coders, Part 2

#63

Post by pixelHD »

jph00 wrote: post #62
pixelHD wrote: post #46

Earlier quoted context omitted.

Well, I looked at the summary, and they're implementing a Seq2Seq model for this. It is what I think of as an archetype for machine translation and chat bot tasks. Quite a few new network architectures in this space have been updates to this model, which uses an RNN encoder and a decoder, along with attention between them and a beam search for better results. I wouldn't call this model a solution for natural language…

Yup it's multi-layer bidir seq2seq with attention, and a few tricks like teacher forcing. Same as Google Translate. Their version takes a long time to train on a lot of GPUs, so we simplify it by using less layers, and a smaller, simplified corpus (it only contains questions, and limits them to 30 words long). By "solve end-to-end problems" I only mean that we show how to do the whole process from beginning to end -…

Oh, nice!

Yeah, I understood the intention behind that statement. Great work with the course!

While you're here, what do you think about using temporal convolution for sequence tasks? I've read a few articles, this particular one by my professor comes to mind now [0], which say CNNs could work extremely well for the tasks traditionally done with RNNs. A recent paper by the people at Google Brain [1] mentioned that their CNN with attention network beats traditional RNN approaches. More surprising is that the network is 130+ layers deep, and yet trains faster than RNNs. Do you think we can potentially switch most machine translation tasks to CNNs?

[0]: https://towardsdatascience.com/the-fall-of-rnn-lstm-2d1594c7... [1]: https://twitter.com/lmthang/status/989261575482560513

Re: Cutting Edge Deep Learning for Coders, Part 2

#64

Post by ScottBurson »

jph00 wrote: post #59
zengid wrote: post #54

Earlier quoted context omitted.

Thanks Jeremy (and Rachel) for everything y'all have done; I learned a lot just from watching part 1! Somewhat tangential question, but how do you feel about sticking with PyTorch for future lessons/research work? What are your thoughts on the new Swift integration with TensorFlow?

We're so early in the software and tools cycle for DL that it's unlikely anything that exists today will remain relevant in its current form for very long. You can see that every couple of years the entire DL ecosystem is getting replaced - I don't think that will change. I'm glad you asked about Swift. It's one of the two directions I'm most excited about for the next stage of DL software (the other being javascript…

Here's a vote for Julia over Swift.

Re: Cutting Edge Deep Learning for Coders, Part 2

#65

Post by jph00 »

pixelHD wrote: post #63
jph00 wrote: post #62

Earlier quoted context omitted.

Yup it's multi-layer bidir seq2seq with attention, and a few tricks like teacher forcing. Same as Google Translate. Their version takes a long time to train on a lot of GPUs, so we simplify it by using less layers, and a smaller, simplified corpus (it only contains questions, and limits them to 30 words long). By "solve end-to-end problems" I only mean that we show how to do the whole process from beginning to end -…

Oh, nice! Yeah, I understood the intention behind that statement. Great work with the course! While you're here, what do you think about using temporal convolution for sequence tasks? I've read a few articles, this particular one by my professor comes to mind now [0], which say CNNs could work extremely well for the tasks traditionally done with RNNs. A recent paper by the people at Google Brain [1] mentioned that th…

Yup potentially. The jury is still out on which will win, or whether both have their place!

Re: Cutting Edge Deep Learning for Coders, Part 2

#66

Post by apohn »

gcmac wrote: post #20
apohn wrote: post #18

Jeremy - First of all, I just want to say thanks to you and Rachel for the course(s). I worked through part 1 (2017 version) last year and it was fantastic. I've got part 2 on my to-do list for this year. Looking at the 2018 part 1 course, quite a lot has changed (from Keras to Pytorch?) from a code perspective in comparison to the 2017 part 1 course. If I want to start part 2, does it make sense to go-ahead with the…

Not Jeremy (obviously) but having taken both versions of the course I would definitely recommend jumping into the 2018 version. The core content is the same (image recognition, recommender systems, NLP) but the 2018 version uses more cutting edge tools and has a tighter feel (as a v2 of anything usually does).

Thanks. I'll go straight to 2018 part 2. Looking forward to it!

Re: Cutting Edge Deep Learning for Coders, Part 2

#67

Post by rashkov »

bitL wrote: post #56
rashkov wrote: post #48

Earlier quoted context omitted.

Since you appreciate candid feedback, I would like to explain why I down-voted you. Your post comes off as super rude and dismissive. You may have some valid criticism to give but, take more care in how you deliver it. It's a bad look for HN.

I appreciate your feedback! :) It's difficult to convey mental state over a single line of text, giving rise to various interpretations that weren't in the intent. Imagine I am your best friend and I say the same in a playful fashion - would that be more acceptable?

Absolutely! That's actually in the hacker news commenting guidelines: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith." https://news.ycombinator.com/newsguidelines.html

So to answer you question, yes, it would be more acceptable if you were saying that to me in person. In fact, I agree with you about the flow of the lectures, and I was looking for someone to bring that up and I'm glad that it's getting discussed here.

However, Jeremy and Rachel are both reading these comments and we should strive to provide thoughtful, fleshed out feedback. The work they've done on Fast.ai is a tremendous lift, and deserves more than a drive-by comment.

Re: Cutting Edge Deep Learning for Coders, Part 2

#68

Post by geezerjay »

jph00 wrote: post #58
geezerjay wrote: post #53

Earlier quoted context omitted.

> because it felt like I was pounding my head on the wall trying to learn abstract math when what I really wanted to do was build stuff. But you do need to know abstract math to build stuff, particularly if what you want to build is deep learning stuff, both models and implementations. For example, how do you expect to understand how to minimize an utility function if you have no idea of what a gradient is, how you c…

> For example, how do you expect to understand how to minimize an utility function if you have no idea of what a gradient is, how you calculate it, and why you want to descend through it. The course teaches all those things - as the comment you're replying to states, you go deeper and deeper during the course to understand all the details. There's been a lot of research into teaching strategies that shows that this i…

> There's been a lot of research into teaching strategies that shows that this is often a more effective approach for many people than the bottom up approach widely used in math and CS.

I seriously doubt that anyone can effectively learn linear algebra, multivariate calculus, optimization and regression models from an onlone tutorial on deep learning. These are subjects whose basics alone take multiple semester-long courses. If a bottom-down approach was remotely effective, no one would bother teaching the basics.

Re: Cutting Edge Deep Learning for Coders, Part 2

#69

Post by neves »

bitL wrote: post #39
ghufran_syed wrote: post #30

Earlier quoted context omitted.

Personally, I find the questions helpful, and for everyone who's watching on video, you can skip whatever portion you like, can't you?

The thing is you can visibly see how it interrupts Jeremy; he is in the middle of explaining something, then has the face of a surprised person, loses context for a few seconds etc. Better IMO would be if he just finished a small section in its entirety then did Q&A, instead of allowing himself being interrupted all the time. And often those questions are missing the point, which is to be expected with newbies, so th…

I like the questions. Some of the doubts I also have. They improve a lot the course.

Re: Cutting Edge Deep Learning for Coders, Part 2

#70

Post by nonytoms »

Hire a professional hacker that can help you with all your hacking problems.Clear criminal records, university upgrade, Facebook hack, instagram hack, WhatsApp hack, bank transfer,credit card top etc Contact us via geraldhacks30@gmail.com or ICQ: 703132619 or WhatsApp +14696660879
Post reply on HN