Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cutting Edge Deep Learning for Coders, Part 2

course.fast.ai

Post reply on HN
Print view View original

71–77 of 77 posts

Re: Cutting Edge Deep Learning for Coders, Part 2

#71

Post by YeGoblynQueenne »

jph00 wrote: post #62
pixelHD wrote: post #46

Earlier quoted context omitted.

Well, I looked at the summary, and they're implementing a Seq2Seq model for this. It is what I think of as an archetype for machine translation and chat bot tasks. Quite a few new network architectures in this space have been updates to this model, which uses an RNN encoder and a decoder, along with attention between them and a beam search for better results. I wouldn't call this model a solution for natural language…

Yup it's multi-layer bidir seq2seq with attention, and a few tricks like teacher forcing. Same as Google Translate. Their version takes a long time to train on a lot of GPUs, so we simplify it by using less layers, and a smaller, simplified corpus (it only contains questions, and limits them to 30 words long). By "solve end-to-end problems" I only mean that we show how to do the whole process from beginning to end -…

>> By "solve end-to-end problems" I only mean that we show how to do the whole process from beginning to end (...)

Then why not write just that? What is the point of using language that implies you can teach people how to solve a very hard problem that nobody knows how to solve yet?

I find it extremely disreputable to claim to be able to accomplish feats that go far beyond the limits of current technology. That is the tactic of charlatans and snake oil salesmen, not of scientists and technologists.

Re: Cutting Edge Deep Learning for Coders, Part 2

#72

Post by YeGoblynQueenne »

stochastic_monk wrote: post #45
YeGoblynQueenne wrote: post #35

>> Welcome to the new 2018 edition of fast.ai's second 7 week course, Cutting Edge Deep Learning For Coders, Part 2, where you'll learn the latest developments in deep learning, how to read and implement new academic papers, and how to solve challenging end-to-end problems such as natural language translation. I would really like to know how to solve natural language translation. I think everyone would. Many people h…

Machine translation is not solved, but it's reached some surprisingly improved benchmarks for accuracy, so while it's a little presumptuous to call it solved, it's not the most egregious exaggeration I've heard about machine learning this week.

Everyone's doing it, so it's OK?

Re: Cutting Edge Deep Learning for Coders, Part 2

#73

Post by YeGoblynQueenne »

pixelHD wrote: post #46
YeGoblynQueenne wrote: post #35

>> Welcome to the new 2018 edition of fast.ai's second 7 week course, Cutting Edge Deep Learning For Coders, Part 2, where you'll learn the latest developments in deep learning, how to read and implement new academic papers, and how to solve challenging end-to-end problems such as natural language translation. I would really like to know how to solve natural language translation. I think everyone would. Many people h…

Well, I looked at the summary, and they're implementing a Seq2Seq model for this. It is what I think of as an archetype for machine translation and chat bot tasks. Quite a few new network architectures in this space have been updates to this model, which uses an RNN encoder and a decoder, along with attention between them and a beam search for better results. I wouldn't call this model a solution for natural language…

>> I wouldn't call this model a solution for natural language translation, nor would anyone else.

Well, the passage I quote, by fast.ai, does exactly that.

Re: Cutting Edge Deep Learning for Coders, Part 2

#74

Post by gumballhead »

geezerjay wrote: post #68
jph00 wrote: post #58

Earlier quoted context omitted.

> For example, how do you expect to understand how to minimize an utility function if you have no idea of what a gradient is, how you calculate it, and why you want to descend through it. The course teaches all those things - as the comment you're replying to states, you go deeper and deeper during the course to understand all the details. There's been a lot of research into teaching strategies that shows that this i…

> There's been a lot of research into teaching strategies that shows that this is often a more effective approach for many people than the bottom up approach widely used in math and CS. I seriously doubt that anyone can effectively learn linear algebra, multivariate calculus, optimization and regression models from an onlone tutorial on deep learning. These are subjects whose basics alone take multiple semester-long…

Do you think it's necessary to have a rigorous understanding of all of those topics before creating a machine learning model? And that you don't learn from interacting with it, even if you don't fully understand how it works? For machine learning in particular I think that's pretty ironic.

Re: Cutting Edge Deep Learning for Coders, Part 2

#75

Post by pixelHD »

YeGoblynQueenne wrote: post #73
pixelHD wrote: post #46

Earlier quoted context omitted.

Well, I looked at the summary, and they're implementing a Seq2Seq model for this. It is what I think of as an archetype for machine translation and chat bot tasks. Quite a few new network architectures in this space have been updates to this model, which uses an RNN encoder and a decoder, along with attention between them and a beam search for better results. I wouldn't call this model a solution for natural language…

>> I wouldn't call this model a solution for natural language translation, nor would anyone else. Well, the passage I quote, by fast.ai, does exactly that.

My bad

Re: Cutting Edge Deep Learning for Coders, Part 2

#76

Post by kriro »

I have nothing to say other than that I loved DL1 and I'll dive into DL2 right away. I really like the overall philosophy ("see DL is approachable and doable, just do it") and I love that they encourage you to read papers even if they seem hard and to translate math to code etc. etc. Fantastic job fast.ai team.

It's pretty damn amazing that they build a library (and also the new fastai.text) that goes one level of abstraction beyond PyTorch with the goal of implementing interesting/helpful papers ASAP and because they wanted to make their teaching even more efficient.

Re: Cutting Edge Deep Learning for Coders, Part 2

#77

Post by olingern »

geezerjay wrote: post #68
jph00 wrote: post #58

Earlier quoted context omitted.

> For example, how do you expect to understand how to minimize an utility function if you have no idea of what a gradient is, how you calculate it, and why you want to descend through it. The course teaches all those things - as the comment you're replying to states, you go deeper and deeper during the course to understand all the details. There's been a lot of research into teaching strategies that shows that this i…

> There's been a lot of research into teaching strategies that shows that this is often a more effective approach for many people than the bottom up approach widely used in math and CS. I seriously doubt that anyone can effectively learn linear algebra, multivariate calculus, optimization and regression models from an onlone tutorial on deep learning. These are subjects whose basics alone take multiple semester-long…

> If a bottom-down approach was remotely effective, no one would bother teaching the basics.

This is a purist approach.

Sometimes, you sacrifice the details to broaden the audience. This has the result of getting more people interested.

Post reply on HN