Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fast.ai Part 2: Deep Learning from the Foundations

course.fast.ai

Post reply on HN
Print view View original

21–30 of 33 posts

Re: Fast.ai Part 2: Deep Learning from the Foundations

#21

Post by jph00 »

enraged_camel wrote: post #14

I've been meaning to learn ML/DL. My problem is that I can't think of any use cases for it, either in my personal life or work life. I understand that the technologies have a lot of potential, and are currently used in many major projects and endeavors. I just keep drawing a blank when trying to answer the question, "what would I do with this?" If I am someone who is goal-driven (rather than, say, learning something…

You might find it helpful to see what other people are doing with deep learning - see if any of it seems relevant to your interests. For instance, here's some examples of folks from diverse backgrounds using DL in their domains of interest: https://www.fast.ai/2019/02/21/dl-projects/ . Or here's a rather deep rabbit hole - hundreds of replies from people showing their learning projects: https://forums.fast.ai/t/share-your-work-here/27676 .

I find it useful to think of DL as just another way to get computers to do what you want. Rather than focusing on control flow and setting/reading variables, you focus on providing examples to learn from. Both approaches can do many of the same things, but each has areas that they're better at. Eg DL is better for things that are hard to explain just how you do them (e.g. seeing pictures, hearing sounds, reading text) and traditional coding is better for things that need specific logical steps. A combination of the two is often best for solving end-to-end problems in practice.

Re: Fast.ai Part 2: Deep Learning from the Foundations

#22

Post by ScoutOrgo »

dpflan wrote: post #7

Can someone who has completed Fast.ai courses comment on the experience and then how you used the newly gained knowledge?

I've gone through all previous iterations of their courses, so have familiarity with using pytorch and the fastai library, but haven't done the best at keeping up with their rapid pace. This course helped me understand the building blocks of the library, which in turn helped me out in a few projects.

For one project, I adopted what Jeremy taught about building code for processes, which our team has then used for a couple other things as well. For another project, I was able to easily put together the code for training on a set of data with a somewhat complex structure.

Re: Fast.ai Part 2: Deep Learning from the Foundations

#23

Post by codesushi42 »

jph00 wrote: post #20
codesushi42 wrote: post #19

Earlier quoted context omitted.

That doesn't really make sense. The code you implement in Swift when in Jupyter needs to also be available at runtime to execute. Meaning you can do the exact same thing in Python, because your model architecture is going to be embedded in the exported model. For custom kernel code, what you really want to use is a custom TF op. But I doubt that's what you're getting at anyway, because that's for more advanced use ca…

The goal is to allow Swift to be used for writing MLIR and XLA kernels. The new LazyTensor under development already allows for fused XLA operations to be created in Swift. There's an awful lot you can do in Swift which is very very hard to do properly in Python. I've got a bit more background on this here: https://www.fast.ai/2019/03/06/fastai-swift/ Edit: HN isn't letting me reply deeper, so I'll reply to "what are…

Thanks. What is the advantage of using Swift over implementing a custom TF op in C++, and using its generated Python wrapper in Jupyter? Just not having to deal with C++?

Re: Fast.ai Part 2: Deep Learning from the Foundations

#24

Post by kevinbird15 »

dpflan wrote: post #7

Can someone who has completed Fast.ai courses comment on the experience and then how you used the newly gained knowledge?

I have taken all of the fast.ai courses available and they have been life-changing for me. The style of teaching that Jeremy uses really clicks with my learning style (Code then explain) and I have gone from knowing nothing about machine learning to finishing pretty highly on a few different competitions mostly involving NLP and Computer vision. I have been working on Super Resolution more recently and I started working as a Data Scientist at work which has been great. I actually didn't even have a ton of programming experience when I started fastai 3 years ago, but now I am pretty confident when I need to code something. I feel like I have moved through the stages from script-kiddie of data science to a point where I can actually intuitively understand what is happening and trust my instincts to change well-established architectures in ways that it works better for the problem that I am focusing on.

Re: Fast.ai Part 2: Deep Learning from the Foundations

#26

Post by kevinbird15 »

enraged_camel wrote: post #14

I've been meaning to learn ML/DL. My problem is that I can't think of any use cases for it, either in my personal life or work life. I understand that the technologies have a lot of potential, and are currently used in many major projects and endeavors. I just keep drawing a blank when trying to answer the question, "what would I do with this?" If I am someone who is goal-driven (rather than, say, learning something…

I agree with looking at projects that other people have done. The other thing I would say is: just do part 1 version 3. That will give you a really good idea of what is currently being done in this area and if it isn't interesting to you at that point, you will at least have enough information to know that for sure. Make sure you don't just watch the videos though. You have to actually implement something after each lesson.

Re: Fast.ai Part 2: Deep Learning from the Foundations

#28

Post by codesushi42 »

jph00 wrote: post #20
codesushi42 wrote: post #19

Earlier quoted context omitted.

That doesn't really make sense. The code you implement in Swift when in Jupyter needs to also be available at runtime to execute. Meaning you can do the exact same thing in Python, because your model architecture is going to be embedded in the exported model. For custom kernel code, what you really want to use is a custom TF op. But I doubt that's what you're getting at anyway, because that's for more advanced use ca…

The goal is to allow Swift to be used for writing MLIR and XLA kernels. The new LazyTensor under development already allows for fused XLA operations to be created in Swift. There's an awful lot you can do in Swift which is very very hard to do properly in Python. I've got a bit more background on this here: https://www.fast.ai/2019/03/06/fastai-swift/ Edit: HN isn't letting me reply deeper, so I'll reply to "what are…

MLIR is going to be orthogonal to C++ performance. You're talking about the efficiency of an intermediate representation. But that IR turns into TF opcodes, which then need to execute natively.

You can achieve the same efficiency in C++ via a custom TF op. You end up with native instructions either way. And you have access to the entire memmapped model in the op, allowing you to do as you please.

You're only debugging in one place, because the same C++ code runs during training in your notebook that runs during inference on your device.

You're getting ease of implementation and usage of Swift. But your code will be less portable; you won't be able to run the same model on Android or on the server. And there's not necessarily any performance benefit over doing the same in C++, definitely not if your kernel is simple.

Re: Fast.ai Part 2: Deep Learning from the Foundations

#29

Post by saynay »

codesushi42 wrote: post #23
jph00 wrote: post #20

Earlier quoted context omitted.

The goal is to allow Swift to be used for writing MLIR and XLA kernels. The new LazyTensor under development already allows for fused XLA operations to be created in Swift. There's an awful lot you can do in Swift which is very very hard to do properly in Python. I've got a bit more background on this here: https://www.fast.ai/2019/03/06/fastai-swift/ Edit: HN isn't letting me reply deeper, so I'll reply to "what are…

Thanks. What is the advantage of using Swift over implementing a custom TF op in C++, and using its generated Python wrapper in Jupyter? Just not having to deal with C++?

Not needing to know both Python and C++.

If you need to debug something, you don't need to use some mixture of pdb and gdb.

Swift is a relatively young language, meaning it does not (yet) have weird hairy bits to work around design decisions made 20 years ago.

Similarly, Swift is still getting defined in many areas. There is (theoretically) the opportunity to influence language design decisions to patterns that mesh better with ML needs.

This is largely my paraphrasing of the reasons stated by Jeremy https://www.fast.ai/2019/03/06/fastai-swift/

Re: Fast.ai Part 2: Deep Learning from the Foundations

#30

Post by codesushi42 »

saynay wrote: post #29
codesushi42 wrote: post #23

Earlier quoted context omitted.

Thanks. What is the advantage of using Swift over implementing a custom TF op in C++, and using its generated Python wrapper in Jupyter? Just not having to deal with C++?

Not needing to know both Python and C++. If you need to debug something, you don't need to use some mixture of pdb and gdb. Swift is a relatively young language, meaning it does not (yet) have weird hairy bits to work around design decisions made 20 years ago. Similarly, Swift is still getting defined in many areas. There is (theoretically) the opportunity to influence language design decisions to patterns that mesh…

Yes, but at the cost of portability. You won't be able to run the same model on Android for instance.

You're still debugging in two places; in Swift and Python. Debugging Swift is probably easier than C++ though.

I think using Swift is a valid solution for special cases, but not the best solution for most cases. The TF authors already provide a suitable, general solution in the form of custom TF ops.

And if you don't need a custom kernel, and the chances are you don't, then stick with pure Python for maximum ease of use and portability.

Post reply on HN