Here is how it's going for me. For background, over the last few months I've very quickly become adept in deep learning, being able to understand current research papers, read through textbooks, implement my own models with Tensorflow and other libraries, and train real models on remote servers (e.g. AWS). For reference, I have an engineering background but no formal schooling passed an undergraduate degree.
The situation: my early-stage startup is fundraising right now, which can be kind of a time sink. Lots of accelerator/grant/angel applications. There's a good chance we hit our seed round, but also a good chance I'm unemployed next quarter when/if runway runs out.
In either case, I decided that AI, specifically deep learning, would be incredibly important to my career. The startup will need the expertise in the future (so I'll have to understand how to hire people with it), and should I need to find another job in a few months, this is a pretty cool field to learn and I find the work enjoyable (previously was a data scientist but foused more on vanilla regression and convex methods).
Therefore since November I've portioned out 20 hrs/week to the startup focusing on its fundraising and BD needs, which leaves a whole lot of other hours for skills development. Here has been roughly my curriculum:
- Mathematics review, and basic neural networks. For this I went over multivariable calculus and lin alg, which I've always been fairly strong, by essentially trying to derive the backpropagation derivatives for simple vanilla neural networks. Then make sure I understand derivatives and matrix data organization for convolution, which is a key component of modern ML.
Sources: pencil and paper, and lots of Google to answer any of my questions.
Time: 1-2 weeks.
- CS231n online course: http://cs231n.stanford.edu/
Great summary of modern methods in deep learning, plus more foundational level stuff. I read through all the lecture content and made sure I could work through derivations, because for me at least this cements technical understanding. Some of them are sort of tedious, e.g. manual RNN backprop. Also this course has great and simple software examples, I read through the code to make sure I understood the numerical computation and data organization parts. I also ran a few software examples and played around with parameters for fun (and learning).
Time: 1 month.
- Reading research papers (and online lectures) on applications that interest me. For this phase, I found 10 initial research papers that interested me. The topics for myself included image classification (starting w/ classic 2012 Hinton paper), reinforcement learning, robotics applications, video prediction. This step was harder, can be like learning a new language. Not every paper is going to make sense at first. But go through enough of them and you'll build up familiarity.
Sources: can start by searching through reddit.com/r/machinelearning
Time: 2 weeks.
- Learning software frameworks. From the above step I came up with my own small sample problem related to stuff I read that I could test even on my weak laptop (remember, training these big networks requires big computing power). So in this step I started researching different frameworks, and settled on starting a small project with Keras.
Sources: google around for deep learning libraries, read up on them, see what you like, and most importantly, have a motivating sample problem that you wanna code up.
Time: 2 weeks.
- Harder problems, more software, more papers. This is where I'm at now, it's sort of like an iterative research loop where I 1) come up with new problems I want to solve, 2) learn more about the software I need to implement it, and 3) search more prior work to gain insights on how I can solve the harder problems. In particular, I've switched over to learning and using Tensorflow, and also learning how to use AWS for stronger computing. So I had to dust off some linux scripting and command line skills too. Like I said, this is fairly iterative and probably closer to "modern research" where learning from my (virtual) peers and experimentation and production are closely linked.
Time: from the last month to present.
Overall, in the last 3 months or so at about 30 hrs/week I've added an extremely powerful new skillset to my arsenal that I've been meaning to do for quite some time. I can understand 90% of all modern research in the field, and create useful software to solve data-driven problems. Completely for free as well, aside from the $0.81/hr I pay to AWS for training some networks overnight. This is the type of thing I'd have wanted from a Master's (or even PhD) program, but who wants to go back to school...
Hope this helps someone :) Remember, AI/ML is more approachable than most people think, you just need to start with a solid mathematics background. After that you'll be flying, the field is relatively quick to learn, especially if you like learning through doing.