11–20 of 108 posts
Maybe I should start a PhD in ML as a university drop out, it sounds like a list of basic stuff taught within the first two years of any math/CS program, but I might miss something as it's far from being detailed...
Now is a terrible time to start a PhD in ML. If you do a CS PhD, pick literally any other subfield.
The Ph.D. period is the time when you have some time available to acquire additional skills, and my advice is: try to strengthen those aspects of your education where you currently have the most glaring deficits. For instance, take a statistics course if that is your weak spot, or learn a foreign language if you have evaded that topic so far in your educational journey.
Read the main textbooks of your field and read and re-read ALL relevant papers for your actual Ph.D. topic, once you've been able to identify it (which may well take you most or all of your first year). Take details notes because nobody can remember most of that much highly concentrated advanced material. Try to find gaps: ask questions and find out if people have tried to answer them yet or not. Interact with others e.g. at conferences, after meeting some people there by attending and networking in a prior year.
"Just know stuff" deeply resonates. Even much below the author's level, as is the case for me, technical knowledge dominates everything else by orders of magnitude. I'm still appalled at how people can manage to gather the courage to utter they don't need math. A great list, too. I guess I have stuff to brush up on for the next 10 to 20 years?
> I'm still appalled at how people can manage to gather the courage to utter they don't need math. I try to (re-)learn math periodically because I feel like I should, like how one ought to eat one's vegetables and one ought to exercise, but extrinsic motivation is the thing that's lacking. I usually end up on recreational math puzzles or something, before dropping it, since at least those are fun. Probably made three…
Personally, if it were “just relearn calculus and move from there” I think I’d be able to motivate myself. At that point I’m still pretty close to the problems and tasks that interest me. But the reality is I spent much of middle school and high school wasting my time doing things other than math, so in reality I’d have to go much further back and relearn all the prerequisite stuff, and the prerequisite stuff to that, etc. By that point Im so divorced from the reason I wanted to try and relearn math in the first places and I just get bored and give up.
I know most of the stuff listed there and do not have a published textbook or papers. It sounds like the author achieved success because they work hard on things they’re passionate about. Knowledge and published works are a byproduct of that.
Maybe you too could be a successful phd-student? :) At least you know the basics!
That's probably the optimal strategy. I'm skeptical of first-principles learning. It's great to immerse yourself in theory, but when you've gone all the way to "topology" you've probably gone beyond the limit of what most ML devs care about on a day-to-day basis.
It's still useful to know. I've applied lots of ideas from other fields. But can you force that knowledge by forcing yourself to study other fields? Maybe. We all have a finite amount of time though.
That said, lots of items on this list are key, and it'd be worth ranking them. There's no need to memorize formulas for Adam, but knowing the concept of momentum-per-weight is pretty crucial.
Earlier quoted context omitted.
Now is a terrible time to start a PhD in ML. If you do a CS PhD, pick literally any other subfield.
Why?
But if you actually care about ML -- I think it's one of the best things in the world -- go for it! ML has never been more accessible.
I didn't expect this to be as helpful as it actually was. Great list. Can anyone suggest me something similar for HPC domain?
For finance, firewall-to-firewall time was an important concept. (The time it takes a signal to get into your datacenter, be processed, then emit a signal back out.)
But e.g. massive data processing is an entirely different beast. Latency isn't too important, whereas parallelization is crucial.
So it's kind of hard to make a list without being pointed in a vague direction.
"Just know stuff" deeply resonates. Even much below the author's level, as is the case for me, technical knowledge dominates everything else by orders of magnitude. I'm still appalled at how people can manage to gather the courage to utter they don't need math. A great list, too. I guess I have stuff to brush up on for the next 10 to 20 years?
> I'm still appalled at how people can manage to gather the courage to utter they don't need math. I try to (re-)learn math periodically because I feel like I should, like how one ought to eat one's vegetables and one ought to exercise, but extrinsic motivation is the thing that's lacking. I usually end up on recreational math puzzles or something, before dropping it, since at least those are fun. Probably made three…
If all you have is a hammer, everything looks like a nail—but the converse of that is that if you have never seen a hammer, nails will be invisible and incomprehensible to you, they will just blend into the background of noise.
When I learn about something, I suddenly see it everywhere.
Hypothetically, if you wanted to learn it, I would recommend devoting the first hour of every day to it. Before your brain fully wakes up and starts asking why you are doing it, just do it.
I assume you have the standard CS background, with decent knowledge of discrete math, calculus, linear algebra. I would recommend starting with a graduate-level linear algebra textbook. A friend of mine used to say “linear algebra is the new addition”, it permeates everything, and knowing it well produces massive dividends.
Maybe I should start a PhD in ML as a university drop out, it sounds like a list of basic stuff taught within the first two years of any math/CS program, but I might miss something as it's far from being detailed...
CS often has the problem that the math basics are not teached as rigorously as it should (I think calculus I/II and linear algebra I/II should be exactly the same as the lectures for math). Then it misses measure theory (usually in calc III) and therefore you are going to miss solid fundamentals in probability. Optimization is also usually absent, it's sometimes squeezed into the numerical math. lecture but it shouldn't since numerics (or better scientific computing) is so important. You also want some statistics course.
Math usually lacks the whole machine learning canon, from SVM to NNs, from bayesian methods to statistical learning theory. I just see the same statistics lecture everywhere building upon probability and deriving good estimators and their properties and that's all. Also, you might miss CS basics like algorithms and struggle with the basics of scientific computing (programming in C, knowing all your matrix decompositions etc.). Also, no knowledge of git, how to navigate a server just with a shell etc.
I think you will either have to learn the missing parts after your undergraduate, for example as your master, first year in your phd, or be really lucky! I think most miss a significant chunk of the topics after completing their undergraduate.