Just know stuff (or, how to achieve success in a machine learning PhD)

Just know stuff (or, how to achieve success in a machine learning PhD)
kidger.site

Hear hear! How did ML get saddled with such awful notation?

I try to (re-)learn math periodically because I feel like I should, like how one ought to eat one's vegetables and one ought to exercise, but extrinsic motivation is the thing that's lacking. I usually end up on recreational math puzzles or something, before dropping it, since at least those are fun. Probably made three or four cracks at it in a decade, each has gone the same way.

I literally don't know what I'd use most of it for—I can't find that need, even when I try. I'm sure I could make something up just to have an excuse to apply what I was learning, but... why? Probably there are other jobs I could find where knowledge of math was absolutely key, but... why? I've been paid to write code for about 23 years, my pay's great, and I've repeatedly gained a reputation at companies for being the guy to go to for tough or low-level problems. But if you gave me an intro to linear algebra final or calculus 1 final, today, I'd be lucky to score 25% on either (hell, I'd be lucky to get anything right on the Calc final, but maybe there'd be a couple easy questions at the beginning—I've never, once, ever applied anything I learned in calc, for any purpose, so what little I knew about it to begin with is long gone)

However, "Just know stuff" I think is a secondary requirement (although an important one) to be successful. People really struggle to "just know stuff" if they aren't interested in the subject in the first place. People who aren't interested will settle for knowing only what things do, and not dive into how they work.

I am interested in this stuff, and actually self taught (bachelors in mathematics here). I have experience with a lot of stuff on this list, not through work or academia, or because I want to make money, but through fiddling on my desktop at home. I too got the "coveted tech job" as a machine learning engineer, but I never would have if I wasn't legitimately interested in this stuff, studying for fun in my spare time. I have seen lots of people fail to progress in this field because they _don't care_, they just want a good job.

Kind of off topic, but this is actually an integral part of my interviewing process. We give candidates a simple dataset to model, and we receive their script. The performance on the hold-out set is only weighted ~20%. The candidates ability to talk about their process, about the internals of the model they used, about the feature engineering quirks to work around model limitations, about their parameter tuning scheme - these conversations reveal how much someone is actually interested in the field, and is a great indicator for whether or not they are going to be a good contributor to the team. I've had candidates who couldn't tell me _anything_ about how the models they used actually worked. PhDs included!