Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Just know stuff (or, how to achieve success in a machine learning PhD)

kidger.site

Post reply on HN
Print view View original

31–40 of 108 posts

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#31

Post by steppi »

Knowing things is good, but I think the real benefit of a PhD is in developing the skill to learn new things quickly and fill in any gaps as needed. It’s important to get practice learning many things in depth but the goal isn’t to become a storehouse of knowledge, it’s to develop the ability assimilate existing knowledge and apply it to figuring new things out.

The amount of topics I’ve studied in depth dwarfs this list (and the same is certainly true of its author) but the set of things I could teach a class on today without preparation is much smaller. The important thing is that if I have a problem, I can use the impressions from all I’ve learned before to get a sense of where to look next. My memory isn’t great but it doesn’t matter because I can refresh, learn, and figure things out as needed.

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#32

Post by joxel »

yamtaddle wrote: post #25
Dr_Birdbrain wrote: post #19

Earlier quoted context omitted.

It’s one of those things where if you don’t have the math knowledge, the opportunities to apply it will be literally invisible to you. If all you have is a hammer, everything looks like a nail—but the converse of that is that if you have never seen a hammer, nails will be invisible and incomprehensible to you, they will just blend into the background of noise. When I learn about something, I suddenly see it everywher…

Why should I think I'll find uses this time, when I already burned time and money learning (some of) it once, and lost all that precisely because I never encountered any need for it? An hour a day is a huge time investment for something that's already failed to prove its worth once. Like I'm sure I could go find some jobs that I can't get now because I lack math skills, and I'm sure some (far from all!) of those pay…

Clearly there are people that don’t need to know math. You happen to be one of them, congratulations. Though I know I’d be bored out of my mind if I did software work that only used high school level math and logic.

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#34

Post by sillysaurusx »

extasia wrote: post #24
sillysaurusx wrote: post #15

If anyone's reading over this and feels "Gosh, I'll never be an ML dev; this is way too much": I don't know most of that list, and still manage to be a productive researcher. I learn what I need as I go. That's probably the optimal strategy. I'm skeptical of first-principles learning. It's great to immerse yourself in theory, but when you've gone all the way to "topology" you've probably gone beyond the limit of what…

The author and you seem to be talking to different audiences, you're talking about ML eng and OP is talking about ML researchers. Researchers absolutely need to know a lot, not necessarily all the way to topology or w/e but definitely the underlying mathematical principles in order to advance the field (IMO).

You're right, I wasn't clear. But I'm a full-time ML researcher. In terms of advancing the field, my contributions so far have been modest, but they're there. Some of my favorite ideas were swarm training (https://battle.shawwn.com/swarm-training-v01a.pdf), stop loss for stabilizing GAN training (https://twitter.com/search?q=from%3Atheshawwn%20stop%20loss&...), and getting GPT to play chess (https://www.theregister.com/2020/01/10/gpt2_chess/) back when that was a shocking idea. And in terms of cited research, https://arxiv.org/abs/2101.00027 has been the most impactful.

There's significant overlap between ML research and ML dev. If I can do it without most of that list, it should give people here some hope of joining the field without needing to immerse themselves in theory.

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#35

Post by lostmsu »

While this surely seems required for a PhD program in one of the top universities, more down-to-earth programs would require you only about 1/4-1/2 of whats in the list depending on the area of research. For instance, graph neural networks are very niche.

And if you just want to do research around current SoTA, that would be more like 1/8th.

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#37

Post by yamtaddle »

joxel wrote: post #32
yamtaddle wrote: post #25

Earlier quoted context omitted.

Why should I think I'll find uses this time, when I already burned time and money learning (some of) it once, and lost all that precisely because I never encountered any need for it? An hour a day is a huge time investment for something that's already failed to prove its worth once. Like I'm sure I could go find some jobs that I can't get now because I lack math skills, and I'm sure some (far from all!) of those pay…

Clearly there are people that don’t need to know math. You happen to be one of them, congratulations. Though I know I’d be bored out of my mind if I did software work that only used high school level math and logic.

Sure, some people need it, I was just responding to:

> I'm still appalled at how people can manage to gather the courage to utter they don't need math.

When... well, the vast majority of people really don't. They promptly forget almost everything back to about 6th grade math, shortly after finishing formal education, because they truly never need it, so that knowledge and those skills quickly rust.

If these people in-fact could make great use of it, such that it's "appalling" that they don't think they need it, then that's probably what school should focus on teaching, at least for non-math-majors. Laser-focus on application in everyday life. Especially in k-12. If it's actually useful and people are being forced to spend hundreds to thousands of hours learning junior high, high school, and college math, but then losing most of it because they never see any use for it, that's a tremendous failing of curriculum that should be addressed as directly as possible. If such a program wouldn't succeed because it's actually true that most people don't really need most of that math for anything, then we ought not be "appalled" at their correctly assessing that truth.

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#39

Post by verdverm »

thwayunion wrote: post #10
Foobar8568 wrote: post #7

Maybe I should start a PhD in ML as a university drop out, it sounds like a list of basic stuff taught within the first two years of any math/CS program, but I might miss something as it's far from being detailed...

Now is a terrible time to start a PhD in ML. If you do a CS PhD, pick literally any other subfield.

I'd argue that cross field is a good avenue for a PhD in CS. Work in collaboration with one of the sciences. There are plenty of papers to be had in application. Not everything in ML research needs to be algorithms. You'll likely find places where the models break down and need algo work too
Post reply on HN