Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Just know stuff (or, how to achieve success in a machine learning PhD)

kidger.site

Post reply on HN
Print view View original

91–100 of 108 posts

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#91

Post by mkl »

patrickkidger wrote: post #81
SQueeeeeL wrote: post #52

Earlier quoted context omitted.

> Read the main textbooks of your field and read and re-read ALL relevant papers for your actual Ph.D. topic, once you've been able to identify I'm gonna be real as someone in grad school. Basically no PhD, grad students, or professors I know read full text books. I hear these ideas a lot, and they often sound like one of those Instagram influencer diets, that's completely unreasonable if you have any constraints in…

As the author of this article... I have read maybe one textbook cover-to-cover in my life. :D (Hands-on machine learning, by Geron, back when I made the jump math->ML.)

I have an applied maths PhD but no machine learning. Would you still recommend Geron for that purpose?

BTW, I spotted a typo in the first paragraph of your thesis abstract: "neural networks and differential equation are two sides".

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#93

Post by extasia »

sillysaurusx wrote: post #34
extasia wrote: post #24

Earlier quoted context omitted.

The author and you seem to be talking to different audiences, you're talking about ML eng and OP is talking about ML researchers. Researchers absolutely need to know a lot, not necessarily all the way to topology or w/e but definitely the underlying mathematical principles in order to advance the field (IMO).

You're right, I wasn't clear. But I'm a full-time ML researcher. In terms of advancing the field, my contributions so far have been modest, but they're there. Some of my favorite ideas were swarm training ( https://battle.shawwn.com/swarm-training-v01a.pdf ), stop loss for stabilizing GAN training ( https://twitter.com/search?q=from%3Atheshawwn%20stop%20loss&... ), and getting GPT to play chess ( https://www.theregis…

Aha! Nice, I'm actually booking up on transformers as part of my doctoral programme and had a recollection about your gpt chess project earlier today!

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#94

Post by patrickkidger »

mkl wrote: post #91
patrickkidger wrote: post #81

Earlier quoted context omitted.

As the author of this article... I have read maybe one textbook cover-to-cover in my life. :D (Hands-on machine learning, by Geron, back when I made the jump math->ML.)

I have an applied maths PhD but no machine learning. Would you still recommend Geron for that purpose? BTW, I spotted a typo in the first paragraph of your thesis abstract: "neural networks and differential equation are two sides".

Aha, I have probably read that sentence literally hundreds of times, and never spotted the typo. I will never be able to unsee that.

Geron is good but now a bit out-of-date. No transformers (just CNNs/RNNs/etc.) and the coding component is all in scikit-learn and TensorFlow (rather than PyTorch or JAX).

FWIW I did ask this question recently over at https://twitter.com/PatrickKidger/status/1602776438159339521, in case any of the responses are helpful.

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#95

Post by l33t233372 »

burnished wrote: post #85
l33t233372 wrote: post #83

Earlier quoted context omitted.

If they are as educated on these topics as they claim, adding references is a trivial matter and generally greatly improves the value of the work.

I am going to disagree with you there. I have physical copies of a couple books I recommend and making them easily referenced can be a bit of a pain - not difficult but may be not worth the effort if I was compiling a list of topics to serve as my general answer to a question I receive frequently

It’s not hard to drop a title and an author at all. I heavily disagree.

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#96

Post by burnished »

l33t233372 wrote: post #95
burnished wrote: post #85

Earlier quoted context omitted.

I am going to disagree with you there. I have physical copies of a couple books I recommend and making them easily referenced can be a bit of a pain - not difficult but may be not worth the effort if I was compiling a list of topics to serve as my general answer to a question I receive frequently

It’s not hard to drop a title and an author at all. I heavily disagree.

Ok, where is your personal list of important topics to understand ? I'd like to check that you properly referenced every recommendation.

I think my real gripe here is the implicit demand for more free labor from some one else. We're talking references here, we both know its not a huge deal if you already have your reference manager fired up, but in the context of this thread the note of 'needs references' feels like a demand/like some kind of obviously missing thing vs a neutral observation that it would make a thing better.

Kinda like if some one makes a cake and when you get a slice all you say is 'needs strawberries'. Maybe y'aint wrong, still rude though.

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#97

Post by [deleted] »

gautamcgoel wrote: post #74

I just wrapped up a machine learning PhD at Caltech (now doing a postdoc in ML at Berkeley) and I disagree strongly with this article. What matters isn't knowing a bunch of random stuff, but rather writing/speaking skills, a willingness to learn new things, perseverance in the face of setbacks, creativity, having enough EQ to navigate the advisor-advisee relationship and departmental politics, and most of all, an abi…

[deleted]

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#98

Post by l33t233372 »

burnished wrote: post #96
l33t233372 wrote: post #95

Earlier quoted context omitted.

It’s not hard to drop a title and an author at all. I heavily disagree.

Ok, where is your personal list of important topics to understand ? I'd like to check that you properly referenced every recommendation. I think my real gripe here is the implicit demand for more free labor from some one else. We're talking references here, we both know its not a huge deal if you already have your reference manager fired up, but in the context of this thread the note of 'needs references' feels like…

The effort required is minimal. Here, watch!

Linear algebra is important! Check out Linear Algebra Done Right by Axler.

If the author can’t do this minimal extra step then I question why they’re writing their post since they’re either unqualified or don’t really care about helping people.

Re: Just know stuff (or, how to achieve success in a machine learning PhD)

#99

Post by j7ake »

gautamcgoel wrote: post #74

I just wrapped up a machine learning PhD at Caltech (now doing a postdoc in ML at Berkeley) and I disagree strongly with this article. What matters isn't knowing a bunch of random stuff, but rather writing/speaking skills, a willingness to learn new things, perseverance in the face of setbacks, creativity, having enough EQ to navigate the advisor-advisee relationship and departmental politics, and most of all, an abi…

These examples right here are the real skills to go beyond being stuck in purely technical roles.

These communication skills are also much more emphasized in American phd programs than European.

I have seen in general the American phd programs produce much more mature, well-rounded and broad scientists than Europe.

It could also be because the blog poster is only 2.5 years into their post graduate years, while American PhDs can easily be double.

Post reply on HN