My story as a self-taught AI researcher • Hacker News Board

Re: My story as a self-taught AI researcher

#11

Post by Gimpei » Mon, Jan 20, 2020, 8:15 PM UTC

Nasrudith wrote: post #7

qntty wrote: post #6

"Many are realizing that education is a zero-sum credential game." Can this silly meme die already? Maybe it's understandable coming from an economist who values education for no other reason than it's economic effects, but it's strange coming from someone who clearly understands the value of personal development.

It is pretty strange even from an economist really - they of all people should be able to understand and articulate the difference between signaling value and direct utility value of a given good or service.

It's not a majority view among economists. Caplan is the only person I can think of who holds this view.

Re: My story as a self-taught AI researcher

#12

Post by currymj » Mon, Jan 20, 2020, 8:15 PM UTC

bluetwo wrote: post #8

The thing that disappoints me about the aspirations of being a researcher is that the goal is to get paid to study AI, not solve real-world problems. I would rather build a small company by solving a real problem than work for a big company spinning my wheels.

for a lot of people who end up in research-type jobs, a sense of curiosity is one of their strongest motivators, and they want work that will let them pursue their curiosity. it sounds like you're motivated by something else.

Re: My story as a self-taught AI researcher

#13

Post by lern_too_spel » Mon, Jan 20, 2020, 8:16 PM UTC

qntty wrote: post #6

"Many are realizing that education is a zero-sum credential game." Can this silly meme die already? Maybe it's understandable coming from an economist who values education for no other reason than it's economic effects, but it's strange coming from someone who clearly understands the value of personal development.

It seems correct to me. If I get a degree from MIT, somebody else can't. They have limited spots. He is promoting models of education for signaling employers that are not zero sum.

Re: My story as a self-taught AI researcher

#14

Post by exdsq » Mon, Jan 20, 2020, 8:19 PM UTC

Survivorship bias or reality:

3 months learning FastAI, 3-12 months personal projects and consulting, 2 months flashcards of ~100 papers, 6 months to publish a paper

What does he mean by ‘paper’? A Medium post? NeurIPS?

Re: My story as a self-taught AI researcher

#15

Post by cygaril » Mon, Jan 20, 2020, 8:20 PM UTC

TrainedMonkey wrote: post #5

woah wrote: post #3

Earlier quoted context omitted.

On the other hand, the lack of data for independent researchers may encourage the development of low data techniques which is much more exciting in the long term since humans are able to learn with much less data than required by most machine learning techniques

Arguably humans have a lifetime of data which was used to develop a model of the world that is amazingly efficient at interpreting new data.

Or our entire evolutionary history of data.

Re: My story as a self-taught AI researcher

#16

Post by hogFeast » Mon, Jan 20, 2020, 8:22 PM UTC

bluetwo wrote: post #8

The thing that disappoints me about the aspirations of being a researcher is that the goal is to get paid to study AI, not solve real-world problems. I would rather build a small company by solving a real problem than work for a big company spinning my wheels.

Why I didn't go into academia but GL convincing other people of the value in that. I am sure there are cultural differences here but where I am, the goal of most people who study CS is: leave me alone while I mess about with X (evidence: the local college was doing speech processing/nlp in the 60s, they actively turned down paid work...unsurprisingly, they got left in the dust, professors are now being encouraged into doing commercial work but, of course, most of it is totally nonviable and is just more messing about with complex nonsense that doesn't work).

I think if you look at history this is also evident: the inventions of the late 18th century were a function of necessity, the invention of semis (not just in the US but how Taiwan developed)...this isn't to say academia is pointless but there is just far more going on (I think if you look at some of the East Asian nations that get great academic results, their progress on actual R&D innovation is far less impressive).

Re: My story as a self-taught AI researcher

#17

Post by gwern » Mon, Jan 20, 2020, 8:24 PM UTC

qntty wrote: post #6

"Many are realizing that education is a zero-sum credential game." Can this silly meme die already? Maybe it's understandable coming from an economist who values education for no other reason than it's economic effects, but it's strange coming from someone who clearly understands the value of personal development.

No one doubts the value of personal development, least of all the interviewee.

But I'm not sure what that has to do with buying expensive formal education credentials.

Re: My story as a self-taught AI researcher

#18

Post by andreyk » Mon, Jan 20, 2020, 8:27 PM UTC

narenst wrote: post #2

This is a really good time to be a Independent Scientist (aka Gentleman scientist) in this field because how nascent deep learning and similar techniques are. It requires a lot of trial and error and time/cost investment to bring the AI techniques to the masses. The FAANGs are trying to hire all the top talent (including Emil who wrote the post) but I believe these independent researchers will be the one finding new…

wrt the data point, to be fair most research is still coming out of universities where students have access to the same data as anyone else. So from a research perspective it's not a huge deal, much as with compute industry can scale up known techniques while individual researchers do more interesting stuff.

Re: My story as a self-taught AI researcher

#19

Post by codebolt » Mon, Jan 20, 2020, 8:34 PM UTC

qntty wrote: post #6

"Many are realizing that education is a zero-sum credential game." Can this silly meme die already? Maybe it's understandable coming from an economist who values education for no other reason than it's economic effects, but it's strange coming from someone who clearly understands the value of personal development.

My prediction is that whoever comes up with the next forward leap in AI will be someone who at minimum has a firm grasp on the various branches of undergraduate level maths. Naively tinkering with heuristic statistical ML methods like neural nets and hoping that higher level intelligence somehow magically pops out isn't the way forward. We need a more sophisticated approach.

Re: My story as a self-taught AI researcher

#20

Post by K0SM0S » Mon, Jan 20, 2020, 8:43 PM UTC

andreyk wrote: post #18

narenst wrote: post #2

This is a really good time to be a Independent Scientist (aka Gentleman scientist) in this field because how nascent deep learning and similar techniques are. It requires a lot of trial and error and time/cost investment to bring the AI techniques to the masses. The FAANGs are trying to hire all the top talent (including Emil who wrote the post) but I believe these independent researchers will be the one finding new…

wrt the data point, to be fair most research is still coming out of universities where students have access to the same data as anyone else. So from a research perspective it's not a huge deal, much as with compute industry can scale up known techniques while individual researchers do more interesting stuff.

So if I understand correctly, to reformulate in my own words/views:

while the "big data" (datasets) formed and thus owned by big-tech, big-ads, big-brother, etc. may be instrumental to build at-scale solutions for real-world usage (for profit, knowledge, control, whatever actionable goal),

fundamental research itself, as done in universities, can move forward without these datasets: using what's publicly available is enough.

Did I read this right? It would effectively add much needed nuance to the common perception that big data is necessary to train innovative models, that there might be some sort of monopoly on oil (data, the 'fuel' of ML) by a few champions of data collection.