Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My story as a self-taught AI researcher

blog.floydhub.com

Post reply on HN
Print view View original

31–40 of 176 posts

Re: My story as a self-taught AI researcher

#31

Post by btrettel »

ssivark wrote: post #9
SQueeeeeL wrote: post #4

Earlier quoted context omitted.

Low data techniques are just another name for algorithms/equations. Dijstras algorithm required 0 training graphs to make. Any other kind of method will get killed by low statistical information in the data (can't get blood from a stone)

Agree with your first statement and disagree with your second; I don’t think the former implies the latter. I think there’s a lot of room to be clever with encoding domain-specific inductive biases into models/algorithms, such that they can perform fast+robust inference. Exploiting this trade off as a design parameter to be tuned, rather than sitting at one of the two extremes is potentially going to generate a lot o…

These types of techniques are already big in certain fields. E.g., in fluid dynamics and heat transfer, "dimensional analysis" is frequently used to simplify and generalize models. Sometimes models can be nearly fully specified up to a constant of proportionality based solely on dimensional considerations. Beyond what is typically seen as "data" the information here is a list of variables involved in the problem and the dimensions of the variables.

As far as I can tell "dimensions" in this sense are a purely human construct. For two variables to have different dimensions, it means that they can not be meaningfully added, e.g., apples and oranges.

Re: My story as a self-taught AI researcher

#32

Post by JamesBarney »

K0SM0S wrote: post #29

This was a great read (and great nuggets, like that paper on Intelligence by Chollet). I wonder: — Is math a problem for non-academic researchers? Most papers strike me as requiring a non-trivial knowledge of linear algebra, for instance; and topology sits right behind; the bold seem to take it one up on category theory as we speak, and geometric algebra is quickly gaining traction too. Lots of math, cool math but ma…

It's my understanding that dirty datasets that "demand scripting skills to sort out" is pretty common and most data scientists spend 80% of their time "sorting this out".

Re: My story as a self-taught AI researcher

#33

Post by deepnotderp »

K0SM0S wrote: post #29

This was a great read (and great nuggets, like that paper on Intelligence by Chollet). I wonder: — Is math a problem for non-academic researchers? Most papers strike me as requiring a non-trivial knowledge of linear algebra, for instance; and topology sits right behind; the bold seem to take it one up on category theory as we speak, and geometric algebra is quickly gaining traction too. Lots of math, cool math but ma…

To be honest, linear algebra is not that difficult to learn on your own, and plenty of people do. Gilbert Strang's course on OCW has made introductory linear algebra quite accessible.

Things like topology (e.g. TDA, persistent homology, etc.) aren't really mainstream yet, but even then most of it isn't really "hardcore" math in the sense that you can get away with a basic understanding, e.g. what a Vietoris-Rips complex is and why we use it instead of a Cech complex in TDA. Plus most DL research nowadays is pretty (advanced) math-light. That being said, taking the time to understand the math is absolutely worthwhile in my experience.

It should also be noted that a lot of real world ML/AI projects in industry aren't really about brand new algorithms using advanced math, but rather more about applying mostly existing techniques to messy, noisy real world data and taking the time to understand the domain you are applying it to.

Re: My story as a self-taught AI researcher

#34

Post by jph00 »

K0SM0S wrote: post #29

This was a great read (and great nuggets, like that paper on Intelligence by Chollet). I wonder: — Is math a problem for non-academic researchers? Most papers strike me as requiring a non-trivial knowledge of linear algebra, for instance; and topology sits right behind; the bold seem to take it one up on category theory as we speak, and geometric algebra is quickly gaining traction too. Lots of math, cool math but ma…

> Is math a problem for non-academic researchers?

It takes a while to figure out how to read academic papers, but it's largely about learning the notation. In the end, it maps back to the code you write anyway in most cases, so it's just another way of writing stuff you already know.

It's not so much linear algebra you need, since much of that is not relevant to AI. It's really matrix calculus. Which is largely about multiplying things together and adding them up. Terence Parr and I tried to create a "all you need to know" tutorial here: https://explained.ai/matrix-calculus/ .

You certainly don't need topology (unless you happen to be interested in that particular sub-field).

Re: My story as a self-taught AI researcher

#35

Post by gdubs »

woah wrote: post #3
narenst wrote: post #2

This is a really good time to be a Independent Scientist (aka Gentleman scientist) in this field because how nascent deep learning and similar techniques are. It requires a lot of trial and error and time/cost investment to bring the AI techniques to the masses. The FAANGs are trying to hire all the top talent (including Emil who wrote the post) but I believe these independent researchers will be the one finding new…

On the other hand, the lack of data for independent researchers may encourage the development of low data techniques which is much more exciting in the long term since humans are able to learn with much less data than required by most machine learning techniques

This would be a great area, IMHO, for the government to step in and fund an initiative to provide huge, rich datasets for anyone to use for ML research.

Re: My story as a self-taught AI researcher

#37

Post by tprice7 »

K0SM0S wrote: post #25
cygaril wrote: post #15

Earlier quoted context omitted.

Or our entire evolutionary history of data.

It indeed strikes me as particularly domain-narrow when I hear neuro or ML scientists claim as self-evident that "humans can learn new stuff with just a few examples!.." when the hardware upon which said learning takes place has been exposed to such 'examples' likely trillions of times over billions of years before — encoded as DNA and whatever else runs the 'make' command on us. The usual corollary (that ML should "…

"The usual corollary (that ML should "therefore" be able to learn with a few examples) may only apply, as I see it, if we somehow encode previous "learning" about the problem in very the structure (architecture, hardware, design) of the model itself."

Yes, and they do. They aren't choosing completely arbitrary algorithms when they attempt to solve a ML problem, they are typically using approaches that have already been proven to work well on related problems, or at least are variants of proven approaches.

Re: My story as a self-taught AI researcher

#38

Post by rmah »

Is this guy actually a researcher in the way most people would think of it? That is, someone who pushes the boundaries of science; who develops new AI techniques or finds the hard boundaries of existing AI techniques; who finds new ways compose multiple AI techniques cohesively; who explores the theoretical foundations of AI.

Or is he someone who uses AI techniques to solve problems (and then wrote a paper about it)? I can't help but wonder a bit.

Re: My story as a self-taught AI researcher

#39

Post by wigl »

This reeks of survivorship bias to me. I much prefer Andreas Madsen's more sober and self-conscious take on independent research [0].

> I’d spend 1-2 months completing Fast.ai course V3, and spend another 4-5 months completing personal projects or participating in machine learning competitions... After six months, I’d recommend doing an internship. Then you’ll be ready to take a job in industry or do consulting to self-fund your research.

Where are these internships that will hire you based on your completion of Fast.ai (if done in 1-2 months by a beginner I assume it's only part 1) alone, especially in 2020? How many are going to place in a Kaggle competition with just half a year of experience? More importantly, just how many people are privileged/secure enough to put their all into learning, with no sense of security or peer support?

> I started working with Google because I reproduced an ML paper, wrote a blog post about it, and promoted it. Google’s brand department was looking for case studies of their products, TensorFlow in this case. They made a video about my project. Someone at Google saw the video, though my skill set could be useful, and pinged me on Twitter.

So what really mattered was self-promotion, good timing, and luck.

> Tl;dr, I spent a few years planning and embarking on personal development adventures. They were loosely modeled after the Jungian hero’s journey with the influences of Buddhism and Stoicism.

Why does the author have to present his life like one would in a fucking college essay?

[0] https://medium.com/@andreas_madsen/becoming-an-independent-r...

Re: My story as a self-taught AI researcher

#40

Post by anjc »

jph00 wrote: post #34
K0SM0S wrote: post #29

This was a great read (and great nuggets, like that paper on Intelligence by Chollet). I wonder: — Is math a problem for non-academic researchers? Most papers strike me as requiring a non-trivial knowledge of linear algebra, for instance; and topology sits right behind; the bold seem to take it one up on category theory as we speak, and geometric algebra is quickly gaining traction too. Lots of math, cool math but ma…

> Is math a problem for non-academic researchers? It takes a while to figure out how to read academic papers, but it's largely about learning the notation. In the end, it maps back to the code you write anyway in most cases, so it's just another way of writing stuff you already know. It's not so much linear algebra you need, since much of that is not relevant to AI. It's really matrix calculus. Which is largely about…

Your tutorial is very good, but to able read even a few paragraphs you need to be proficient with linear algebra and calculus already.

> Most papers strike me as requiring a non-trivial knowledge of linear algebra

I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge

Post reply on HN