Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My story as a self-taught AI researcher

blog.floydhub.com

Post reply on HN
Print view View original

71–80 of 176 posts

Re: My story as a self-taught AI researcher

#71

Post by anjc »

K0SM0S wrote: post #63
anjc wrote: post #40

Earlier quoted context omitted.

Your tutorial is very good, but to able read even a few paragraphs you need to be proficient with linear algebra and calculus already. > Most papers strike me as requiring a non-trivial knowledge of linear algebra I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge

> I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge Yes, in the context of "a self-taught researcher", I think I intuitively meant anything that precisely requires a degree, typical academic knowledge. E.g. you can become a great business person who won't feel hindered by lack of academic knowledge, you definitely can't…

> I guess I was wondering where math fit in this picture for AI research.

Well you can't do #1 or #2 without having a level of maths proficiency that most college grads do not have.

FWIW, I don't understand the difference between #1 and #2 above. Most academic/industrial research is incremental (i.e. #1), and a tiiiiny % will have any impact in the way something like XGBoost would (the example he gave in another comment). That doesn't mean that the non-impactful research isn't 'basic'. You could alternatively just call #2 "groundbreaking research" and #1 "non groundbreaking research, but you need mathematics knowledge for both imo.

Re: My story as a self-taught AI researcher

#72

Post by NWM123 »

I personally found this article to be very interesting. I don't know much about AI, but I was fascinated by the discussion of peer to peer educational system. I believe that they will become more prevalent as student loan payments cause debt to so much of our population in order to get an education .

Re: My story as a self-taught AI researcher

#73

Post by JamesBarney »

wigl wrote: post #64
JamesBarney wrote: post #55

Earlier quoted context omitted.

> Where are these internships that will hire you based on your completion of Fast.ai (if done in 1-2 months by a beginner I assume it's only part 1) alone, especially in 2020? I don't think the idea is to look for an internship after the course but an additional 4 months of personal projects. After applying state of the art deep learning for 4 months full time you'll have some very cool projects, and you could probab…

I should have been more specific and did not mean to exclude the other 4-5 months in his half year estimate. I meant to say no other background/experience to the position that would qualify the candidate for the role other than fast.ai v3 part 1. I love fast.ai and would still recommend it to all. I just think that the chances for getting into internships in that period, given the difficulties/inefficiencies/biases i…

Yeah I don't think it would be easy. I'm not in a data science role, but I do hire for software engineers. And if someone came to me with "Hey I'm trying to be a software engineer. I finished this boot camp 4 months ago, and since then have built these really cool projects. Could I work for you as an intern for the next 6 months to break into the field?" I'd totally say yes.

Re: My story as a self-taught AI researcher

#74

Post by jhrmnn »

AlexCoventry wrote: post #58
ineedasername wrote: post #50

I think in these sorts of discussions two concepts with the same name tend to get conflated, so I think it's important to make a distinction between: 1) AI Research as applying/tweaking known ML/DL methods to a novel problem. I would term these something like "AI Engineering Research" 2) AI Research as examining the theoretical frameworks & approaches to ML/DL in a way that may itself lead to shifts in the understand…

Is AlphaFold 1 or 2?

I have a decent understanding of the approach and would vote for #1. I’d say almost all applications of ML in physical sciences are #1. In contrast applying methods of statistical physics to understand how deep learning (as in DNN+SDG) works at all is a good example of #2.

Re: My story as a self-taught AI researcher

#75

Post by ineedasername »

JamesBarney wrote: post #66
rumanator wrote: post #54

Earlier quoted context omitted.

> Mind providing some concrete examples? It's like the difference between, say, applied and pure sciences. One is focused on developing and studying new algorithms, while the other is focused on using algorithms developed by someone else in practical applications. To put it differently, it's like physics vs engineering. A physicist might develop new structural analysis methods, while the engineer would use those meth…

I understand the separation between physics. But most structural analysis methods are discovered by professors of structural engineering and not physicists(and much of it is empirical). But I was asking because I was specifically looking for concrete examples in deep learning.

Yep, this is why I talk about the virtuous feedback loop between these two modes. Empirical methods feed theory which feeds empirical methods ad infinitum.

In the field of ML, a concrete example might be the tool Xgboost (#1) and the original work that led to and developed Gradient Boosting itself (#2), of which Xgboost is an implementation, and probably one that has helped refine the underlying theory as well.

ML has lots of examples where the researcher(s) for #2 were also doing #1. A famous paper in NLP comes to mind as an excellent example of this overlap (PDF: https://www.csie.ntu.edu.tw/~b92b02053/print/good-turing-smo...)

Re: My story as a self-taught AI researcher

#76

Post by YeGoblynQueenne »

codebolt wrote: post #19
qntty wrote: post #6

"Many are realizing that education is a zero-sum credential game." Can this silly meme die already? Maybe it's understandable coming from an economist who values education for no other reason than it's economic effects, but it's strange coming from someone who clearly understands the value of personal development.

My prediction is that whoever comes up with the next forward leap in AI will be someone who at minimum has a firm grasp on the various branches of undergraduate level maths. Naively tinkering with heuristic statistical ML methods like neural nets and hoping that higher level intelligence somehow magically pops out isn't the way forward. We need a more sophisticated approach.

Pragmatically speaking, the majority of machine learning researchers right now are not trying to make a leap in AI, they 're just trying to get in on the money while the current funding frenzy lasts.

That is, for example, why it is possible to find people presumably seriously suggesting to:

  3. Flashcard the Deep Learning Book (4-6m)
  4. Flashcard ~100 papers in a niche (2m)
As a method to "bootstrap yourself into deep learning research".

I mean, it's clear to me that the language deployed in the article is ostensibly about teaching yourself to do machine learning research when what it's really discussing is how to get hired by one of the companies that are curently paying six-figure salaries for machine learning engineers etc.

Or I'm just old and cynical. Wait, let me find my false teeth so I can chew that over.

Re: My story as a self-taught AI researcher

#77

Post by deepnotderp »

K0SM0S wrote: post #62
jph00 wrote: post #34

Earlier quoted context omitted.

> Is math a problem for non-academic researchers? It takes a while to figure out how to read academic papers, but it's largely about learning the notation. In the end, it maps back to the code you write anyway in most cases, so it's just another way of writing stuff you already know. It's not so much linear algebra you need, since much of that is not relevant to AI. It's really matrix calculus. Which is largely about…

> matrix calculus It might be wrong but I tend to see vectors and matrices as two notations for the same mathematical object[1]. So I indeed meant matrices! However I didn't see calculus itself as such a big requirement, as it all felt pretty "linear" to me (regressions etc). Are we talking things that e.g. "Calc 2"[2] should cover? I feel reassured by your first paragraph. This can be done. I'll definitely work on y…

Deep learning is mostly matrix calc. More classic ML methods are generally linear

Re: My story as a self-taught AI researcher

#78

Post by K0SM0S »

deepnotderp wrote: post #33
K0SM0S wrote: post #29

This was a great read (and great nuggets, like that paper on Intelligence by Chollet). I wonder: — Is math a problem for non-academic researchers? Most papers strike me as requiring a non-trivial knowledge of linear algebra, for instance; and topology sits right behind; the bold seem to take it one up on category theory as we speak, and geometric algebra is quickly gaining traction too. Lots of math, cool math but ma…

To be honest, linear algebra is not that difficult to learn on your own, and plenty of people do. Gilbert Strang's course on OCW has made introductory linear algebra quite accessible. Things like topology (e.g. TDA, persistent homology, etc.) aren't really mainstream yet, but even then most of it isn't really "hardcore" math in the sense that you can get away with a basic understanding, e.g. what a Vietoris-Rips comp…

> Gilbert Strang's course on OCW

Thanks for the pointer! (link[1] for those interested)

> you can get away with a basic understanding

Great news to me!

> taking the time to understand the math is absolutely worthwhile in my experience.

Strongly agree — for any topic, any field. My concerns are practical indeed, and less about the 10-year horizon (well enough to become skilled at anything) than the early stages of that, the best way to propel oneself far/fast enough on year 1, then 2, etc.

> applying mostly existing techniques to messy, noisy real world data and taking the time to understand the domain you are applying it to.

I hear that. I actually do like the sound of that, hence concerns that I was biased.

[1]: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra...

Re: My story as a self-taught AI researcher

#79

Post by K0SM0S »

anjc wrote: post #71
K0SM0S wrote: post #63

Earlier quoted context omitted.

> I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge Yes, in the context of "a self-taught researcher", I think I intuitively meant anything that precisely requires a degree, typical academic knowledge. E.g. you can become a great business person who won't feel hindered by lack of academic knowledge, you definitely can't…

> I guess I was wondering where math fit in this picture for AI research. Well you can't do #1 or #2 without having a level of maths proficiency that most college grads do not have. FWIW, I don't understand the difference between #1 and #2 above. Most academic/industrial research is incremental (i.e. #1), and a tiiiiny % will have any impact in the way something like XGBoost would (the example he gave in another comm…

I see what you mean. It seems possible the distinction is perhaps academic at best, a matter of perception. (I certainly don't have a personal opinion, yet! but point taken, and the continuity you speak of seems more realistic tbh).
Post reply on HN