Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TensorFlow, Keras and deep learning, without a PhD

codelabs.developers.google.com

Post reply on HN
Print view View original

91–100 of 156 posts

Re: TensorFlow, Keras and deep learning, without a PhD

#91

Post by Datenstrom »

gas9S9zw3P9c wrote: post #31

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

I have self taught this material and have been working professionally in the field for some years now. It was primarily driven by the need to solve problems for autonomous systems I was creating. When I am asked how to do it I give the progression I followed. First have preferably a CS background but at least Calc 1&2, Linear Algebra, and University statistics, then:

1. Read "Artificial Intelligence A Modern Approach" and complete all the exercises [1]

2. Complete a ML course I recommend the Andrew Ng Coursera one [2]

3. Complete a DL course I recommend the Andrew Ng Coursera one [3]

4. Complete specialization courses in whatever you are interested in such as computer vision, reinforcement learning, natural language processing, etc. These will cover older traditional methods in the introduction also which will be very useful for determining when DL is not the correct solution.

Additionally I suggest a data science course which usually covers other important things like data visualization and how to handle bad/missing data. Also, I learned a lot by simply being surrounded by brilliant people who know all this and being able to ask them questions and see how they approach problems. So not really self taught as much as untraditionally taught.

Unfortunately not a single person has actually followed the advice. Everyone has only watched random youtube bloggers and read blogs. Some have gotten into trouble after landing a job by talking buzzwords and asked for help but my advice does not change.

It does make it rather hard to find a job without a degree though, I would not recommend it. All of mine only come from strong references I have from luckily getting my foot in the door initially.

[1]: http://aima.cs.berkeley.edu/index.html

[2]: https://www.coursera.org/learn/machine-learning

[3]: https://www.coursera.org/specializations/deep-learning

Edit: formatting, typo

Re: TensorFlow, Keras and deep learning, without a PhD

#92

Post by gas9S9zw3P9c »

Datenstrom wrote: post #91
gas9S9zw3P9c wrote: post #31

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

I have self taught this material and have been working professionally in the field for some years now. It was primarily driven by the need to solve problems for autonomous systems I was creating. When I am asked how to do it I give the progression I followed. First have preferably a CS background but at least Calc 1&2, Linear Algebra, and University statistics, then: 1. Read "Artificial Intelligence A Modern Approach…

Thanks for sharing your learning path and congrats to your success. There are so many great resources out there that it's possible for anyone to become an expert. Unfortunately, people like you who are willing to put in the hard work seem to be the minority. All of your success is well-deserved and props to you. Just like you said, most gravitate towards the easy-to-understand videos and blogs instead of confronting their gaps in knowledge. I've had the same experience with giving advice - anything that looks like it requires focused work (solving textbook problems!) or is unsexy is readily ignored in favor of the latest hype demo or high-level framework.

> It was primarily driven by the need to solve problems for autonomous systems I was creating.

I wonder if your success also had something to do with the fact that you had a specific problem you were trying to solve. Did you feel that your specific problem put everything you were learning into context and made it easier to stay motivated?

Re: TensorFlow, Keras and deep learning, without a PhD

#93

Post by p1esk »

currymj wrote: post #90
sabalaba wrote: post #79

Earlier quoted context omitted.

I would disagree that having a lot of interest hurts the field. Where are you seeing “noise and low quality work”? For your sake, I hope you’re not reading random papers from unknown authors on arXiv in your spare time! I think we’ve seen impressive contribution from people without PhDs. Chris Olah and Alec Radford come to mind first. (Note: I’m not implying that you disagree with that statement, just wanted to point…

there are many papers that seem like obvious crap that get into supposedly prestigious conferences (NeurIPS, ICML) every year. I think many people might reasonably believe that a paper in the supposed best venue in the field will be worth reading, but that is not really true. compared to machine learning, other research communities in computer science have much less nonsense. you can really feel the difference in qua…

Link to at least one “obvious crap” paper that got accepted to NeurIPS or ICML?

Re: TensorFlow, Keras and deep learning, without a PhD

#94

Post by dekhn »

Twenty five years ago, this would have been "LINUX, UNIX, and serving, without a PhD" and Matt Welsh's Linux Installation And Getting Started was the intro (https://www.mdw.la/papers/linux-getting-started.pdf). I was one of many who adopted Linux early, using this book (later I read the BSD Unix Design and Implementation, which I would describe as senior undergrad/junior grad student material).

Having those sorts of resources to introduce junior folks to advanced concepts are really great to me- my experience is that I learn a lot more by reading a good tutorial than a theory book, up until I need to do advanced work (this is particular to my style of learning; I can read code that implements math, but struggle to parse math symbology).

Re: TensorFlow, Keras and deep learning, without a PhD

#95

Post by phonebucket »

amelius wrote: post #64
gas9S9zw3P9c wrote: post #31

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

Well, maybe I got the wrong impression but after reading the (very accessible) Yolo V3 paper [1], it seems to me that even the experts do little real math and lots of guesswork, kicking a model until it starts giving results. [1] https://pjreddie.com/media/files/papers/YOLOv3.pdf

> it seems to me that even the experts do little real math and lots of guesswork

Machine Learning is a rich and varied field. Like most applied sciences, there is a spectrum from the heavily applied to the heavily theoretical, to some which try to span both sides of the spectrum at the same time (e.g. https://arxiv.org/abs/1704.04932).

Re: TensorFlow, Keras and deep learning, without a PhD

#96

Post by shrimpx »

gas9S9zw3P9c wrote: post #68
amelius wrote: post #64

Earlier quoted context omitted.

Well, maybe I got the wrong impression but after reading the (very accessible) Yolo V3 paper [1], it seems to me that even the experts do little real math and lots of guesswork, kicking a model until it starts giving results. [1] https://pjreddie.com/media/files/papers/YOLOv3.pdf

Oh yes, I don't think advanced math is required in any way. However, there is a difference in that researchers who have worked with these models for many years (often including having done some of the math) have a very good intuitive understanding of these models. Once you have that, it's fine to be driven by gut feeling. Just like many engineers are driven by gut feeling that come from tacit knowledge through experi…

It sounds like what you’re complaining about is some people who put papers on arxiv and deceivingly claim to be experts. And kind of implying that these blog posts are to blame, so the blog posts should be retracted due to those dishonest academic people? You’re making a very confusing point.

Re: TensorFlow, Keras and deep learning, without a PhD

#97

Post by KineticLensman »

ricksharp wrote: post #77
MrBuddyCasino wrote: post #41

Earlier quoted context omitted.

> Good backend/infrastructure/data engineers are so much harder to find these days, and these skills IMO provide much more value than doing some kind of modeling. After all these years of FOMO on AI, this is music to my ears.

Right, I’ve been reading about machine learning for about 5 years, have read hundreds of articles about different techniques, and have often tried to explore ways it could be used. However, I’ve never found a practical use in software engineering. Every time I think I discover something that could use machine learning, I usually don’t have any data to work with or don’t have a clear definition of what the inputs and…

> In the end, I find a way to develop a solution that doesn’t need AI and often makes me realize that AI would not have been able to provide the required reliability.

This was brought home to me back in the 1990s at a place where we were trying to popularise expert system technology (a positively prehistoric form of AI). It was one of my first jobs in industry after leaving academia. The goal was to create a system that could predict the speed at which a unit of military vehicles could move so that a contact report expert system could decide whether two groups of vehicles could have feasibly moved between two points in a given time. The system could use this to help decide whether two reports referred to two different enemy units or a single unit that had moved from point A to point B.

After quite a lot of time cranking out and debugging rules to describe the movement behaviours, I realised that some simple convoy arithmetic would do just as good a job - units are often constrained by the speed of the slowest vehicle, and respect inter-vehicle distances. For most purposes this simple arithmetic was just as good (and orders of magnitude faster) than the complex rule engine.

Re: TensorFlow, Keras and deep learning, without a PhD

#98

Post by kadder »

Machine learning Academia, is very different from how its applied in a real world setting. The number of people who can launch a model and deal with the intricacies of serving a model in production are still very few. Resources like these are essential for people not familiar with ML. We need more people who can apply successfully at scale.

A top down approach to learning ML in my opinion is more efficient than a bottom's up approach, that most people try to advocate

Re: TensorFlow, Keras and deep learning, without a PhD

#99

Post by mumbisChungo »

gas9S9zw3P9c wrote: post #31

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

>At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field.

This seems extremely short-sighted to me.

If the barrier to entry (on an already relatively young technology) has come down so far that there are a bunch of noobs running rampant right now, does that not bode extremely well for future advancements in the field (assuming some non-zero conversion rate from noobs to productive members of the field over time)?

Also, let's not pretend that people with relevant degrees don't make shitty contributions to arXiv all the time.

Re: TensorFlow, Keras and deep learning, without a PhD

#100

Post by sdinsn »

gas9S9zw3P9c wrote: post #31

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

> At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field.

Note that this is not a problem exclusive to AI or Computer Science.

I have a hobbyist interest in Entomology, and I was so disappointed to see that people are still pumping out papers that are minor tweaks off old population modeling papers from the 1980s. The field is shockingly stagnant. I've read random PhD theses from the 70s that are written on damn typewriters that are higher quality than modern papers from so-called "top tier" research universities.

Post reply on HN