TensorFlow, Keras and deep learning, without a PhD • Hacker News Board

Re: TensorFlow, Keras and deep learning, without a PhD

#81

Post by zurfer » Fri, Jul 17, 2020, 1:48 PM UTC

s1t5 wrote: post #21

amrx101 wrote: post #4

Have a simple rant here. All these BIG $ companies every now and then come out with statements and what not, that doing AI ML is very easy and every one including their cats should do AI, ML courses and training(preferably on their platform). Once that is done the job market is yours. Reality is far from this. - Today AI|ML does not have the capability marketed by these big companies. Incidentally marketing is target…

Another side of this that I've seen is people with PhDs in applied mathematics on a research team where a PhD is a hard requirement, spending all of their time passing dataframes around and convincing the sales team that the bar charts in the product are in fact correct.

Have we been working on the same team? :P The worst part (or best part of the joke) is that with a mathematical PhD you're very ill equiped for a discussion with your typical sales rep.

Re: TensorFlow, Keras and deep learning, without a PhD

#82

Post by devalgo » Fri, Jul 17, 2020, 1:58 PM UTC

gas9S9zw3P9c wrote: post #31

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

>At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field.

I'm sorry but given that many papers in NeurIPS, ICML, etc are exactly what you described I find your criticism a bit lacking.

Re: TensorFlow, Keras and deep learning, without a PhD

#83

Post by ausjke » Fri, Jul 17, 2020, 2:02 PM UTC

m0zg wrote: post #26

Mandatory plug: do consider using PyTorch instead. It's far easier to pick up and work with. Easy things are easy, hard things are possible.

I read somewhere saying TF2 is good for production while Pytorch is good for research(and papers), is this true? I'm more interested in putting one of them into real products, esp standalone embedded devices.

Re: TensorFlow, Keras and deep learning, without a PhD

#84

Post by gas9S9zw3P9c » Fri, Jul 17, 2020, 2:11 PM UTC

devalgo wrote: post #82

gas9S9zw3P9c wrote: post #31

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

>At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. I'm sorry but given that many papers in NeurIPS, ICML, etc are exactly what you descri…

The peer review system is so broken, especially with the number of submissions that the top journals get in ML now, that I don't even pay attention to what's accepted where anymore. It's all the same as arXiv to me. The best way to figure out what's useful and what's not is to wait and see which papers pass the test of time. But if you're an academic you don't necessarily have that luxury.

Re: TensorFlow, Keras and deep learning, without a PhD

#85

Post by dekhn » Fri, Jul 17, 2020, 2:12 PM UTC

amelius wrote: post #64

gas9S9zw3P9c wrote: post #31

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

Well, maybe I got the wrong impression but after reading the (very accessible) Yolo V3 paper [1], it seems to me that even the experts do little real math and lots of guesswork, kicking a model until it starts giving results. [1] https://pjreddie.com/media/files/papers/YOLOv3.pdf

The "real math" comes in when you're debugging a huge network that occasionally emits NaNs deep in the reverse pass. At that point, you need to understand the details of gradient descent and a whole lot more practical analytic theory.

Re: TensorFlow, Keras and deep learning, without a PhD

#86

Post by devalgo » Fri, Jul 17, 2020, 2:15 PM UTC

gas9S9zw3P9c wrote: post #84

devalgo wrote: post #82

Earlier quoted context omitted.

>At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result on arXiv so they can put AI research on their resume. This has resulted in so much noise and low quality work that it really hurts the field. I'm sorry but given that many papers in NeurIPS, ICML, etc are exactly what you descri…

The peer review system is so broken, especially with the number of submissions that the top journals get in ML now, that I don't even pay attention to what's accepted where anymore. It's all the same as arXiv to me. The best way to figure out what's useful and what's not is to wait and see which papers pass the test of time. But if you're an academic you don't necessarily have that luxury.

Well that definitely speaks more to the field in general and not necessarily to "beginners" who tune a pretrained model and post the results to arXiv. There's massive incentives to post papers with incremental progress when there's potentially billions of dollars of Grants and VC money for folks with their names on those papers.

Re: TensorFlow, Keras and deep learning, without a PhD

#87

Post by gotzmann » Fri, Jul 17, 2020, 2:23 PM UTC

echelon wrote: post #8

I'm doing deep learning without a data science background. Some of my current results are: * https://vo.codes * https://trumped.com The voices need better data curation and longer training, but some speakers such as David Attenborough are quite good. I've also built a real time streaming voice conversion system. I want to generalize it better so that it can be an actual product. I think it could be a killer app for D…

Could you provide some examples of good work in the field of singing nets?

Re: TensorFlow, Keras and deep learning, without a PhD

#88

Post by gumby » Fri, Jul 17, 2020, 2:26 PM UTC

gas9S9zw3P9c wrote: post #31

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

I consider this equivalent to the democratization of web site development and then app development. It’s certainly lead to an explosion of crummy and security-nightmare apps but in exchange has been an on-ramp for some good developers and exciting products as well.

The massive hype surrounding anything “AI” has caused the literature to become a dumpster fire, yes, but a handful of good papers still appear. Just use a low pass filter as with most things these days.

Re: TensorFlow, Keras and deep learning, without a PhD

#89

Post by sibmike » Fri, Jul 17, 2020, 2:29 PM UTC

I am constantly puzzled by people saying that AI is overhyped and fresh grads won't have enough jobs for them. Almost every real life industry: retail, logistics, construction, farming, heavy industries, mining, medicine have just recently started to try AI. The amount of manual and suboptimal tasks that have to be automated and optimized is enormous. I am pretty sure there is more the enough work for applied DSs with domain knowledge in mentioned industries.

Re: TensorFlow, Keras and deep learning, without a PhD

#90

Post by currymj » Fri, Jul 17, 2020, 2:31 PM UTC

sabalaba wrote: post #79

gas9S9zw3P9c wrote: post #31

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

I would disagree that having a lot of interest hurts the field. Where are you seeing “noise and low quality work”? For your sake, I hope you’re not reading random papers from unknown authors on arXiv in your spare time! I think we’ve seen impressive contribution from people without PhDs. Chris Olah and Alec Radford come to mind first. (Note: I’m not implying that you disagree with that statement, just wanted to point…

there are many papers that seem like obvious crap that get into supposedly prestigious conferences (NeurIPS, ICML) every year. I think many people might reasonably believe that a paper in the supposed best venue in the field will be worth reading, but that is not really true.

compared to machine learning, other research communities in computer science have much less nonsense. you can really feel the difference in quality if you look at papers accepted to equivalently competitive non-ML conferences.

i think the broader problem may be that people hiring for very lucrative, competitive jobs are effectively outsourcing their hiring decisions to conference reviewers, which causes the whole system to break down.