Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TensorFlow, Keras and deep learning, without a PhD

codelabs.developers.google.com

Post reply on HN
Print view View original

11–20 of 156 posts

Re: TensorFlow, Keras and deep learning, without a PhD

#11

Post by axegon_ »

wirrbel wrote: post #7

The whole AI/ML stuff has become so hyped up that its probably time to find another topic of interest in software engineering for me. Its a weird melange nowadays where frameworks and "academic credentials" are fused together by major tech companies and leaves me - who has deployed a dozen of classical ML models into production that are still running after couple of years - wondering what this is all about. Overall,…

Really familiar territory. I think the hype has poisoned the minds of many and at this state "AI/ML" has turned into a simple buzzword. Much like "blockchain" 2 years ago. And while I'm still as fascinated about ml as I was 5 years ago, like many others, I've decided to stay in the shadows and do my own thing just for the fun of it. Especially since marketing and ego started playing a big role around those communities. It genuinely makes me sad but I think I always knew in the back of my mind that this would likely turn out to be a nail in the coffin of AI/ML, not robots taking over the world.

The way I see it, ML/AI is nothing more than a marketing campaign for much of the industry and few people realize that it's often a small component and rarely a major selling point for anything. Like "ml-powered kitchen blender" or whatever. As you said, few people discuss evaluating predictions, tackling biases. I suspect because most people are a lot more interested in snatching a piece of the cake.

Re: TensorFlow, Keras and deep learning, without a PhD

#12

Post by s1t5 »

totorovirus wrote: post #9

It is totally possible and easy to use tensorflow/torch if you havn't skipped linear algebra classes. Ph.D is needed if you are going for a job where you design a sophisticated model (not just adding layers, but experimenting with activation, attention etc).

A PhD isn't even a requirement for doing the more advanced stuff. Obviously you need a lot of math and ML specific knowledge but there's no reason why you can't have that knowledge with an undergraduate math degree (for example). Spending 3-6 years doing research in a very narrow and possibly unrelated branch of mathematics will give you a PhD, but the linear algebra and multivariable calculus that you actually need for the ML stuff are covered in a bunch of undergrad/masters courses in mathematics, computer science, engineering, physics etc.

Re: TensorFlow, Keras and deep learning, without a PhD

#13

Post by op03 »

wirrbel wrote: post #7

The whole AI/ML stuff has become so hyped up that its probably time to find another topic of interest in software engineering for me. Its a weird melange nowadays where frameworks and "academic credentials" are fused together by major tech companies and leaves me - who has deployed a dozen of classical ML models into production that are still running after couple of years - wondering what this is all about. Overall,…

Well the hype is required to get your grandmom, older execs, or a strategy/biz dev team at a brick and mortar firm who can afford only 1 dev to gain confidence that they too can use ML.

Re: TensorFlow, Keras and deep learning, without a PhD

#14

Post by s1t5 »

axegon_ wrote: post #11
wirrbel wrote: post #7

The whole AI/ML stuff has become so hyped up that its probably time to find another topic of interest in software engineering for me. Its a weird melange nowadays where frameworks and "academic credentials" are fused together by major tech companies and leaves me - who has deployed a dozen of classical ML models into production that are still running after couple of years - wondering what this is all about. Overall,…

Really familiar territory. I think the hype has poisoned the minds of many and at this state "AI/ML" has turned into a simple buzzword. Much like "blockchain" 2 years ago. And while I'm still as fascinated about ml as I was 5 years ago, like many others, I've decided to stay in the shadows and do my own thing just for the fun of it. Especially since marketing and ego started playing a big role around those communitie…

> Much like "blockchain" 2 years ago.

It's different. With the ML stuff there's a bunch of actually useful applications and interesting problems at the core with a lot of fluff and marketing piled on top of it. That's the reason why you're seeing the ML/AI hype last so much longer than blockchain (which was basically a quick cash grab with no substance).

Re: TensorFlow, Keras and deep learning, without a PhD

#15

Post by xvzwx »

wirrbel wrote: post #7

The whole AI/ML stuff has become so hyped up that its probably time to find another topic of interest in software engineering for me. Its a weird melange nowadays where frameworks and "academic credentials" are fused together by major tech companies and leaves me - who has deployed a dozen of classical ML models into production that are still running after couple of years - wondering what this is all about. Overall,…

Its true that with todays Frameworks and easy API calls almost everyone with a little technological background can deploy a ML/AI model and get sufficiant results. But Bootcamps cannot replace an academic education. As soon as you are not able to understand and review new released papers and insights and have to wait for an high level blog entry or video course on that topic you are worth nothing. Without a deeper understanding you can just guess what is going on inside that blackbox NN or ML model and have to rely on blindly change parameters and even worse you are not able to understand your results or compare someoneleses results with yours using statistical tests and so on. So in the end people (maybe not everyone) without academic background are just API callers that will struggle on the long term.

Re: TensorFlow, Keras and deep learning, without a PhD

#16

Post by lonesword »

s1t5 wrote: post #12
totorovirus wrote: post #9

It is totally possible and easy to use tensorflow/torch if you havn't skipped linear algebra classes. Ph.D is needed if you are going for a job where you design a sophisticated model (not just adding layers, but experimenting with activation, attention etc).

A PhD isn't even a requirement for doing the more advanced stuff. Obviously you need a lot of math and ML specific knowledge but there's no reason why you can't have that knowledge with an undergraduate math degree (for example). Spending 3-6 years doing research in a very narrow and possibly unrelated branch of mathematics will give you a PhD, but the linear algebra and multivariable calculus that you actually need…

I second this. I have worked with a bunch of undergrads (I am pursuing a masters degree in CS) and they had a thorough grasp of the math and could really contribute to the research agenda of the group. When I did my undergrad (in 2011-15), I ended up taking a lot of electronics/hardware courses. Turns out undergrads these days just swap them with math/machine learning courses. Good for them.

Re: TensorFlow, Keras and deep learning, without a PhD

#18

Post by axegon_ »

s1t5 wrote: post #14
axegon_ wrote: post #11

Earlier quoted context omitted.

Really familiar territory. I think the hype has poisoned the minds of many and at this state "AI/ML" has turned into a simple buzzword. Much like "blockchain" 2 years ago. And while I'm still as fascinated about ml as I was 5 years ago, like many others, I've decided to stay in the shadows and do my own thing just for the fun of it. Especially since marketing and ego started playing a big role around those communitie…

> Much like "blockchain" 2 years ago. It's different. With the ML stuff there's a bunch of actually useful applications and interesting problems at the core with a lot of fluff and marketing piled on top of it. That's the reason why you're seeing the ML/AI hype last so much longer than blockchain (which was basically a quick cash grab with no substance).

Don't get me wrong, AI/ML is immensely more valuable than blockchain ever was. Not a single doubt in my mind. But in terms of exploitation for marketing purposes - it's a very similar story. That's what I'm referring to.

Re: TensorFlow, Keras and deep learning, without a PhD

#19

Post by echelon »

amrrs wrote: post #10
echelon wrote: post #8

I'm doing deep learning without a data science background. Some of my current results are: * https://vo.codes * https://trumped.com The voices need better data curation and longer training, but some speakers such as David Attenborough are quite good. I've also built a real time streaming voice conversion system. I want to generalize it better so that it can be an actual product. I think it could be a killer app for D…

This is an amazing side project. Would you share some details about how you've hosted them? Also, if possible about the training details like how long did it take for you train them? Did you do it on your local GPU or some cloud provider or free service like Colab?

Thanks!

I'm pretty serious, so I've put some money into it. And even more time.

I've got a 2x1080Ti setup I used locally back in the day, but it's really slow. I still train stuff on it, but only things I know will train successfully for a long time (eg the Melgan model).

I use rented V100 GPUs to train the speaker models. They're quick and allow me to refine the datasets and parameters much more quickly than if I was doing all of it on my own box. Colabs are great and I could probably get along with them if I wasn't running so many experiments in parallel.

I can get reasonable results in a few hours on an 8xV100. Once I hone in on a direction I like, I'll let it train for a few days. (The David Attenborough model is a result of this.)

I still have a ton of refinement to do. I'm also working on singing models, and these should be ready by the weekend.

I've thought about buying beefy GPUs at this point as I've proven to myself it's not just a temporary hobby. Cloud compute is expensive.

The models are hosted on Rust microservices (a frontend proxy that fans out into multiple model servers), and this is deployed to a Kubernetes cluster. I'm planning to add more intelligence to the proxy and individual model containers so they independently scale.

Post reply on HN