Live data from Hacker News

TensorFlow, Keras and deep learning, without a PhD

codelabs.developers.google.com

111–120 of 156 posts

Re: TensorFlow, Keras and deep learning, without a PhD

#111

Since everyone is talking about hype in ML, I wish there was some hype for good ole' conversional scientific computing. Yes, it's not so sexy, you have to build your own model yourself, and then the hard work is in finding and verifying a suitable numerical method and finally devising a solid implementation. It requires a vast number of different skills, anything from pure math to low level programming and it is defi…

It's also a lot of fun. For anyone that likes math, but also likes weird approximations, there's all kinds of juicy stuff in scientific computing. It will make you a better ML coder, too.

Re: TensorFlow, Keras and deep learning, without a PhD

#112
post #26

Mandatory plug: do consider using PyTorch instead. It's far easier to pick up and work with. Easy things are easy, hard things are possible.

Does it train just as fast on gpus?

In my experience, not quite as fast for fully-tuned code, but the difference is small - and given the same project deadline, the PyTorch version will probably be faster.

Re: TensorFlow, Keras and deep learning, without a PhD

#113

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

I have self taught this material and have been working professionally in the field for some years now. It was primarily driven by the need to solve problems for autonomous systems I was creating. When I am asked how to do it I give the progression I followed. First have preferably a CS background but at least Calc 1&2, Linear Algebra, and University statistics, then: 1. Read "Artificial Intelligence A Modern Approach…

Teaching isn't just about presenting the information to the student.

That's basically all you've done. Here, student, read these complex topics and at the end of it all you will have learned machine learning!

The art of being a teacher is much more nuanced. You (apparently) fail to present the material in a way that is accessible, relatable, and not overwhelming.

For example, the first thing you say to do is go read a college textbook front to back, and do all the exercises. And you're surprised that nobody has followed your steps?

Re: TensorFlow, Keras and deep learning, without a PhD

#114
post #64

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

Well, maybe I got the wrong impression but after reading the (very accessible) Yolo V3 paper [1], it seems to me that even the experts do little real math and lots of guesswork, kicking a model until it starts giving results. [1] https://pjreddie.com/media/files/papers/YOLOv3.pdf

The paper is definitely accessible, but that doesn't mean you don't have to have a solid understanding of math to do this stuff behind the hood. They gloss over stuff like "focal loss" plus this is just an update on some small tweaks they've made, so obviously wouldn't be super math heavy.

All in all, not a great paper to prove your point imo.

Re: TensorFlow, Keras and deep learning, without a PhD

#115

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

Yeah, if you're not programming your models in binary then gtfo imposters!!

On a more serious note, yes understanding this and anything really takes time and investment. The problem for me at least originally not only with ML but originally with engineering back when I was trying to learn (and couldn't afford school), was finding quality sources for getting started in that process of learning. By providing simplified resources like this google one, the hope is that many beginners can get that one "aha!" moment where they start the basic understanding that allows them to start tinkering and learning.

People without a decent understanding shouldn't be submitting research papers, full on agree there. It's basically a waste of everyone's time and harmful for the field of research as a whole as it dilutes the overall signal to noise ratio. However there's so much space in the ML world that doesn't involve research, not only for fun hobby projects, but also even professionally. Resources like this are critical to reducing the knowledge gap out there between researchers and the programmers in the field that work on little ML projects for doing things like sentiment analysis for their company.

These sub-research projects are mission critical at a lot of companies yet are held up at the majority of non-FAANG companies because there's only one data scientist while the teams of engineers are clueless as to how to assist.

Re: TensorFlow, Keras and deep learning, without a PhD

#116
post #109
post #102

Earlier quoted context omitted.

For what it's worth, I've found Pytorch to be much more rigid than TF. Maybe I just haven't found the easy way to do things. For example here's a function that applies an N×N box filter to all but the first 2 dimensions of a tensor (apologies to mobile users): def boxfilter(image, N=3): shape = image.shape image = tf.reshape(image, [1, shape[0], shape[1], -1]) C = image.shape[-1] conv = tf.nn.conv2d(image, tf.eye(C,…

You can almost 1:1 translate this by swapping "tf" and "torch". No need to use nn.Conv2d -- there's a functional API for all these layers: https://pytorch.org/docs/master/nn.functional.html#conv2d Torch doesn't have "same" padding, so you have to manually calculate the correct padding value for your input/output shapes.

That looks great, thanks!

Re: TensorFlow, Keras and deep learning, without a PhD

#117

Earlier quoted context omitted.

I have self taught this material and have been working professionally in the field for some years now. It was primarily driven by the need to solve problems for autonomous systems I was creating. When I am asked how to do it I give the progression I followed. First have preferably a CS background but at least Calc 1&2, Linear Algebra, and University statistics, then: 1. Read "Artificial Intelligence A Modern Approach…

Teaching isn't just about presenting the information to the student. That's basically all you've done. Here, student, read these complex topics and at the end of it all you will have learned machine learning! The art of being a teacher is much more nuanced. You (apparently) fail to present the material in a way that is accessible, relatable, and not overwhelming. For example, the first thing you say to do is go read…

Yes, you are right. I'm not trying to teach the the dozen or so people who have asked, only to lay out a progression with prerequisites similar to what I did for self learning. I certainly do not have time to be creating courses and problem sets or anything more than answering specific questions.

The AIMA book has a lot of open resources around it that I always mention including a full open course I believe, it should all be linked on the site. Although, I also mention that while it is probably not a good idea they can possibly skip it and go right on to the ML course. Both of the Coursera courses are complete with lecture videos and work presented in a very accessible manner including interesting projects.

Re: TensorFlow, Keras and deep learning, without a PhD

#118

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

I share your sentiment to a fair extent and I have written about it before here on HN.

Yes, for a lot of this stuff you don't need a PhD (and frankly I find that marketing weird for the above course). But you do need strong intuitions, understanding of some CS, and math "savviness" i.e. you don't have to know All The Math now, but you should be able to pick up stuff as needed, when you're trying to understand your problem and/or structuring your solution.

One could learn all of this stuff online today - the amount of good resources out there is crazy. Frankly, I am jealous, because I began working on ML more than 10 yr ago, and we were relatively starved for resources on pretty much all fronts: resources to study from (reading material or videos), affordable compute power, s/w libraries. But unfortunately, despite their abundance today, most people don't take the time to dive deep. Of all the years of me suggesting courses and books to people (when asked), only ONE (or maybe two) person managed to go through them to a fair extent. But there is a significant fraction of the rest, for which reductive messaging like "become a pro in AI in 3 weeks" has been misleading.

As a hiring manager sometimes they are as surprised as me, when an interview doesn't go well, after the resume seemed promising to both sides. And to be very clear, I don't blame them (sure, there are some pretentious opportunists, who flat out lie, but I've found them not to be the norm); all this messaging seems to have created a bubble where often you don't know what you don't know. It's amusing that thrice, rejected candidates reached out me saying that the interview was quite eye opening! I have been at the receiving end too - where 90% of the interview seemed to be about some very specific setting of a library or a method, because that was the conception of ML the interviewer had.

I think people should learn, by whatever means, and create stuff because they can - this is the best kind of learning. Silly projects are great too, if they are fun - if they don't advance your understanding, they might motivate you to be less silly! What's missing in this ecosystem is honest messaging about where your skill levels really are. I don't know how to fix it in a way that also doesn't harm, in some way, the widespread learning/awareness reg ML. On a smaller scale though, I have accepted that this has increased my scope of work in screening resumes: if someone lists her github repo, or an arxiv paper, I actually need to spend time to go through them. I don't see this as noise, but a widening of the spectrum of available ML skills in the market; and I need to put in some effort to place an applicant in this spectrum. I've accepted that this is the flipside of working in a hot area: for the multiple job opportunities accessible to me, I have to put in more thought for hiring. I can't have the luxury of the former without the responsibility of the latter. Although, being lazy, I'd totally want to ;).

Re: TensorFlow, Keras and deep learning, without a PhD

#119
post #7

The whole AI/ML stuff has become so hyped up that its probably time to find another topic of interest in software engineering for me. Its a weird melange nowadays where frameworks and "academic credentials" are fused together by major tech companies and leaves me - who has deployed a dozen of classical ML models into production that are still running after couple of years - wondering what this is all about. Overall,…

Instead of saying anti-correlated is better to say "inversely correlated" (or if you mean lack of correlation then "uncorrelated")

You mean "negatively correlated"?

Re: TensorFlow, Keras and deep learning, without a PhD

#120

As a researcher in the field I am not quite sure how I feel about these kind of resources. I am all for making research accessible to a wider audience and I believe that you don't need a PhD, or any degree, to do meaningful work. At the same time, the low barrier of entry and hype has resulted in a huge amount of people downloading Keras, copying a bunch of code, tuning a few parameters, and then putting their result…

I am one of those without a PhD, but have taken the time to the learn the math & contribute quite a bit to this area.

That being said, I also don't think deep learning itself is not really a "science". The issue I have is you can't predict if a network will learn.

We're effectively testing deep learning networks the same way the Romans used to test bridges. Send a bunch of elephants over them, if it holds it's good enough.

There's obviously some indicators of success, but on a whole the overall interaction between components is very difficult to calculate and near-impossible to predict. While I think it's important to understand how layers interact and how a given function will impact your optimization, etc. it's not fully required to have a deep understanding of the mathematics, at least for most cases.

I also personally don't view anything on arXiv worth anything. I typically will read articles/papers myself if reviewing a candidate and / or would like to see their publications at conferences or journals. Otherwise, it's essentially a blog post (which IMO is fine, but will require me to review it).

Post reply on HN