Open source deep learning models that programmers can download and run first try • Hacker News Board

Re: Open source deep learning models that programmers can download and run first try

#41

Post by davedx » Sat, Jun 17, 2017, 12:15 PM UTC

Houshalter wrote: post #21

davedx wrote: post #10

Earlier quoted context omitted.

Using matrices to perform the calculations is an optimization over doing a bunch of for loops. This vectorization results in faster code within higher level languages and on certain hardware platforms (SIMD). It's nothing to do with "slicker notation", although having written gradient descent with for loops and matrix operations, the vectorized version is simpler and cleaner to read in my opinion.

He's not complaining about using vectorization in code. The problem is papers and even explanations targeted at non-experts, often use obfuscated math in place of clear explanations. I've complained about this before here: https://news.ycombinator.com/item?id=13953530 Mathematical notation is basically a programming language. A programming language with weird symbols you can't type to search for, single letter variab…

OK, I see what you're saying. I think you have the same issue with "real" programming languages too. If you compare some very concise Clojure or Scala code with the equivalent in Java, it can be quite hard to understand if you're not very familiar with the language. But I wouldn't necessarily say it's "incredibly bad practice". A Scala programmer can write concise and elegant code that to another Scala programmer is actually faster to understand because of that conciseness. Whereas the same code written with for loops and class method calls and all the boilerplate in Java would take more studying to filter out the low level constructions.

It's about the level of abstraction. And yeah if you don't understand the notation or syntax at the level of abstraction you're studying, it will be very hard.

(FWIW I find Scala code quite hard to understand sometimes, but I also find the more I know about the language, the more comprehensible it gets).

Re: Open source deep learning models that programmers can download and run first try

#42

Post by Houshalter » Sat, Jun 17, 2017, 5:45 PM UTC

davedx wrote: post #41

Houshalter wrote: post #21

Earlier quoted context omitted.

He's not complaining about using vectorization in code. The problem is papers and even explanations targeted at non-experts, often use obfuscated math in place of clear explanations. I've complained about this before here: https://news.ycombinator.com/item?id=13953530 Mathematical notation is basically a programming language. A programming language with weird symbols you can't type to search for, single letter variab…

OK, I see what you're saying. I think you have the same issue with "real" programming languages too. If you compare some very concise Clojure or Scala code with the equivalent in Java, it can be quite hard to understand if you're not very familiar with the language. But I wouldn't necessarily say it's "incredibly bad practice". A Scala programmer can write concise and elegant code that to another Scala programmer is…

It's not necessarily the conciseness that's a problem. Using foreach instead of a full for loop is one thing. What I'm complaining about is code in place of an explanation. E.g. imagine coming across some nasty piece of code like this: https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overv...

It doesn't matter how familiar you are with the language. Without an explanation of what the hell is going on, just looking at the code is useless.

Re: Open source deep learning models that programmers can download and run first try

#43

Post by davedx » Sun, Jun 18, 2017, 4:42 PM UTC

Houshalter wrote: post #42

davedx wrote: post #41

Earlier quoted context omitted.

OK, I see what you're saying. I think you have the same issue with "real" programming languages too. If you compare some very concise Clojure or Scala code with the equivalent in Java, it can be quite hard to understand if you're not very familiar with the language. But I wouldn't necessarily say it's "incredibly bad practice". A Scala programmer can write concise and elegant code that to another Scala programmer is…

It's not necessarily the conciseness that's a problem. Using foreach instead of a full for loop is one thing. What I'm complaining about is code in place of an explanation. E.g. imagine coming across some nasty piece of code like this: https://en.wikipedia.org/wiki/Fast_inverse_square_root#Overv... It doesn't matter how familiar you are with the language. Without an explanation of what the hell is going on, just look…

Now we're talking about documentation. You are correct, no code is conpletely self documenting. But that Quake code is very low level, the opposite of what I think the grandparent doesn't like (very high level abstract notation)

Re: Open source deep learning models that programmers can download and run first try

#44

Post by nilkn » Thu, Jun 22, 2017, 5:42 PM UTC

minkzilla wrote: post #40

nilkn wrote: post #36

Earlier quoted context omitted.

I pretty much agree with you. I have a bachelor's degree in pure mathematics from a top 15 university -- I even published research as an undergraduate. Deep learning strikes me as an experimental science and even a subset of ordinary programming more than it does a subset of mathematics or statistics or statistical learning. I've read the entirety of the Deep Learning book by Goodfellow et. al. It's a fantastic book,…

Do you have any recommendations for books to get into deep learning? I want some theory and math, but not too much because I do not have that much mathamitical background. I have only gone through calculus 2. (I will be taking calc 3 this coming semester). I am a computer science major. I have dabbled in writing a super simple neural network to solve the MNIST. Using an example written in python and porting it to go,…

Once you take calculus 3, I'd recommend diving right into the Deep Learning book: http://www.deeplearningbook.org/

It's definitely the best reference on the subject. With only calculus 3 under your belt the math won't be trivial, but it should overall be fairly approachable and certainly much more so than something like "The Elements of Statistical Learning".