Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Build a Neural Network

enlight.nyc

Post reply on HN
Print view View original

41–49 of 49 posts

Re: Build a Neural Network

#41

Post by fartcannon »

peterhj wrote: post #8
fartcannon wrote: post #2

As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…

I'm curious if you have a particular language/microcontroller/microwave/badger you have in mind? Depending on which, YMMV.

No, not off hand I don't. It's just something I've noticed in all these make a neural network posts. Feels a little like they're just drawing the rest of the owl, if you know what I mean. But thank you.

Re: Build a Neural Network

#42

Post by fartcannon »

felipellrocha wrote: post #22
fartcannon wrote: post #2

As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…

Yep. I would like to see an article that implements everything without using matrices first, then creates the matrices library with you, and refactors everything over. So much learning that we're missing by not going through this step.

To me it feels a bit like that joke about drawing instructions. "1. Draw some circles. 2. Now draw the rest of the owl."

Re: Build a Neural Network

#43

Post by fartcannon »

whatshisface wrote: post #6
fartcannon wrote: post #2

As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…

The complicated parts of Numpy are themselves a wrapper for the seminal LAPACK: http://www.netlib.org/lapack/ . It has C language APIs, so that might help you with what you need to do.

That's very interesting, thank you.

Re: Build a Neural Network

#44

Post by fartcannon »

pilooch wrote: post #21
fartcannon wrote: post #2

As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…

Exactly the reason why my colleagues and myself do all deep learning in C++, performance and portability, from cloud to RPie. We've even modified caffe2 so we could build the training graph from pure C++. We know this is not the current doxa :) It's also all open sourced just in case others might need it...

Link? I love C++ and would love to see it.

Re: Build a Neural Network

#45

Post by fartcannon »

cscheid wrote: post #31
fartcannon wrote: post #2

As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…

Hm. I just finished teaching an ML course where all of the assignments were pure Python (on purpose, so students would actually have the chance to see all of the code). One of the assignments included implementing reverse-mode autodiff and a NN classifier on top. It can be done in ~600 lines of clear python, serious!

Which course?

Re: Build a Neural Network

#46

Post by fartcannon »

gnulinux wrote: post #9
fartcannon wrote: post #2

As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…

Matrix math is easy peasy. Freshman level programming. Just lookup algorithms on Wikipedia and you're all set. The problem is it's extremely hard to make it efficient. Dozens of men-years are spent trying to optimize linear algebra libraries. There are handful linalg libraries that have competitive performance. It was my college project to make a fast linalg library, and boy it is fast. There are some things like mat…

Easy peasy but not optimized sounds like it would fit with these from scratch tutorials that pop up everyone now and then.

Perhaps it is overkill. It's just not actually from scratch without it, you know?

Re: Build a Neural Network

#48

Post by animal531 »

asdfman123 wrote: post #17
fartcannon wrote: post #2

As someone who has read a lot of implementing neural networks from articles, the massive problem with all of them is that they import numpy. You may think that it is silly to reimplement the matrix math but with out that part of the code, you can't easily port it to other languages/microcontrollers/microwaves/badgers. It's a legitimately valid part of machine learning, and its not easy to do for novices. And I need h…

It is easy to do unless you don't code at all, or are completely confused by the math. I'm a C# developer and I'm sure it would take me all of about 30 seconds to install a matrix multiplication package through nuget. I'm sure it would be immediately obvious how to add items to matrices or do a dot product.

I'm a C# developer who wrote my own implementation (with help from random tutorials etc).

It was dead easy to get code examples as needed.

Re: Build a Neural Network

#49

Post by cscheid »

fartcannon wrote: post #45
cscheid wrote: post #31

Earlier quoted context omitted.

Hm. I just finished teaching an ML course where all of the assignments were pure Python (on purpose, so students would actually have the chance to see all of the code). One of the assignments included implementing reverse-mode autodiff and a NN classifier on top. It can be done in ~600 lines of clear python, serious!

Which course?

https://cscheid.net/courses/spr19/csc665/

The assignments are not directly available, but my email is easy to find.

Post reply on HN