How to build your own neural network from scratch in Python • Hacker News Board

Re: How to build your own neural network from scratch in Python

#41

Post by [deleted] » Fri, Oct 05, 2018, 5:24 PM UTC

ShorsHammer wrote: post #16

a_bonobo wrote: post #2

Too often you see articles like this and they start with $ import a_whole_bunch_of_stuff Good to see that this is not the case here :) The fast.ai course has a similar exercise in the beginning, but you'll still import the weights from somewhere else. Their fast.ai v1 library has a very short implementation too (loading the MINIST example dataset and then using Resnet18): from fastai import * from fastai.data import…

It's not mentioned in the article but he is importing numpy. It has around 100k LOC.

[deleted]

Re: How to build your own neural network from scratch in Python

#42

Post by MrUnderBridje » Fri, Oct 05, 2018, 5:24 PM UTC

sprobertson wrote: post #11

a_bonobo wrote: post #2

Too often you see articles like this and they start with $ import a_whole_bunch_of_stuff Good to see that this is not the case here :) The fast.ai course has a similar exercise in the beginning, but you'll still import the weights from somewhere else. Their fast.ai v1 library has a very short implementation too (loading the MINIST example dataset and then using Resnet18): from fastai import * from fastai.data import…

How is your example not "$ import a_whole_bunch_of_stuff"?

You always have to start somewhere.

Re: How to build your own neural network from scratch in Python

#43

Post by mos_basik » Fri, Oct 05, 2018, 5:43 PM UTC

MrUnderBridje wrote: post #38

FartyMcFarter wrote: post #29

Earlier quoted context omitted.

> I guess that there's always a software layer which may be considered "backbone". The point is that you don't need any software layers at all to code up a basic neural network implementation. A programming language with basic floating-point operations is all you need. The algorithms are not complicated so even x86 Assembly is practical for this purpose if you're already experienced with it. So the "backbone" can sim…

> The point is that you don't need any software layers at all to code up a basic neural network implementation. I grafted my answer to the wrong post, I meant to respond to the guy talking about the project implicitely using Numpy. Offtopic remark: It would be nice to be able to move a post to a different thread, or make a single answer to several messages.

>single answer to several messages

Always surprises me that out of all the different kinds of internet communities, it's only the oft-maligned imageboards that consistently provide that feature.

I was kind of expecting Discourse to play with the idea, since they seem to be the modern-reboot-of-forums with the most traction + willingness to experiment, but they haven't so far, afaik.

Re: How to build your own neural network from scratch in Python

#44

Post by spicytunacone » Fri, Oct 05, 2018, 9:16 PM UTC

mos_basik wrote: post #43

MrUnderBridje wrote: post #38

Earlier quoted context omitted.

> The point is that you don't need any software layers at all to code up a basic neural network implementation. I grafted my answer to the wrong post, I meant to respond to the guy talking about the project implicitely using Numpy. Offtopic remark: It would be nice to be able to move a post to a different thread, or make a single answer to several messages.

>single answer to several messages Always surprises me that out of all the different kinds of internet communities, it's only the oft-maligned imageboards that consistently provide that feature. I was kind of expecting Discourse to play with the idea, since they seem to be the modern-reboot-of-forums with the most traction + willingness to experiment, but they haven't so far, afaik.

How would you represent that? On HN and similar every response is its own thread. This is true with email as well, but I don't have experience with any significant threads or mailing lists.

The naïve solution would be to just limit nesting and order nested comments chronologically. Another solution may be to CC @postid in your messages and have posts display "responding to: id1, id4" at the top and "responses: id91" at the bottom with appropriate links. There's a question here about which post to write or display your response under. Finally, an extravagant solution would be to really represent discussion as a graph and I think a lot of time would need to be spent creating an interface with minimal frustration as I feel it can get really involved (i.e., a lot of clicking).

Curious what you or anyone else thinks.

Re: How to build your own neural network from scratch in Python

#45

Post by jononor » Sat, Oct 06, 2018, 10:47 AM UTC

claytonjy wrote: post #40

plg wrote: post #32

Honest question: what are the reasons to code this up using OOP, creating a neural network object plus methods, instead of data structures and functions that operate on them? If it’s just personal preference, I’m fine with that, I’m not trying to start a flame war.

Python, both the language and community, are very strong proponents of OOP. While you can do a lot of more functional stuff, esp. w/ functools, the community at large tends to discourage that. "Never use map/filter" is a weirdly common phrase among pythonistas. So this, like most python-driven examples, is doing things in a pythonic way. Coming from the R side, I tend to prefer structures & functions as well, but if…

The never use map/filter is in favor of list comprehensions, which is not really a OOP construct.

But sure, it's a conventional thing. And for a long time the built-in alternatives to a class for such a datastructure tuples and dicts, neither which are very nice for functions to operate on (dict values have to be addressed with d['key'] instead of d.key). With a class and method there is also no doubt as to what the function operates on, which is convenient when there type hints and IDE support is missing. This is changing since Python 3.5 and type checking tools like MyPy.

Since Python 3.7 there are also data classes, a primitive for classes which just hold values. https://www.linuxjournal.com/content/introducing-python-37s-... But it will take a while before programming conventions change.

Re: How to build your own neural network from scratch in Python

#46

Post by zodPod » Thu, Oct 11, 2018, 6:59 PM UTC

a_bonobo wrote: post #2

Too often you see articles like this and they start with $ import a_whole_bunch_of_stuff Good to see that this is not the case here :) The fast.ai course has a similar exercise in the beginning, but you'll still import the weights from somewhere else. Their fast.ai v1 library has a very short implementation too (loading the MINIST example dataset and then using Resnet18): from fastai import * from fastai.data import…

There's a guy who makes youtube videos like this a lot. He's a good guy and a smart guy but seems to either not care or not realize he isn't helping. I've seen videos like "Computer Vision In 5 lines" where the first line is "import helperclass.py" or something like that and that helper class has like 1500 lines of code that he wrote to implement. Sure people need to be aware of what they're learning but if someone has no programming experience and finds that video they're going to just think it's magic.