[deleted]
Neural networks in JavaScript – free 19-part course
51–60 of 74 posts
Re: Neural networks in JavaScript – free 19-part course
#52Hey guys! I'm the creator of the course and lead developer of brain.js and would love to answer any questions you may have.
Hey Axel! Just started watching but the interactive guide is incredible. It's like having Screenhero with a private tutor.
Only thing is, I kind of wish there were something like the time-coded comments SoundCloud has. At the end of the second guide, we're asked to play around with the tests.
I added: console.log(net.run([0, 4])); console.log(net.run([3, 3])); console.log(net.run([8, 4]));
Based on the training data, I would expect this to resolve to ~4(or 1), ~0, ~8(or 1) by standard logic expectancies (if same return 0, else return the higher number or 1). But instead I received ~0, ~0, ~0.
It's not immediately obvious what is causing this. But it seems like the model created is inherently ignorant of basic logic (at least by my narrow definition), and there isn't any immediate discussion of caveats as to error margin.
I'll admit this might be a n00bish concern based on never programing neural nets before, but as this guide seems focused on introducing NN's to n00bs like me: a way to discuss concerns with other viewers/the author would be amazing.
Aside from that, incredible work! I'll keep watching to see if I can figure out my misunderstandings.
Update: Just discovered the Q&A tab, this should likely be adequate for my concerns. Well done. This may be the best online demo/tutorial I've ever seen.
Re: Neural networks in JavaScript – free 19-part course
#53Earlier quoted context omitted.
There is nothing wrong to use js, like there is nothing wrong to call it impractical. JS is faster than Python, but in the land of DL, there is C++ and everyone else. No one is using Python to do the actual computation anyway.
"No one is using Python to do the actual computation anyway." Depends what "actual" computation is. If you're definition of "actual computation" is something which requires extreme optimization then your definition precludes the question. In the real world there is an extraordinary amount of computation done with inefficient languages though simply because development time costs very often outweighs run time costs.
Python is used because there are C/C++ bindings for working with NNs.
The point is that none of computations are done in pure Python. Python just provides a convenient wrapper over non-Python code.
Re: Neural networks in JavaScript – free 19-part course
#54why would you do ML in JS?
exactly, JS for ML sounds like a stretch indeed.
Re: Neural networks in JavaScript – free 19-part course
#55Earlier quoted context omitted.
Cost as in money. To deploy a model in js (as a web page) all you need is a static S3 or GCS bucket. You don't even need a webserver and it can automatically handle infinite scale. Show me a python solution that can do the same.
> To deploy a model in js (as a web page) all you need is a static S3 or GCS bucket. First of all, what you have described is far from the reality. Had this come true, only inference will be in javascript, through some language agnostic standardized model format, not training. The model is just a blackbox function for the js runtime to call. The amount of javascript to make this happen will be surprisingly slim anywa…
> First of all, what you have described is far from the reality.
Did you try any of the links I included? This is the reality for all of them and they are a few years old. They have a model file loaded from bucket url and never make another network request thereafter.
It's actually the recommended workflow from https://js.tensorflow.org/ where you'd find tons of other examples.
But you're right, this is for inference only. I would not do training in JS.
Re: Neural networks in JavaScript – free 19-part course
#56Earlier quoted context omitted.
Cost as in money. To deploy a model in js (as a web page) all you need is a static S3 or GCS bucket. You don't even need a webserver and it can automatically handle infinite scale. Show me a python solution that can do the same.
There is currently no NN learning algorithm that can handle massively paralleled training. We can use some simple fixes such as mean gradient but they have severe limitations, and their limitation grows as you scale up. Currently even with a handful paralleled GPU training, the gradient computation needs to wait for all GPU batch to complete and then return to CPU before the next batch, so your idea of infinite scali…
Yea the infinite scaling was referring to deployment of inference, not training.
Re: Neural networks in JavaScript – free 19-part course
#57Earlier quoted context omitted.
One very practical reason is to save on computation costs. GPU servers are not cheap and if you're going to run on CPU anyway, you might as well run it locally on the user's browser. Some demos such as real time object detection isn't possible at all if you had to pay roundtrip server latency (not to mention the complexity of streaming video to and from your server): https://github.com/ModelDepot/tfjs-yolo-tiny And a…
Are you suggesting distributed NN training through user's browsers? This is a very sketchy area along the lines of browser based bitcoin mining. You don't want to run any computation heavy code on client side without user's knowledge & approval. Though if you only meant performing inference in user's browsers, then the challenge would be to find a way for TF/pytorch pre-trained models to port and perform accurately i…
See https://js.tensorflow.org/ for automatically porting tensorflow models to the browser
Re: Neural networks in JavaScript – free 19-part course
#58Why the heck Javascript?
Personally I've found that Python - or perhaps more Numpy et al - are impenetrable for a learner. I am sure the data structures that are used in Python + Numpy et al are powerful and well-suited for the task, but as a learner coming to this with minimal knowledge trying to ALSO learn the idiosyncrasies and weirdness (IMHO) of how Numpy does things and the weirdness (IMHO) of how Numpy even names things (e.g. in decades of programming, the term "shape" for an array was new on me) was an extra burden.
Doing the same thing in Javascript with good old-fashioned arrays using good old-fashioned terminology clears the fog and makes things simpler for people who are not already fluent in numpy's data structures and terminology
Javascript is - in my view - the equivalent of a "business english" of programming, i.e. even if you aren't fluent, its syntax and terminology is familiar to C/C++/Java/C#/Golang/ObjectiveC/Perl/etc that most people will at least be able to understand what is going on in the same way that business people who might not be fluent in English will at least be able to understand and basically communicate with each other even if they perhaps will not be writing Sonnets. Python feels like a niche language that developed in isolation and is only readable to people who have actually gone out of their way to learn it.
Re: Neural networks in JavaScript – free 19-part course
#59Is there a Numpy equivalent for JS yet? How easy is it to use (considering that JS doesn't have operator overloading)?
Not effectively. From what I've read, JS lacks SIMD support which makes Numpy-like vectorizations unachievable. Would be nice though...
I think Numpy is written in C, so in principle they could use the same approach for JS (and invoke SIMD functions from C). The problem, however, is that there is no operator overloading in JS. So you can't write things like a[:,:,3] *= 2.