Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep Learning in JavaScript

github.com

Post reply on HN
Print view View original

61–70 of 92 posts

Re: Deep Learning in JavaScript

#61

Post by foolswisdom »

4hg4ufxhy wrote: post #53
QuadmasterXLII wrote: post #51

Earlier quoted context omitted.

In python when I try a new gpu accelerated array library, to write norm_arr = arr / lib.sqrt( lib.sum( arr*2, axis=-1, keepdims=True) , I have to read documentation for sum to see whther they use axis or dim in sum. In javascript, to write the same thing I need to read documentation for a sum function, a broadcasted division function, a multiply function. I can probably assume that the sqrt function behaves and is na…

Why can you make assumptions about operation overloads but not functions?

Because there is nothing to make assumptions about. In the example code, both multiplication and division have a scalar on one side, there's no possible ambiguity of behavior. But there is the eternal question of terminology: do you specify dimensions by "axis" or "dim" and does your API actually use both terms in different places?

(that's what I think the GP meant, anyway).

Re: Deep Learning in JavaScript

#62

Post by eiriklv »

Edit: Great work! I'd love to have a nice alternative to PyTorch in JavaScript :)

Edit: formatting

Making JavaScript look like Python in this case could potentially bite you in the ass.

From the example:

    const torch = require("js-pytorch");

    // Instantiate Tensors:
    x = torch.randn([8,4,5]);
    w = torch.randn([8,5,4], requires_grad = true);
    b = torch.tensor([0.2, 0.5, 0.1, 0.0], requires_grad = true);
And:

    class Transformer extends nn.Module {
      constructor(vocab_size, hidden_size, n_timesteps, n_heads, p) {
        //.....
        this.b1 = new nn.Block(hidden_size, hidden_size, n_heads, n_timesteps, dropout_p=p);
For both `requires_grad` and `dropout_p` you wouldn't be able to change the ordering + you're creating global variables.

    /**
     * All of the arguments for this function are positional
     * and cannot be provided in a different order than defined
     */
    function performOperation(values, arg1 = false, arg2 = -1) {
        //....
    }

    /**
     * This works, but only because of the order
     */
    const result = performOperation([1, 2, 3], arg1 = true, arg2 = 10);

    /**
     * This does not work
     */
    const result = performOperation([1, 2, 3], arg2 = 10, arg1 = true);

    /**
     \* What is actually happening
     \*/
    arg1 = true; // global variable
    arg2 = 10;   // global variable

    const result = performOperation([1, 2, 3], 10, true);

Re: Deep Learning in JavaScript

#63

Post by eduardoleao052 »

eiriklv wrote: post #62

Edit: Great work! I'd love to have a nice alternative to PyTorch in JavaScript :) Edit: formatting Making JavaScript look like Python in this case could potentially bite you in the ass. From the example: const torch = require("js-pytorch"); // Instantiate Tensors: x = torch.randn([8,4,5]); w = torch.randn([8,5,4], requires_grad = true); b = torch.tensor([0.2, 0.5, 0.1, 0.0], requires_grad = true); And: class Transfor…

Thats true, it’s a limitation of working between these languages. I tried to mitigate it by using clear JSDoc, so that each variable pops up alongside an explanation when calling a function.

Re: Deep Learning in JavaScript

#64

Post by eiriklv »

eduardoleao052 wrote: post #63
eiriklv wrote: post #62

Edit: Great work! I'd love to have a nice alternative to PyTorch in JavaScript :) Edit: formatting Making JavaScript look like Python in this case could potentially bite you in the ass. From the example: const torch = require("js-pytorch"); // Instantiate Tensors: x = torch.randn([8,4,5]); w = torch.randn([8,5,4], requires_grad = true); b = torch.tensor([0.2, 0.5, 0.1, 0.0], requires_grad = true); And: class Transfor…

Thats true, it’s a limitation of working between these languages. I tried to mitigate it by using clear JSDoc, so that each variable pops up alongside an explanation when calling a function.

I feel you - Python has much better (more flexible) argument support than JavaScript in this case. Converting the entire set of arguments into a keyed object is usually what happens, but then it wouldn't look like PyTorch anymore.

Re: Deep Learning in JavaScript

#65

Post by throwaway4aday »

QuadmasterXLII wrote: post #51
throwaway4aday wrote: post #48

Earlier quoted context omitted.

What's wrong with creating a function that does those things? It would be less surprising to people new to the library, would be self-documenting by having a name and an easily inspected declaration with named arguments, and it would be idiomatic JS. You could also have variants that are purely functional and return a new value or ones that mutate in place that you could use depending on your needs.

In python when I try a new gpu accelerated array library, to write norm_arr = arr / lib.sqrt( lib.sum( arr*2, axis=-1, keepdims=True) , I have to read documentation for sum to see whther they use axis or dim in sum. In javascript, to write the same thing I need to read documentation for a sum function, a broadcasted division function, a multiply function. I can probably assume that the sqrt function behaves and is na…

If each of those operators were implemented as functions then you'd have different names for different implementations in order to avoid confusion over what type of division or multiplication they were performing. It's more verbose but that's a good thing since it prevents you from making incorrect assumptions about what's going to happen when you do a * b.

Re: Deep Learning in JavaScript

#66

Post by throwaway4aday »

goatlover wrote: post #55
modeless wrote: post #26

Someone needs to do a TypeScript compiler plugin to add multidimensional array slicing and operator overloading to the language, so these libraries can actually work the way PyTorch does. I know operator overloading is controversial, but the way it allows automatic differentiation to work transparently through regular arithmetic expressions is very helpful. Without it these libraries will never feel like PyTorch. Jav…

Would JS be faster than Python when it comes to Pytorch? For example, I seriously doubt that would be the case for Numpy, since it's a wrapper for C code, with the ability to use Fortran libraries for optimization.

The benefit of having it in JS is not speed but portability and access to the JS ecosystem of tools. Having the code run in the browser without needing a complex setup is a huge benefit for sharing demos. Node.js provides a way to use native code as well and it's quite commonly used https://github.com/nodejs/node-gyp so there's no reason you couldn't use those same or similar libraries in a JS implementation.

Re: Deep Learning in JavaScript

#67

Post by fulafel »

goatlover wrote: post #55
modeless wrote: post #26

Someone needs to do a TypeScript compiler plugin to add multidimensional array slicing and operator overloading to the language, so these libraries can actually work the way PyTorch does. I know operator overloading is controversial, but the way it allows automatic differentiation to work transparently through regular arithmetic expressions is very helpful. Without it these libraries will never feel like PyTorch. Jav…

Would JS be faster than Python when it comes to Pytorch? For example, I seriously doubt that would be the case for Numpy, since it's a wrapper for C code, with the ability to use Fortran libraries for optimization.

Very much the opposite, since this is pure JS. PyTorch uses tuned native-code and GPU components for the heavy lifting and can in some cases compile Python code using PyTorch JIT / torch.compiler / torch.fx.

Re: Deep Learning in JavaScript

#68

Post by _ea1k »

goatlover wrote: post #55
modeless wrote: post #26

Someone needs to do a TypeScript compiler plugin to add multidimensional array slicing and operator overloading to the language, so these libraries can actually work the way PyTorch does. I know operator overloading is controversial, but the way it allows automatic differentiation to work transparently through regular arithmetic expressions is very helpful. Without it these libraries will never feel like PyTorch. Jav…

Would JS be faster than Python when it comes to Pytorch? For example, I seriously doubt that would be the case for Numpy, since it's a wrapper for C code, with the ability to use Fortran libraries for optimization.

It'd be shocking if it was. PyTorch isn't particularly slow.

Re: Deep Learning in JavaScript

#69

Post by _ea1k »

ralusek wrote: post #30

Many people seem to be unaware of tensorflow.js, an official JS implementation of TF https://github.com/tensorflow/tfjs I'd love to see PyTorch in JS, but I think unless you get it running on the GPU it won't be able to do much.

I had the same thought, but it does seem that tensorflow usage is in steady decline.

Re: Deep Learning in JavaScript

#70

Post by throwaway4aday »

ipsum2 wrote: post #47
fire_lake wrote: post #45

Earlier quoted context omitted.

> nodejs if you wanted to, but why? Node.js is better backend than something like Flask.

Performance is a lot worse on NodeJS with a WebAssembly/WebGL backend versus Flask with a PyTorch/CUDA backend.

If you're using Node you can write whatever you want in C++ and then add a binding to call it from within your Node app. Don't need WebGL.
Post reply on HN