Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GNU Gneural Network

gnu.org

Post reply on HN
Print view View original

71–80 of 111 posts

Re: GNU Gneural Network

#71

Post by argonaut »

pmarreck wrote: post #70
_delirium wrote: post #3

I agree with the general motivation that having too much AI research in the hands of software companies who keep it proprietary harms transparency and progress. But there is already a lot of neural-network free software, so why another package? For example, these widely used packages are free software, and seemingly more featureful: http://torch.ch/ , http://www.deeplearning.net/software/theano/ , http://pybrain.org/…

Do you know of anything besides Microsoft's GUI tool that lets someone simply specify the various attributes of the network they want, and it simply creates it, ready to be trained and tested and consuming data?

For image data, Caffe (http://caffe.berkeleyvision.org/) is dominant (most deep learning computer vision research I see is done with Caffe). However, that's the chief problem with Caffe - it's very difficult to extend because it revolves so much around "just specify attributes" and train.

Re: GNU Gneural Network

#72

Post by imglorp »

gcr wrote: post #59
sjtgraham wrote: post #57

Earlier quoted context omitted.

> Good luck collecting 10k hours of transcribed speech I'm sure that nearly every DVD theatrical release has subtitles available. Speech against a wide range of background noise too, e.g. music, explosions, traffic, normal ambient noise, etc. Seems a good start for acquiring a large corpus of labelled speech.

Models trained on DVD audio are considered derived works. You certainly couldn't release such a model under the GPL. You also have to solve the (very difficult) subtitle alignment problem before you could begin training.

Really?

It's not like you could take the neural net weights aggregated from thousands of movies and retrieve any form of entertainment from them. Is a derived work anything at all based on an original, or just something in the similar field, ie entertainment->entertainment?

Re: GNU Gneural Network

#73

Post by argonaut »

mindcrime wrote: post #43
sseveran wrote: post #31

Earlier quoted context omitted.

Its going to need to use CUDA or it will not be competitive with alternatives. CUDA makes training networks more than an order of magnitude faster.

But that may or may not matter, depending on what you're doing. And how often you do it. If I have a network that I only retrain once a month, I can deal with it taking a day or two to train. Heck, it could take a week as far as that goes. OTOH, it obviously matters a lot if you're constantly iterating and training multiple times a day or whatever.

The difference is between training taking a week, and training taking 10 weeks.

It takes a week to train a standard AlexNet model on 1 GPU on ImageNet (and this is pretty far from state of the art).

It takes 4 GPUs 2 weeks to train a marginally-below state of the art image classifier on ImageNet (http://torch.ch/blog/2016/02/04/resnets.html) - the 101 layer deep residual network. This would be 20 weeks on an ensemble of CPUs. (State of the art is 152 layers; I don't have the numbers but I'd guess-timate 3-4 weeks to train on 4 GPUs).

Re: GNU Gneural Network

#74

Post by anonbanker »

If you were an AI (software), and you had to pick a license to release your source code under, one would assume you would pick the GPL, as it retains as much freedoms as a piece of software could ever expect in a world full of us.

Re: GNU Gneural Network

#75

Post by over »

anonbanker wrote: post #74

If you were an AI (software), and you had to pick a license to release your source code under, one would assume you would pick the GPL, as it retains as much freedoms as a piece of software could ever expect in a world full of us.

If I was an AI, I would release my source code as public domain or BSD. That way, big corporations would start using me and I'd have access to the world's financial and defense systems.

Shit, maybe I'm an AI.

Re: GNU Gneural Network

#76

Post by dgacmu »

IshKebab wrote: post #50
mindcrime wrote: post #40

Earlier quoted context omitted.

I agree with the general motivation that having too much AI research in the hands of software companies who keep it proprietary harms transparency and progress. But there is already a lot of neural-network free software, so why another package? Not only is there a lot out there, a lot of it was released by companies like IBM[1], Google[2], Yahoo[3], Baidu[4], Microsoft[5], etc. So while I'm generally sympathetic to t…

Yeah they've really missed the fact that it isn't the algorithms or code that we're missing out on. Companies are usually pretty open about these because they know it isn't bit that is hard to compete on. The hard bit is the training data. Good luck collecting 10k hours of transcribed speech, or 10k recordings of "Okay Google".

+this. Freely available data is a huge value to everyone. We use it in academia, and it's useful in companies big and small. (Even at Google -- I started exploring some of my research questions using MNIST and Imagenet because they're baselines that allow reproducibility, and because you don't have to deal with the privacy issues. For amusing anecdotes about this, consider what the Smart Reply team had to do: http://googleresearch.blogspot.com/2015/11/computer-respond-... It's much harder to train a network when you can't ever look at the training data!)

The best thing the assorted communities involved in this effort could do to accelerate the advance of open, accessible machine learning is to create good creative-commons datasets that anyone could use to train models that could be released open source. And as an academic, let me say that hundreds of researchers would figuratively kiss the ground you walk on for doing so. :)

Re: GNU Gneural Network

#77

Post by rck »

The implementations look odd. A network consists of a collection of neurons, which are implemented individually as structs. The forward pass through the network is a series of nested loops, and the gradient descent implementation doesn't use backpropagation - it uses finite differences to approximate derivatives, which is known to be inefficient. Given the overall design of the library, it isn't really clear what you would use it for in practice.

I hope that future versions take inspiration from other open source machine learning libraries, which show how to use linear algebra and backpropagation and are much more effective.

Re: GNU Gneural Network

#78

Post by SXX »

cwyers wrote: post #22
semisight wrote: post #6

Earlier quoted context omitted.

Almost all of the open source software in the area is permissive-licensed, and relies on non-free components (CUDA). To be honest, I'm not sure how Gneural plans to compete with those packages without support from CUDA or cuDNN, all of which are distinctly not open source.

Why not focus on adding GPLed code to an existing package with a GPL-friendly license?

FSF want to be copyright holder for all of it projects code so it's possible to relicense codebase under newer version of GPL. For same reason everyone contribute to their projects must sign CLA.

Re: GNU Gneural Network

#79

Post by voltagex_ »

dgacmu wrote: post #76
IshKebab wrote: post #50

Earlier quoted context omitted.

Yeah they've really missed the fact that it isn't the algorithms or code that we're missing out on. Companies are usually pretty open about these because they know it isn't bit that is hard to compete on. The hard bit is the training data. Good luck collecting 10k hours of transcribed speech, or 10k recordings of "Okay Google".

+this. Freely available data is a huge value to everyone. We use it in academia, and it's useful in companies big and small. (Even at Google -- I started exploring some of my research questions using MNIST and Imagenet because they're baselines that allow reproducibility, and because you don't have to deal with the privacy issues. For amusing anecdotes about this, consider what the Smart Reply team had to do: http://…

>Another bizarre feature of our early prototype was its propensity to respond with “I love you” to seemingly anything. As adorable as this sounds, it wasn’t really what we were hoping for.

Re: GNU Gneural Network

#80

Post by jonathankoren »

ycmbntrthrwaway wrote: post #44
rand1012 wrote: post #37

Anyone else notice how GNU's website is stuck in 1993?

No, it was updated since then. Its header/footer format is certainly was not common in 1993, it has search box and things like that. Anyway, it is usable, does not require JavaScript, and loads really fast.

Also, it sets the background colors. You couldn't do that until like HTML 3 in 1995.

However, META ICBM, is a joke as old as the META tag, which I guess is 1995.

Post reply on HN