Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GNU Gneural Network

gnu.org

Post reply on HN
Print view View original

51–60 of 111 posts

Re: GNU Gneural Network

#51

Post by mindcrime »

frisco wrote: post #45
mindcrime wrote: post #43

Earlier quoted context omitted.

But that may or may not matter, depending on what you're doing. And how often you do it. If I have a network that I only retrain once a month, I can deal with it taking a day or two to train. Heck, it could take a week as far as that goes. OTOH, it obviously matters a lot if you're constantly iterating and training multiple times a day or whatever.

For state of the art work "a day or two" is pretty fast for a production network, and that's on one or more big GPUs. Not using CUDA is definitely a dealbreaker for any kind of real deep learning beyond the mnist tutorials. It's common to leave a Titan X to run over a weekend; that would be weeks on a CPU.

Well not using CUDA isn't necessarily synonymous with "use a CPU". There is OpenCL. But still, you have a point even if we might quibble over details. This is why I am very much hoping AMD gets serious about Machine Learning and hoping for OpenCL on AMD chips will eventually reach a level of parity (or near parity) with the CUDA on nVidia stuff.

Re: GNU Gneural Network

#52

Post by stevenaleach »

Funny.. The majority of AI research is currently using open source libraries (Theano, Lasagne, Torch, Keras, Scikit-Learn, Nolearn, etc. etc. etc.)

Now Google does have access to a whole lot of data that the rest of the world doesn't. and FB, Google, and etc. have more than a bit of a hardware advantage... for now, at least. Distribute a shared system over a P2P infrastructure, and you can change that. Perhaps rather significantly.

Re: GNU Gneural Network

#53

Post by RealityVoid »

IshKebab wrote: post #50
mindcrime wrote: post #40

Earlier quoted context omitted.

I agree with the general motivation that having too much AI research in the hands of software companies who keep it proprietary harms transparency and progress. But there is already a lot of neural-network free software, so why another package? Not only is there a lot out there, a lot of it was released by companies like IBM[1], Google[2], Yahoo[3], Baidu[4], Microsoft[5], etc. So while I'm generally sympathetic to t…

Yeah they've really missed the fact that it isn't the algorithms or code that we're missing out on. Companies are usually pretty open about these because they know it isn't bit that is hard to compete on. The hard bit is the training data. Good luck collecting 10k hours of transcribed speech, or 10k recordings of "Okay Google".

Huh, I wonder how much truth there is to your words. As an outsider to Machine learning and neural networks, it seems to me that algorithms can be very valuable and big companies do not lack training data. Of course, training data is expensive and very important, but if training data were that important, their most important resource would not be machine learning scientists, but an army of do-monkeys that provide training data. It won't be the victory of the smartest scientist but of the one with the most employees. And that does not seem to be the case.

Re: GNU Gneural Network

#54

Post by _delirium »

gcr wrote: post #49
semisight wrote: post #6

Earlier quoted context omitted.

Almost all of the open source software in the area is permissive-licensed, and relies on non-free components (CUDA). To be honest, I'm not sure how Gneural plans to compete with those packages without support from CUDA or cuDNN, all of which are distinctly not open source.

It's possible to run any of the "major" neural network toolkits (Caffe, Torch, Theano) on CPU-only systems. All of them are permissively licensed (to my knowledge). It will be prohibitively difficult to train the model without some kind of hardware assistance (CUDA). This means that if we're building an ImageNet object detector, even if the code implements the model correctly the first time, training it to have close…

> This is about as permissively as it could be licensed because the ImageNet training data itself is under an academic-only license anyway.

Does this necessarily follow, that a machine-learning model is a derived work of all data it's trained on? As far as I know, the law in this area isn't really settled. And many companies are operating on the assumption that this isn't the case. It would lead to some absurd conclusions in some cases, for example if you trained a model to recognize company logos, you'd need permission of the logos' owners to distribute it.

(This is assuming traditional copyright law; under jurisdictions like the E.U. that recognize a separate "database right" it's another story.)

Re: GNU Gneural Network

#56

Post by DigitalJack »

_delirium wrote: post #3

I agree with the general motivation that having too much AI research in the hands of software companies who keep it proprietary harms transparency and progress. But there is already a lot of neural-network free software, so why another package? For example, these widely used packages are free software, and seemingly more featureful: http://torch.ch/ , http://www.deeplearning.net/software/theano/ , http://pybrain.org/…

Not only that, but GNU has deliberately obfuscated code (GCC) to prevent people from using it in ways GNU doesn't approve (but are allowed by license).

Re: GNU Gneural Network

#57

Post by sjtgraham »

IshKebab wrote: post #50
mindcrime wrote: post #40

Earlier quoted context omitted.

I agree with the general motivation that having too much AI research in the hands of software companies who keep it proprietary harms transparency and progress. But there is already a lot of neural-network free software, so why another package? Not only is there a lot out there, a lot of it was released by companies like IBM[1], Google[2], Yahoo[3], Baidu[4], Microsoft[5], etc. So while I'm generally sympathetic to t…

Yeah they've really missed the fact that it isn't the algorithms or code that we're missing out on. Companies are usually pretty open about these because they know it isn't bit that is hard to compete on. The hard bit is the training data. Good luck collecting 10k hours of transcribed speech, or 10k recordings of "Okay Google".

> Good luck collecting 10k hours of transcribed speech

I'm sure that nearly every DVD theatrical release has subtitles available. Speech against a wide range of background noise too, e.g. music, explosions, traffic, normal ambient noise, etc.

Seems a good start for acquiring a large corpus of labelled speech.

Re: GNU Gneural Network

#58

Post by gcr »

akhilcacharya wrote: post #55

Is there more being done to promote GPU acceleration on non-CUDA platforms? I feel like this would be more useful than yet another FOSS NN library.

Torch has rudimentary OpenCL support. Some things "sort of" work. https://github.com/hughperkins/cltorch Theano has been slowly working on integrating OpenCL support too for several years, but I'm not aware if it's supported or not.

Nvidia has a complete monopoly on all deep learning hardware and tooling. With the possible exception of Google (and maybe Facebook), 100% of all serious academic researchers are training their models on Nvidia hardware with Nvidia's propietary CUDA toolkit. Using anything else is currently completely unthinkable. Amazon and Nvidia have even teamed up to make CUDA training cheap (on the short term) for EC2 users.

I'd love to be able to switch to OpenCL, but there's so much momentum and very little perceived benefit when your lab already has four (very expensive) Titan X cards.

Re: GNU Gneural Network

#59

Post by gcr »

sjtgraham wrote: post #57
IshKebab wrote: post #50

Earlier quoted context omitted.

Yeah they've really missed the fact that it isn't the algorithms or code that we're missing out on. Companies are usually pretty open about these because they know it isn't bit that is hard to compete on. The hard bit is the training data. Good luck collecting 10k hours of transcribed speech, or 10k recordings of "Okay Google".

> Good luck collecting 10k hours of transcribed speech I'm sure that nearly every DVD theatrical release has subtitles available. Speech against a wide range of background noise too, e.g. music, explosions, traffic, normal ambient noise, etc. Seems a good start for acquiring a large corpus of labelled speech.

Models trained on DVD audio are considered derived works. You certainly couldn't release such a model under the GPL.

You also have to solve the (very difficult) subtitle alignment problem before you could begin training.

Re: GNU Gneural Network

#60

Post by gcr »

_delirium wrote: post #54
gcr wrote: post #49

Earlier quoted context omitted.

It's possible to run any of the "major" neural network toolkits (Caffe, Torch, Theano) on CPU-only systems. All of them are permissively licensed (to my knowledge). It will be prohibitively difficult to train the model without some kind of hardware assistance (CUDA). This means that if we're building an ImageNet object detector, even if the code implements the model correctly the first time, training it to have close…

> This is about as permissively as it could be licensed because the ImageNet training data itself is under an academic-only license anyway. Does this necessarily follow, that a machine-learning model is a derived work of all data it's trained on? As far as I know, the law in this area isn't really settled. And many companies are operating on the assumption that this isn't the case. It would lead to some absurd conclu…

I'm not aware of the formal legality of it, but I don't see why it wouldn't be the case. Without the training data, the model can't work. That seems to fit the definition of "derivative work".
Post reply on HN