Basics of Neural Networks with example codes and illustrations • Hacker News Board

Re: Basics of Neural Networks with example codes and illustrations

#21

Post by [deleted] » Fri, Aug 30, 2013, 5:00 PM UTC

level09 wrote: post #15

JDDunn9 wrote: post #10

Does anyone have any examples of areas where neural networks beat out statistical based methods, other than maybe image recognition? I can't even think of another major area where they dominate. - Search engines use algorithms, not neural nets. - The most popular algorithm on Kaggle (data analysis competitions) is random forests - Google's self-driving car uses statistical-based methods I can't imagine commercial air…

We use Neural Networks to extract text transcripts from our video stream, the process is extremely complicated as the stream is in Arabic which is a fairly complex language. the system is fed with the actual transcript for some videos so it can learn with time and minimize error. the overall system is incredibly useful as you can search the video stream of a TV channel for a specific keyword. I have also used self or…

[deleted]

Re: Basics of Neural Networks with example codes and illustrations

#22

Post by mrbrowning » Fri, Aug 30, 2013, 6:30 PM UTC

JDDunn9 wrote: post #10

Does anyone have any examples of areas where neural networks beat out statistical based methods, other than maybe image recognition? I can't even think of another major area where they dominate. - Search engines use algorithms, not neural nets. - The most popular algorithm on Kaggle (data analysis competitions) is random forests - Google's self-driving car uses statistical-based methods I can't imagine commercial air…

They (specifically neural networks trained on the cross-entropy error, with the logistic function as your activation function) work very well in classification tasks where you want to interpret the output as a probability, which is useful when you're more concerned with predicting across datasets rather than individual instances. My lab uses neural networks towards this end in predicting pharmacological activity across large libraries of molecules and all the prospective validation we've done indicates that neural networks are far and away the best-performing option out there for this domain as compared to e.g. SVMs or random forests.

Re: Basics of Neural Networks with example codes and illustrations

#23

Post by bcuccioli » Fri, Aug 30, 2013, 6:55 PM UTC

I wrote a simple neural network about a year ago for doing optical character recognition as a class project. I think looking over the code could be good for learning, as it has a pretty simple OOP structure: https://github.com/bcuccioli/neural-ocr

Re: Basics of Neural Networks with example codes and illustrations

#24

Post by aryastark » Fri, Aug 30, 2013, 8:47 PM UTC

JDDunn9 wrote: post #10

Does anyone have any examples of areas where neural networks beat out statistical based methods, other than maybe image recognition? I can't even think of another major area where they dominate. - Search engines use algorithms, not neural nets. - The most popular algorithm on Kaggle (data analysis competitions) is random forests - Google's self-driving car uses statistical-based methods I can't imagine commercial air…

> I can't imagine commercial aircraft would use a neural net. What happened if one crashed? They would analyze the data and ask questions like, Q: "What happened?" A: "I don't know" Q: "Can we fix it so it doesn't happen again?" A: "I don't know".

We already do. The pilot and co-pilot(s). Though I get what you're saying, I just find it humorous that we would worry about it in such manner.

AI neural nets aren't quite there yet. But we attribute that great unknown to vague catch-all "human error" all the time. I suspect one day we will simply attribute AI screw ups to "computer error" or something like that.

Re: Basics of Neural Networks with example codes and illustrations

#25

Post by joe_the_user » Fri, Aug 30, 2013, 9:45 PM UTC

JDDunn9 wrote: post #10

Does anyone have any examples of areas where neural networks beat out statistical based methods, other than maybe image recognition? I can't even think of another major area where they dominate. - Search engines use algorithms, not neural nets. - The most popular algorithm on Kaggle (data analysis competitions) is random forests - Google's self-driving car uses statistical-based methods I can't imagine commercial air…

Well,

The thing is that (at least several kinds of) neural networks and support vector machines are equivalent in their operations. SVMs themselves seem to be something like a general form of binary regression (a clever way to separate two complex sets with a complex curve on a high-dimensional space). So the choice of neural network or support vector machine or some other statistical device seems like it comes down to which problem is easier to formulate how, which tricks are available for what problem, etc.

For example: http://blenny.ncl.ac.uk/peter.andras/PAnpl2002.pdf (that's what I google just now, think there are results on "deep" networks also).

Re: Basics of Neural Networks with example codes and illustrations

#26

Post by aheilbut » Sat, Aug 31, 2013, 12:25 AM UTC

JDDunn9 wrote: post #10

Does anyone have any examples of areas where neural networks beat out statistical based methods, other than maybe image recognition? I can't even think of another major area where they dominate. - Search engines use algorithms, not neural nets. - The most popular algorithm on Kaggle (data analysis competitions) is random forests - Google's self-driving car uses statistical-based methods I can't imagine commercial air…

Despite the name, there's nothing mystical about neural networks that make them not 'statistical based methods'

It's just a particular way of describing a mixture of nonlinear regression models. There are some nice, although very abstract analogies to biology, which is both a blessing and a curse.

Re: Basics of Neural Networks with example codes and illustrations

#27

Post by joe_the_user » Sat, Aug 31, 2013, 12:50 AM UTC

aheilbut wrote: post #26

JDDunn9 wrote: post #10

Does anyone have any examples of areas where neural networks beat out statistical based methods, other than maybe image recognition? I can't even think of another major area where they dominate. - Search engines use algorithms, not neural nets. - The most popular algorithm on Kaggle (data analysis competitions) is random forests - Google's self-driving car uses statistical-based methods I can't imagine commercial air…

Despite the name, there's nothing mystical about neural networks that make them not 'statistical based methods' It's just a particular way of describing a mixture of nonlinear regression models. There are some nice, although very abstract analogies to biology, which is both a blessing and a curse.

Yes,

"...there's nothing mystical about neural networks", yes, there's nothing "not statistical" in the reality of the construct (the NN classes are often given without reference to stat).

But in the way that people present them or think about them, there is tendency for them to be seen or used as indeed "black boxes"(per the gp) or mystical constructs.

A NN might indeed be a good fit for a given class of problems and its the mechanism might be revealing for said problems. But the external impression that their development gives, that their development is that of a "thinking process", is kind of a weakness for the field.

Re: Basics of Neural Networks with example codes and illustrations

#28

Post by atlbeer » Sat, Aug 31, 2013, 2:33 AM UTC

level09 wrote: post #15

JDDunn9 wrote: post #10

Does anyone have any examples of areas where neural networks beat out statistical based methods, other than maybe image recognition? I can't even think of another major area where they dominate. - Search engines use algorithms, not neural nets. - The most popular algorithm on Kaggle (data analysis competitions) is random forests - Google's self-driving car uses statistical-based methods I can't imagine commercial air…

We use Neural Networks to extract text transcripts from our video stream, the process is extremely complicated as the stream is in Arabic which is a fairly complex language. the system is fed with the actual transcript for some videos so it can learn with time and minimize error. the overall system is incredibly useful as you can search the video stream of a TV channel for a specific keyword. I have also used self or…

Is the product publicly available?

Re: Basics of Neural Networks with example codes and illustrations

#29

Post by pests » Sat, Aug 31, 2013, 4:15 AM UTC

coderjack wrote: post #7

pests wrote: post #2

In regards to the first interactive demo, it seems to be adjusting the line to be parallel to the one drawn on the background. Was this intentional or are the supposed to converge?

The line in the background is the function towards which the neural net is expected to converge.

When I first posted my reply every time I watched that demo the neural net line was converging on a line perpendicular to the line in the background.

(rereading my original comment I think I described what I was seeing incorrectly but this new reply correctly explains what I was originally seeing)

Re: Basics of Neural Networks with example codes and illustrations

#30

Post by nomailing » Sat, Aug 31, 2013, 9:58 AM UTC

coderjack wrote: post #12

JDDunn9 wrote: post #10

Does anyone have any examples of areas where neural networks beat out statistical based methods, other than maybe image recognition? I can't even think of another major area where they dominate. - Search engines use algorithms, not neural nets. - The most popular algorithm on Kaggle (data analysis competitions) is random forests - Google's self-driving car uses statistical-based methods I can't imagine commercial air…

Yes its true that neural nets are not used in commercial systems because its computation intensive and also needs huge infrastructure even for things like finding cat images out of youtube videos but the results are far more accurate then any of the statistical methods used in commercial systems. There is time-performance trade-off when it comes to choosing statistical methods over neural nets and also it's a field y…

Why do you think that they are not used in commercial systems? As far as I remember from leckeres of Geoffrey Hinton they are now used quite extensively in speech recognition systems on smartphones, etc.