Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning Confronts the Elephant in the Room

quantamagazine.org

Post reply on HN
Print view View original

71–80 of 116 posts

Re: Machine Learning Confronts the Elephant in the Room

#71

Post by unstuckdev »

sorokod wrote: post #56
Qworg wrote: post #53

Earlier quoted context omitted.

Errors in precision and accuracy.

Hmm... there is no doubt about the sensor input, the elephant is there. The issue is with precision and accuracy of the model itself. Edit: I need to rephrase, the interesting case for me ( and the one the article is describing) is when the model fails with accurate input data due to assumptions that are intrinsic to the model itself.

Optical sensors don't have anywhere near the dynamic range of a retina. I think poor low light performance has been implicated in at least one self-driving crash. Eyes can at least render objects and shapes in extreme low light even if the brain doesn't have enough information to identify them. The tiny sensors in a camera compact enough to be practical still have a long way to go.

Re: Machine Learning Confronts the Elephant in the Room

#72

Post by vinayms »

I am not an expert in AI/ML, just a casual observer, but I didn't like the tone of the article. Not only it fixates on the speed of processing, it also seems smug and acts as if we know how human brain sees and processes images, while all we have is a bunch of conjectures. That was distracting.

Re: Machine Learning Confronts the Elephant in the Room

#73

Post by sweezyjeezy »

unstuckdev wrote: post #63
sweezyjeezy wrote: post #47

Earlier quoted context omitted.

Try this one : https://i.ytimg.com/vi/I11Vlcpz3jc/hqdefault.jpg I tried it on a couple of online demos, neither spotted that there was a car in the picture.

Is a car still a car if it's taken out of context? To us, yes. To an AI, it might not matter. A space collision avoidance AI that identifies a road vehicle as a road vehicle and expects it to behave as one normally behaves would create problems. A human can look at a 1936 Ford truck floating in space and know it's not going to make a sudden left turn. An AI working in space would still treat it as debris if a human t…

Sure, but this is a narrow view of the issue, which is that neural nets are solving a somewhat superficial version of the problem. This may not be an issue for simple image classification, but if we're looking to attack more complex tasks (e.g. describing what happens in a movie), it's clear that we need neural nets with deeper understanding.

Re: Machine Learning Confronts the Elephant in the Room

#74

Post by brilee »

TeMPOraL wrote: post #62
skierscott wrote: post #46

Earlier quoted context omitted.

> a low confidence score Neural nets should return a low confidence score. But, the popular approach (described below) ignores that. Neural nets ignore confidence because of a technique called softmax [1]. This happens as the final operation of a neural net, and is required for training. Softmax is a tool to make an array of positive numbers look like a probability distribution: out = x / x.sum() x[i] is a class pred…

Is this what softmax is? Simply dividing a vector by sum of its components? If so, then how does it deserve a name , not to mention a long Wikipedia page full of formulas?

It also includes the exponentiation step before the vector normalizations. There are connections to statistical mechanics here, where the relative energy population numbers are proportional to the softmax of the energy levels divided by temperature. (so as temperature goes up, the relative energy differences get smaller and the states are more equally populated.) That idea has been ported over as "softmax temperature" in some places.

Re: Machine Learning Confronts the Elephant in the Room

#75

Post by Tenobrus »

TeMPOraL wrote: post #62
skierscott wrote: post #46

Earlier quoted context omitted.

> a low confidence score Neural nets should return a low confidence score. But, the popular approach (described below) ignores that. Neural nets ignore confidence because of a technique called softmax [1]. This happens as the final operation of a neural net, and is required for training. Softmax is a tool to make an array of positive numbers look like a probability distribution: out = x / x.sum() x[i] is a class pred…

Is this what softmax is? Simply dividing a vector by sum of its components? If so, then how does it deserve a name , not to mention a long Wikipedia page full of formulas?

No, it's not. It's actually e ^ x_i / sum(e ^ x_j for x_j in x), which is in fact different. Simply dividing by the sum wouldn't work for "squashing to a probability distribution" in a large number of cases.

Re: Machine Learning Confronts the Elephant in the Room

#76

Post by sgt101 »

unstuckdev wrote: post #63
sweezyjeezy wrote: post #47

Earlier quoted context omitted.

Try this one : https://i.ytimg.com/vi/I11Vlcpz3jc/hqdefault.jpg I tried it on a couple of online demos, neither spotted that there was a car in the picture.

Is a car still a car if it's taken out of context? To us, yes. To an AI, it might not matter. A space collision avoidance AI that identifies a road vehicle as a road vehicle and expects it to behave as one normally behaves would create problems. A human can look at a 1936 Ford truck floating in space and know it's not going to make a sudden left turn. An AI working in space would still treat it as debris if a human t…

If the net stops thinking that unicyclists are objects when it rains that could present a challenge for unicyclists.

Re: Machine Learning Confronts the Elephant in the Room

#78

Post by mcguire »

vinayms wrote: post #72

I am not an expert in AI/ML, just a casual observer, but I didn't like the tone of the article. Not only it fixates on the speed of processing, it also seems smug and acts as if we know how human brain sees and processes images, while all we have is a bunch of conjectures. That was distracting.

Well, we do know that the brain does something that CV systems don't, and that it was a part of a recent self-driving car accident.

Re: Machine Learning Confronts the Elephant in the Room

#79

Post by PavlikPaja »

First, the chair wasn't replaced with a couch - you can see there is yet another rectangle just a few pixels to the left, that likely says "chair".

Second, even many people are surprisingly bad at decoding incongruous scenes, which is why hidden object games are a thing.

Re: Machine Learning Confronts the Elephant in the Room

#80

Post by [deleted] »

gok wrote: post #52
skierscott wrote: post #46

Earlier quoted context omitted.

> a low confidence score Neural nets should return a low confidence score. But, the popular approach (described below) ignores that. Neural nets ignore confidence because of a technique called softmax [1]. This happens as the final operation of a neural net, and is required for training. Softmax is a tool to make an array of positive numbers look like a probability distribution: out = x / x.sum() x[i] is a class pred…

Note that you can get a form of confidence by just not applying softmax to the output during inference. Softmax is primarily to aid in training.

[deleted]
Post reply on HN