Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning Confronts the Elephant in the Room

quantamagazine.org

Post reply on HN
Print view View original

51–60 of 116 posts

Re: Machine Learning Confronts the Elephant in the Room

#51

Post by sp332 »

ivan_gammel wrote: post #37
sp332 wrote: post #35

Earlier quoted context omitted.

Yes, check out "attention" mechanisms. The network can use wide convolutions for the image as a whole, effectively mixing data kind of like a blur function, and also use finer convolutions for a small piece of the image that it chooses to focus on.

Does the problem of elephants still exist with such approach? It doesn't seem right that an unexpected feature in a corner of the image would have a dramatic impact on the recognition of the features elsewhere.

Well if there's a thing/elephant/chair floating over someone's head in one part of the room, that does change the context for interpreting other things in the room.

Re: Machine Learning Confronts the Elephant in the Room

#52

Post by gok »

skierscott wrote: post #46
hydrox24 wrote: post #40

I know very little about machine vision, so forgive the naïvete of this question: > an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance. Wouldn't the machine return a low confidence score when a scene is confusing? If not, why is this difficult to get around? If so, why can't we just call this a computation difficulty problem, simply requiring…

> a low confidence score Neural nets should return a low confidence score. But, the popular approach (described below) ignores that. Neural nets ignore confidence because of a technique called softmax [1]. This happens as the final operation of a neural net, and is required for training. Softmax is a tool to make an array of positive numbers look like a probability distribution: out = x / x.sum() x[i] is a class pred…

Note that you can get a form of confidence by just not applying softmax to the output during inference. Softmax is primarily to aid in training.

Re: Machine Learning Confronts the Elephant in the Room

#53

Post by Qworg »

sorokod wrote: post #36
antpls wrote: post #27

Earlier quoted context omitted.

This is a vast engineering field studied before the existence of neural networks. Keywords are "control theory", "sensor fusion", "automated decision under uncertainty" that you can look on Google, Wikipedia and arXiv. Also "Simultaneous localization and mapping" which is a good example of using uncertain data points from different sensors to build a representation of the reality. In those systems, a neural network i…

Thank you, I will have a look. Just out of curiosity, what do you mean by 'uncertain' in this context?

Errors in precision and accuracy.

Re: Machine Learning Confronts the Elephant in the Room

#55

Post by sgt101 »

dukoid wrote: post #54

Wouldn't humans who haven't seen Elephants before or who are not trained to recognize the general category of mammals / animals be prone to make similar errors?

I don't think so - I think that the human may fail to find the correct label for the Elephant (although if they have some knowledge about Elephants they might), but the machine fails to see it - to identify it as a thing, and also it changes the labels it gives other things; so cats become dogs, mugs become books and so on. Humans are not prone to this.

Re: Machine Learning Confronts the Elephant in the Room

#56

Post by sorokod »

Qworg wrote: post #53
sorokod wrote: post #36

Earlier quoted context omitted.

Thank you, I will have a look. Just out of curiosity, what do you mean by 'uncertain' in this context?

Errors in precision and accuracy.

Hmm... there is no doubt about the sensor input, the elephant is there. The issue is with precision and accuracy of the model itself.

Edit: I need to rephrase, the interesting case for me ( and the one the article is describing) is when the model fails with accurate input data due to assumptions that are intrinsic to the model itself.

Re: Machine Learning Confronts the Elephant in the Room

#57

Post by 21 »

hydrox24 wrote: post #40

I know very little about machine vision, so forgive the naïvete of this question: > an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance. Wouldn't the machine return a low confidence score when a scene is confusing? If not, why is this difficult to get around? If so, why can't we just call this a computation difficulty problem, simply requiring…

> simply requiring a way to go back and spend more effort later when required

Present neural nets have no "more effort" or "less effort" knob. For the same input they always produce the same output.

The article says a different thing: humans rerun the "algorithm" again, but this time with the knowledge that something was wrong in the previous run (lets say color), and this knowledge will tweak the way it runs again, maybe even running a completely different algorithm the second time (a network more specialized in color but which is worse at shapes)

Re: Machine Learning Confronts the Elephant in the Room

#58

Post by Qworg »

sorokod wrote: post #56
Qworg wrote: post #53

Earlier quoted context omitted.

Errors in precision and accuracy.

Hmm... there is no doubt about the sensor input, the elephant is there. The issue is with precision and accuracy of the model itself. Edit: I need to rephrase, the interesting case for me ( and the one the article is describing) is when the model fails with accurate input data due to assumptions that are intrinsic to the model itself.

There are doubts in both, frankly.

Re: Machine Learning Confronts the Elephant in the Room

#59

Post by singularity2001 »

ivan_gammel wrote: post #43
ThePhysicist wrote: post #38

Earlier quoted context omitted.

Yes but children can pass these tests when presented with a single photo as well (if I understand the setup correctly), so I'd say it's a fair comparison. The algorithm can "look" at the picture as long as it wants as well, e.g. using many different convolutional filters that can "see" features in different parts of the image and then combine these in later stages of the analysis.

How close are those convolutional filters to the physical sight of animals? Were they designed specifically to mimic nature or they are artificial concept with similar purpose?

First they were designed to mimic nature, then these patterns (gabor filters) astonishingly automatically developed themselves via backpropagation (which gave rise to the recent enthusiasm).

Modern networks are instantiated with random noise but end up pretty much the same as previously hardcoded filters.

Re: Machine Learning Confronts the Elephant in the Room

#60

Post by deltron3030 »

dmichulke wrote: post #25
deltron3030 wrote: post #4

Can't they implement a "virtual representation" or sort of ideal or archetypical scene in an AIs memory that's quickly accessible, and then use that to diff "new unexpected stuff" and refocus on that difference somehow?

If we had such a representation and the mapping from real world (images) to it, we'd basically solved all image recognition tasks because the above is the definition of an image recognition task

In the real world you wouldn't need to know the exact shape of an object to determine what it is, unique features could be enough. For an elephant this could be just a range of skin colors that are unique to elephants.

The problem with videos and images vs. the real world is the fidelity, you can't trust images or videos that are previously captured through other equipment. This barrier wouldn't exist if the AI can "see" into the real world, the skin colors of animals can be trusted there!

Post reply on HN