Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning Confronts the Elephant in the Room

quantamagazine.org

Post reply on HN
Print view View original

31–40 of 116 posts

Re: Machine Learning Confronts the Elephant in the Room

#31

Post by weregiraffe »

hnhg wrote: post #21
weregiraffe wrote: post #13

Why can't you just train the network on artificially altered photos, with "elephants" randomly scattered around, until it is robust to them?

Because there's always another type of 'elephant' to be thrown in. The networks are just not inherently robust against unexpected features (I'll call them outliers for want of a better term), and this is what needs to be fixed, and not increasing the variety and frequency of outliers.

>Because there's always another type of 'elephant' to be thrown in.

Well, you could randomly generate and render 3D objects, and paste them into photos. This will give you a huge space of 'elephants'.

Re: Machine Learning Confronts the Elephant in the Room

#32

Post by dpwm »

antpls wrote: post #11

The title is a bit misleading / clickbait. It should be "Neural networks Confronts...". Machine Learning isn't all about neutral networks and deep learning. As another comment said, "second take" is not what neural networks are made for. Neural networks are a building block of more complex decision systems, where the weakness of the neural networks are taken into account before automatically commiting to decisions. O…

> It should be "Neural networks Confronts...".

I disagree, for the following reasons:

- The paper is called The Elephant in the Room. Technically the paper confronted the popular machine learning approaches, so the reversal would have been a more eye catching headline. "The Elephant in the Room confronts Machine Learning" is a rare opportunity to honestly play with word order in a way that would arguably be more attention-grabbing.

- Machine learning is more general than Convolutional Neural Networks, but less general than AI. This seems appropriate for this type of popular science publication.

- There are approaches that use attention and RNNs and they are an interesting approach, but by no means mainstream nor producing state-of-the-art classification accuracies. Actually, second take is exactly what an attention-based RNN can do.

- The last time I checked the state-of-the-art solutions to image classification involve Convolutional Neural Networks.

If there was an approach that was even close to approaching ConvNet performance, I would agree that it is perhaps unfair to condemn the entire field. Even then, we would be in the territory of generalizations in headlines, which are often fair. "{party} proposing legislation" is a probably bigger generalization of headline that would generally pass without criticism.

Re: Machine Learning Confronts the Elephant in the Room

#33

Post by antpls »

LeanderK wrote: post #15
antpls wrote: post #11

The title is a bit misleading / clickbait. It should be "Neural networks Confronts...". Machine Learning isn't all about neutral networks and deep learning. As another comment said, "second take" is not what neural networks are made for. Neural networks are a building block of more complex decision systems, where the weakness of the neural networks are taken into account before automatically commiting to decisions. O…

i wouldn't be so critical. It's journalism and I would intrepret the title more as "Machine Learning-community confronts...". And while ML isn't all about NNs and deep learning, we're quite enticed by them right now and try to push them to see where their limitations are. So I think with a bit more generous interpretation the title is acceptable. I think it's certainly not clickbait, since NNs are right now a very si…

Well, to me, this is not journalism. Journalists have a strong ethic and respect for the meaning of words to relay high quality information.

This is science vulgarisation, or a magazine article. Not that there is no value in it, but it's an over-generalization.

Re: Machine Learning Confronts the Elephant in the Room

#34

Post by YeGoblynQueenne »

weregiraffe wrote: post #28
throwaway2048 wrote: post #20

Earlier quoted context omitted.

Are you going to repeat this with every object that exists, or could exist?

What I'd do is train it on a large diverse set of foreign objects, then test it with a different diverse set of foreign objects, to see if it is possible to make it generally robust. Basically, the goal is that any feature which is not overlapping with the "elephant" should not be influenced by it.

Well, CNNs are already trained on "a large diverse set" of objects, in many different configurations.

By the way, something that is not described in the article- the referenced paper also describes experiments where objects where translated inside the source image itself - so not in another image. And the tested networks were still confused.

So it's not "foreign" objects that are the problem. It's that neural nets generalise very poorly. Show them something they haven't seen before and their ability to recognise it falls off a cliff.

Re: Machine Learning Confronts the Elephant in the Room

#35

Post by sp332 »

ivan_gammel wrote: post #30

Please, correct me if I'm wrong, but isn't the whole machine learning testing approach based on the recognition the whole image at once in full resolution in one attempt? If so, it's not how animals see and observe the world and this could be a key difference between AI and living beings. What if an algorithm actually had a focused sight? Look at the brightest feature, blur the rest, then try to find another distinct…

Yes, check out "attention" mechanisms. The network can use wide convolutions for the image as a whole, effectively mixing data kind of like a blur function, and also use finer convolutions for a small piece of the image that it chooses to focus on.

Re: Machine Learning Confronts the Elephant in the Room

#36

Post by sorokod »

antpls wrote: post #27
sorokod wrote: post #18

Earlier quoted context omitted.

The problem described in the article is taken into account when building systems (like autonomous cars) using neural networks. Where can I find more information on this?

This is a vast engineering field studied before the existence of neural networks. Keywords are "control theory", "sensor fusion", "automated decision under uncertainty" that you can look on Google, Wikipedia and arXiv. Also "Simultaneous localization and mapping" which is a good example of using uncertain data points from different sensors to build a representation of the reality. In those systems, a neural network i…

Thank you, I will have a look. Just out of curiosity, what do you mean by 'uncertain' in this context?

Re: Machine Learning Confronts the Elephant in the Room

#37

Post by ivan_gammel »

sp332 wrote: post #35
ivan_gammel wrote: post #30

Please, correct me if I'm wrong, but isn't the whole machine learning testing approach based on the recognition the whole image at once in full resolution in one attempt? If so, it's not how animals see and observe the world and this could be a key difference between AI and living beings. What if an algorithm actually had a focused sight? Look at the brightest feature, blur the rest, then try to find another distinct…

Yes, check out "attention" mechanisms. The network can use wide convolutions for the image as a whole, effectively mixing data kind of like a blur function, and also use finer convolutions for a small piece of the image that it chooses to focus on.

Does the problem of elephants still exist with such approach? It doesn't seem right that an unexpected feature in a corner of the image would have a dramatic impact on the recognition of the features elsewhere.

Re: Machine Learning Confronts the Elephant in the Room

#38

Post by ThePhysicist »

ivan_gammel wrote: post #30

Please, correct me if I'm wrong, but isn't the whole machine learning testing approach based on the recognition the whole image at once in full resolution in one attempt? If so, it's not how animals see and observe the world and this could be a key difference between AI and living beings. What if an algorithm actually had a focused sight? Look at the brightest feature, blur the rest, then try to find another distinct…

Yes but children can pass these tests when presented with a single photo as well (if I understand the setup correctly), so I'd say it's a fair comparison. The algorithm can "look" at the picture as long as it wants as well, e.g. using many different convolutional filters that can "see" features in different parts of the image and then combine these in later stages of the analysis.

Re: Machine Learning Confronts the Elephant in the Room

#39

Post by antpls »

dpwm wrote: post #32
antpls wrote: post #11

The title is a bit misleading / clickbait. It should be "Neural networks Confronts...". Machine Learning isn't all about neutral networks and deep learning. As another comment said, "second take" is not what neural networks are made for. Neural networks are a building block of more complex decision systems, where the weakness of the neural networks are taken into account before automatically commiting to decisions. O…

> It should be "Neural networks Confronts...". I disagree, for the following reasons: - The paper is called The Elephant in the Room. Technically the paper confronted the popular machine learning approaches, so the reversal would have been a more eye catching headline. "The Elephant in the Room confronts Machine Learning" is a rare opportunity to honestly play with word order in a way that would arguably be more atte…

> - The paper is called The Elephant in the Room.

How is this argument related to the confusion of the terms Machine Learning and Neural Networks...?

> - Machine learning is more general than Convolutional Neural Networks, but less general than AI. This seems appropriate for this type of popular science publication.

The most appropriate term is the most precise one, in this case : neural networks

> - There are approaches that use attention and RNNs and they are an interesting approach, but by no means mainstream nor producing state-of-the-art classification accuracies. Actually, second take is exactly what an attention-based RNN can do.

Correct, but you are still talking about neural networks...

> - The last time I checked the state-of-the-art solutions to image classification involve Convolutional Neural Networks.

And again neural networks.

‍️

Re: Machine Learning Confronts the Elephant in the Room

#40

Post by hydrox24 »

I know very little about machine vision, so forgive the naïvete of this question:

> an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance.

Wouldn't the machine return a low confidence score when a scene is confusing? If not, why is this difficult to get around?

If so, why can't we just call this a computation difficulty problem, simply requiring a way to go back and spend more effort later when required? A problem like this would simply require better computers, and patience.

In other words, Is this problem as deeply rooted as the article suggests? Or is it simply a problem with the popular approach to machine vision?

Post reply on HN