Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning Confronts the Elephant in the Room

quantamagazine.org

Post reply on HN
Print view View original

61–70 of 116 posts

Re: Machine Learning Confronts the Elephant in the Room

#61

Post by ThePhysicist »

ivan_gammel wrote: post #43
ThePhysicist wrote: post #38

Earlier quoted context omitted.

Yes but children can pass these tests when presented with a single photo as well (if I understand the setup correctly), so I'd say it's a fair comparison. The algorithm can "look" at the picture as long as it wants as well, e.g. using many different convolutional filters that can "see" features in different parts of the image and then combine these in later stages of the analysis.

How close are those convolutional filters to the physical sight of animals? Were they designed specifically to mimic nature or they are artificial concept with similar purpose?

Actually the artificial neural networks used for vision tasks are not so different from our visual cortex, there's an interesting Nature paper on this:

https://www.nature.com/articles/s42003-018-0110-y

Of course the neurons in our brain work in a very different way, but at an architecture level the strategies of natural and artificial neural networks seem to match.

Re: Machine Learning Confronts the Elephant in the Room

#62

Post by TeMPOraL »

skierscott wrote: post #46
hydrox24 wrote: post #40

I know very little about machine vision, so forgive the naïvete of this question: > an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance. Wouldn't the machine return a low confidence score when a scene is confusing? If not, why is this difficult to get around? If so, why can't we just call this a computation difficulty problem, simply requiring…

> a low confidence score Neural nets should return a low confidence score. But, the popular approach (described below) ignores that. Neural nets ignore confidence because of a technique called softmax [1]. This happens as the final operation of a neural net, and is required for training. Softmax is a tool to make an array of positive numbers look like a probability distribution: out = x / x.sum() x[i] is a class pred…

Is this what softmax is? Simply dividing a vector by sum of its components? If so, then how does it deserve a name, not to mention a long Wikipedia page full of formulas?

Re: Machine Learning Confronts the Elephant in the Room

#63

Post by unstuckdev »

sweezyjeezy wrote: post #47
salty_biscuits wrote: post #41

Earlier quoted context omitted.

I just spent 15 minutes putting a picture of a bear with a top hat riding Abraham Lincoln with laser eyes into all the online demos I could find. They weren't unintelligible responses, just a bit crap. They said "weapon", "animal", "person", etc. I'm not sure how much global context they really use, in my experience it has been latching onto a weird bit of local texture. More work to do...

Try this one : https://i.ytimg.com/vi/I11Vlcpz3jc/hqdefault.jpg I tried it on a couple of online demos, neither spotted that there was a car in the picture.

Is a car still a car if it's taken out of context? To us, yes. To an AI, it might not matter.

A space collision avoidance AI that identifies a road vehicle as a road vehicle and expects it to behave as one normally behaves would create problems. A human can look at a 1936 Ford truck floating in space and know it's not going to make a sudden left turn. An AI working in space would still treat it as debris if a human told it "that's a truck, it's not going to do anything different unless acted upon."

Re: Machine Learning Confronts the Elephant in the Room

#64

Post by PurpleBoxDragon »

dukoid wrote: post #54

Wouldn't humans who haven't seen Elephants before or who are not trained to recognize the general category of mammals / animals be prone to make similar errors?

>who are not trained to recognize the general category of mammals / animals

Does such a human exist? Other than perhaps someone blind from birth who had their sight restored, I don't think you could fine such a person give how conditioned we are to recognize animals, perhaps even on a biological level. Closest I can think of is animals that are camouflaged, and I think we would see many humans making mistakes on those. I've definitely have, especially with regards to insects that look like plants. But those animals specifically evolved to throw off predator detection systems.

Re: Machine Learning Confronts the Elephant in the Room

#65

Post by TeMPOraL »

ivan_gammel wrote: post #43
ThePhysicist wrote: post #38

Earlier quoted context omitted.

Yes but children can pass these tests when presented with a single photo as well (if I understand the setup correctly), so I'd say it's a fair comparison. The algorithm can "look" at the picture as long as it wants as well, e.g. using many different convolutional filters that can "see" features in different parts of the image and then combine these in later stages of the analysis.

How close are those convolutional filters to the physical sight of animals? Were they designed specifically to mimic nature or they are artificial concept with similar purpose?

They may be close to first stages of sight pipeline. Maybe even close to full pipe for some simple animals. For humans, they're pretty much input processing stage. Human vision (and presumably animal vision too) has runtime feedback that uses the the output of the vision process to fix up the input. That's what makes you suddenly see things if someone tells you what to look for, or suddenly unsee things. That's why you didn't notice I repeated the the word "the" in few places in this comment.

As the intro to the the article says (you now noticed, didn't you?), the way DNN vision systems are different is that "unlike humans, they can’t do a double take". And those "double takes" is what our vision pipeline does all the time.

Re: Machine Learning Confronts the Elephant in the Room

#66

Post by TeMPOraL »

weregiraffe wrote: post #13

Why can't you just train the network on artificially altered photos, with "elephants" randomly scattered around, until it is robust to them?

Because that's not the problem. The issue is that networks can't recognize their output doesn't make sense, reconfigure themselves and try again, and repeat that process until the output starts matching expectations (while simultaneously weakening those expectations the longer this continues). The way our vision does.

Re: Machine Learning Confronts the Elephant in the Room

#67

Post by PurpleBoxDragon »

kakarot wrote: post #49

I'm confused about what's so special here. This is analogous to a person from the early days of photography having no knowledge of the possibility of image doctoring, and us feeling "smug" because a photo like this would elicit confusion. It seems to me you just have to run your NN on pairs of doctored and undoctored images, and create a separation of concern between scene continuity and object recognition. It's like…

>It seems to me you just have to run your NN on pairs of doctored and undoctored images, and create a separation of concern between scene continuity and object recognition.

This may fix this problem, but what other visual issues will we uncover that might be fixable with the right kind of training but which we haven't yet accounted for.

For example, we see so many optical illusions because of how our eyes and brain (and the neurons between them) work. These are often times caused by our sensation or perception incorrectly picking up data, but on purpose (well, as much purpose as an evolved system could have). For example, how neurons in the eye exaggerate differences in brightness or how we are overly sensitive to perceiving faces so that we see them even where they don't exist. What impact does the lack of these have on an artificial vision system? For starters, it would likely indicate the system wouldn't recognize the optical illusion unless trained for it, but are there any other impacts that we need to take into consideration, especially when considering having human lives depending upon the system?

Re: Machine Learning Confronts the Elephant in the Room

#68

Post by hendzen »

TeMPOraL wrote: post #62
skierscott wrote: post #46

Earlier quoted context omitted.

> a low confidence score Neural nets should return a low confidence score. But, the popular approach (described below) ignores that. Neural nets ignore confidence because of a technique called softmax [1]. This happens as the final operation of a neural net, and is required for training. Softmax is a tool to make an array of positive numbers look like a probability distribution: out = x / x.sum() x[i] is a class pred…

Is this what softmax is? Simply dividing a vector by sum of its components? If so, then how does it deserve a name , not to mention a long Wikipedia page full of formulas?

Because grad students need to publish papers.

Re: Machine Learning Confronts the Elephant in the Room

#69

Post by joejerryronnie »

TeMPOraL wrote: post #65
ivan_gammel wrote: post #43

Earlier quoted context omitted.

How close are those convolutional filters to the physical sight of animals? Were they designed specifically to mimic nature or they are artificial concept with similar purpose?

They may be close to first stages of sight pipeline. Maybe even close to full pipe for some simple animals. For humans, they're pretty much input processing stage. Human vision (and presumably animal vision too) has runtime feedback that uses the the output of the vision process to fix up the input. That's what makes you suddenly see things if someone tells you what to look for, or suddenly unsee things. That's why y…

Very clever young man, but it's turtles all the way down!

Re: Machine Learning Confronts the Elephant in the Room

#70

Post by sgt101 »

hydrox24 wrote: post #40

I know very little about machine vision, so forgive the naïvete of this question: > an ability humans have that AI lacks: the ability to understand when a scene is confusing and thus go back for a second glance. Wouldn't the machine return a low confidence score when a scene is confusing? If not, why is this difficult to get around? If so, why can't we just call this a computation difficulty problem, simply requiring…

Well - you want to believe the best method you have, if the best method produces a low confidence approach then that's that really - you can't "believe" the next best classifier instead - because by definition it's likely to be wrong. Humans re-appraise the scene in light of their surprise or confusion.

We need a cognitive model for vision... This is all leading back to David Marr, who wuda thunk.

Post reply on HN