CaptionBot by Microsoft • Hacker News Board

1024core wrote: post #77

I worked in Image Processing and Vision for a long time. If you'd asked me 2 years ago that something like this could be possible, I would have laughed you out of the room. But in the last year or so, I've been stunned beyond belief at how well these networks work.

[deleted]

Re: CaptionBot by Microsoft

#134

Post by Cornelia_Kara » Wed, Apr 13, 2016, 11:51 PM UTC

andreyk wrote: post #94

Links to a couple of the initial super impressive research papers on generating captions for images from 2014 and 2015: http://googleresearch.blogspot.com/2014/11/a-picture-is-wort... http://cs.stanford.edu/people/karpathy/deepimagesent/ As far as I know this was the first research to do the super cool thing to combine multiple neural nets trained on different data in super cool ways: "Now, what if we replaced that f…

Here's a whitepaper on the tech behind the CaptionBot.ai demo: http://research.microsoft.com/pubs/264408/ImageCaptionInWild...

This demo uses the Vision API and the Emotion API from here: https://www.microsoft.com/cognitive-services/

Re: CaptionBot by Microsoft

#135

Post by asib » Thu, Apr 14, 2016, 12:25 AM UTC

This made me laugh: https://imgur.com/PhbyAyK

Re: CaptionBot by Microsoft

#136

Post by zarify » Thu, Apr 14, 2016, 12:39 AM UTC

So I looked for a random photo on my phone and fed it a picture of a spot my leg that I'm keeping an eye on. Close-up of a cat apparently. Damn these hairy legs.

Re: CaptionBot by Microsoft

#137

Post by ilaksh » Thu, Apr 14, 2016, 12:49 AM UTC

satysin wrote: post #89

Hmm I can't help but think it should have done a little better with this image http://i.imgur.com/yBNJWKf.png

I got the same response not really confident but think it's a cell phone. Mine was a cluster of little buildings with gardens on top. Your is a ketchup bottle. Maybe 'cell phone' is the default response when it doesn't know.

Re: CaptionBot by Microsoft

#138

Post by colllectorof » Thu, Apr 14, 2016, 12:59 AM UTC

jnky wrote: post #98

I found it curious that this Bot is really bad at recognizing apes: chimpanzees and gorillas specifically. I fed it a lot of the images from a Google image search for these animals and more often than not it either doesn't recognize anything or considers them bears. I don't mean to offend, but I'm left wondering if the creators of image recognition services disincentivize their neural nets from recognizing something…

Reading the blog post and the Twitter thread.

Wow. 10 years ago it would have been seen as a comical blunder of a stupid AI. Something to be fixed for sure, but not a Serious Social Issue by any means. Nowadays, it apparently warrants several follow-up articles in the mainstream media, "social" commentary and 3,309 retweets.

From the blog post: “The bias of the Internet reflects the bias of society,” she said.

In some cases - yes, but this one seems more like society deliberately projecting human motivations onto a primitive algorithm and jumping to conclusions about what its errors "really" mean.

-

For people who disagree, here is a scenario your might want to consider. Imagine that you built an image tagging service. Imagine that someone found a glitch in your service that they consider offensive. Imagine them tweeting about it (before or instead of contacting you directly) and getting a similar kind of reaction, complete with extensive social commentary and media coverage. Nice, big crowd of people using your company and your service as a convenient example of things-that-are-wrong-with-our-society. How would you feel in that case?

Re: CaptionBot by Microsoft

#139

Post by mc32 » Thu, Apr 14, 2016, 1:52 AM UTC

readams wrote: post #114

jnky wrote: post #98

I found it curious that this Bot is really bad at recognizing apes: chimpanzees and gorillas specifically. I fed it a lot of the images from a Google image search for these animals and more often than not it either doesn't recognize anything or considers them bears. I don't mean to offend, but I'm left wondering if the creators of image recognition services disincentivize their neural nets from recognizing something…

People in general look very much like gorillas. They are close cousins after all. Telling humans and other apes apart can't be very easy for a poor, overworked neural network.

Also why are people offended by gorillas, or apes. Imagine being mislabeled as a bear or a giraffe, would people get as offended? What about bring identified as a parrot?

Re: CaptionBot by Microsoft

#140

Post by jnky » Thu, Apr 14, 2016, 1:56 AM UTC

colllectorof wrote: post #138

jnky wrote: post #98

I found it curious that this Bot is really bad at recognizing apes: chimpanzees and gorillas specifically. I fed it a lot of the images from a Google image search for these animals and more often than not it either doesn't recognize anything or considers them bears. I don't mean to offend, but I'm left wondering if the creators of image recognition services disincentivize their neural nets from recognizing something…

Reading the blog post and the Twitter thread. Wow. 10 years ago it would have been seen as a comical blunder of a stupid AI. Something to be fixed for sure, but not a Serious Social Issue by any means. Nowadays, it apparently warrants several follow-up articles in the mainstream media, "social" commentary and 3,309 retweets. From the blog post: “The bias of the Internet reflects the bias of society,” she said. In som…

I couldn't agree more.

I'm all for not discriminating people, but on the other hand I firmly believe that technical progress should not be second to people's feelings.