Live data from Hacker News

CaptionBot by Microsoft

captionbot.ai

91–100 of 170 posts

Re: CaptionBot by Microsoft

#91
post #23

It's only a matter of time before a repeat of Microsoft's last AI experiment (Tay), when the Internet teaches CaptionBot all of the positions in the Kama Sutra.

It looks like it refuses to comment on a certain toothbrush-mustached dictator

Edit: recognizes Stalin though http://i.imgur.com/9W6wqUd.png

Re: CaptionBot by Microsoft

#92
My results ranged from impressive to awful. It recognized Pete Carroll with 96% accuracy from a meme picture where he struts and chews gum. Then it thought a picture of the super bowl field before the game was boats on a table.

Re: CaptionBot by Microsoft

#94
Links to a couple of the initial super impressive research papers on generating captions for images from 2014 and 2015: http://googleresearch.blogspot.com/2014/11/a-picture-is-wort... http://cs.stanford.edu/people/karpathy/deepimagesent/

As far as I know this was the first research to do the super cool thing to combine multiple neural nets trained on different data in super cool ways:

"Now, what if we replaced that first RNN and its input words with a deep Convolutional Neural Network (CNN) trained to classify objects in images? Normally, the CNN’s last layer is used in a final Softmax among known classes of objects, assigning a probability that each object might be in the image. But if we remove that final layer, we can instead feed the CNN’s rich encoding of the image into a RNN designed to produce phrases. We can then train the whole system directly on images and their captions, so it maximizes the likelihood that descriptions it produces best match the training descriptions for each image."

AND

"Our alignment model is based on a novel combination of Convolutional Neural Networks over image regions, bidirectional Recurrent Neural Networks over sentences, and a structured objective that aligns the two modalities through a multimodal embedding"

Re: CaptionBot by Microsoft

#95

Ohh I got a good one: "I am not really confident, but I think it's a close up of a plane with a blue umbrella." http://imgur.com/FYucrda

I got a pretty good one too: "I am not really confident, but I think it's a close up of a man with a cow."

http://imgur.com/DmQHP7W

Re: CaptionBot by Microsoft

#96
post #90

Earlier quoted context omitted.

Hah, yeah but when I got it working again, this time with a screenshot of Jules from Pulp Fiction, it thinks his gun is a camera. > I am not really confident, but I think it's a man holding a camera. Source Image: http://www.cinemablend.com/images/news_img/79237/pulp_fictio...

Imagine this tech matures and it can be incorporated with bodycams for police, when confronting a subject with objects in their hands it may be able to confidently estimate the probability of being a firearm or not, with better predictability than the police/people.

...and better proveability. Most people are actually unarmed. http://www.theguardian.com/us-news/2015/jun/01/black-america...

While we're here, let's go the full way and set up a proveable and public way to train a robocop, and I'd trust that more than a human cop. The awkward moment when AIs have more brains than cops (at least under the US system).

Re: CaptionBot by Microsoft

#97
post #23

It's only a matter of time before a repeat of Microsoft's last AI experiment (Tay), when the Internet teaches CaptionBot all of the positions in the Kama Sutra.

I chose goatse and it says "I am not really confident, but I think it's a man holding a cat."

Re: CaptionBot by Microsoft

#98
I found it curious that this Bot is really bad at recognizing apes: chimpanzees and gorillas specifically. I fed it a lot of the images from a Google image search for these animals and more often than not it either doesn't recognize anything or considers them bears.

I don't mean to offend, but I'm left wondering if the creators of image recognition services disincentivize their neural nets from recognizing something as an ape, gorilla or chimpanzee so as to avoid the same mistake Google made when it falsely recognized black people as gorillas [1].

[1] http://blogs.wsj.com/digits/2015/07/01/google-mistakenly-tag...

Re: CaptionBot by Microsoft

#99

CaptionBot team here. Thanks for the images and captions! Please keep sharing them and give us feedback.

First of all congratulations on a) the science (built on the shoulder of giants...) b) the accessibility / interface and service

Wondering if you plan to open up a caption API of any sort? Can definitely use something like this. If you desire the training feedback, then that could be added as well as part of the API. I'd be willing to do that for some images. So if you do add a training feedback API, please make it optional.

Re: CaptionBot by Microsoft

#100

Pretty impressive - gave it a few profile photos and it did suprisingly well, correctly identifying "A couple walking on a beach at sunset," "a man looking out a window", etc. It struggled with wildlife photos - a pack of arctic wolves was "a sheep standing in the snow", and penguins swimming was "a bird flying over a body of water" (close but no cigar).

Flying over, flying through... Minor detail.
Post reply on HN