Sharing Pre built models are so cool, and definitely important to the advance machine learning science. Especially when you consider how mixing weight layers allows you to do things like understand Portuguese text better through English text.
Neural Networks That Describe Images
11–20 of 63 posts
Re: Neural Networks That Describe Images
#12Idk if this is a daft question, but in the Visual-Semantic Alignment section, are those objects in the colored boxes actually being directly recognized by the software? Or are they inputted in some other way?
From the paper: Our core insight is that we can leverage these large image-sentence datasets by treating the sentences as weak labels, in which contiguous segments of words correspond to some particular, but unknown location in the image. Our approach is to infer these alignments and use them to learn a generative model of descriptions.
so basically you have a database of pre existing sentences to match whats roughly likely to be on the page, not actually seeing individual objects and generating a grammatically correct and accurate description based on the individual objects?
Re: Neural Networks That Describe Images
#13It is interesting how the neural network labeled the woman in the lower right photo with the rectangle that includes the body only, without the head.
Re: Neural Networks That Describe Images
#14I really appreciate that the Stanford group is always willing to post mistakes/mislabels. Kudos.
Re: Neural Networks That Describe Images
#15Idk if this is a daft question, but in the Visual-Semantic Alignment section, are those objects in the colored boxes actually being directly recognized by the software? Or are they inputted in some other way?
From the paper: Our core insight is that we can leverage these large image-sentence datasets by treating the sentences as weak labels, in which contiguous segments of words correspond to some particular, but unknown location in the image. Our approach is to infer these alignments and use them to learn a generative model of descriptions.
This gets a little more detailed into the work, but compared to other papers that have sprung up in this area recently, our paper slightly frowns on the idea of distilling a complex image into a single short sentence description. In that sense we are a little more ambitious and we're trying to produce snippets of text that cover the full image with descriptions on level of image regions. I would call our results encouraging, but there is certainly more work to be done here. And I think one of the limitations right now to do a good job is the amount of training data available to us.
Re: Neural Networks That Describe Images
#16Presentations like these make me realize how close we are to developing law enforcement (/police state) technology that will be very effective. I figure when the kinks are smoothed out, that we could run this on a video feed, and have crimes prevented right as they're about to happen. It's almost scary. Imagine 20 years from now, some guy pulls a gun on you, and a video feeds identifies his action, and immediately shoots a tranquilizer straight into his jugular with perfect aim.
Re: Neural Networks That Describe Images
#17Presentations like these make me realize how close we are to developing law enforcement (/police state) technology that will be very effective. I figure when the kinks are smoothed out, that we could run this on a video feed, and have crimes prevented right as they're about to happen. It's almost scary. Imagine 20 years from now, some guy pulls a gun on you, and a video feeds identifies his action, and immediately sh…
Effective and also scary. That same video feed could be archiving (for eternity) every single citizen's movement, action, or even words:
"Citizen #5135: In 2015, spit on sidewalks 28x this year, jaywalked 49x, etc..."
Re: Neural Networks That Describe Images
#18This is one of the coolest things i've seen in a while. I'd guess this is super similar to what the folks on the DeepMind team at Google are working on now, with the overall vision of being to classify images that have no metadata and add them to a dynamically learning knowledge graph:
http://www.newscientist.com/article/dn24946-google-buys-ai-f...
http://en.wikipedia.org/wiki/Knowledge_Graph
http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=H...
http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=H...
Re: Neural Networks That Describe Images
#19They forgot to put the link to the npm module ;)
Re: Neural Networks That Describe Images
#20Fascinating. Also interesting to see the failure modes. Any human would quickly realize that the "boy doing backflip on wakeboard" is actually playing on a trampoline. Or the "two young girls playing with legos toy". Great stuff!
I think I'm a human, but I did not realize that boy was on a trampoline instead of a wakeboard until you pointed it out. I think the water in the background confused me. I was also probably biased by reading the network's label before reaching my own judgment.