Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cloud Video Intelligence API

cloud.google.com

Post reply on HN
Print view View original

51–60 of 91 posts

Re: Cloud Video Intelligence API

#51

Post by chipperyman573 »

muzakthings wrote: post #44
imh wrote: post #36

The demo picture they chose is interesting. It's obviously a tiger, and is identified as such with only 90% probability. I appreciate the difficulty of the problem and how big of a success it is to achieve even that level of confidence, but that low level of confidence really shows how far we are from being able to simply trust computer vision. Still useful from an information retrieval perspective, I expect.

You realize that softmax scores aren't probabilities, right? It's just a relative measure of confidence, scaled such that they all sum to 1.0.

You can't add any of the numbers in the picture to equal 1.0 (or 100)

Re: Cloud Video Intelligence API

#53

Post by ar15saveslives »

pgodzin wrote: post #39
ar15saveslives wrote: post #24

Correct me if I'm wrong, but this is just a frame-by-frame labeling. You can download whatever pre-trained CNN, pass individual frames through it and get the same result.

It also mentions verbs like swimming so it may do those across several frames? Not sure how much more accurate that is than frame-by-frame

I suspect it's not a detected action, but just a picture labeled "swimming", like one of these (random image from internet): http://www.ambientegallerie.com/blog/wp-content/uploads/2016...

Re: Cloud Video Intelligence API

#54

Post by chrisballinger »

tyre wrote: post #50

I think their model should take a second pass on the words and probabilities, independent of the video. Look at their example: Animal: 97.76% Tiger: 90.11% Terrestrial animal: 68.17% So we are 90% sure it is a tiger but only 68% sure it is a land animal? I don't think that makes sense. It could be that this is a weakness of seeding AI data with human inputs. I can believe that 90% of people who saw the video would ag…

Perhaps it's confused by the many images and videos of tigers swimming in water?

Re: Cloud Video Intelligence API

#55

Post by chimtim »

catshirt wrote: post #33
chimtim wrote: post #29

what is the "video" bit here? This is just running image recognition on a bunch of frames.

how do you know the implementation details? it would be completely naive to implement it that way, considering there is an entirely new attribute video applies over images which of course is "time". I don't know shit about ML- talking out of my ass here- but I'd be surprised if the algorithms didn't account for changes over time or canonical entity recognition (is this the same boat that was in the last image)?

The linked press release shows an animal is detected -- tiger etc. It does not say tiger running or hunting, which is where the time component would have been used.

Re: Cloud Video Intelligence API

#56

Post by BrandonY »

joaoaccarvalho wrote: post #52

When you use these Google APIs, can Google keep/ use your data in any way?

Here's a link to the Google Cloud Terms of Service: https://cloud.google.com/terms/ (you may be interested in section 5.2: Use of Customer Data).

Disclaimer: I work for Google but am definitely not a lawyer and can't authoritatively speak for Google here.

Re: Cloud Video Intelligence API

#57

Post by timc3 »

komali2 wrote: post #11
timc3 wrote: post #4

I have been on the beta program for this and generally the results in our testing have been very good. I particularly like how granular the data can get.

What do you guys plan to do with it? Another poster mentioned how it seems hard to imagine a business that has a model around "finding the sailboats in this batch of pics."

We build systems that let our customer manage their video, and usually in large quantities. Some of this video spans a very long period of time and the only description that they have of it is in the filename.

Getting proper structured metadata from content has traditionally been expensive as it has required humans, sometimes trained as librarians, so providing the ability to extract some meaning from video becomes valuable.

Even for systems that have trained librarians, it can still help to have the ability to have a system that highlights the general content so they can further refine it and bring it inline with a taxonomy.

Google Vision API also helps because the metadata can be placed against timecode, further helping the ability to search for particular subjects within a video that might otherwise not be found easily from quickly browsing the videos.

There are other use cases depending on the customers need (such as a customer making sure that certain things are not in video about to go to air), but none of it has to do with marketing.

Re: Cloud Video Intelligence API

#58

Post by timc3 »

hartator wrote: post #25

It's awesome, but I can't really see any application beside content filtering and supericial content classification.

It can help more than superficially, it can become a starting point for humans to go and refine classification into a controlled taxonomy.

The Google implementation can also detect when different people are speaking which is useful - though it takes someone to tag who is who.

As I mentioned in another post, it can also highlight where things are happening in a video - for instance a 2hr video where a scene suddenly appears. Like in a nature video rushes where an animal appears for only a few seconds.

Other types of video analysis can also detect problems in the video/audio, such as dropped frames, noise or colour gamut issues.

Re: Cloud Video Intelligence API

#59

Post by timc3 »

chimtim wrote: post #55
catshirt wrote: post #33

Earlier quoted context omitted.

how do you know the implementation details? it would be completely naive to implement it that way, considering there is an entirely new attribute video applies over images which of course is "time". I don't know shit about ML- talking out of my ass here- but I'd be surprised if the algorithms didn't account for changes over time or canonical entity recognition (is this the same boat that was in the last image)?

The linked press release shows an animal is detected -- tiger etc. It does not say tiger running or hunting, which is where the time component would have been used.

I have seen it detect that a car is drifting..

Re: Cloud Video Intelligence API

#60

Post by timc3 »

aub3bhat wrote: post #6

I think there is a need for a comprehensive system for image and video data analytics. Much like how we today have relational databases (postgres, MYSQL) and full text search engines (lucene/Solr). The approach Google or Amazon have been taking which involves providing a "tagging" API is frankly unimaginative. I am working on Deep Video Analytics an Open Source Visual Search and Analytics platform for images and vide…

I would be interested to know more about this, particularly the database and what you plan to do with it in the future (I am thinking the license on the GitHub project is obviously restrictive for a purpose at the moment).
Post reply on HN