11–19 of 19 posts
one of cool google projects that does not take off because its so hard to run and integrate with other cool things...
If we take the less cynical take, it'll end up in Youtube as audio description for people with visual impairments.
"the employee is talking to X, the employee is working, the employee is going to the toilet"
Had this idea about 5 years ago, seems like it might be viable now, would love to see a video analyzer that creates relationship graphs using sentiment analysis. Ex. who responds to whom, what their tone is, how often. Hopefully with modern methods, it wouldn't take too many examples to pick out the dimensionality of expression (cheating out vs in, loudness + arousal, whining on the spectrum of playful to hurt, confi…
My first thoughts were also how this would effect the porn industry…
Had this idea about 5 years ago, seems like it might be viable now, would love to see a video analyzer that creates relationship graphs using sentiment analysis. Ex. who responds to whom, what their tone is, how often. Hopefully with modern methods, it wouldn't take too many examples to pick out the dimensionality of expression (cheating out vs in, loudness + arousal, whining on the spectrum of playful to hurt, confi…
> when they create context from unrelated scenes by cutting from one to the other. Do you mean juxtaposition?
That's along the same vein tho
one of cool google projects that does not take off because its so hard to run and integrate with other cool things...
If we take the less cynical take, it'll end up in Youtube as audio description for people with visual impairments.
Had this idea about 5 years ago, seems like it might be viable now, would love to see a video analyzer that creates relationship graphs using sentiment analysis. Ex. who responds to whom, what their tone is, how often. Hopefully with modern methods, it wouldn't take too many examples to pick out the dimensionality of expression (cheating out vs in, loudness + arousal, whining on the spectrum of playful to hurt, confi…
Some of this: https://vi.microsoft.com/en-us
But no, that's not it at all. The sentiment analysis buried at the bottom there is generic. It's advertised for aggregate customer mood bc that's the only thing it's sensitive enough to detect, thumbs up or down.
I mean something like https://m.youtube.com/watch?v=8dagzaFjHU4&t=2m, the expressions in the eyes, ex. concern - solicitous - complacent/satisfied - steely from 2:10 - 2:20.
That + the same for voice tone would be everything.
one of cool google projects that does not take off because its so hard to run and integrate with other cool things...
This isn’t really a “google project” in the way I think about that term, but it’s a research project. Google’s research is constantly advancing and when things get far enough along they do tend to get used in production. Individual research papers are just a step along the way. This research seems useful for training video generation systems like transformers and especially multi modal systems. Imagine you have a rob…
one of cool google projects that does not take off because its so hard to run and integrate with other cool things...
https://github.com/google-research/scenic/tree/main/scenic/p... has an example showing how to "train Vid2Seq on YouCook2" using "python -m scenic.projects.vid2seq.main", but I couldn't see the recipe for using it against a video to return a description.