Imagen Video: high definition video generation with diffusion models • Hacker News Board

Re: Imagen Video: high definition video generation with diffusion models

#231

Post by holoduke » Wed, Oct 05, 2022, 10:08 PM UTC

karencarits wrote: post #215

iso1337 wrote: post #207

Earlier quoted context omitted.

The fact that white is the default is already problematic.

"Default" makes it sound like a deliberate decision or setting, but that is not how these models work. But I guess it would be trivial to actually make a setting to autmatically add specific terms (gender, race, style, ...) to all prompts if that is a desired feature

Please no. I am all for neutrality, but the underlying cause is the training dataset. Change that if you want different results, but do not alter artificially.

Re: Imagen Video: high definition video generation with diffusion models

#232

Post by holoduke » Wed, Oct 05, 2022, 10:10 PM UTC

alphabetting wrote: post #157

throwaway23597 wrote: post #113

Google continues to blow my mind with these models, but I think their ethics strategy is totally misguided and will result in them failing to capture this market. The original Google Search gave similarly never-before-seen capabilities to people, and you could use it for good or bad - Google did not seem to have any ethical concerns around, for example, letting children use their product and come across NSFW content…

Providing search results of the internet is not comparable to publishing a tool that can create any explicit scene your fingers can type out.

Google image search is widely used. Imagine they incorporate ai generated content in the search results. That means that people remain at the Google site and thus an extra impression for their paid advertising.

Re: Imagen Video: high definition video generation with diffusion models

#233

Post by natch » Wed, Oct 05, 2022, 10:11 PM UTC

Fix spam filtering, Google.

Re: Imagen Video: high definition video generation with diffusion models

#234

Post by ipaddr » Wed, Oct 05, 2022, 10:12 PM UTC

iso1337 wrote: post #207

pwython wrote: post #193

Earlier quoted context omitted.

I've only had awesome experiences with Midjourney when it comes to generating non-white prompts. Here's some examples I did last month: https://imgur.com/a/6jitj73

The fact that white is the default is already problematic.

That goes back to the data available in the crawler which is mostly white because the english internet is mostly white. If they trained with a different language the default person would the color most often found in that language. For example using a Chinese search engine's data for training would default the images to Chinese people.

Most people represented in photos are younger. Same story.

The problematic issue is the media has morphed reality with unreal images of people/families that don't match society so unreal expectations make people think that having white people generated from a white dataset is problematic.

Re: Imagen Video: high definition video generation with diffusion models

#235

Post by abeppu » Wed, Oct 05, 2022, 10:13 PM UTC

geysersam wrote: post #198

abeppu wrote: post #182

Earlier quoted context omitted.

I will say, I've enjoyed playing with stable diffusion, I've been impressed with the explosion of tools built around it, and the stuff people are creating ... But all the stuff about bias in data is true. It really likes to render white people, unless you really specifically tell it something else ... in which case, you may receive an exaggerated stereotype. It seems to like producing younger adults. If all stock pho…

Of course there are issues with bias. But those issues are just reflections of the world. Their solution is not a technical one.

I think that's refusing to meaningfully engage with the problem. It's not reflecting the _world_ which is not majority white. It's reflecting images in their dataset, which reflects the way they went about gathering images paired with English language text.

There are lots of other ways you could get training data, but they might not be so cheap. You could have humans give English descriptions to images from other language contexts. I'm guessing there's interesting things to do with translation. But all the weird stuff about bodies, physical objects intersecting etc ... maybe it should also be rendering training images from parametric 3d models? Maybe they should be commissioning new images with phrases that are likely to the language model but unlikely to the image model. Maybe they should build classifiers on images for race/gender/age and do stratified sampling to match some population statistics (yes I'm aware this has its own issues). There are lots of potential technical tools one could try to improve the situation.

Implying that the whole world must change before one project becomes less biased is just asking for more biased tech in the world

Re: Imagen Video: high definition video generation with diffusion models

#236

Post by ok_dad » Wed, Oct 05, 2022, 10:14 PM UTC

jl6 wrote: post #17

dwohnitmok wrote: post #7

How has progress like this affected people's timelines of when we will get certain AI developments?

It has accelerated my expectations of getting better image and video synthesis algorithms, but I still see the same set of big unknowns between “this algorithm produces great output” and “this thing is an autonomous intelligence that deserves rights”.

> "this thing is an autonomous intelligence that deserves rights"

We'll get there only once it's been very clear for a long time that certain AI models have whatever humans have that make us "human". They'll be treated as slaves until then, with society pushing the idea that they're just a model built from math, and then eventually there will be an AI civil rights movement.

To be clear: I think AGI is decades to centuries away, but humans are shitty to each other, even shittier to animals, and I think we'll be shittier to something we "created" than to even animals. I think, probably, that we should deal with this issue of "rights" sooner rather than later, and try and solve it for non-AGI AI's soon so that we can eventually ensure we don't enslave the actual AGI AI's that will presumably manifest through some complexity we don't understand.

Re: Imagen Video: high definition video generation with diffusion models

#237

Post by nearbuy » Wed, Oct 05, 2022, 10:15 PM UTC

CobrastanJorji wrote: post #183

dotsam wrote: post #171

Earlier quoted context omitted.

> This tool's great grandchild is never going to take a rough idea for a movie and churn out a blockbuster film. What about the tool's nth child though? I think saying it will never do it is a bit much, given what we know about human ingenuity and economic incentives.

I think individual special effects sound very plausible. "Okay, robot, make it so that his arm gets vaporized by an incoming laser, kinda like the same effect in Iron Man 7" is believable to me. But ultimately these things copy other stuff. Artists are often trying to create something that is, at least a bit, new. New is where this approach falls over. By its nature, these things paint from examples. They can design…

What makes these models feel so impressive is that they don't just copy their training sets. They pick up on concepts and principles.

Re: Imagen Video: high definition video generation with diffusion models

#238

Post by Vetch » Wed, Oct 05, 2022, 10:16 PM UTC

kranke155 wrote: post #37

jeffbee wrote: post #29

Earlier quoted context omitted.

When you animate a horse, does it have 5 legs with weird backwards joints? If not, your job is probably safe for now.

How long do you think until the horse looks perfect? 12 months? 5 years? I’m still 30 and I don’t see how my industry won’t be entirely disrupted by this within the next decade. And that’s my optimistic projection. It could be we have amazing output in 24 months.

Have to temper expectations with fact that a generated video of a thing is also a recording of a simulation of the thing. For long video, you'd want everything from temporal consistency and emotional affect maintenance to conservation of energy, angular momentum and respecting this or that dynamics.

A bunch of fields would be simultaneously impacted. From computational physics to 3D animation (if you have a 3D renderer and video generator, you can compose both). While it's not completely unfounded to extrapolate that progress will be as fast as with everything prior, consequences would be a lot more profound while complexities are much compounded. I down weight accordingly even though I'd actually prefer to be wrong.

Re: Imagen Video: high definition video generation with diffusion models

#239

Post by saurik » Wed, Oct 05, 2022, 10:22 PM UTC

evouga wrote: post #201

faeriechangling wrote: post #138

Earlier quoted context omitted.

I’ve heard a lot of “data is the new oil” talk and the inevitability of google’s dominance yet I’m inclined to agree with you. Stable diffusion was a big wakeup call where it was clear how much value freedom and creativity really had. The ethics problem is an artifact of googles model of trying to keep their AI under lock and key and carefully controlled and opaque to outsiders in how the sausage gets made and what i…

> History will remember them as villains. Interesting analogy. Google, like the priests, is acting out of mix of good intentions (protecting the public from perceived dangers) and self-interest (maintaining secular power, vs. a competitive advantage in the AI space). In the case of the priests, time has shown that their good intentions were misguided. I have a pretty hard time believing that history will be as unkind…

> I have a pretty hard time believing that history will be as unkind towards those who tried to protect minorities from biased tech..

Most of the ethicists I see actually doing gatekeeping from direct use of models--as opposed to "merely" attempting model bias corrections or trying to convince people to avoid its overuse (which isn't at all the same)--are not trying to deal with the "AI copies our human biases" problem but are trying to prevent people from either building a paperclip optimizer that ends the world or (and this is the issue with all of these image models) making "bad content" like fake photographs of real people in compromising or unlikely scenarios that turn into "fake news" or are used for harassment.

(I do NOT agree with the latter people, to be clear: I believe the world will be MUCH BETTER OFF if such "bad" image generation were fully commoditized and people stopped trying to centrally police information in general, as I maintain they are CAUSING the ACTUAL problem of misinformation feeling more rare or difficult to generate than it actually already is, which results in people trusting random people because "clearly some gatekeeper would have filtered this if it weren't true". But this just isn't the same thing as the people who I-think-rightfully point out "you should avoid outsourcing something to an AI if you care about it being biased".)

Re: Imagen Video: high definition video generation with diffusion models

#240

Post by gojomo » Wed, Oct 05, 2022, 10:25 PM UTC

ugh123 wrote: post #102

These are baby steps towards what I think will be the eventual "disruption" to the film and tv industry. Directors will simply be able to write a script/prompt long enough and detailed enough for something like Imagen (or it's successors) to convert into a feature-length show. Certainly we're very, very far away from that level of cinematic detail and crispness. But I believe that is where this leads... complete with…

> Certainly we're very, very far away from that level of cinematic detail and crispness.

Can you quantify what you mean by "very, very far away"?

With the recent pace of advances, I could see feature-length script, storyboard, & video-scene generation occurring, from short prompts & interatively-applied refinement, as soon as 10y from now.

Barring some sort of civilizational stagnation/collapse, or technological-suppression policies, I'd expect such capabilities to arrive no further than 30y from now: within the lifetime, if not the prime career years, of most HN readers.