Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Imagen Video: high definition video generation with diffusion models

imagen.research.google

Post reply on HN
Print view View original

271–280 of 500 posts

Re: Imagen Video: high definition video generation with diffusion models

#271

Post by splatzone »

javchz wrote: post #267
fassssst wrote: post #241

How long until the AI just generates the entire frame buffer on a device? Then you don’t need to design or program anything; the AI just handles all input and output dynamically.

Imagine you click a youtube video in a bad network envoirment, then the server sends like an alt tag equivalent for the video as a promnt, and the Neural Engine chip inside your phone create the first seconds of the video while it loads. We're fay away from it now, but I've seen less sketchy solutions being implemented.

I wonder if this is an area actively being researched, using models like these for video compression?

Re: Imagen Video: high definition video generation with diffusion models

#272

Post by seydor »

armchairhacker wrote: post #168
oceanplexian wrote: post #112

Earlier quoted context omitted.

I actually think it's the opposite, AI will probably be writing the stories and humans might occasionally film a few scenes. ~95% of TV shows and movies are cookie-cutter content, with cookie-cutter acting and production values, with the same hooks and the same tropes regurgitated over and over again. Heck they can't even figure out how to make new IP so they keep making reruns of the same old stuff like Star Wars, M…

The last-mile problem applies here too. GPT-3 text is convincing at a distance but when you look closely there is no coherence, no real understanding of plot or emotional dynamics or really anything. TV shows and movies are filled with plot holes and bad writing but it's not that bad. Also I think "a good algorithm" is more than just repetitive content. The plots are reused and generic, but there's real skill involve…

yes someone will need to hand-pick the best versions of each episode . over time a large enough dataset will have been generated that a model can be trained to the task of curation

Re: Imagen Video: high definition video generation with diffusion models

#273

Post by mcbuilder »

splatzone wrote: post #271
javchz wrote: post #267

Earlier quoted context omitted.

Imagine you click a youtube video in a bad network envoirment, then the server sends like an alt tag equivalent for the video as a promnt, and the Neural Engine chip inside your phone create the first seconds of the video while it loads. We're fay away from it now, but I've seen less sketchy solutions being implemented.

I wonder if this is an area actively being researched, using models like these for video compression?

I believe they are already state of the art for image compression.

That being said these shitty video models I believe are just an arms race between Meta and Google after the release of stable diffusion. Microsoft has a video version of CLIP that I believe will really change the game, but unless you have trained a model with video embeddings it's all going to look devoid of any narrative. Right now the models just look like a sequence of images with the same promt and some sort of continuity to make it look more video like.

Re: Imagen Video: high definition video generation with diffusion models

#274

Post by fassssst »

i_like_apis wrote: post #261

The concern trolling and gatekeeping about social justice issues coming from the so-called "ethicists" in the AI peanut gallery has been utterly ridiculous. Google claims they don't want to release Imagen because it lacks what can only be called "latent space affirmative action". Stability or someone like it will valiantly release this technology, again and there will be absolutely no harm to anyone. Stop being so to…

Google is a bigger lawsuit target since they have more $, probably simple as that. I’m not a lawyer but I’m sure there will be many copyright challenges.

Re: Imagen Video: high definition video generation with diffusion models

#276

Post by seydor »

throwaway23597 wrote: post #113

Google continues to blow my mind with these models, but I think their ethics strategy is totally misguided and will result in them failing to capture this market. The original Google Search gave similarly never-before-seen capabilities to people, and you could use it for good or bad - Google did not seem to have any ethical concerns around, for example, letting children use their product and come across NSFW content…

google can be sued for billions of money for a product that is not making any money yet. SD probably can't that s how i see it. So of course they ll cover their ass rather than trying to make something cool

Re: Imagen Video: high definition video generation with diffusion models

#278

Post by fzysingularity »

What's next? Dreamfusion Video = Imagen Video (this) + Dreamfusion (https://dreamfusion3d.github.io/)

Fundamentally, I think we have all the pieces based on this work and Dreamfusion to make it work. From the looks of it, there's a lot of SSR (spatial SR) and TSR (temporal SR) going on at multiple levels to upsample (spatially) and smoothen (temporally) images that won't be needed for NERFs.

What's impressive is the ability to leverage billion-scale image-text pairs for training a base model that can be used to super-resolve over space and time. And that they're not wastefully training video models from scratch, and instead separately training TSR, SSR models for turning the diffused images to video.

Re: Imagen Video: high definition video generation with diffusion models

#280

Post by Garlef »

evouga wrote: post #249

> We train our models on a combination of an internal dataset consisting of 14 million video-text pairs The paper is sorely lacking evaluation; one thing I'd like to see for instance (any time a generative model is trained on such a vast corpus of data) is a baseline comparison to nearest-neighbor retrieval from the training data set.

In the end: Who cares? Sure, from an academic perspective this might be interesting. But in the end it's the users who will pick the tool. And they will figure out the things the tool can not express in the first week after release.

So: Focusing on increasing expressiveness and ergonomics should beat academic rigour.

Post reply on HN