It's funny that still don't really have video wallpapers on most devices (I'm only aware of Wallpaper Engine on Windows)
I had a video wallpaper on my Motorola Droid back in 2010.
Stable Video Diffusion
181–190 of 316 posts
Re: Stable Video Diffusion
#182It makes sense that they had to take out all of the cuts and fades from the training data to improve results. I’m the background section of the research paper they mention “temporal convolution layers”, can anyone explain what that is? What sort of training data is the input to represent temporal states between images that make up a video? Or does that mean something else?
It means that instead of (only) doing convolution in spatial dimensions, it also(/instead) happens in the temporal dimension.
A good resource for the "instead" case: https://unit8.com/resources/temporal-convolutional-networks-...
The "also" case is an example of 3D convolution, an example of a paper that uses it: https://www.cv-foundation.org/openaccess/content_iccv_2015/p...
Re: Stable Video Diffusion
#183Earlier quoted context omitted.
Controlnet is adapted to video today, the issues are that it's very slow. Haven't you seen the insane quality of videos on civitai?
> Haven't you seen the insane quality of videos on civitai? I have not, so I went to https://civitai.com/ which I guess is what you're talking about? But I cannot find a single video there, just images and models.
A small percentage of the images are animations. This id (for obvious reasons) particularly common for images used on the catalog pages for animation-related tools and models, but also its not uncommon for (AnimateDiff-based, mostly) animations to be used to demo the output of other models.
Re: Stable Video Diffusion
#184Earlier quoted context omitted.
It also somewhat depends on open legal questions like whether models are copyrightable and, if so, whether model outputs are derivative works of the model. Suppose that models are not copyrightable, due to their not being the product of human creativity (this is debatable). Then the creator can still require people to agree to contractual terms before downloading the model from them, presumably including the usage li…
> Then the creator can still require people to agree to contractual terms before downloading the model from them, presumably including the usage limitations as well as an agreement not to redistribute the model to anyone else who does not also agree. I don't think it's possible to invent copyright-like rights.
Why not? Two willing parties can agree to bind themselves to all kinds of obligations in a contract as long as they're not explicitly illegal.
Copyleft is an example of someone successfully inventing a copyright-like right by bootstrapping off existing copyright with a specially engineered contract.
Re: Stable Video Diffusion
#185Looks like I'm still good for my bet with some friends that before 2028 a team of 5-10 people will create a blockbuster style movie that today costs 100+ million USD on a shoestring budget and we won't be able to tell.
I wouldn't bet either way. Back in the mid 90s to 2010 or so, graphical improvements were hailed as photorealistic only to be improved upon with each subsequent blockbuster game. I think we're in a similar phase with AI[0]: every new release in $category is better, gets hailed as super fantastic world changing, is improved upon in the subsequent Two Minute Papers video on $category, and the cycle repeats. [0] all of…
> Back in the mid 90s to 2010 or so, graphical improvements were hailed as photorealistic
Whenever I saw anybody calling those graphics "photorealistic", I always had to roll my eyes and question if those people were legally blind.
Like, c'mon. Yeah, they could be large leaps ahead of the previous generation, but photorealistic? Get real.
Even today, I'm not sure there's a single game that I would say has photo-realistic graphics.
Re: Stable Video Diffusion
#186Earlier quoted context omitted.
In a few years' time, teenagers will be consuming shows and films made by their peers, not by streaming providers. They'll forgive and perhaps even appreciate the technical imperfections for the sake of uncensored, original content that fits perfectly with their cultural identity. Actually, when processing power catches up, I'm expecting a movie engine with well-defined characters, scenes, entities, etc., so people w…
They do that now (forget the name there's a popular one my niece uses to make animated comics, others do similar things in Minecraft etc), and have been doing that since forever - nearly 30 years ago my friends and I were scribbling comic panels into our notebooks and sharing them around class.
ms comic chat for the win
Re: Stable Video Diffusion
#187Earlier quoted context omitted.
> Has anyone come across a solution where model can iterate (eg, with prompts like "move the bicycle to the left side of the photo")? It feels like we're close. I feel like we're close too, but for another reason. For although I love SD and these video examples are great... It's a flawed method: they never get lighting correctly and there are many incoherent things just about everywhere. Any 3D artist or photographer…
> you'll describe something and you'll get a full 3D scene, with 3D models, source of lights set up, etc. I'm always confused why I don't hear more about projects going in this direction. Controlnets are great, but there's still quite a lot of hallucination and other tiny mistakes that a skilled human would never make.
I think the bottleneck is data
For single 3D object the biggest dataset is ObjaverseXL with 10M samples
For full 3D scenes you could at best get ~1000 scenes with datasets like ScanNet I guess
Text2Image models are trained on datasets with 5 billion samples
Re: Stable Video Diffusion
#188The rate of progress in ML this past year has been breath taking. I can’t wait to see what people do with this once controlnet is properly adapted to video. Generating videos from scratch is cool, but the real utility of this will be the temporal consistency. Getting stable video out of stable diffusion typically involves lots of manual post processing to remove flicker.
What was the big “unlock” that allowed so much progress this past year? I ask as a noob in this area.
I mean, you probably didn't pay much attention to battery capacity before phones, laptops, and electric cars, right? Battery capacity has probably increased though at some rate before you paid attention. It's just when something actually becomes relevant that we notice.
Not that more advances don't happen with sustained hype, just there's some sort of tipping point involving usefulness based either on improvement of the thing in question or it's utility elsewhere.
Re: Stable Video Diffusion
#189This field moves so fast. Blink an eye and there is another new paper. This is really cool and the learning speed of us humans is insane! Really excited on using it for downstream tasks! I wonder how easy it is to integrate animatediff with this model?
Also, can someone benchmark it on m3 devices? It would be cool to see if it is worth getting on to run these diffusion inferences and development. If m3 pro can allow finetuning it would be amazing to use it on downstream tasks!
Re: Stable Video Diffusion
#190Very excited to play with this. Some of my latest experiments - https://www.jasonfletcher.info/vjloops/