Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Make-A-Video: AI system that generates videos from text

makeavideo.studio

Post reply on HN
Print view View original

291–300 of 399 posts

Re: Make-A-Video: AI system that generates videos from text

#291

Post by mlsu »

tmjdev wrote: post #125
Filligree wrote: post #44

Earlier quoted context omitted.

So you need to optimise on compressed version, not the whole thing. What they’re doing right now is akin to a human trying to hold an entire picture - or entire movie - in their head all at once. We can’t do it. AIs can sort of do it. Latent diffusion models already demonstrated that operating on a compressed representation gives far better results, faster, but I don’t think we’re anywhere near the limit for what’s p…

I am curious if there has been any research on temporal attention in humans. I'm not sure how you'd quantify it. But in myself I know that I'm constantly predicting where something will be or what it will look like based on how it did a second ago. It's probably the root of reflexes.

Your comment reminded me of this video [1]

They put an eye tracker on someone and captured their motion when walking in some rough terrain. You can sort of see that the person is focusing on the most likely place their foot will go next.

[1] https://www.youtube.com/watch?v=ph6uUHq3a-g

I think that we will discover that there is a more efficient way to encode temporal relationships, which appears to be "just throw transformers at it." My guess is that it will be in a more conceptual latent space that this attention will be applied.

Re: Make-A-Video: AI system that generates videos from text

#293

Post by permo-w »

anigbrowl wrote: post #270
permo-w wrote: post #222

Earlier quoted context omitted.

> A lot of people saying it's over for traditional movie-making who is saying that?

Me, film-industry vet. You'll see an excellent mostly- or all-AI feature within 5-10 years. There will be terrible ones before that, maybe 2-3 years. The first really good one will be enjoyed on its own merits, ad the artificiality of it will come to light after it has gained popularity.

I would imagine you’ll see one or a few that are barely AI-produced as soon as marketing teams get wind of this. “come and see the first ever AI-produced* blockbuster! ⠀ ⠀ ⠀

*produced, assisted, or discussed on set one time during lunchbreak”

Re: Make-A-Video: AI system that generates videos from text

#294

Post by treis »

agitator wrote: post #233

What's mind blowing is that you can extrapolate where this is going to go. Eventually, you will be able to generate full movie scenes from descriptions. What's interesting to me is how this is so similar to human imagination. Give me a description and I will fabricate the visuals in my mind. Some aspects will be detailed, others will be vague, or unimportant. Crazy to see how fast AI is progressing. Machines are appr…

IMHO this particular avenue is a dead end. It's an extraordinarily impressive dead end but it's clear that there's no real understanding here. Look at this video of the knight riding a horse:

>https://makeavideo.studio/assets/A_knight_riding_on_a_horse_...

The horse's face is all wrong

The gait is wrong

The interface with the ground & hooves is wrong

The knight's upper body doesn't match with the lower and they're not moving correctly

I think ultimately the right path is something like AI automated Blender. AI creates the models & actions while Blender renders it according to a rules based physics engine.

Re: Make-A-Video: AI system that generates videos from text

#295

Post by lossolo »

chatterhead wrote: post #249
permo-w wrote: post #240

I really don’t understand the fear people have about these things. have I missed something and everyone else was placing huge value in out of context videos and pictures? if you read “France declares war on Canada”, you’re not gonna believe it unless it’s coming from an extremely reputable source. so why would you trust a random unsourced video? the absolute worst thing that’s gonna happen is that video-based social…

We are quickly approaching a point where these independently created AI systems will be wrangled together the same away protocols were to create a computer network and the AI that emerges will likely be able to create its own code, solve its own problems and generate its own recursive processes. Everything at a conscious level without the ability to self recognize. If AI ever does wake up; we'll never know - first th…

> first thing that happens is it will hide from us.

To do this it would need to want to live and know what fear is. It's just a piece of code, without being conscious there is no need to be online/alive for it. What people call AI today is just a few/hundreds GBs of data that reacts to some text and push it through a heuristic like system to get an answer doing fancy pattern matching. It doesn't do anything when there is no input, there are no processes, no thinking, no anything, it's "dead". Fear and instinct is not taught in any species, it's inherited prior and biological process, you would need to explicitly code feeling into AI for it to be able to "feel" anything.

Re: Make-A-Video: AI system that generates videos from text

#296

Post by nemo44x »

consumer451 wrote: post #194
mjr00 wrote: post #140

Earlier quoted context omitted.

A long time. Much like self-driving cars, AI can take you 90% of the way there, but the last 10% is the difference between music that sort-of-seems-competent, and music that people will actually listen to and enjoy.

Is it really the same though? A driving system which can only create a good outcome 90% of the time is not really useful as irl human safety is a factor. However, a creative system curated by a human could end up creating useful outputs, could it not?

Why humans? Build an AI that can judge good outcomes from bad. Build another AI that can judge good outcome AIs, etc. Its AIs all the way down.

Re: Make-A-Video: AI system that generates videos from text

#297

Post by pmarreck »

slhomme wrote: post #121

As an owner of a Video Production studio, this kind of tech is blowing my mind and makes me equally excited and scared. I can see how we could incorporate such tools in our workflows, and at the same time I'm worried it'll be used to spam the internet with thousands and thousands of souless generated videos, making it even harder to look through the noise. A fun related experiment, I thought it was fun to see what ki…

https://thismoviedoesnotexist.org/movie/the-yetis-last-stand

The description text does not convert sentences with carriage returns (or probably, newlines) into separate div's or whatever html element you'd prefer, FYI! Otherwise, very cool!

Re: Make-A-Video: AI system that generates videos from text

#298

Post by masterspy7 »

j_m_b wrote: post #28

What I want to see is a model that can generate 3D models for use in applications such as Blender. It would provide a good starting point for someone with talent to make beautiful. Or just save people like me time for making games.

Funnily enough, you might not even need a 3D model if the txt2video is good enough. Whatever you wanted to render in Blender could just be rendered via text prompt (when this becomes 100x better).

Re: Make-A-Video: AI system that generates videos from text

#299

Post by inerte »

tpmx wrote: post #8
Datenstrom wrote: post #2

The > A golden retriever eating ice cream on a beautiful tropical beach at sunset, high resolution example is terrifying.

In case anyone else had problems finding it - it's at the bottom of the page: https://makeavideo.studio/assets/a_golden_retriever_eating_i... (webp) That grasp though. These things still feel a bit like e.g. Google/GCP services to me: Super appealing at first glance, quite close to what you want, but somehow never quite there. Maybe they'll asymptotically get there, eventually? Perhaps that statistical model can't re…

I’ve found that replacing the bad parts with new ones, like Dalle Outpainting, can remove the worst parts of the image, like the hands here… doesn’t make it perfect, but certainly removes the worst offenders that instantly bring attention to themselves.

Re: Make-A-Video: AI system that generates videos from text

#300

Post by pkaye »

ericd wrote: post #284
pkaye wrote: post #253

Earlier quoted context omitted.

Then the rich will have no consumers.

Why do you need consumers when you have machines making most of what you want? Then you just need to be able to trade things with other people who have different machines and resources from yourself to get the bits you can't have made by your machines.

So you are saying everyone will have those machines?
Post reply on HN