Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Experiment: Can 3D improve AI video consistency?

backdroptech.github.io

Post reply on HN
Print view View original

21–30 of 38 posts

Re: Experiment: Can 3D improve AI video consistency?

#21

Post by BubbleRings »

TobiasEnholmX wrote: post #6
daef wrote: post #3

i am very confused... what is 3D? to me 3D means three dimensions - but it looks like 3D refers to a product? this page doesn't help with explaining anything

Good question! When we say '3D' here, we mean using 3D models and scenes as a base to guide AI video generation. The AI isn’t just making things up from scratch—it follows the structure of a 3D-rendered scene to keep things consistent across frames. It’s not a product, just an approach to solving the problem of AI-generated video drifting too much between frames.

I’d suggest adjusting your demo, I also couldn’t figure out if “3D” meant a technique or a product here.

Re: Experiment: Can 3D improve AI video consistency?

#22

Post by TobiasEnholmX »

echelon wrote: post #19
TobiasEnholmX wrote: post #2

AI-generated video struggles with consistency. Flickering, weird proportions, and characters changing. I tried using 3D as a way to get more consistency. Overall, it worked. No sudden changes in proportions, clothing, or style. Still, there are some limitations, especially with fine details. We’re looking into whether this could be useful as a tool and would love to hear what you think: Has anyone experimented with 3…

We and several other startups tried this, and we even filed a few patents. 1. The users with experience and patience for this are slim. Blocking out a scene and the animation are tough, and the users with this skill and inclination are using Blender and ComfyUI already or are submitting renders to RunwayML V2V. We're still too early for AI auto rigging and animation to work, though those technologies will make this a…

First off, I’d love to hear more about your experience. If you’re up for a chat, shoot me an email at tobias@backdrop.tech, or let me know where I can learn more about what you’ve worked on!

1. I assume you mean patience when setting up a 3D scene? That’s definitely a factor, but it’s getting easier with image-to-3D tools, and AI can even assist with object placement to speed things up.

2.Yeah, predictability is key. Our approach is about making it easier to generate high-quality, consistent images, which are then fed into video models—rather than relying on direct video-to-video style transfer, which can be more chaotic.

3. Agreed! AI can animate things that traditional methods struggle with, but consistency is still a challenge. This workflow helps strike a balance between AI flexibility and user control.

Re: Experiment: Can 3D improve AI video consistency?

#23

Post by erichocean »

TobiasEnholmX wrote: post #2

AI-generated video struggles with consistency. Flickering, weird proportions, and characters changing. I tried using 3D as a way to get more consistency. Overall, it worked. No sudden changes in proportions, clothing, or style. Still, there are some limitations, especially with fine details. We’re looking into whether this could be useful as a tool and would love to hear what you think: Has anyone experimented with 3…

Yes, it's better to use AI to automate individual steps in the actual animation process itself, with AI workers.

The quality is identical to human animators (because: same tools, same process).

Just the cost is lower (although training the AI workers is a new cost).

The only companies that can do this are those that have a very strong workflow, because AI workers operate on individual steps, NOT the entire workflow.

2D animation already has this, so it's easier to adapt to AI workers than 3D (for that reason).

Re: Experiment: Can 3D improve AI video consistency?

#27

Post by TobiasEnholmX »

nailer wrote: post #24

This is great tech, but I was expecting more from “I’ll show you how I used 3D to make AI generations more predictable” - I wanted to know how, instead we essentially got an ad.

Fair point! We debated whether to post this here since it's more about examples rather than the full how-to and technical breakdown. Next post, we’ll aim to go deeper into the process and techniques behind it. Appreciate the feedback!

Re: Experiment: Can 3D improve AI video consistency?

#29

Post by TobiasEnholmX »

weinzierl wrote: post #26

It would be surprising if it didn't, but it's a question of diminishing returns. The complexity and cost likely increase approximately with the third power. Does the result justify this effort?"

We’re actually finding that leveraging 3D reduces complexity rather than increasing it. Setting up a 3D scene is getting easier with tools like image-to-3D, and once it's set up, you can reuse it flexibly across different shots without having to recreate assets from scratch.

By passing a structured 3D-rendered image to the AI, you increase the chances of the video model generating exactly what you want, rather than relying on unpredictable, frame-by-frame AI improvisation.

Re: Experiment: Can 3D improve AI video consistency?

#30

Post by TobiasEnholmX »

ncfausti wrote: post #28

This is one of the most impressive demos I've ever seen. Truly incredible. I have been dreaming about working on something like this for a while. Open to help?

Glad you like it! In the blog post you can find my email, or join the discord https://discord.gg/backdrop
Post reply on HN