How far are we from Ender's Game?
Rendering a game in real time with AI
51–60 of 115 posts
Re: Rendering a game in real time with AI
#52I do not get the point of this at all, why not just generate game assets and run them in an engine? With this format there would be no regularity that the thing you saw before will look the same (and that is not a fixable problem). Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past…
The tech will improve to far exceed the capabilities of a game engine. Real time improvisation and infinite choices, scope, etc. It makes no sense when people say AI can't do this or that. It will do it next week.
Having spent 20 years making game engines and the past 4 years playing with AI image gen, I believe you are right.
There have been musings for a while now that 3D rendering is going to switch from “lay down the scene’s albedo & specular parameters then do the lighting pass” to “lay down the scene’s latent parameters and then do the diffusion pass”.
Recently, the advances in “real time AI world models” have been coming ridiculously fast.
Put these together and it’s no stretch at all imagining a game built by having artists go nuts doing whatever they want with whatever Maya can handle as long as they also make proxy geometry of trivial complexity that can be conceptually associated with the final renders. Train the AI on the association. Render the proxy geometry the old fashioned way. AI that up in real time to the associated Maya-final-render approximation.
It’s not going to happen this week. But, in 5 years? Somebody’s gonna pull it off.
Re: Rendering a game in real time with AI
#53Earlier quoted context omitted.
"It makes no sense when people say AI can't do this or that. It will do it next week." So full self driving vecicles will be finally ready next week then? Great to hear, though to be honest, I remain sceptical.
Waymos are driving themselves around several cities right now.
Yes, go to rome. I will be impressed when we have self driving cars in rome.
Re: Rendering a game in real time with AI
#54Cool, if the game is promptable, one may even be able to make the game accessible.
Re: Rendering a game in real time with AI
#55Earlier quoted context omitted.
You're comparing apples to oranges, holding up today's practical real-time rendering techniques against a hypothetical future neural method that runs many orders of magnitude faster than anything available today, and solves the issues of temporal stability, directability and overall robustness. If we grant "equation based" methods the same liberty then we should be looking ahead to real-time pathtracing research, whi…
The question was "why does it feel more real", and I answered that - because the best AI generated images today feel more real than the best 3D renders, even when they take all the compute in the world to finish. So I can imagine that trend going forward into real-time rendering as well. I did not claim that AI-based rendering will overcome traditional methods, and have even explicitly said that this is a heavy assum…
I think we'll have to agree to disagree about well done 3D renders not feeling real. Movie studios still regularly underplay how much CGI they use for marketing purposes, and get away with it, because the CGI looks so utterly real that nobody even notices it until much later when the VFX vendors are allowed to give a peek behind the curtain.
e.g. Top Gun Mavericks much lauded "practical" jet shots, which were filmed on real planes, but then the pilots were isolated and composited into 100% CGI planes with the backdrops also being CGI in many cases, and huge swathes of viewers and press bought the marketing line that what they saw was all practical.
Re: Rendering a game in real time with AI
#56I'm pretty sure the generation could easily run locally on a low-to-mid tier graphics card. While it might take a bit longer to generate, you're still saving network and authentication latency.
I do not see how it replaces or substitutes network and authentication latency, especially in terms of a single player game in which neither is necessary.
Re: Rendering a game in real time with AI
#57I do not get the point of this at all, why not just generate game assets and run them in an engine? With this format there would be no regularity that the thing you saw before will look the same (and that is not a fixable problem). Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past…
> I do not get the point of this at all Dunno, this seems like an avenue definitely worth exploring. Plenty of game applications today already have a render path of input -> pass through AI model -> final image. That's what the AI-based scaling and frame interpolation features like DLSS and FSR are. In those cases, you have a very high-fidelity input doing most of the heavy lifting, and the AI pass filling in gaps. E…
*Edit: misread the post I replied to, disregard comment contents*
> Plenty of game applications today already have a render path of input -> pass through AI model -> final image.
Where "plenty" equals zero?
> In those cases, you have a very high-fidelity input doing most of the heavy lifting, and the AI pass filling in gaps.
That is, in these cases you already have high-fidelity input in the form of an actual game, and "AI" contributions to the output are dubious at best.
Do you really believe it's DLSS that's doing all the heavy lifting in a game like Expedition 33 or Cyberpunk 2077?
Re: Rendering a game in real time with AI
#58Earlier quoted context omitted.
"It makes no sense when people say AI can't do this or that. It will do it next week." So full self driving vecicles will be finally ready next week then? Great to hear, though to be honest, I remain sceptical.
You really need to update your language model because self driving cars have been driving around on their own for at least a year now
So have they stopped having the >1 average remote drivers for each self driving vehicle as well?
The problem with these statements is language has so much context implicit in it. "driving around on their own" to me means with zero active oversight. "driving around" to me means not just in a small set of city streets, but as a replacement for human driving (eg anywhere a vehicle can physically fit). Obviously to you it means other things, but it's what makes these conversations and statements of fact challenging.
Re: Rendering a game in real time with AI
#59Earlier quoted context omitted.
Yeah, but I find this fascinating regardless. This is getting into the direction of a kind of simulation where stuff is not determined by code but a kind of "real" physics.
Why does using a language/vision model feel more “real” to you than using equations which directly describe our understandings of physics?
Because our ability to simulate/render a realistic world in real time using direct equations is still very limited. We’re accustomed to these limitations and often feel “graphics are good enough”. But, we’ll always be decades behind “ILM in real time”.
The AI route has a good chance of moving us from decades behind ILM to merely “years behind ILM”.
Re: Rendering a game in real time with AI
#60I do not get the point of this at all, why not just generate game assets and run them in an engine? With this format there would be no regularity that the thing you saw before will look the same (and that is not a fixable problem). Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past…
Visually speaking, there's always visual issues in tying disparate assets together in a seamless fashion. I can see how AI could be easily used to "hide the seams" so to speak. I think a hybrid approach would be an improvement definitely.
AI can't even hide its own seams. The seams are kind of the defining characteristic of AI.