Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rendering a game in real time with AI

blog.jeffschomay.com

Post reply on HN
Print view View original

41–50 of 115 posts

Re: Rendering a game in real time with AI

#41

Post by Janicc »

lukan wrote: post #32
abbycurtis33 wrote: post #28

Earlier quoted context omitted.

The tech will improve to far exceed the capabilities of a game engine. Real time improvisation and infinite choices, scope, etc. It makes no sense when people say AI can't do this or that. It will do it next week.

"It makes no sense when people say AI can't do this or that. It will do it next week." So full self driving vecicles will be finally ready next week then? Great to hear, though to be honest, I remain sceptical.

You really need to update your language model because self driving cars have been driving around on their own for at least a year now

Re: Rendering a game in real time with AI

#42

Post by 127 »

abbycurtis33 wrote: post #28
coolKid721 wrote: post #23

I do not get the point of this at all, why not just generate game assets and run them in an engine? With this format there would be no regularity that the thing you saw before will look the same (and that is not a fixable problem). Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past…

The tech will improve to far exceed the capabilities of a game engine. Real time improvisation and infinite choices, scope, etc. It makes no sense when people say AI can't do this or that. It will do it next week.

An interactive feedback loop that handles various edge cases of AI, rendering it, asset loading and display, keeping track of global data, user input, etc. -- is still a game engine.

Re: Rendering a game in real time with AI

#43

Post by sjsdaiuasgdia »

curl-up wrote: post #8
roxolotl wrote: post #5

Earlier quoted context omitted.

Why does using a language/vision model feel more “real” to you than using equations which directly describe our understandings of physics?

Not OP, but I have long thought of this type of approach (underlying "hard coded" object tracking + fuzzy AI rendering) to be the next step, so I'll respond. The problem with using equations is that they seem to have plateaued. Hardware requirements for games today keep growing, and yet every character still has that awful "plastic skin", among all the other issues, and for a lot of people (me included) this creates…

I find it odd that you're that bothered by uncanny valley effects from game rendering but apparently not by the same in image model outputs. They get little things wrong all the time and it puts me off the image almost instantly.

Re: Rendering a game in real time with AI

#44

Post by jdiff »

Janicc wrote: post #41
lukan wrote: post #32

Earlier quoted context omitted.

"It makes no sense when people say AI can't do this or that. It will do it next week." So full self driving vecicles will be finally ready next week then? Great to hear, though to be honest, I remain sceptical.

You really need to update your language model because self driving cars have been driving around on their own for at least a year now

"Full self driving" was the term used and I believe the distinction is relevant to the point being made.

Re: Rendering a game in real time with AI

#45

Post by GPerson »

abbycurtis33 wrote: post #28
coolKid721 wrote: post #23

I do not get the point of this at all, why not just generate game assets and run them in an engine? With this format there would be no regularity that the thing you saw before will look the same (and that is not a fixable problem). Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past…

The tech will improve to far exceed the capabilities of a game engine. Real time improvisation and infinite choices, scope, etc. It makes no sense when people say AI can't do this or that. It will do it next week.

I’m looking forward to the day when magical thinking such as this gets grounded again. That is when the real work will start anew.

Re: Rendering a game in real time with AI

#46

Post by sho_hn »

jdiff wrote: post #44
Janicc wrote: post #41

Earlier quoted context omitted.

You really need to update your language model because self driving cars have been driving around on their own for at least a year now

"Full self driving" was the term used and I believe the distinction is relevant to the point being made.

I understand the point you're making, but I think it's not a good one.

The failure mode for getting a self-driving car right is grave. The failure mode for rendering game graphics imperfectly is to require a bit of suspension of disbelief (it's not a linear spectrum given the famous uncanney valley, etc., I'm aware). Games already have plenty of abstract graphics, invisible walls, and other cludges that require buy-in from users. It's a lot easier to scale that wall.

Re: Rendering a game in real time with AI

#47

Post by bob1029 »

The power consumption of modern gaming is getting a bit out of hand. This AI stuff is taking it to the next level.

Ray tracing and other forms of real time global illumination are extremely resource intensive approaches to lighting a scene. Every client machine has to figure out how to light everything every single frame. Contrast this with baked global illumination where the cost is incurred exactly once and is amortized across potentially millions of machines.

We need more things like baked GI in gaming. This class of techniques makes the development iterations slower and more deliberate, but it also produces a far more efficient and refined product experience. I'd be very interested in quantifying the carbon impact of realtime vs baked lighting in gaming. It is likely a non-trivial figure at scale. Also bear in mind that baked GI is why games like the batman series still look so good in 2025, even when running on period hardware. You cannot replace your art team by consuming more electricity.

Re: Rendering a game in real time with AI

#48

Post by Topfi »

hiatus wrote: post #29
steveruizok wrote: post #13

We did a similar thing at tldraw with Draw Fast ( https://drawfast.tldraw.com/ ) and it was very fun. Inspired a few knock offs too. We had to shut it down because it was getting popular on Russian Reddit. A related project Lens ( https://lens.tldraw.com ) also used the same technique, but in a collaborative drawing app. At the peak, when we were streaming back video from Fal and getting <100ms of lag, the setup prod…

Is there any chance you'd open up the source for those projects so others can play with them?

They've already shared it under a none-commercial use license: https://github.com/tldraw/draw-fast

The TLDraw team, from what I have seen, is really open in sharing their experiments under their own license. Not FOSS strictly, but I feel their licensing approach and decision is fair considering how they fund development, know that it's a contentious topic though for some.

Post reply on HN