Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rendering a game in real time with AI

blog.jeffschomay.com

Post reply on HN
Print view View original

61–70 of 115 posts

Re: Rendering a game in real time with AI

#61

Post by Workaccount2 »

coolKid721 wrote: post #23

I do not get the point of this at all, why not just generate game assets and run them in an engine? With this format there would be no regularity that the thing you saw before will look the same (and that is not a fixable problem). Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past…

> (and that is not a fixable problem)

Genie 3 is incredible (relative to previous models) in this regard. Not a solve, but it is doing what it is not supposed to be able to do.[1]

[1]https://deepmind.google/discover/blog/genie-3-a-new-frontier...

Re: Rendering a game in real time with AI

#62

Post by suddenlybananas »

kayamon wrote: post #35
lukan wrote: post #32

Earlier quoted context omitted.

"It makes no sense when people say AI can't do this or that. It will do it next week." So full self driving vecicles will be finally ready next week then? Great to hear, though to be honest, I remain sceptical.

Waymos are driving themselves around several cities right now.

Wait til it snows.

Re: Rendering a game in real time with AI

#63

Post by johnisgood »

suddenlybananas wrote: post #62
kayamon wrote: post #35

Earlier quoted context omitted.

Waymos are driving themselves around several cities right now.

Wait til it snows.

The discussion around self-driving cars often feels like shifting goalposts: each time one feature is achieved, a new requirement is added, perpetually delaying the "final" answer.

Self-driving cars are "here"... until someone adds another requirement.

Re: Rendering a game in real time with AI

#64

Post by Workaccount2 »

SCdF wrote: post #58
Janicc wrote: post #41

Earlier quoted context omitted.

You really need to update your language model because self driving cars have been driving around on their own for at least a year now

So have they stopped having the >1 average remote drivers for each self driving vehicle as well? The problem with these statements is language has so much context implicit in it. "driving around on their own" to me means with zero active oversight. "driving around" to me means not just in a small set of city streets, but as a replacement for human driving (eg anywhere a vehicle can physically fit). Obviously to you i…

That >1 spec is from Cruise, who went defunct in 2023.

Tesla's have >1 but they are not really self-driving, but more "100% human supervised self-driving."

Re: Rendering a game in real time with AI

#65

Post by jdiff »

sho_hn wrote: post #46
jdiff wrote: post #44

Earlier quoted context omitted.

"Full self driving" was the term used and I believe the distinction is relevant to the point being made.

I understand the point you're making, but I think it's not a good one. The failure mode for getting a self-driving car right is grave. The failure mode for rendering game graphics imperfectly is to require a bit of suspension of disbelief (it's not a linear spectrum given the famous uncanney valley, etc., I'm aware). Games already have plenty of abstract graphics, invisible walls, and other cludges that require buy-i…

Not my point, but I agree with it so.

The statement was one of capability. There are some things that the tech is flatly not capable of, and that it will take time to develop the capability of. Even if there were no safety concerns at all and we lived in a cotton candy bubble world, self driving cars still have hard failure modes. The tech is not capable, and will not develop the capability next week, either.

The point being made is that the tech is moving fast, at least according to the marketing, but a revolution is not happening ever week. "This is the worst it'll ever be" is an increasingly tired refrain when things seem to be stagnating more than ever. The mentioned behavior will take longer a good amount of time, it's silly to wait around for it when it is not unlikely it may never come.

Re: Rendering a game in real time with AI

#66

Post by fzeroracer »

abbycurtis33 wrote: post #28
coolKid721 wrote: post #23

I do not get the point of this at all, why not just generate game assets and run them in an engine? With this format there would be no regularity that the thing you saw before will look the same (and that is not a fixable problem). Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past…

The tech will improve to far exceed the capabilities of a game engine. Real time improvisation and infinite choices, scope, etc. It makes no sense when people say AI can't do this or that. It will do it next week.

Why would I want any of that? Games are interesting because of the deliberate choices and limitations made by a developer. When you have a game that tries to do everything, you have a game that actually does nothing.

Re: Rendering a game in real time with AI

#67

Post by recursive »

johnisgood wrote: post #63
suddenlybananas wrote: post #62

Earlier quoted context omitted.

Wait til it snows.

The discussion around self-driving cars often feels like shifting goalposts: each time one feature is achieved, a new requirement is added, perpetually delaying the "final" answer. Self-driving cars are "here"... until someone adds another requirement.

For me it's always been cars without steering wheels built on a factory line.

Re: Rendering a game in real time with AI

#68

Post by corysama »

coolKid721 wrote: post #23

I do not get the point of this at all, why not just generate game assets and run them in an engine? With this format there would be no regularity that the thing you saw before will look the same (and that is not a fixable problem). Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past…

Here’s a recent demo made by researchers at Nvidia trying to render the most detailed, realistic scene possible using AI tech to a small degree -mostly as asset compression, but also in material modeling beyond what direct approaches can currently handle in real time: https://m.youtube.com/watch?v=0_eGq38V1hk

Here’s a video from a rando on Reddit conveniently posted today after playing around for an afternoon: https://www.reddit.com/r/aivideo/comments/1n1piz4/enjoy_nano...

The Nvidia team carefully selected a scene to be feasible for their tech. The rando happened to select a similar-ish scene that is largely well suited for game tech.

Which scene looks more realistic? The rando’s by far. And, adding stylization is some AIs are very well established as being excellent at.

Yes, there are still lots of issues to overcome. But, I find the negativity in so many comments in here to be highly imbalanced.

Re: Rendering a game in real time with AI

#69

Post by coolKid721 »

corysama wrote: post #68
coolKid721 wrote: post #23

I do not get the point of this at all, why not just generate game assets and run them in an engine? With this format there would be no regularity that the thing you saw before will look the same (and that is not a fixable problem). Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past…

Here’s a recent demo made by researchers at Nvidia trying to render the most detailed, realistic scene possible using AI tech to a small degree -mostly as asset compression, but also in material modeling beyond what direct approaches can currently handle in real time: https://m.youtube.com/watch?v=0_eGq38V1hk Here’s a video from a rando on Reddit conveniently posted today after playing around for an afternoon: https:…

that rando's one looks like fucking garbage like a bunch of shitty B roll footage from an AI generated advertisement for some kind of pharmaceutical, the space and layout of the world clearly is constantly shifting and makes no sense. just generate a fucking map we already know how to do this, tracking geometry is an EASY problem for computers, don't force some AI shit to try to do it for some reason.

Re: Rendering a game in real time with AI

#70

Post by coolKid721 »

sho_hn wrote: post #37
coolKid721 wrote: post #23

I do not get the point of this at all, why not just generate game assets and run them in an engine? With this format there would be no regularity that the thing you saw before will look the same (and that is not a fixable problem). Actually figuring out and improving AI approaches for generating consistent and decent quality game assets is actually something that will be useful, this I have no idea the point of past…

> I do not get the point of this at all Dunno, this seems like an avenue definitely worth exploring. Plenty of game applications today already have a render path of input -> pass through AI model -> final image. That's what the AI-based scaling and frame interpolation features like DLSS and FSR are. In those cases, you have a very high-fidelity input doing most of the heavy lifting, and the AI pass filling in gaps. E…

Compared to just coming up with and having solid systems for generating game assets? Actually having decent quality style consistent 3d models, texture work, animation, sound effects, etc (especially if it were built into say a game engine or something) would actually revolutionize indie game dev. Games are fundamentally artistic works so yes anything decent will actually require tailoring and crafting, AI set up to serve those people actually makes sense and is totally technically feasible are way easier problems to solve.

And no if you heavily visually modify something with AI models to the extent it significantly alters the appearance it simply has no way of being consistent unless you include the previously generated thing somehow which then has the huge problem of how do you maintain that over an 80 hour game? How do you inform the AI what visual elements (say text, interactive elements) are significant and can be modified and which can't? (You can't)

Actually using AI to generate assets, having a person go in to make sure they look good together and make sure they match then just saving them as textures so they function like normal game assets makes 10000x more sense then generating a user image then trying to extract "hey what did the wrapper of this candy bar look like" from one ai generated image and figuring out how to make sure that is consistent across that type of candy bar in the world and maintains that consistency throughout the entire game, instead of just you know, generating a texture of a candybar?

Post reply on HN