Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Stadia uses Style Transfer ML to change video game art in real time

turingtribe.com

Post reply on HN
Print view View original

61–70 of 96 posts

Re: Google Stadia uses Style Transfer ML to change video game art in real time

#61

Post by jmathai »

smaili wrote: post #30

Is anyone else concerned this will be yet another Google product that goes by the wayside in a few years? This has essentially become my biggest fear now with any new product they announce/release.

I understand that it's inconvenient when services shut down but I see 3 options.

1) Wait until it seems there's enough comittment from Google before using it. 2) Start using it right away knowing that it might not be around forever. 3) Don't use it because you don't trust Google to maintain it.

Given these 3 options, does Google launching a service really have that much impact on consumer's lives? I'm a huge Gmail user and have been for over a decade. If they shut it down it will suck but I'll find another way to email people.

I get the philosophical arguments but curious why some people have the level of concern or angst when Google announces a product.

(for the sake of this comment let's forget about competition and monopolies)

Re: Google Stadia uses Style Transfer ML to change video game art in real time

#62

Post by muststopmyths »

heavenlyblue wrote: post #23
caprese wrote: post #18

Earlier quoted context omitted.

Onlive - required other people's infrastructure and a hardware play, and a software client. Playstation Now - required Sony to buy/use a cloud service, maintain it, and expensive Sony hardware just as a force of habit. Google - negligible instance running on their existing infrastructure, maintained under existing processes, playable on chromecasts already in hotels and homes everywhere, alongside browsers and mobile…

Are you implying Google doesn’t need to maintain an expensive cloud infrastructure for it as opposed to Playstation? The only reason they can do that today without specialised hardware a-la OnLive is because Nvidia and co. now have specialised GPU circuits allowing one to stream the encoded video output to memory. When OnLive started they had to do it themselves. I don’t believe you have any idea what you are speakin…

> Nvidia and co. now have specialised GPU circuits allowing one to stream the encoded video output to memory.

Could you clarify what you (and the comments mentioning NVENC below) are talking about ?

As far as I understand it, NVENC and the ilk are solutions that capture video output and encode it to H264. So they are x264 encoding accelerators, nothing more.

If you were running a datacenter this way, you'd be much better of having a bank of Matrox capture cards (which support multiple simultaneous inputs) in dedicated hardware converting your video game output to x264 streams for broadcast. They even support capturing, x264 encoding and streaming to IP addresses on your network for further distribution.

What would be interesting is if they bypass the video display output phase completely and render the finished framebuffers to memory (or an encoder chip via DMA over PCI) instead. I assume this would cut down manufacturing (+licensing for HDMI ?) costs a bit.

I know AMD has DirectGMA that allows other devices on the PCI bus access to limited chunks of GPU memory. There are signalling mechanisms in DirectGMA so that devices can basically implement producer-consumer pairs.

As far as I know this doesn't exist in consumer chips though. You need "workstation" GPUs. Which might explain Google's particular GPU choice, now that I think about it.

Re: Google Stadia uses Style Transfer ML to change video game art in real time

#63

Post by _bxg1 »

A neat tech demo, but as with so many things coming out of The Valley these days, I don't think anybody actually wants this. No actual game is so bland that you'd want the ability to overlay your own "themes" on it after the fact. AI has lots of potential for assisting artists, and maybe part of that will include post-processing effects, but putting such things in the player's hands is pointless.

Re: Google Stadia uses Style Transfer ML to change video game art in real time

#64

Post by m3kw9 »

lwb wrote: post #3

To me this was the least interesting part of the Stadia reveal. State share was the real innovation imo. Would any serious game studio use style transfer ml to actually replace their visual artists? Didn't seem like the quality was there. That said, it is an awesome PoC. Just not something I see being practically applicable.

Maybe used as a player entering dream like equivalent after eating magic mushrooms. Other than that you’d be hard pressed to see this in any game in memory

Re: Google Stadia uses Style Transfer ML to change video game art in real time

#65

Post by currymj »

Sephr wrote: post #60
currymj wrote: post #50

this is certainly just a neat gimmick, but it's interesting to me for a particular reason: the Stadia box uses an AMD GPU, and per the Google announcement, all the style transfer is done in real time. this suggests that we may soon have better AMD support in TensorFlow.

Style transfer is most likely not running on the GPU in this case. It's probably running on Google TPUs, given that the GPUs are likely already under high load in order to play AAA games at 4K HDR 60fps.

where can I find out more about this? all I'm going by is them saying in their keynote that its running "on the Stadia instance".

Re: Google Stadia uses Style Transfer ML to change video game art in real time

#69

Post by nabergh »

alkonaut wrote: post #67

The useful application for AI in streamed gaming seems like it would be for predicting the next 30ms to avoid input lag. This seems gimmicky at best. Prediction has to take place on the client though - so the challenge is making it cheap.

This is really cool to think about. In practice though I would imagine every incorrect prediction to be incredibly jarring. Additionally my intuition is that latency is easier to decrease than predicting the next 30ms. If there is some level of latency that is very difficult to reduce (more so than training an AI to predict the next 30ms with very high accuracy), I'd be curious to know why.

Re: Google Stadia uses Style Transfer ML to change video game art in real time

#70

Post by alkonaut »

nabergh wrote: post #69
alkonaut wrote: post #67

The useful application for AI in streamed gaming seems like it would be for predicting the next 30ms to avoid input lag. This seems gimmicky at best. Prediction has to take place on the client though - so the challenge is making it cheap.

This is really cool to think about. In practice though I would imagine every incorrect prediction to be incredibly jarring. Additionally my intuition is that latency is easier to decrease than predicting the next 30ms. If there is some level of latency that is very difficult to reduce (more so than training an AI to predict the next 30ms with very high accuracy), I'd be curious to know why.

Between e.g. central and west US you'd have a physical ping of ~10ms so 20 round trip just based on the speed of light (i.e. very hard to get rid of). Meanwhile the input lag you want is one or two frames so on the same order. But this lag is additive, when the game receives the new state it has to compute and render the world, the display has its lag before it shows up etc. So the new frame has to start computing as soon as the input is registered in order to hit reasonable input lag.

Predicting 2-3 frames should't beyond what we can do at least roughly. Especially for some important inputs such as turning in first person view it should basically just start moving the screen. I belive John Carmack tried some kind of rotational prediction (Not based on AI) for Oculus.

Post reply on HN