Live data from Hacker News

OpenAI Universe

universe.openai.com

71–80 of 139 posts

Re: OpenAI Universe

#71
Disclaimers: I cannot see the future. These are just my opinions. I really appreciate the work and money that SamA, Elon, and others have put into the OpenAI project. The Universe work in particular might help encourage young people, many of whom love video games, to study AI.

But I feel that contrarians, such as myself, have an ethical commitment to young people to voice our doubts and criticisms, so that they can avoid making a long journey down a career/research path that leads to a dead end. That being said, I think this project leads in a very unpromising direction. Here are some reasons:

1. Games aren't a good testbed for studying intelligence. In a game the main challenge is to map an input percept to an output action (am I drifting off the side of the road? Okay swerve right). The real challenge of intelligence is to find hidden abstractions and patterns in large quantities of mostly undifferentiated data (language, vision, and science all share this goal).

2. This platform is not going to help "democratize" AI. To succeed in one of these domains, contestants will need to use VAST amounts of computing power to simulate many games and to train their DL and/or RL algos. DeepMind and others will sufficient CPU/GPU power will almost certainly dominate in all of these settings.

3. Deep Learning, as it is practiced, isn't intellectually deep. With a few exceptions, there is nothing comparable to the great discoveries of physics, not even anything comparable to the big ideas of previous AI work (A*, belief propagation, VC theory, MaxEnt, boosting, etc). Progress in DL mostly comes from architecture hacking: tweak the network setup, run the training algo, and see if we get a better result. The apparent success of DL doesn't depend on any special scientific insight, but on the fact that DL algos can run on the GPU. That, combined with the fact that, except for the GPU, Moore's Law broke down roughly 10 years ago, means that relative to everything else, DL looks amazingly successful - because all other approaches to AI are frozen in time in terms of computing power.

Re: OpenAI Universe

#72
post #49

From my initial reading, the end user can't create environments? Is that a feature that I can expect will eventually come?

It looks like the image from the server and control information to the server is sent through the VNC protocol. Other information such as the reward signal from the environment server is sent through a WebSockets protocol using JSON:

https://github.com/openai/universe/blob/master/doc/protocols...

You should be able to implement this protocol for your environment and run a VNC server for the rest. A new class for the client representing your environment can be based on this:

https://github.com/openai/universe/blob/master/universe/envs...

Then register the class with OpenAI Gym:

https://github.com/openai/universe/blob/master/universe/__in...

After creating the environment using gym.make you need to add information about your remote in the call to configure:

env = gym.make('gtav.SaneDriving-v0')

env.configure(remotes="vnc://localhost:vnc_port+rewarder_port")

https://github.com/openai/gym/blob/master/gym/core.py#L234

https://github.com/openai/universe/blob/master/universe/envs...

This is only based on a cursory reading, but it should be possible to use custom environments with OpenAI Universe as it is today.

Re: OpenAI Universe

#73

Unless I missed something it looks like the AI has to learn from screen pixels instead of getting game state data. I don't like that approach at all. I understand that it's easy to implement for OpenAI but I think having the game developers provide a real bot-capable API is much better. I hope the latter is what Blizzard will provide for their DeepMind collaboration.

If your goal is to build General AI, getting access to the game state information is cheating and ultimately self defeating because it does not generalize across games. Expert humans are able to pick up and play a brand new game with a high degree of initial success despite zero knowledge of the memory states within the game. We do this by reliance on our vast knowledge of video game and literary tropes as well as experience playing past games. I have yet to see any video game bot make use of this stuff to figure out how to play Zelda, for example.

Re: OpenAI Universe

#74
post #69
post #67

Earlier quoted context omitted.

Seems unlikely. The focus seems to be on improving AI through "vision". The idea is to make the AI learn skills the same way a human would (at least in the first years of life). Google's AlphaGo also learned from screen pixels. So these would be human-like bots, rather than bot-like bots, like you normally have in games. The bot would simply learn by doing, until it masters the game, not by getting access to game alg…

> Google's AlphaGo also learned from screen pixels. Source? That literally seems to make zero sense to me. Go can be represented in a super-simple state. Why make it spend millions of cycles learning to categorize pixels into that state you already have?

I would guess that they trained AlphaGo from many thousands of hours of match footage. Writing a computer vision script to segment / extract the data may cost cycles as you say, but would save many human hours by eliminating the need to re-watch the footage and literally type out state information for each move.

Re: OpenAI Universe

#75
jesus man the constant switching of inflection due to jump cuts and shitty jokes was irritating. This is the stuff I always hate about your videos, and in general any "how to code an AI that does ___ with only < 10 lines of python". No you're giving me an extremely generic framework, and some vague direction that i can read off the linked page (the 9 lines), not how to actually write all these things, those are like afterthoughts in the video, more work goes into the click-baity titles and thumbnails then actual substantive information.

Re: OpenAI Universe

#76

Disclaimers: I cannot see the future. These are just my opinions. I really appreciate the work and money that SamA, Elon, and others have put into the OpenAI project. The Universe work in particular might help encourage young people, many of whom love video games, to study AI. But I feel that contrarians, such as myself, have an ethical commitment to young people to voice our doubts and criticisms, so that they can a…

1. Games are great, especially the closer they get to the real 3d worlds with all the basic visual transforms in play. You can generate training data cheaper that helps you bootstrap the AI.

2. Your argument, which boils down to large organizations can accomplish more than individuals is in general true. But then that isn't saying anything new. Still, I'd prefer the car company give its blueprints than not. My factory (my cpu) can then at least build the car, albeit at a smaller scale. And soon my factory will be bigger/cheaper. Yeah I know, I'd like to be wealthy like Google too.

3. This is your own value judgement. The insight that we simplify AI architecture to some addition and multiplication is a big idea - in my opinion. Transistors are getting cheaper and I believe they will continue to do so. DNN are better suited to take advantage of this new computing power. Turns out, all those great discoveries were just some multiplications and adds the whole time ;)

Re: OpenAI Universe

#77
post #73

Unless I missed something it looks like the AI has to learn from screen pixels instead of getting game state data. I don't like that approach at all. I understand that it's easy to implement for OpenAI but I think having the game developers provide a real bot-capable API is much better. I hope the latter is what Blizzard will provide for their DeepMind collaboration.

If your goal is to build General AI , getting access to the game state information is cheating and ultimately self defeating because it does not generalize across games. Expert humans are able to pick up and play a brand new game with a high degree of initial success despite zero knowledge of the memory states within the game. We do this by reliance on our vast knowledge of video game and literary tropes as well as e…

An AI which can learn any game from "looking" at a screen is a very ambitious goal. I doubt it is achievable in the near term. Humans can do it because of years of learning about the world. Personally, I prefer the more modest and achievable goal of teaching an AI to play complex real-time games such as Starcraft, LoL, and Dota--especially the latter 2 since they are team games.

Re: OpenAI Universe

#79

Earlier quoted context omitted.

Sure, just define a good score function...

That should be easy, I'll need a 5 million dollar grant and 5 years.

How much do you need to train a model to write grant applications for 5 millions dollars or more over 5 years.

Re: OpenAI Universe

#80

Unless I missed something it looks like the AI has to learn from screen pixels instead of getting game state data. I don't like that approach at all. I understand that it's easy to implement for OpenAI but I think having the game developers provide a real bot-capable API is much better. I hope the latter is what Blizzard will provide for their DeepMind collaboration.

Getting the game state data means deciding a-priori what features the AI should learn on. The whole point of the deep learning paradigm is to allow a machine to learn such features that enable good prediction, visualization, generation (aka. hallucination), etc. Instead, researchers have provided the raw feed input data to these agents with the hope that the learned features could be interpreted as game state data by…

I would say that it is a point of deep learning rather than "the whole point". For an AI to interact with the real-world then building models from vision (as we do) makes a lot of sense. In the virtual world, however, it makes no sense. Model data is already available and the AI has no need for something as inefficient as vision. We humans have to use vision (and sound, etc) in games because we have do not have access to direct data feeds, computers have no such limitation. Why cripple the AI by imposing human limitations on it?
Post reply on HN