Is AlphaZero really a breakthrough in AI? • Hacker News Board

Re: Is AlphaZero really a breakthrough in AI?

#81

Post by jraut » Fri, Dec 15, 2017, 7:05 AM UTC

dingo_bat wrote: post #64

jraut wrote: post #62

An AI which excels in imperfect information games (card games, Starcraft) would be a real breakthrough. Raw calculation power is bound to win games with a finite set of possibilities. The huge leap would be the ability to handle probabilites: taking guesses, making assumptions and coming into some kind of successful conclusions based on those.

Go cannot be won with raw power though. It has a lot of the constraints of incomplete information problems.

So every move cannot be precalculated? Some of the constraints may be there. The completeness of information available would be the definition of breakthrough for me. In this case, the first state of the game is presented in total with 100% accuracy. So are all the steps from there onward. In my opinion, the challenge for a breakthrough comes when most of the actions of other players happen in dark and no feedback is presented to the AI.

In a lifelike situation the AI will not have access to the inner state of the game, but instead has to gather the information via the same (restricted) mechanisms as other players.

edit: I should probably clarify that the above is about competitive StarCraft. I should probably learn to play GO, too.

Re: Is AlphaZero really a breakthrough in AI?

#82

Post by mannykannot » Fri, Dec 15, 2017, 2:20 PM UTC

YeGoblynQueenne wrote: post #78

mannykannot wrote: post #52

The author is actually claiming something more serious than the title suggests: "...all the concerns added together cast reasonable doubts about the current scientific validity of the main claims." To me, what follows does not seem to justify this claim, but it is not my field. In addition, some of his arguments seem to be beside the point - for example, he asks "Does AlphaZero completely learn from self-play?", and…

As many others have pointed out, self-play in adversarial AI goes back to TD Gammon, in 1992. The fact alone that DeepMind is making such a big todo about self-play is a bit iffy in and of itself. It's probably a sign that they're more interested in catching the attention of the popular press and the general public, than of anyone who has at least read through Russel and Norvig [i.e. a popular AI textbook that mentio…

To a typical insurance salesman, the fact that self-play has been around for a while raises the question of why not until now, for chess? Maybe self-play in chess had already come within a whisker of this result? Has the state of the art reached the point where a bespoke self-play solution for any given chess/go -like game is now unremarkable? Is chess regarded as a sideshow that is given more attention than it deserves from the press and public? Even if this outcome is completely unremarkable from a technical point of view, the question remains, but as a social one.

The author can certainly scrutinize whatever claims he likes, but does he make his case? I suppose he can certainly say that without further information, the outcome simply cannot be independently evaluated.

Re: Is AlphaZero really a breakthrough in AI?

#83

Post by eutectic » Fri, Dec 15, 2017, 2:27 PM UTC

eutectic wrote: post #56

seanwilson wrote: post #39

Can anyone summarise how self play works here if AlphaZero only starts out being told the rules of the game? Does it initially plays games using completely random moves as both players? Is it only told who the winner is with no other feedback? How is it able to learn e.g. that certain moves at the start eventually lead to a win?

There is a value network which estimates the win rate for the current player from a given board state, and a policy network which estimates the probability that each move should be played. As of the more recent iterations, these networks share their bottom layers for greater computational and training efficiency. The value network is simply updated to match the real outcomes of games of self-play. The policy network…

Oops, I obviously meant 'P is the policy'

Re: Is AlphaZero really a breakthrough in AI?

#84

Post by disgruntledphd2 » Sat, Dec 16, 2017, 12:58 PM UTC

simonh wrote: post #17

When can we have an AI that plays Third Reich? But not too well, I want to at least have a chance. I'm actually not joking. I wonder how much different it would be to teach an AI like this how to play more complex games. I imagine Axis and Allies wouldn't take much, but Third Reich is notoriously complicated. The quickest war-length game I've played took a week of playing 3-4 hours per day and games like that seem to…

The hard part is probably writing a program to encode the rules of the game. Then, you can figure out how to represent the state (i.e. the board and the player's cards/resources etc) as a matrix or array (i like how this is done in NLP, they tend to represent each character/word/token as a 1-hot vector and then reduce the dimensionality of these (normally really sparse) inputs.

OpenAI's gym is probably a good place to start, as you can crib how they do it for a whole bunch of games. https://github.com/openai/gym