How much did AlphaGo Zero cost? (2018) • Hacker News Board

Re: How much did AlphaGo Zero cost? (2018)

#161

Post by embrassingstuff » Thu, Jun 11, 2020, 10:05 PM UTC

Symmetry wrote: post #120

ipsum2 wrote: post #2

Alpha Go Zero*, which was trained from scratch, without human games. I've also heard rumors that AlphaStar ( https://deepmind.com/blog/article/alphastar-mastering-real-t... ) was essentially put on hold because it was too expensive to improve/train. The bot wasn't able to beat StarCraft champions and _only_ got to a grandmaster level.

Alpha Go Zero used a combination of deep learning within a classical AI Monte Carlo simulation framework. Without a similarly effective framework its not surprising that Alpha Star wasn't able to achieve similar success. I've watched a lot of Alpha Star replays and the lapses in forethought glare through pretty regularly, though other aspects of its judgement (not just reflexes!) seem frankly super human.

Why was MCTS (or some search variant) not used in alphastar ?

(Sure, u need to somehow roll forward and rollback the StarCraft world, but for Atari using MCTS was shown to be an order of magnitude more efficient )

I have also seen comments that the search width is too large, or maybe academic purity consideration?

Re: How much did AlphaGo Zero cost? (2018)

#162

Post by tuxiano » Thu, Jun 11, 2020, 10:21 PM UTC

vecter wrote: post #160

tuxiano wrote: post #156

Earlier quoted context omitted.

In official correspondence games the computer assistance is allowed so most (if not all) of the players usually start their analysis with the computer suggestions (Stockfish, Lc0 or others). Some players limit themselves to this and play the engine's move, others try to improve with their own contribution. If no human contribution was possible, correspondence chess would become an hardware fight while results show th…

Isn't it possible that you're not improving upon the engine's suggestions, but instead, your opponent is choosing suboptimal non-engine lines, and your engine is beating their weakened engine?

Occasionally it is possible. After seven years and more than one hundred games played I can tell you that I have been surprised by my opponent's reply not more than an handful of times. For "surprised" I mean he didn't play the top choice of the engine. In fact most of the times the best move in a given position is easily agreed on by any reasonable engine on any decent hardware. In few critical moments in the game, the best move is not clear and there are two or three or more playable alternatives that take into very different positions. In these cases the computer, after a long thought (one or more hours) usually converges to one suggestion and sticks to it even if given more time (a sort of "horizon effect"). These are the moments where a human, after a long thought, can overcome the computer suggestion and favor the 2nd or 3rd choice of the engine. So in brief no, I can't recall a game where I've been gifted the win by my opponent "weakened" move while most of the time I have confronted with the "engine's approved" suggestion and had to build my win by refuting it.

Re: How much did AlphaGo Zero cost? (2018)

#163

Post by tuxiano » Thu, Jun 11, 2020, 10:23 PM UTC

JamesBarney wrote: post #158

tuxiano wrote: post #151

Earlier quoted context omitted.

The rating you are referring to are typically based on tournament or rapid games, where the limited time induces the human players to mistakes that the computer capitalizes on. Given enough time or with a “blunder check” option, the best human players are still strategically stronger. In correspondence chess, where the is much more time at disposal, the human players can still improve the computer suggestions. Source…

Yeah I was thinking about classic or standard time controls. The last big cyborg tournament a few years ago I remember a computer coming in 1st and 2nd. I wasn't thinking about correspondence but what was the latest large cyborg correspondence tournament?

I don't know the last one but I recall the matches of Hydra chess machine [0] in the early 2000s against GM Adams in tournament condition (5½ to ½ for the machine) and against GM Nickel in correspondence condition (2 to 0 for the human). Both Grandmaster were top players in their relative field so it showed very clearly how the time limitation impacted the competitive results. Nobody in the chess elite would claim that Hydra understood chess better than GM Adams but still he lost resoundigly due to the inevitable mistakes caused by the relatively fast time control.

[0] https://en.wikipedia.org/wiki/Hydra_(chess)

Re: How much did AlphaGo Zero cost? (2018)

#164

Post by tuxiano » Thu, Jun 11, 2020, 10:26 PM UTC

roenxi wrote: post #154

scheveningen wrote: post #140

Earlier quoted context omitted.

It's much more difficult than it used to be, but I think there is still some value to human guidance, more as a "referee" than anything else. Right now we have essentially two top tier engines -- traditional brute force with alpha beta pruning (stockfish), and ML (leela). Both alone are incredibly strong, but they are strongest and weakest in different types of positions. A computer chess expert, who knows what kind…

Computer mastery of Go has reached the point where it is a difficult task for an expert (read: grandmaster level) human to even follow what is happening. It is totally implausible that a human could resolve a conflict between top engines in a meaningful way. It is unlikely that Chess is any different. Any superficial understanding by a human of which move is 'better' is just ignorance of the issues around evaluating…

I can't speak for Go but in Chess the best players in the world understand the nuances of a position still better than the computer engines and - if occasionally proven wrong by the computer analysis - are able to understand the refutation and refine their strategic eveluation. I know this because it's what I've been doing in the past seven years in the realm of correspondence chess to gain the title of international master.

Re: How much did AlphaGo Zero cost? (2018)

#165

Post by vecter » Thu, Jun 11, 2020, 10:32 PM UTC

tuxiano wrote: post #162

vecter wrote: post #160

Earlier quoted context omitted.

Isn't it possible that you're not improving upon the engine's suggestions, but instead, your opponent is choosing suboptimal non-engine lines, and your engine is beating their weakened engine?

Occasionally it is possible. After seven years and more than one hundred games played I can tell you that I have been surprised by my opponent's reply not more than an handful of times. For "surprised" I mean he didn't play the top choice of the engine. In fact most of the times the best move in a given position is easily agreed on by any reasonable engine on any decent hardware. In few critical moments in the game,…

I assume that when you come across one of these novel moves, plug it into the computer, and give it time to search, it ultimately decides that it's superior?

Relatedly, can you give some examples of novel non-engine lines that turned out to be better than engine lines?

Re: How much did AlphaGo Zero cost? (2018)

#166

Post by tuxiano » Thu, Jun 11, 2020, 10:45 PM UTC

roenxi wrote: post #159

tuxiano wrote: post #149

Earlier quoted context omitted.

I suppose he can’t because it isn’t true at all. The best correspondence players usually improve significantly over the computer suggestions. Source: I’m a corrspondence chess international master

I'm interested because the experience in Go is humans simply can't keep up. What is the evidence that it isn't a hardware or software differential between the players? I can't think of an easy way to ensure that both players started with computer-suggested moves of the same quality.

There are a lot of engines with rating on the chart way higher than the best humans, so every suggestion on their part should be in theory enough to overcome any human opponent. In practice most (if not all) of the players rely on Stockfish and Lc0 (both open source). During a game, most of the time the "best" move is easily agreed on by every reasonable engine on any decent hardware. Only in few cases during a game, the position offers two or three or more playable choices. In these cases a stronger hardware or a longer thought rarely makes the computer change his idea. It's a sort of horizon effect where more power doesn't translate into a really better analysis.

For example in a given position you could have 3 moves M1 - a calm continuation with a good advantage M2 - an exchange sacrifice (a rook for a bishop or a knight) for an attack M3 - a massive exchange of pieces entering into a favorable endgame. If the three choices are so different, the computer usually can't dwell enough to settle on a clear best move. Instead the human can evaluate the choices until one of them shows up as clearly best (for example the endgame can be forcefully won). In these cases the computer suggestion becomes almost irrelevant and only a naive player would make the choice on some minimal score difference (that can unpredictably vary on hardware, software version or duration of analysis). So the quality of the starting suggestion is somehow irrelevant if you plan to make a thoughtful choice.

Re: How much did AlphaGo Zero cost? (2018)

#167

Post by JamesBarney » Thu, Jun 11, 2020, 10:53 PM UTC

tuxiano wrote: post #163

JamesBarney wrote: post #158

Earlier quoted context omitted.

Yeah I was thinking about classic or standard time controls. The last big cyborg tournament a few years ago I remember a computer coming in 1st and 2nd. I wasn't thinking about correspondence but what was the latest large cyborg correspondence tournament?

I don't know the last one but I recall the matches of Hydra chess machine [0] in the early 2000s against GM Adams in tournament condition (5½ to ½ for the machine) and against GM Nickel in correspondence condition (2 to 0 for the human). Both Grandmaster were top players in their relative field so it showed very clearly how the time limitation impacted the competitive results. Nobody in the chess elite would claim th…

But wasn't Hydra 2005 ~2800 ELO where as the current best chess engines like Leela Chess Zero or Stockfish are ~4000 ELO?

Just realized that correspondence chess is cyborg chess, I didn't know computers were legal in correspondence chess, but it makes sense now. Reading about it, it sounds like it's less about knowing chess, and more about understanding the applications you're using.

Re: How much did AlphaGo Zero cost? (2018)

#168

Post by nl » Thu, Jun 11, 2020, 11:04 PM UTC

tuxiano wrote: post #157

nl wrote: post #107

Earlier quoted context omitted.

If by "it's far from a solved problem" you mean that chess isn't a solved game that's true. But Kasparov and others have given up on the idea that a human provides any unique insight into chess anymore. Computers are just better.

You are right if by "better" you mean "competitively stronger at tournament or rapid conditions". Humans are still way stronger strategically and competitively if given enough time and resources to avoid tactical mistakes. So yes, humans still provide unique insight into chess every day in correspondence chess or analytic research.

Humans aren't stronger strategically anymore either, under any conditions.

In 2014 a heavily handicapped Stockfish beat the 5th ranked player in the world (Nakamura) under tournament conditions despite no access to its opening or closing books and a one pawn handicap.

Re: How much did AlphaGo Zero cost? (2018)

#169

Post by tuxiano » Thu, Jun 11, 2020, 11:18 PM UTC

vecter wrote: post #165

tuxiano wrote: post #162

Earlier quoted context omitted.

Occasionally it is possible. After seven years and more than one hundred games played I can tell you that I have been surprised by my opponent's reply not more than an handful of times. For "surprised" I mean he didn't play the top choice of the engine. In fact most of the times the best move in a given position is easily agreed on by any reasonable engine on any decent hardware. In few critical moments in the game,…

I assume that when you come across one of these novel moves, plug it into the computer, and give it time to search, it ultimately decides that it's superior? Relatedly, can you give some examples of novel non-engine lines that turned out to be better than engine lines?

Sometimes if you play a move and the first plies (i.e. half moves) of the main variation the computer starts "understanding" and its score changes accordingly. Those are the cases where more hardware power could be useful and make the engine realize the change from the starting position. More often, the "non-engine" move relies on some blindness of the engine, so the computer starts understanding its strength only when it's too late. In these cases is unlikely that more power could bring benefits. Typical cases are

- fortresses [0]. One side has more material but the position can't be won by the superior side. As the chess rules declare the draw only after 50 moves without captures or pawn pushes, current engines can't look this far away and continue manouvering without realizing the blocked nature of the position. Some engines have been programmed to solve this problem but their overall strength decreases significantly.

- Threefold repetitions [1]. The engine believes the position is equal and move the pieces in - let me say - pseudorandom way. Only at some point it realizes the repetition can be avoided favourably by one side. Also this topic is frequently discussed in the programming forums but no clearcut solution has still emerged.

If you are looking for positions where human play is still better than engine's, the opening phase is the most fruitful. Most theoretical lines were born by human creativity and I doubt a chess engine will ever be able to navigate the intricacies of the Poisoned Pawn Variation of the Sicilian Najdorf [2] or the Marshall Attack of the Ruy Lopez [3]. Neural networks engines are strategically stronger than classical AB programs in the opening phase but they suffers from occasional tactical blindness. Engine-engine competitions often use opening books to force the engines to play a prearranged variation to increase the variabililty and reduce the draw percentage.

[0] https://en.wikipedia.org/wiki/Fortress_(chess) [1] https://en.wikipedia.org/wiki/Threefold_repetition [2] https://en.wikipedia.org/wiki/Poisoned_Pawn_Variation [3] https://en.wikipedia.org/wiki/Ruy_Lopez#Marshall_Attack

Re: How much did AlphaGo Zero cost? (2018)

#170

Post by nhooyr » Thu, Jun 11, 2020, 11:27 PM UTC

geodel wrote: post #127

sukilot wrote: post #114

Earlier quoted context omitted.

Note that everything in software development is R&D. Building (not operating) Gmail and Android and Office and Azure are R&D.

Right. That's the Development part of Research and Development.

I thought R&D was mostly research so I found his comment clarifying.