Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grandmaster level in StarCraft II using multi-agent reinforcement learning

deepmind.com

Post reply on HN
Print view View original

301–310 of 321 posts

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#301

Post by simonh »

YeGoblynQueenne wrote: post #258
emerongi wrote: post #201

Earlier quoted context omitted.

Most of the approaches used are re-usable, which is a big part of why we can develop new AIs for games faster than ever. You can take the algorithm(s) that was used in one game and use it in another. Yes, a human is still needed to decide which approach to use, but we are slowly approaching a world where building an AI becomes easier and faster. It will become absolutely irrelevant that a single AI can not play all g…

>> The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend! So far I haven't seen Google sending their AIs to a friend.

With transfer learning, you can download some of the worlds most advanced pre-trained NLP models and adapt them to a specific data set.

https://www.analyticsvidhya.com/blog/2019/03/pretrained-mode...

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#302

Post by Shaddox »

justicezyx wrote: post #88
floitsch wrote: post #16

Earlier quoted context omitted.

These extremely difficult/impossible things didn't really give an advantage. For example, AlphaStar would sometimes click on an object at the border of the screen. For humans that would be almost impossible, because the screen would scroll when the mouse approaches the border. Similarly, AlphaStar would not play with group hotkeys, but use a different technique. However, in none of the analyses, people noticed things…

What you said is a meaningful advtange. Scrolling screen is a major action to gather info and control units. If one can control near edge units without scrolling it gives more stable view and lower chance of making mistakes.

There's an in-game option to disable scrolling the screen with the mouse and doing it with only the keyboard.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#303

Post by Strom »

halflings wrote: post #289
Strom wrote: post #212

Earlier quoted context omitted.

The mistakes it makes are due to bad decisions. There have been no claims by DeepMind that they have some sort of chaos engineering [1] going on where the AI decides one thing and then the output system actually does another thing. Also I think you overestimate the AI/IT knowledge of these top players that they're consulting. I have great respect towards them, but they're not renaissance men [2] who both play 10 hour…

>[2] The problem with both parties (the pros & deepmind) is that they're so overspecialized. I'm nowhere near as good as them at their respective fields, but I am a professional programmer and diamond in StarCraft II. In addition I've built StarCraft II AI myself, although with different goals related to finding optimal strategies. So many things wrong with this comment. You're nowhere near a professional StarCraft p…

You quoted me saying that I'm not as good as them and then you say that my statement is wrong because I'm not as good as them. We seem to be in agreement. Maybe you misread? Anyway that was just a footnote to point towards my generalist nature, because I think that's a fundamental reason why I immediately see these things while for them it takes time to realize. Also to be even more fair towards them, my generalist knowledge around computer controlled gaming goes way beyond StarCraft II as I've written bots for many different games.

My main argument revolves around APM. Oriol Vinyals might be great, but I've also seen him make a statement on video in 2019 [1] how restricting the AI based on average APM during the whole game is reasonable. He has blindspots that someone like me can immediately spot.

--

[1] https://www.twitch.tv/videos/369062832

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#304

Post by PeterisP »

the_af wrote: post #128
PeterisP wrote: post #107

Earlier quoted context omitted.

In strategy games like Civilization the "preferred" difficulty level is that hard/easy to beat because it gets extra resources. It would be preferrable to have the same difficulty level through opponents that play better/smarter while having the same game mechanic consequences as players if they make the same actions, but we currently can't, so they get artificial production multipliers and such.

Yes, I understand this. I guess I disagree better/smarter would be better, because in videogames what matters is the illusion of challenge, not a real challenge. So spending resources into developing a real AI for Civilization is probably not the best idea; as long as it tricks casual players into believing it's putting up a fight, that's good enough.

This approach creates a mismatch between singleplayer and multiplayer modes - this means that playing against a computer opponent rewards/requires different strategies than a human; a challenging computer opponent has more income and units but poor usage of them, while a similarly challenging human opponent has less income and units but uses them very differently; so playing against challenging computer opponents doesn't help you improve against other players but possibly is even counterproductive as you learn to adopt strategies that are bad in the other environment.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#305

Post by 0x008 »

YeGoblynQueenne wrote: post #258
emerongi wrote: post #201

Earlier quoted context omitted.

Most of the approaches used are re-usable, which is a big part of why we can develop new AIs for games faster than ever. You can take the algorithm(s) that was used in one game and use it in another. Yes, a human is still needed to decide which approach to use, but we are slowly approaching a world where building an AI becomes easier and faster. It will become absolutely irrelevant that a single AI can not play all g…

>> The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend! So far I haven't seen Google sending their AIs to a friend.

What do you mean? Basically all Neural-network models (from Google and others) are freely available. And to top that you can also get pre-trained weights for the models for free.

https://www.tensorflow.org/hub/

https://github.com/tensorflow/models

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#306

Post by YeGoblynQueenne »

0x008 wrote: post #305
YeGoblynQueenne wrote: post #258

Earlier quoted context omitted.

>> The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend! So far I haven't seen Google sending their AIs to a friend.

What do you mean? Basically all Neural-network models (from Google and others) are freely available. And to top that you can also get pre-trained weights for the models for free. https://www.tensorflow.org/hub/ https://github.com/tensorflow/models

I mean the Alpha* family- AlphaGo, AlphaStar etc. I don't think they're sharing their analytics models either, or their DMT models, etc.

"All" neural network models is a stretch. Some researchers, including some that work in the industry, do releast their models. The majority don't.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#307

Post by the_af »

PeterisP wrote: post #304
the_af wrote: post #128

Earlier quoted context omitted.

Yes, I understand this. I guess I disagree better/smarter would be better, because in videogames what matters is the illusion of challenge, not a real challenge. So spending resources into developing a real AI for Civilization is probably not the best idea; as long as it tricks casual players into believing it's putting up a fight, that's good enough.

This approach creates a mismatch between singleplayer and multiplayer modes - this means that playing against a computer opponent rewards/requires different strategies than a human; a challenging computer opponent has more income and units but poor usage of them, while a similarly challenging human opponent has less income and units but uses them very differently; so playing against challenging computer opponents doe…

True. I don't know many videogames in which playing against the computer really helps you against human opponents.

I don't know whether a more capable non-cheating AI would help though. Not unless it specifically imitated how a (good) human opponent would play, which I guess is an additional and difficult to implement constraint.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#308

Post by TaupeRanger »

drchewbacca wrote: post #257
TaupeRanger wrote: post #251

Earlier quoted context omitted.

Not at all. We've known machines are better than humans at machine-like tasks for hundreds of years. IMO responses like yours are defense mechanisms of AI researchers/enthusiasts who want to protect the illusion that they are creating things that do anything remotely resembling what the human brain does.

Kasparov comments on chess computers in an interview with Thierry Paunin on pages 4-5 of issue 55 of Jeux & Stratégie (published in 1989): ‘Question: ... Two top grandmasters have gone down to chess computers: Portisch against “Leonardo” and Larsen against “Deep Thought”. It is well known that you have strong views on this subject. Will a computer be world champion, one day ...? Kasparov: Ridiculous! A machine will a…

The fact that Kasparov was obviously wrong about a computer's ability to solve a concrete optimization problem better than him says nothing of value whatsoever and essentially proves my original point. We already knew machines were better than humans at these kinds of tasks, but people (like Kasparov) who didn't understand what computers were capable of will make wrong statements.

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#309

Post by falcor84 »

yters wrote: post #105

What would be interesting is to limit the AI processing speed to human capacity, which is something like 60 bits per second. In all these AI v. Human games I see, it is really apples to oranges because the human consumes vastly less resources and compute cycles to perform at the same level as the AI. And when I say 'vast' I mean Vast. There is like a quintillion factor difference between the AI and the human. There i…

> What would be interesting is to limit the AI processing speed to human capacity, which is something like 60 bits per second.

What part of human processing capacity is this slow?

Re: Grandmaster level in StarCraft II using multi-agent reinforcement learning

#310

Post by posterboy »

Strom wrote: post #205

> Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM. I'm happy to see that they've greatly improved the APM cap. During the earlier showmatch they had an extremely…

an SC robot with alphastar brain--learning to use a mouse for a start--would probably only shift the blame to those super human materials. They should put alpha* into a human! That's where it's going anyway, isn't it?
Post reply on HN