Sorry but this is extremely shoddy work. M:tG will never be "solved" like this. >> Using data from drafts carried out by humans, I trained a neural network to predict what card the humans would take out of each pack. It reached 60% accuracy at this task. Going by what's in the linked notebook, the model was evaluated on its ability to match the decks in its training set card-for-card. Without any attempt to represent…
Just to add a bit to this: >> Using data from drafts carried out by humans, I trained a neural network to predict what card the humans would take out of each pack. It reached 60% accuracy at this task. And in the 40% when the human and the model differ, the model’s predicted choice is often better than what the human picked. How the model's pick is "better than what the humn picked" is never made clear, but since acc…
Teaching an AI to Draft Magic: The Gathering
21–30 of 53 posts
Re: Teaching an AI to Draft Magic: The Gathering
#22Sorry but this is extremely shoddy work. M:tG will never be "solved" like this. >> Using data from drafts carried out by humans, I trained a neural network to predict what card the humans would take out of each pack. It reached 60% accuracy at this task. Going by what's in the linked notebook, the model was evaluated on its ability to match the decks in its training set card-for-card. Without any attempt to represent…
Re: Teaching an AI to Draft Magic: The Gathering
#23Earlier quoted context omitted.
Just to add a bit to this: >> Using data from drafts carried out by humans, I trained a neural network to predict what card the humans would take out of each pack. It reached 60% accuracy at this task. And in the 40% when the human and the model differ, the model’s predicted choice is often better than what the human picked. How the model's pick is "better than what the humn picked" is never made clear, but since acc…
The author was saying that, in their personal playing skill-limited estimation, the model made a stronger pick than the human drafter did. They propose that individuals overrate and underrate cards, but the model collects that and appropriate rates them. But it could also just be that the author is overrating cards - that’s why they asked for other opinions.
Re: Teaching an AI to Draft Magic: The Gathering
#24This seems to me to be similar to poker, and considered a game of luck. But i am not familiar enough.
I also found this https://boardgamegeek.com/article/29969194#29969194
Re: Teaching an AI to Draft Magic: The Gathering
#25Sorry but this is extremely shoddy work. M:tG will never be "solved" like this. >> Using data from drafts carried out by humans, I trained a neural network to predict what card the humans would take out of each pack. It reached 60% accuracy at this task. Going by what's in the linked notebook, the model was evaluated on its ability to match the decks in its training set card-for-card. Without any attempt to represent…
> The author suggests that the model "understands" colour because it drafts decks of specific colours. Well, its dataset consists of decks with cards of specific colours. It learned to reproduce those decks. It didn't learn anything about why those decks pick particular cards, or what particular cards are. All it has is a list of numbers that it has to learn to put together in specific ways. > This is as far from "un…
Re: Teaching an AI to Draft Magic: The Gathering
#26Earlier quoted context omitted.
> The author suggests that the model "understands" colour because it drafts decks of specific colours. Well, its dataset consists of decks with cards of specific colours. It learned to reproduce those decks. It didn't learn anything about why those decks pick particular cards, or what particular cards are. All it has is a list of numbers that it has to learn to put together in specific ways. > This is as far from "un…
>> In this case there isn't enough training data for that kind of resolution, but it has learned that blue cards go with blue cards, and red cards with red cards, and there's no hard lines from there to the concept of color. A card is blue (resp. red, etc) because it has a blue mana symbol in its casting cost. Not because it is found in the company of other blue cards. That is the concept of colour that a model must…
A blue card is found in the company of other blue cards, because humans picked them, because of the blue mana symbol in its casting cost.
With proper training, you end up with exactly the "end-to-end model with an internal representation of cards' charactersitics"
Since it can't see the cards, it can't say anything useful about a card it hasn't seen during training, but if you added some new cards and started training again, a pre-trained net might learn the new cards faster than one you train from scratch. That would be evidence that the network has learnt a meaningful embedding.
There is no proof that this network has done so, but I think word2vec shows that it's a feasible approach.
Re: Teaching an AI to Draft Magic: The Gathering
#27Sorry but this is extremely shoddy work. M:tG will never be "solved" like this. >> Using data from drafts carried out by humans, I trained a neural network to predict what card the humans would take out of each pack. It reached 60% accuracy at this task. Going by what's in the linked notebook, the model was evaluated on its ability to match the decks in its training set card-for-card. Without any attempt to represent…
Just to add a bit to this: >> Using data from drafts carried out by humans, I trained a neural network to predict what card the humans would take out of each pack. It reached 60% accuracy at this task. And in the 40% when the human and the model differ, the model’s predicted choice is often better than what the human picked. How the model's pick is "better than what the humn picked" is never made clear, but since acc…
Maybe it's too different of an idea, but in I draft I absolutely evaluate (part of) my skill by how well I've reproduced important components of a good deck. Did I find my seat? Good curve? Enough removal? Then there are format specific things - did I include enough 1/3s for 2, knowing that I'm likely to lose to fast decks with 2/1s if I don't?
Re: Teaching an AI to Draft Magic: The Gathering
#28Earlier quoted context omitted.
>> In this case there isn't enough training data for that kind of resolution, but it has learned that blue cards go with blue cards, and red cards with red cards, and there's no hard lines from there to the concept of color. A card is blue (resp. red, etc) because it has a blue mana symbol in its casting cost. Not because it is found in the company of other blue cards. That is the concept of colour that a model must…
By observing what cards people tend to pick together, you can infer that certain cards have certain properties, even if you never get see the card face. A blue card is found in the company of other blue cards, because humans picked them, because of the blue mana symbol in its casting cost. With proper training, you end up with exactly the "end-to-end model with an internal representation of cards' charactersitics" Si…
You're assuming way too much capability that is not present. Just because a human can make this inference, it doesn't mean that a neural net can. Neural networks are notoriously incapable of inference, or anything that requires reasoning.
>> There is no proof that this network has done so, but I think word2vec shows that it's a feasible approach.
Word2vec (word embeddings in general) are actually a good example why this kind of thing doesn't work the way you think it does. A word embedding model represents information about the context in which tokens (words, sentences, etc) are found but it does not, in and of itself, represent the meaning of words. The only reason why we know that words it places in the general vicinity of each other have similar meaning is because we already understand meaning and we can interpret the results. But the model itself does not have anything like "understanding". It only models collocations.
Same thing here. You seem pretty certain that with more data (perhaps with a deeper model) you can represent something that the model doesn't have an internal representation for. But just because the behaviour of the model partially matches the behaviour of a system that does have an internal representation for such a thing, in other words, a human, that doesn't mean that the model also behaves the way it behaves because it models the world in the same way that the human does.
And you can see that very clearly if you try to use a model like the one in the article, or one trained on all the magic drafts ever, to draft a set of cards it hasn't seen before. It should be obvious that such a model would be entirely incapable of doing so. That's because it doesn't represent anything about the characteristics of cards it hasn't seen and so can't handle new cards. A human understands what the cards' characteristics means and so can just pick up and play a new card with little trouble.
As to what I mean by "internal representation"; machine learning models that are trained end-to-end and that are claimed to learn constituent concepts in the process of learning a target concept actually have concrete representations of those constituent concepts as part of their structure. For example, CNNs have internal representations of each layer of features they learn in the process of classifying an image. Without such an internal reprsentation all you have is some observed behaviour and some vague claims about understanding this or learning that, at which point you can claim anything you like.
Re: Teaching an AI to Draft Magic: The Gathering
#29This is cool stuff. Have you checked out mtgtop8.com? Some code for it from back in Kaladesh: https://github.com/ivodopyanov/MTGML . I asked the dev a couple of years ago and he said he had a newer version, but it wasn't on github. Thinking about your jellybean guessing (wisdom of the crowds) analogy, I don't want an unweighted average of a Ben Stark and a random player. Years ago mtgo elo ratings were public - seems…
Re: Teaching an AI to Draft Magic: The Gathering
#30Sorry but this is extremely shoddy work. M:tG will never be "solved" like this. >> Using data from drafts carried out by humans, I trained a neural network to predict what card the humans would take out of each pack. It reached 60% accuracy at this task. Going by what's in the linked notebook, the model was evaluated on its ability to match the decks in its training set card-for-card. Without any attempt to represent…
This is a lot of anger over a choice of vocabulary. The author certainly didn’t mean that the model had a deep understanding of the nuances of the concept of color - just that it had identified clusters that in real life correspond to the color of cards.