I did something like this with Dota2 where I'd have it monitor the opposing team to save their locations when they appeared on the minimap in a shaded color on a 2nd monitor on that 2nd computer. I used an HDMI splitter and HDMI capture card to send the input to the second computer. I did the CV stuff with opencv library. Taking it a step further as shown in the article could be done to do stuff like auto creep killi…
The meta game of “cheating” sounds way more fun to me than playing any of these games.
There is no sport and few other pre-computer competitive games that even approach this level of depth and complexity.
But speaking of playing the meta-game more than the game, I had a ton of fun making a "hero picker".
At the beginning of each match there's a mini-game where each of the ten players gets to choose their hero out of the available pool of 119. Some heroes combo well others on their own team, and others counter the enemy choices well. This leads to an explosion of possibilities, with something like 4e20 possible distinct combinations, so choosing well is... difficult.
I downloaded the outcome of 30 million games (just a few days' worth!) and tried a bunch of ML/AI algorithms to come up with an optimal hero picker: Given 'n' current picks, choose the best hero for the nest pick.
That produced high win rates but boring games: it just picked the overall strongest characters with very little variance.
I then tried to make it pick the hero with the highest differential win rate. That is, it would sometimes recommend a hero that would decrease your chance of winning, but increase your chance for that hero relative to other games. This would give the opportunity to play weak heroes in games that suited them best.
That was okay, but still not optimal, because it used the win rates from very bad players.
I then took my 30M games, and produced an automatic chess-style ranking for all players. I threw away all "weird" games and players only seen a few times, giving me a clean learning set of regular players with a known skill set. I then filtered it to my personal ID number +/- a 10% skill range (still millions of games!) and trained the ML on that, hoping to give optimal recommendations for my own skill set.
That was a good try, but it turned out that people in my skill range are terrible at the meta-game of picking heroes.
I then threw up my hands and simply had the ML train on the top 10% of players. That was amazing! It was like having the knowledge of the best players distilled into an AI. It recommended some amazing combos, and generally stopped recommending heroes that are strong in the mid-tier "pub" games but weak against skilled opponents. My skill went up, and I slowly climbed the ranks to about the top 20% or so.
Fun times, fun times...