Non-profit AI research company OpenAI has developed an AI Dota 2 team capable of winning against skilled human opponents, and have announced plans to put it to the test against pro players at The International in August.
“Our team of five neural networks, OpenAI Five, has started to defeat amateur human teams at Dota 2,” a blog post on their website reads.
“While today we play with restrictions, we aim to beat a team of top professionals at The International in August subject only to a limited set of heroes.We may not succeed: Dota 2 is one of the most popular and complex esports games in the world, with creative and motivated professionals who train year-round to earn part of Dota’s annual $40M prize pool (the largest of any esports game).”
OpenAI have also announced plans to hold a showmatch versus top Dota 2 players on July 28 to benchmark the bot’s progress. They will be streaming that on their Twitch channel, as well as holding an event allowing people to watch in person.
The technology works through what’s termed “reinforcement learning,” an area of machine learning that aims to mimic the way humans and animals learn. However, this process is far less efficient in computers than it is in nature. The major advantage that AI has over humans is volume: with enough processing power, the technology can play against itself for far longer than any human would be able to practice – OpenAI says that its bot plays 180 years worth of games itself every day. During this time, the bot builds up its idea of what optimal play looks like as it experiences more scenarios in-game.
Strengths and weaknesses
They’ve had considerable success already. OpenAI Five has played several test games against teams of different skill levels, decisively beating OpenAI employees, the best audience players watching that initial test game and a Valve employee team. The AI has also faced off against an Amateur team and a Semi-pro team, winning two of its first three games against both.
The team behind Open AI Five note that it is stronger in some areas than others. In particular, last-hitting is weak – Open AI say on their blog that professional Dota commentator Blitz estimated the skill-level in this area to be around the median for Dota players. The bot’s main strengths lie in teamwork and objective-prioritisation, both of which require sacrificing short-term rewards for more lasting, long-term advantages.
Especially interesting is how the bots have seemingly independently come up with strategies that are mainstays in the professional scene. “[The bot] Repeatedly sacrificed its own safe lane (top lane for dire; bottom lane for radiant) in exchange for controlling the enemy’s safe lane, forcing the fight onto the side that is harder for their opponent to defend,” the blog post notes. “This strategy emerged in the professional scene in the last few years, and is now considered to be the prevailing tactic. [Dota caster] Blitz commented that he only learned this after eight years of play, when Team Liquid told him about it.
All of which seems to indicate that, if the current rate of growth continues, the bot could start discovering strategies that haven’t even occurred to human players yet. There are even indications this could already be happening. The Open AI team write that the bot “deviated from current playstyle in a few areas, such as giving support heroes (which usually do not take priority for resources) lots of early experience and gold.”
Applications outside gaming
OpenAI’s stated mission is to develop safe Artificial general intelligence (AGI), a term that is generally considered to describe an AI capable of completing any task a human can. The aim of developing AIs for Dota and other games is to ultimately refine the technology to the point where it could be used to tackle real-world applications. In this sense, Dota 2 represents a meaningful progression from games like Chess and GO, which are comparatively simple for a computer to master due to their absence of hidden information and finite number of variables. Dota, by contrast, has plenty of hidden information and a near-infinite array of variables. Perhaps not quite as many as you’d find in real-world scenarios – but definitely more than chess.
“Our underlying motivation reaches beyond Dota,” OpenAI says. “Real-world AI deployments will need to deal with the challenges raised by Dota which are not reflected in Chess, Go, Atari games, or Mujoco benchmark tasks. Ultimately, we will measure the success of our Dota system in its application to real-world tasks. ”
And while the betterment of humankind is a noble aim, in the shorter term I’m still happy to settle for watching Liquid get taken apart by a team of robots. For science, of course.