AI team competes in Dota 2 competition

Ilya Sutskever, co-founder and Research Director of OpenAI, and his team are developing AI bots smart enough to battle some of the world’s best human gamers. This brings AI closer to wider use in real-world applications. NVIDIA has been an early supporter of the company, with CEO Jensen Huang personally delivering the first DGX-1 AI supercomputer in a box to OpenAI.

In August, OpenAI Five, a team of five neural networks, were defeated by some of the world’s top professional players of Dota 2, the multiplayer online battle arena game at an international competition run by Valve in Canada. This was quite an achievement as OpenAI Five played a nearly unrestricted version of Dota 2 - a complex game - at the professional level.

In Dota 2, players can deploy all kinds of tactics, strategies and interactions to win. The game layout — only partially observable — requires both short-term tactics and long-term strategy, over a match that can last 45 minutes.

In comparison, the average number of actions in chess and go, typically used in AI challenges, is 35 and 250, respectively. In Dota 2, there are about 170,000 actions per move and 20,000 moves per game.

“Professional players dedicate their lives to this game,” said Sutskever at NTECH, an annual engineering conference at NVIDIA’s Silicon Valley campus in the US. “It’s not an easy game to play.”

A reliable reinforcement learning (RL) algorithm called proximal policy optimisation was used to train OpenAI Five at massive scale. The team ran on more than 1,000 NVIDIA Tesla P100 GPUs on the Google Cloud platform.

Researchers at OpenAI are also making strides on a project called Dactyl, which aims to increase the dexterity of a robot hand. The team there has been working on domain randomisation — an old concept — with significant results. They have been able to train the robot hand to manipulate objects in simulation, and then transfer that knowledge to real-world manipulation. This is important, because simulation is the only way to get enough training experience for these robots.

OpenAI is a nonprofit that was formed in 2015 to develop and release artificial general intelligence aimed at benefiting humanity. Its founding members include Tesla CEO Elon Musk, Y Combinator President Sam Altman and other tech luminaries who have collectively committed US$1 billion to its mission.