AI system for Minecraft: MIPT team wins MineRL international contest

Researchers from the Moscow Institute of Physics and Technology have dominated the MineRL contest. They proposed a new method for reinforcement learning with demonstrations, which enables rapidly and efficiently solving hierarchical problems in the Minecraft game environment, namely, searching for resources and crafting new tools. The developed algorithms can be used in autonomous mobile robots and manipulator robots. The contest took place during the Neural Information Processing Systems 2019 international conference.

Reinforcement learning is a promising method for artificial intelligence, showing amazing performance in tasks involving video games and robots. The method requires considerable computation resources and long training times, though. Staff researchers and interns of MIPT’s Cognitive Dynamic Systems Lab and the Artificial Intelligence Research Institute (full name below text) came up with a new approach using a mechanism for forgetting incorrect expert demonstrations of solving the task at hand. The team comprised Alexey Skrynnik, Ermek Aitygulov, Aleksey Staroverov, Kirill Aksenov, Vasilii Davydov, and Aleksandr Panov.

“The lab’s team achieved an excellent result, showing the high standards at MIPT and in Russia. The Minecraft gaming environment, in which we could implement intelligent agent behavior, provides for a wide range of interactions between a human player and a complex, unpredictable external environment. That makes it an excellent, fairly realistic simulator, which is more challenging when it comes to behavior learning method development, compared with the game of go, Dota, or StarCraft. The method we proposed opens up opportunities for applying reinforcement learning to real-world tasks, such as autonomous vehicle and manipulator control,” said Aleksandr Panov, who heads the Cognitive Dynamic Systems Lab at the MIPT Center for Cognitive Modeling.

MineRL lasted five months and comprised two rounds. In the first one, the MIPT team placed third, but then it overtook nine finalists, ultimately winning the competition with a score considerably ahead of the other contestants. More than 40 teams from around the world participated in MineRL.

“MineRL is a competition in intelligent algorithms rather than computational resources. Many of the impressive artificial intelligence results that came before were achieved by ‘brute force,’ that is, using large numbers of GPUs and CPUs to train complex systems,” added Andrey Raigorodskiy, the director of the Phystech School of Applied Mathematics and Informatics at MIPT. “Once the computation efficiency of the developed methods became one of the requirements, the chances of the teams became even. That allowed out contestants to triumph over teams from the U.S., China, Japan, Finland, Germany, and other countries. This is yet another testament to the fact that our Phystech School and MIPT in general are global leaders in the field of artificial intelligence. This is actually to be expected, since the school has a unique partner environment encompassing both academic institutions and industry partners. In that regard, one has to mention the success of our joint project with the Institute for Systems Analysis of the Russian Academy of Sciences and AIM Tech.”

MineRL was organized by Microsoft and Carnegie Mellon University. The organizing and program committees featured Katja Hofmann of Microsoft Research, Ruslan Salakhutdinov from Carnegie Mellon University, Sergey Levine from the University of California at Berkeley, and Chelsea Finn of Google Brain and UC Berkeley.

We congratulate the MIPT team on their much-deserved victory and express our gratitude to the laboratory’s industry partner AIMTech for the computation resources and financial support provided. A video recording with solution presentations at NeurIPS is available online. Fast forward to 01:01:39 for an introduction about the MineRL; the team presentations start at 01:22:10 with a talk by Alexey Skrynnik. A description of the algorithm proposed by the MIPT team is outlined in the summary.

###

The Artificial Intelligence Research Institute is part of the Federal Research Center for Computer Science and Control of the Russian Academy of Sciences.