“Strategy and coordination in Quake are simple.” “3-D environments are designed to make navigation easy,” Dr. Games like this are not nearly as complex as the real world.
“But one agent will sit in the opponent’s base camp, waiting for the flag to appear, and that is only possible if it is relying on its teammates.” “How you define teamwork is not something I want to tackle,” said Max Jaderberg, another DeepMind researcher who worked on the project. (Even mere ants can collaborate by trading chemical signals.)Īlthough the result looks like collaboration, the agents achieve it because, individually, they so completely understand what is happening in the game. They are merely responding to what is happening in the game, rather than trading messages with one another, as human players do. DeepMind’s agents are not really collaborating, said Mark Riedl, a professor at Georgia Tech College of Computing who specializes in artificial intelligence. “I was absolutely blown away.”Īs impressive as such technology has been among gamers, many artificial-intelligence experts question whether it will ultimately translate to solving real-world problems. “I didn’t think it would be possible for the machine to play five-on-five, let alone win,” he said. But as the agents continued to learn the game and he played them as a team, he was shocked by their skill.
Last year, William Lee, a professional Dota 2 player and commentator known as Blitz, played against an early version of the technology that could play only one-on-one, not as part of a team, and he was unimpressed. In April, a team of five autonomous agents beat a team of five of the world’s best human players. And at OpenAI, researchers built a system that mastered Dota 2, a game that plays like a souped-up version of capture the flag. Since completing this project, DeepMind researchers also designed a system that could beat professional players at StarCraft II, a strategy game set in space. But they gradually picked up the nuances of the game, like when to follow teammates as they raided an opponent’s home base: DeepMind’s autonomous agents learned capture the flag by playing roughly 450,000 rounds of it, tallying about four years of game experience over weeks of training. Many experts had thought this would not be accomplished for another decade, given the enormous complexity of the game.įirst-person video games are exponentially more complex, particularly when they involve coordination between teammates. In 2016, using the same fundamental technique, DeepMind researchers built a system that could beat the world’s top players at the ancient game of Go, the Eastern version of chess. If an agent consistently wins more points by moving toward an opponent’s home base when a teammate is about to capture a flag, it adds this tactic to its arsenal of tricks.
But over the past several years, DeepMind, OpenAI and other labs have made significant advances, thanks to a mathematical technique called “reinforcement learning,” which allows machines to learn tasks by extreme trial and error.īy playing a game over and over again, an automated agent learns which strategies bring success and which do not. Until recently, building a system that could match human players in a game like Quake III did not seem possible. “If you can’t solve games, you can’t expect to solve anything else.” “Games have always been a benchmark for A.I.,” said Greg Brockman, who oversees similar research at OpenAI, a lab based in San Francisco. Many researchers believe that success in the virtual arena will eventually lead to automated systems with improved abilities in the real world.įor instance, such skills could benefit warehouse robots as they work in groups to move goods from place to place, or help self-driving cars navigate en masse through heavy traffic. As human players know, the moment the opposing flag is brought to one’s home base, a new flag appears at the opposing base, ripe for the taking.ĭeepMind’s project is part of a broad effort to build artificial intelligence that can play enormously complex, three-dimensional video games, including Quake III, Dota 2 and StarCraft II. Through thousands of hours of game play, the agents learned very particular skills, like racing toward the opponent’s home base when a teammate was on the verge of capturing a flag.
“They can adapt to teammates with arbitrary skills,” said Wojciech Czarnecki, a researcher with DeepMind, a lab owned by the same parent company as Google.