319 — Human-level performance in first-person multiplayer games with population-based deep reinforcement learning

DeepMind has published a beautiful web-paper which you can read here. The visuals themselves are art — to say nothing of the merit of the research itself:

The one thing I found most fascinating reading this blog post is this: there were 6 people working on visualizations. Six, designers and web developers, on visualizations - at least this is my understanding. They are arguably extremely nicely done visuals indeed. https://t.co/3bJwwrhlVB

— Ferenc Huszár (@fhuszar) July 4, 2018

This agent-based learning architecture is dubbed the “For the Win” FTW agent — and it’s an appropriate title, because these agents are trained to play Capture-the-Flag (CTF) video games. But unlike prior training mechanisms, these agents are trained in large group cohorts with “peer” agents. As a result, each agent learns not only to play the CTF game (i.e. the rules and the general gameplay mechanisms, such as map navigation) but also to cooperate with and rely upon teammates — even though the agents have no realtime knowledge of their teammates’ ‘rationales.’ As a result, humans and machines are interchangeable in the team-based game: A team might have only one human on it — and the cooperative aspect would not be lost.

But the FTW agents actually outperformed humans in gameplay: Teams that had AI teammates performed better than all-human teams, and teams comprised solely of AI outperformed all others.

But perhaps more interestingly, in survey results, humans rated the FTW agents higher in “collaborativeness” than their other human players.