113 — Nintendo Super Smash Bros. Melee: An 'Untouchable' Agent

What better way to celebrate a later-in-the-evening #365papers than with a paper all about Super Smash Bros.?

Using emulators, Super Smash Bros. Melee can be executed using controlled hardware, where the memory space can be evaluated for optimizing a neural network. This enabled the researchers to optimize an agent that computes individual players’ locations in the world and on screen.

Using reinforcement learning, the agent-based approach to SSB is able to remain “untouched” by enemies for long periods of time using only dodges and moves. The agent was rewarded with 0 points for every shielding action and $\frac{1}{60}$ points for every frame without movement (to avoid rewards for constant shielding behavior).

As soon as the agent is hit and suffers any damage, the match is reset; this encourages the agent to learn from the early frames of the match and only progress to later frames when it has “mastered” the beginning frames.

The agent is able to dodge all hits for a minute or longer around 3/4 of the time, at the highest AI level available in the SSB game.

Code is available here.