The cost to train machine learning models has been increasing exponentially,
making exploration and research into the correct features and architecture a
costly or intractable endeavor at scale. However, using a technique named
"surgery" OpenAI Five was continuously trained to play the game DotA 2 over the
course of 10 months through 20 major changes in features and architecture.
Surgery transfers trained weights from one network to another after a selection
process to determine which sections of the model are unchanged and which must
be re-initialized. In the past, the selection process relied on heuristics,
manual labor, or pre-existing boundaries in the structure of the model,
limiting the ability to salvage experiments after modifications of the feature
set or input reorderings.
We propose a solution to automatically determine which components of a neural
network model should be salvaged and which require retraining. We achieve this
by allowing the model to operate over discrete sets of features and use
set-based operations to determine the exact relationship between inputs and
outputs, and how they change across tweaks in model architecture. In this
paper, we introduce the methodology for enabling neural networks to operate on
sets, derive two methods for detecting feature-parameter interaction maps, and
show their equivalence. We empirically validate that we can surgery weights
across feature and architecture changes to the OpenAI Five model.
Jonathan Raiman, Susan Zhang, Christy Dennison