Reproducibility and Replicability in Deep Reinforcement Learning (and Other Deep Learning Methods)

Abstract

In recent years, significant progress has been made in solving challenging problems using deep learning. Reproducing existing work and accurately judging the improvements offered by novel methods is vital to sustaining this progress. Unfortunately, reproducing results for state-of-the-art deep learning methods is sometimes not straightforward. Several recent works examine these issues in generative adversarial networks (GANs), language models, reinforcement learning (RL), dialogue systems, and other fields. Here we present a brief survey of general problems highlighted by these recent works. We then focus on deep RL methods, emphasizing issues which are particularly problematic to the field. Some recommendations and suggestions for proper experimental practices will be discussed based on recent empirical research. Such suggestions are geared to make deep learning methods easily reproducible and replicable, ensuring that such methods continue to matter to the broader community.