Code

GitHub profile

Highlighted projects developed by my collaborators and me:

• Procgen Benchmark (2019-future): GitHub / blog post. I strongly recommend using this for RL research & benchmarking.
• Gym Retro (2018-future): GitHub / blog post on dataset / contest
• OpenAI Baselines (2016-future): GitHub / original post (DQN) / ACKTR + A2C / PPO
• OpenAI Gym (2016-future): homepage / GitHub / blog post / article on NVIDIA blog
• Computation Graph Toolkit (2015): GitHub / docs. Computation Graph Toolkit (CGT) is an automatic differentiation library, intended to be "Theano reloaded" with fast compilation, multithreading, improved compile-time inference, and a simpler codebase. I stopped developing it after Tensorflow came out and turned out to be excellent.
• TrajOpt (developed 2012-2013): GitHub / docs. TrajOpt is a software framework for generating robot trajectories by local optimization. The following core capabilities are included: a solver for generic nonlinear optimization problems by sequential quadratic programming, cost and constraint functions for kinematics and collision avoidance, and a JSON-based problem specification format for trajectory optimization problems. The core libraries are implemented in C++, and a Python API using Boost.Python is provided.