OpenAI today announced the launch of Spinning Up, a program designed to teach anyone deep reinforcement learning. OpenAI is well known for making funky-looking agents in virtual environments that learn how to walk on their own such as Humanoid v2 or POLO, a collaboration with University of Washington.
Reinforcement learning involves providing reward signals to an agent in an environment incentivized to maximize its reward to meet a goal.
RL has played a role in major AI breakthroughs such as Google DeepMind’s AlphaGo and agents trained in environments like Dota 2.
Spinning Up includes a collection of important reinforcement learning research papers, a glossary of terminology necessary to understand RL, and a collection of algorithms for running exercises.
The program is being launched not just to help people learn how reinforcement learning works, but to make progress towards OpenAI’s general goal of safely creating artificial general intelligence (AGI) by involving more people from fields beyond computer science.
“Solving AI safety will require people with a wide range of expertise and perspectives, and many relevant professions have no connection to engineering or computer science at all. Nonetheless, everyone involved will need to learn enough about the technology to make informed decisions, and several pieces of Spinning Up address that need,” according to Spinning Up documentation text.
Spinning Up was informed by working with OpenAI scholars and fellows, people with an inquisitive interest in AI but little experience in the field.
In addition to making progress toward the goal of safely deployed AGI, the program was created because there’s no single textbook available today that can deliver the knowledge necessary to implement reinforcement learning; the knowledge currently is locked away in a series of disparate lectures and influential research papers.
“So our package here is designed to serve as the missing middle step for people who are excited by deep RL, and would like to learn how to use it or make a contribution, but don’t have a clear sense of what to study or how to transmute algorithms into code. We’ve tried to make this as helpful a launching point as possible,” the project page reads.
The Spinning Up initiative is part of a larger set of education services OpenAI intends to make available. A Spinning Up workshop will be held in February 2019, and another workshop will be co-hosted with the Center for Human-Compatible AI (CHAI) at the University of California at Berkeley.