# OpenAI: Spinning Up in Deep RL

> Source: <https://spinningup.openai.com/en/latest/>
> Published: 2026-09-02 06:57:12+00:00

# Welcome to Spinning Up in Deep RL![¶](#welcome-to-spinning-up-in-deep-rl)

[Spinning Up as a Deep RL Researcher](spinningup/spinningup.html)[Key Papers in Deep RL](spinningup/keypapers.html)[1. Model-Free RL](spinningup/keypapers.html#model-free-rl)[2. Exploration](spinningup/keypapers.html#exploration)[3. Transfer and Multitask RL](spinningup/keypapers.html#transfer-and-multitask-rl)[4. Hierarchy](spinningup/keypapers.html#hierarchy)[5. Memory](spinningup/keypapers.html#memory)[6. Model-Based RL](spinningup/keypapers.html#model-based-rl)[7. Meta-RL](spinningup/keypapers.html#meta-rl)[8. Scaling RL](spinningup/keypapers.html#scaling-rl)[9. RL in the Real World](spinningup/keypapers.html#rl-in-the-real-world)[10. Safety](spinningup/keypapers.html#safety)[11. Imitation Learning and Inverse Reinforcement Learning](spinningup/keypapers.html#imitation-learning-and-inverse-reinforcement-learning)[12. Reproducibility, Analysis, and Critique](spinningup/keypapers.html#reproducibility-analysis-and-critique)[13. Bonus: Classic Papers in RL Theory or Review](spinningup/keypapers.html#bonus-classic-papers-in-rl-theory-or-review)

[Exercises](spinningup/exercises.html)[Benchmarks for Spinning Up Implementations](spinningup/bench.html)