Taylor expansion policy optimization

Yunhao Tang, Michal Valko, Rémi Munos

2020 · in (ICML 2020) and (MS RL Day 2021)

Abstract

In this work, we investigate the application of Taylor expansions in reinforcement learning. In particular, we propose Taylor expansion policy optimization, a policy optimization formalism that generalizes prior work (e.g., TRPO) as a first-order special case. We also show that Taylor expansions intimately relate to off-policy evaluation. Finally, we show that this new formulation entails modifications which improve the performance of several state-of-the-art distributed algorithms.

PDF · International Conference on Machine Learning · arXiv preprint · talk · image · bibtex · DOI