Summary
This course describes theory and methods for Reinforcement Learning (RL), which revolves around decision making under uncertainty. The course covers classic algorithms in RL as well as recent algorithms under the lens of contemporary optimization.
Content
Keywords
Reinforcement Learning (RL)
Markov Decision Process (MDP)
Dynamic Programming
Linear Programming
Policy Gradients
Deep Reinforcement Learning (Deep RL)
Imitation Learning
Markov Games
Robust Reinforcement Learning
RL Algorithms (e.g., Q-Learning, SARSA, TRPO, PPO)
Offline Reinforcement Learning
Behavior Cloning
Inverse Reinforcement Learning
Equilibria
Robustness
Learning Prerequisites
Required courses
Previous coursework in optimization, calculus, linear algebra, and probability is required. Familiarity with
optimization is useful. Familiarity with python, and basic knowledge of pytorch deep learning framework is needed.
Recommended courses
EE-556 Mathematics of Data: From Theory to Computation
Important concepts to start the course
2023-2024 COURSE BOOKLET
Reinforcement learning Page 1 / 2
Familiarity with optimization algorithms, linear programming and convex duality.
Learning Outcomes
By the end of the course, the student must be able to:
• Define the key features of RL that distinguishes it from standard machine learning.
• Assess / Evaluate strengths, limitations and theoretical properties of RL algorithms.
• Recognize the common, connecting boundary of optimization and RL.
• Formulate and solve sequential decision-making problems by applying relevant RL tools.
Teaching methods
Lectures are comlemented with Jupiter notebook exercises along with a hands-on group project.
Assessment methods
The students are required to solve Jupiter notebook homeworks. They will work in a group to complete a project on the course and present a poster on the project at the end of the semester.