Fundamentals of Reinforcement Learning Course Syllabus

Overview: This course offers a comprehensive introduction to reinforcement learning, focusing on core concepts and practical implementations. You'll explore sequential decision-making, Markov Decision Processes, value functions, and dynamic programming. The course blends theory with hands-on coding exercises, enabling you to build foundational skills in AI-driven decision systems. With approximately 15 hours of content, the course is designed for self-paced learning, ideal for those seeking a structured yet flexible path into reinforcement learning.

Module 1: Welcome to the Course!

Estimated time: 1 hour

• Introduction to course structure and objectives
• Meet the instructors from the University of Alberta
• Course roadmap and learning outcomes
• Navigating the Coursera platform

Module 2: An Introduction to Sequential Decision-Making

Estimated time: 3 hours

• Understanding the exploration-exploitation trade-off
• Implementing incremental algorithms for action-value estimation
• Comparing exploration strategies
• Foundations of reward-based learning

Module 3: Markov Decision Processes

Estimated time: 3 hours

• Translating real-world problems into MDPs
• Understanding goal-directed behavior via reward maximization
• Differentiating episodic and continuing tasks
• Formalizing decision-making problems using MDPs

Module 4: Value Functions & Bellman Equations

Estimated time: 3 hours

• Defining policies and value functions
• Understanding state-value and action-value functions
• Deriving and applying Bellman equations
• Role of Bellman equations in optimal decision-making

Module 5: Dynamic Programming

Estimated time: 3 hours

• Computing value functions using dynamic programming
• Implementing policy evaluation and improvement
• Understanding Generalized Policy Iteration
• Solving MDPs efficiently with iterative methods

Module 6: Final Project

Estimated time: 2 hours

• Design and implement a reinforcement learning agent
• Apply MDPs and dynamic programming to a simulated environment
• Submit code and analysis for peer review

Prerequisites

• Proficiency in Python programming
• Basic understanding of probability and linear algebra
• Familiarity with fundamental machine learning concepts

What You'll Be Able to Do After

• Formulate decision-making problems using MDPs
• Implement and compare exploration strategies
• Apply value functions and Bellman equations to real-world scenarios
• Use dynamic programming to solve reinforcement learning tasks
• Build foundational knowledge for roles in AI, robotics, and data science