Building a Machine Learning Pipeline from Scratch

Module 1: Course Goals & Structure

⏳ 10 minutes

• Topics: Intended audience; course goals; structure & strengths

• Hands-on: Review course roadmap and objectives

Module 2: Getting Started

⏳ 15 minutes

• Topics: Why pipelines vs. notebooks; defining ML training pipelines

• Hands-on: Complete the “Getting Started” quiz

Module 3: Structuring the ML Pipeline

⏳ 30 minutes

• Topics: System architecture; directory layout; code organization; dependency management

• Hands-on: Scaffold a project directory and initial files

Module 4: Directed Acyclic Graphs (DAGs)

⏳ 20 minutes

• Topics: DAG fundamentals; topological sorting

• Hands-on: Implement and sort a DAG for sample pipeline tasks

Module 5: Building the ML Library

⏳ 45 minutes

• Topics: OOP modules; OmegaConf configurations; abstract base classes; datasets; models; reports

• Hands-on: Create library components and configuration schemas

Module 6: The Pipeline Core

⏳ 45 minutes

• Topics: CLI parsing (argparse); experiment tracking; logging; docstrings

• Hands-on: Assemble top-level pipeline script with logging and tracking

Module 7: Extending the Pipeline

⏳ 30 minutes

• Topics: Adding support for new datasets and model types

• Hands-on: Extend pipeline to a second dataset

Module 8: Testing

⏳ 30 minutes

• Topics: Unit testing; pytest; system testing

• Hands-on: Write and execute tests for pipeline functions

Get certificate