Lilly Tech Systems
KubeFlow Pipelines
Build, deploy, and manage reproducible end-to-end machine learning workflows on Kubernetes. Learn the KubeFlow Pipelines SDK, create reusable components, run experiments, and implement production MLOps patterns.
Your Learning Path
Follow these lessons in order, or jump to any topic that interests you.
1. Introduction
Understand what KubeFlow Pipelines is, its architecture, and how it enables reproducible ML workflows on Kubernetes.
2. Installation
Install KubeFlow Pipelines on your Kubernetes cluster using standalone deployment, full KubeFlow, or managed services.
3. Pipeline SDK
Build ML pipelines using the KFP v2 Python SDK with decorators, type annotations, and DAG composition.
4. Components
Create reusable pipeline components, use pre-built Google Cloud components, and package components as containers.
5. Experiments
Run experiments, compare runs, track metrics and artifacts, and implement hyperparameter tuning pipelines.
6. Best Practices
Production patterns for CI/CD integration, pipeline versioning, error handling, caching, and multi-team collaboration.
What You'll Learn
By the end of this course, you'll be able to:
Build ML Pipelines
Create end-to-end ML workflows with data processing, training, evaluation, and deployment stages using Python.
Create Components
Build reusable, containerized pipeline components that can be shared across teams and projects.
Track Experiments
Run experiments, compare metrics, visualize results, and iterate on models systematically.
Production MLOps
Implement CI/CD for ML pipelines, version control workflows, and automate model deployment.