• MLMMI - Machine Learning Model Management and Inference (SS 2026)

    This advanced course explores the system and data management challenges in managing and serving machine learning (ML) models. The lifecycle of an ML model extends far beyond training.
     
    The course begins with recapping standard ML pipelines up to the training stage and then shifts focus to the post-training workloads. Students will be introduced to foundational frameworks for model management, where trained models are treated as core data artifacts. Topics include data management techniques and optimizations such as model selection, versioning, lineage tracking, metadata management, and model monitoring.
     
    Building on these foundations, students will explore the architectures of modern model serving systems for both classical models and large language models (LLMs), along with performance optimizations such as dynamic batching, model compilation, and resource-efficient inference execution. Finally, the course discusses ML agents as an emerging workload.
     
    Throughout the course, students will engage with state-of-the-art research on inference and model management. Through a combination of seminal research papers and hands-on projects, students will gain a comprehensive understanding of the entire ML lifecycle beyond training, preparing them for both academic research and real-world system design.
     
    • Where: E-N 189 (lectures) and MAR 0.001 (exercises)
    • When: Wednesday, 16:15 - 17:45 (lectures) and Monday, 16:15 (consultation with TA).
                  First lecture: April 15, 16:15
     
    Note
    The outline below is tentative. It will evolve as we teach.
     

    Lecture 1: Intro and Logistics [April 15]

     

    Part 1: ML Deployment

     

    Lecture 2: Model Management Systems [April 22]

    • Model and data versioning
    • Lineage tracing
    • Feature stores
    • Readings: ModelDB, ModelHub, DataHub, LIMA, FeathrPO
    • Slides
     

    Lecture 3: ML Platforms and Deployment [April 29]

    • MLOps
    • End-to-end ML platforms and lifecycle management
    • Continuous retraining
    • Data validation
    • Readings: TFX, MODYN, AWS Deequ
    • Slides
     

    Lecture 4: Model Selection Systems [May 06]

    • Model selection systems
    • Transfer learning
    • Readings: Hyperband, Cerebro, VISTA, SHiFT
    • Slides
     

    Guest Lecture: Nils Strassenburg [May 13]

     

    Part 2: ML Serving

     

    Lecture 5: ML Serving Systems [May 20]

    • Traditional model serving
    • Serving optimizations
    • In-database ML serving
    • Readings: Clipper, Pretzel, TVM
    • Slides
     

    Lecture 6: AI Agents and Compound AI Systems [May 27]

    • Compound AI systems and use cases
    • AI agents and use cases
    • Context memory management, tooling, and RAG
    • Readings: ReAct, MemGPT
    • Slides
     

    Lecture 7: LLM Serving [June 17]

    • LLM inference overview
    • Iteration-level scheduling
    • Prefill-decode disaggregation
    • Local LLM serving
    • LoRA
    • Slides
     

    Lecture 8: KV Cache Management [June 24]

    • KV cache memory management
    • Prompt caching
    • Long-context LLM serving
    • Quantization for KV cache
    • Slides
     

    Programming Assignments

    1. Lightweight Model Management System (task description | deadline: 04.05.2026)
    2. Transfer Learning (task description | deadline: 18.05.2026)
    3. Model Serving System (task description | deadline: 01.06.2026)

    Projects

    The majority of the course grade will come from a group project. There are five project topics.
     
    • Projects #1 and #2 focus on the application side of the system and involve building LLM-assisted frameworks. These projects give exposure to modern ML engineering agents and automated pipeline generation techniques.
    • Projects #3 and #4 focus on the internals of Stratum, including optimizer rewrites and high-performance Rust operator implementations. These projects expose core concepts in data systems, query optimization, and runtime.
    • Project #5, which may be assigned to two independent groups, involves building an AutoML framework on top of skrub and stratum. This project requires exploring optimization and search techniques such as successive halving.
     
     
    Note
    Project overviews are shared below so you can start thinking about preferences and possible directions. All projects will be discussed in greater detail during the lecture on May 20.
     
    1. Skrubifier: LLM-Assisted Conversion of ML Pipelines to Skrub DataOps
      Build an LLM-assisted framework that automatically converts tabular machine learning pipelines written using popular Python ML libraries into the Skrub DataOps API. Your framework should analyze existing pipelines and generate runnable Skrub DataOps DAGs compatible with skrub or stratum. Use tabular ML pipelines from MLE-Bench and Kaggle as source workloads. Select representative solutions, convert them using your framework, and evaluate correctness and executability.
      Main deliverables: A robust conversion framework/toolchain; 10 converted pipelines written using the Skrub DataOps API.
    2. AIDE-skrub: Extending AIDE Agent for Skrub DataOps Generation
      Extend the AIDE agent framework to generate pipelines directly in the Skrub DataOps API. Explore methods including modifying existing prompts in AIDE and few-shot demonstrations, tool augmentation, and adding new modules/sub-agents in AIDE. Use tabular datasets and tasks from MLE-Bench and Kaggle to evaluate your system.
      Main deliverables: A modified version of AIDE capable of producing Skrub DataOps code; 10 generated pipelines runnable on skrub or stratum.
    3. Stratum Optimizer: Rule-Based Rewrites for stratum
      Extend stratum with a collection of rule-based optimization rewrites. Each group member must implement five rewrites, where each rewrite consists of a pattern matcher, a rewrite rule, and unit tests validating correctness. Example rewrites include algebraic rewrites, constant folding, projection fusion, and pushdown.
      Main deliverables: Rewrite implementations integrated into stratum; unit tests for every rewrite.
    4. High-Performance Rust Kernels for Stratum
      Implement high-performance Rust kernels for operators used in stratum. Each group member must contribute two kernels. Select a baseline implementation from systems such as Scikit-learn and implement optimized Rust versions targeting a constrained/default hyperparameter configuration. The goal is to achieve at least 2× speedup over the baseline. Potential optimization techniques include SIMD/vectorization, parallelization, cache-aware execution, and zero-copy processing. Example operators include processing methods (normalization, encoding, scaling), learning algorithms (logistic regression, SGD classifier, Naive Bayes), and feature transformation operators.
      Main deliverables: Rust implementations integrated with stratum; unit tests validating correctness; benchmark suite comparing performance against baselines.
    5. AutoML on Skrub DataOps DAGs (two independent groups)
      Design and implement an AutoML framework on top of the Skrub DataOps API. Given (1) a dataset, (2) a problem description, and (3) a YAML specification describing candidate preprocessing techniques and models, your system should parse/compile the YAML into a Skrub DataOps DAG representing all pipeline combinations, execute and evaluate candidate pipelines, iteratively prune poor-performing pipelines using techniques such as successive halving, and continue refinement until a single best-performing pipeline remains. The framework should support arbitrary tabular ML problems from Kaggle. Example configurable components include encoders, imputation methods, scaling methods, linear models, tree-based models, and regularization choices.
      Main deliverables: An AutoML framework that generates skrub DataOps DAGs; two complete problem solutions per person using the framework.
     

    Organization

    • Lecture: Dr.-Ing. Arnab Phani
    • Teaching Assistant: Ellias Strauss