MLOps Patterns
Production ML repeats the same shapes. The page gathers the pattern guides first, the catalogs, roadmaps, courses, and Metaflow reading, and then shows two end-to-end systems that use them.
General patterns
A pattern only helps once you can see where it sits in a product's life, so the catalogs start with the lifecycle itself. ML product lifecycle patterns is Antony Mayi's guide to understanding ML-product lifecycle patterns, and the figure below is its picture of those lifecycles.
Inside that lifecycle, the design choices have names. The ML design patterns book repo is the source code accompanying the O'Reilly book Machine Learning Design Patterns, and the two figures after it are pages from that book.
The book's patterns are about the model; the next catalog is about the system around it. MLOps design patterns is Mercari's set of system design patterns for machine learning, shown in the figure below.
When a single catalog is not enough, Awesome MLOps is the curated list of references for MLOps. The Visenger figure, copied from the original hosted image, and the table of contents after it show how that list is organized.
Lists describe the field as it is; roadmaps describe where it is going. The MLOps Roadmap is the 2022 roadmap of the CD Foundation's SIG MLOps. Google's Practitioners Guide to MLOps: a framework for continuous delivery and automation of machine learning is written for ML leaders and enterprise architects who want to understand MLOps in theory and in practice. The State of MLOps page on ml-ops.org comes with the template shown below.
The roadmaps become concrete in two hands-on pieces. Easy MLOps with PyCaret and MLflow is by Moez Ali. Challenges and solutions by Iguazio is by Yaron Haviv, and the Iguazio figure below is credited to that article. The general-patterns figure after it is copied from the original hosted image.
For a full course instead of articles, Stanford CS329 is CS 329S: Machine Learning Systems Design. The course goes into how ML systems are built, and how to debug them, find a root cause, and monitor them.
One framework keeps coming up in these patterns, so the Metaflow reading on Medium follows, from a high-level review and the schema to docs on loading and storing data:
A pipeline also has to be tuned and watched once it runs. HyperparameterHunter is the thread of Hyperopt, MLflow, unit tests, concept drift, using Python and Kafka, the logging-and-monitoring side of putting ML in production.
The same notes are in Drift and ML Experiment Management.
Patterns in practice
With the catalogs in hand, two systems show the patterns running end to end.
The same notes are in MetaFlow and Prefect.
The first is an MLOps end-to-end system, "You dont need a bigger boat", using Metaflow, Snowflake, DBT, Prefect, Great Expectations, Weights & Biases, SageMaker, and Lambda. The second strips that down: A simplistic end-to-end system joins the modern data stack with the modern ML stack using Snowflake, DBT, S3, CometML, Reclist, and SageMaker.
Both systems follow the lifecycle this page opened with, and Antony Mayi's guide is also at its original address: A Guide to Classifying Operational Lifecycles of ML-Driven Products with an Overview of their Notable Patterns, ML product lifecycle patterns. https://towardsdatascience.com/understanding-ml-product-lifecycle-patterns-a39c18302452
Deprecated links
These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.
Easy mlops with pycaret and mlflow. This address no longer opens: https://towardsdatascience.com/easy-mlops-with-pycaret-mlflow-7fbcbf1e38c6
Challenges and solutions by iguazio. This address no longer opens: https://towardsdatascience.com/ml-ops-challenges-solutions-and-future-trends-d2e59b74dc6b
4 (amazing). This address no longer opens: https://towardsdatascience.com/learn-metaflow-in-10-mins-netflixs-python-r-framework-for-data-scientists-2ef124c716e4
5 (extra). This address no longer opens: https://towardsdatascience.com/be-more-efficient-to-produce-machine-learning-pipeline-with-metaflow-db5f943ebbe7
HyperparameterHunter, Hyperopt, mlflow, unit test, concept drifts, using python and kafka. This address no longer opens: https://towardsdatascience.com/putting-ml-in-production-ii-logging-and-monitoring-algorithms-91f174044e4e
Last updated
Was this helpful?