> For the complete documentation index, see [llms.txt](https://www.mlcompendium.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.mlcompendium.com/ai-engineering/mlops.md). # MLOps Patterns Production ML repeats the same shapes. The page gathers the pattern guides first, the catalogs, roadmaps, courses, and Metaflow reading, and then shows two end-to-end systems that use them. ## General patterns A pattern only helps once you can see where it sits in a product's life, so the catalogs start with the lifecycle itself. [ML product lifecycle patterns](https://medium.com/data-science/understanding-ml-product-lifecycle-patterns-a39c18302452) is Antony Mayi's guide to understanding ML-product lifecycle patterns, and the figure below is its picture of those lifecycles.

Inside that lifecycle, the design choices have names. The [ML design patterns book](https://github.com/GoogleCloudPlatform/ml-design-patterns) repo is the source code accompanying the O'Reilly book Machine Learning Design Patterns, and the two figures after it are pages from that book.

The book's patterns are about the model; the next catalog is about the system around it. [MLOps design patterns](https://github.com/mercari/ml-system-design-pattern/tree/master) is Mercari's set of system design patterns for machine learning, shown in the figure below.

When a single catalog is not enough, [Awesome MLOps](https://github.com/visenger/awesome-mlops) is the curated list of references for MLOps. The Visenger figure, copied from the original hosted image, and the table of contents after it show how that list is organized.

Lists describe the field as it is; roadmaps describe where it is going. The [MLOps Roadmap](https://github.com/cdfoundation/sig-mlops/blob/main/roadmap/2022/MLOpsRoadmap2022.md) is the 2022 roadmap of the CD Foundation's SIG MLOps. [Google's Practitioners Guide to MLOps: a framework for continuous delivery and automation of machine learning](https://cloud.google.com/resources/mlops-whitepaper) is written for ML leaders and enterprise architects who want to understand MLOps in theory and in practice. The [State of MLOps](https://ml-ops.org/content/state-of-mlops) page on ml-ops.org comes with the template shown below.

The roadmaps become concrete in two hands-on pieces. [Easy MLOps with PyCaret and MLflow](https://medium.com/data-science/easy-mlops-with-pycaret-mlflow-7fbcbf1e38c6) is by Moez Ali. [Challenges and solutions by Iguazio](https://medium.com/data-science/ml-ops-challenges-solutions-and-future-trends-d2e59b74dc6b) is by Yaron Haviv, and the Iguazio figure below is credited to that article. The general-patterns figure after it is copied from the original hosted image.

For a full course instead of articles, [Stanford CS329](https://stanford-cs329s.github.io/syllabus.html) is CS 329S: Machine Learning Systems Design. The course goes into how ML systems are built, and how to debug them, find a root cause, and monitor them. One framework keeps coming up in these patterns, so the Metaflow reading on Medium follows, from a high-level review and the schema to docs on loading and storing data: A pipeline also has to be tuned and watched once it runs. HyperparameterHunter is the thread of [Hyperopt, MLflow, unit tests, concept drift, using Python and Kafka](https://medium.com/data-science/putting-ml-in-production-ii-logging-and-monitoring-algorithms-91f174044e4e), the logging-and-monitoring side of putting ML in production. The same notes are in [Drift](/ai-engineering/mlops-monitoring-and-alerts.md#drift) and [ML Experiment Management](/ai-engineering/experiment-management.md). ## Patterns in practice With the catalogs in hand, two systems show the patterns running end to end. The same notes are in [MetaFlow](/ai-engineering/full-stack-and-ops.md#metaflow) and [Prefect](/ai-engineering/full-stack-and-ops.md#prefect). The first is an MLOps end-to-end system, "[You dont need a bigger boat](https://github.com/jacopotagliabue/you-dont-need-a-bigger-boat)", using Metaflow, Snowflake, DBT, Prefect, Great Expectations, Weights & Biases, SageMaker, and Lambda. The second strips that down: [A simplistic end-to-end system](https://github.com/jacopotagliabue/post-modern-stack) joins the modern data stack with the modern ML stack using Snowflake, DBT, S3, CometML, Reclist, and SageMaker. Both systems follow the lifecycle this page opened with, and Antony Mayi's guide is also at its original address: A Guide to Classifying Operational Lifecycles of ML-Driven Products with an Overview of their Notable Patterns, ML product lifecycle patterns. ## Deprecated links {% hint style="warning" %} These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above. {% endhint %} * Easy mlops with pycaret and mlflow. This address no longer opens: * Challenges and solutions by iguazio. This address no longer opens: * 4 (amazing). This address no longer opens: * 5 (extra). This address no longer opens: * HyperparameterHunter, Hyperopt, mlflow, unit test, concept drifts, using python and kafka. This address no longer opens: --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://www.mlcompendium.com/ai-engineering/mlops.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.