Feature Stores & Feature Pipelines
Training and serving need the same features. The page starts with the pipeline that builds them, then moves to the store that serves them: what a store is, how it differs from a warehouse, the products, and why a store alone is not enough.
The same notes are in FEATURE STORE and Monitoring & alerts.
Feature calculation and serving are the heart of every classical machine learning system, and that is where Feature store and feature pipelines, Assaf Pinhasi, begins: a deep dive into the system-engineering and analytical tradeoffs of both.
FEATURE STORE
With the pipeline named, the next question is where its output lives, so this section is the store products and the articles that compare them.
The same notes are in Feature Stores & Feature Pipelines.
The case for a store starts with The importance of having one - medium, an article that is now kept at the end of the page. Feast's What is? is the comprehensive guide to feature stores, their role in machine learning operations, and the key challenges they solve in the ML lifecycle. The obvious objection is that a warehouse already stores data, and KDnuggets' Feature store vs data warehouse answers it: a feature store is a data warehouse of features, but dual-database, one side serving features at low latency to online applications and the other storing large volumes of features.
Once the idea is clear, the open-source reference is Feast, the introduction to Feast, the open source feature store. The same Feast guide doubles as what is 1, and what is 2 is the second explainer on feature stores as components of a data science factory. The figure below, copied from the original hosted image, shows the features a store holds.
If you would rather not run the store yourself, there are managed products. Tecton.ai managed feature store is the first; the address now lands on Databricks, which pitches a unified platform for data, analytics, and AI. Its figure follows.
The Iguazio feature store is the second ML feature store product, with its own figure below.
A store, managed or not, still does not make the whole infrastructure, which is the point of Why feature store is not enough. Almog Baku's "Effective AI Infrastructure or Why Feature Store Is Not Enough" argues that modern AI infrastructure could accelerate the ML lifecycle and ease the interaction between data scientists and engineers. Why feature store is not enough. https://towardsdatascience.com/effective-ai-infrastructure-or-why-feature-store-is-not-enough-43bc2d803401
Deprecated links
These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.
- **The importance of having one -** medium. This address no longer opens: https://towardsdatascience.com/the-importance-of-having-a-feature-store-e2a9cfa5619f
Last updated
Was this helpful?