Data Mesh

A central data team can become the bottleneck, so mesh treats domains as owners of data products. The page goes from what a data mesh is, through Zhamak Dehghani's own introductions and principles, to comparisons with lakes and fabric, topologies, and the practice notes of vendors and teams who built one.

Mesh is an architecture choice and a sibling of fabric, so it is worth reading with both. The same notes are in Data Architecture and Data Fabric.

The first question is what a data mesh is, and in practice: the Databricks session on how Europe's leading online fashion platform went beyond the data lake.

The idea is Zhamak Dehghani's, so her own material comes next. Introduction to Data Mesh is her Thoughtworks introduction. how to move from data lake to distributed data mesh is the article that names the problems with the centralized data lake and says a data mesh needs domains, self-service platforms, and product thinking. principles and logical architecture is the follow-up with the four principles that drive a mesh's logical architecture. The (good) Keynote - Data Mesh by Zhamak Dehghani makes the core claim that the pipeline of oltp→etl→olap is broken. lessons from the trenches is Zhamak Dehghani and Sina Jahan on what they learned implementing it.

With the principles in hand, the next step is placing mesh next to its neighbors. lake vs mesh, he probably means fabric vs mesh is the data lakehouse vs data mesh comparison, read with that correction in mind. data mesh 101 — by Confluent is the hands-on episode on creating a data product. The (good) mesh topologies — by Piethein Strengholt covers topologies and domain granularity, drawn from his customer conversations and architecture sessions at Microsoft where mesh kept coming up. The figure below is from it.

mesh topologies

The remaining sources are practice. Data Mesh & MLOps using a data platform. ties the mesh to MLOps through a data platform. Explained is an in-depth explainer written by someone learning data mesh from scratch and sharing it. The next gen of architecture argues that architecture defines the efficiency, scalability, and usability of data, and presents mesh as the transition to the next generation. how not to mesh it - monte carlo is Monte Carlo's beginner's guide to implementing a mesh without messing it up. building data mesh using lake house approach by AWS is the tech talk on building a mesh architecture with AWS Lake Formation. The enterprise data mesh by oracle is Oracle's ebook on the enterprise version. deconstructing data mesh principles closes by taking the principles apart again.

Deprecated links

These links and images no longer work. The original wording is kept here. A same-resource copy, when one was checked, is used above.

• (good) mesh topologies - by Piethein Strengholt. This address no longer opens: https://towardsdatascience.com/data-mesh-topologies-and-domain-granularity-65290a4ebb90