Data Mesh: Game Changer or Just Hot Air?

Data mesh is a hot buzzword, but what is behind it? Regardless of how one feels about it, the concept addresses some highly relevant areas. It is about a focus on data products and the decentralized creation of and responsibility for data products in the line of business, which are based on a self-service analytics infrastructure and federated data governance with a high degree of automation.

But does data mesh really address the central pains that many companies are struggling with? Are the principles of data mesh suitable for solving these challenges in practice? What functional, technical and organizational measures are companies applying and how far have they come? Are there special demands on the technology?

This study examines how companies view data mesh and how relevant the concept is in terms of their data democratization journey. It explores the organizational and technical directions enterprises are currently choosing and the experience they have gained so far. We examine how consistently data mesh advocates act and how that differs from the actions of those who do not consider data mesh relevant. We also consider data intelligence in the context of data mesh.

It has clearly been confirmed that all the pillars of data mesh are highly relevant to most organizations and are being implemented by many, regardless of whether they are called data mesh or not. However, there is no one-size-fits-all approach. The data mesh toolbox must be adapted to the reality of the company and supplemented by other measures. Each company must plot its own individual data democratization journey – it won’t be found in the data mesh book.

Management summary

Should business domains own data products?
Executives in particular believe that centralized data teams will not meet future data & analytics requirements. They do not scale sufficiently and lack domain expertise. In practice, the shift towards more data ownership in business domains is well underway. Ownership of reports/dashboards and ad hoc and visual analytics already lies primarily in the business domain. In terms of data preparation, the responsibility currently lies mainly with technical users – but there is also a notable ownership share in the business, especially for consumer-oriented and ML/AI-oriented data products.
To become a more data-driven enterprise, you need a shift towards more decentralized data ownership in the business domains. A decentralized model ensures scalability and thus flexibility and agility. In addition, domain expertise creates even better data products that meet business requirements. In this scenario, the central data teams take on a new, very important role. They support the business domains as advisors, and ensure the availability and stability of the data & analytics platform.
What sharing data as a product implies
When data is a revenue driver, it needs to be treated as an asset. The most important measure to improve the quality of data & analytics assets in the enterprise is to treat data as a product. This should apply not only to analytical data, but above all to operational source data. Continuous monitoring helps ensure the targeted improvement of data assets. However, these conclusions are not reflected in practice.
Handle your data as you handle your products. Data products, like any product, should be valuable, feasible and desirable. They need to adhere to a set of usability characteristics determined by data product consumers. Finding the right balance requires monitoring and targeted improvement driven by business priorities. Therefore, place the responsibility for the appropriate alignment of data products in the hands of the business units.
Does self-service empowerment require a central platform?
It is undisputed that business users need to be empowered to perform their data & analytics tasks through a self-serve data platform. However, there is no one-size-fits-all self-service offering. For most users, elementary analytical functions such as report/dashboard authoring, data visualization and ad hoc analysis are very important. Data & analytics experts place more value on advanced analytics and their integration into operational systems than the average user. In doing so, they obviously aim to generate more value from analytics than just with passive numbers.
Shape your self-serve data platform according to the needs of different target groups. Casual users need to be supported differently than data-savvy users. The latter are increasingly demanding to be empowered in the direction of data preparation and advanced analytics. These divergent needs do not necessarily have to be addressed in a single platform. Plan your platform and architecture in such a way that you can make data assets from different platforms available and linkable as needed.
Federated data governance on the rise
When data & analytics are used extensively in every business domain, a purely centralized governance concept threatens to reach its limits. While the chosen approaches to data governance are diverse, central and federated approaches are currently much more popular and successful than local ones. Process and data experts from business units as well as managers are more satisfied with the results of their measures than participants from central data & analytics teams or IT in general.
Whether centralized or federated, choose a data governance approach that fits current requirements, governance maturity and corporate culture. Avoid purely local approaches, however, as these are less successful according to our study. Be aware that your data governance approach will need to evolve over time, as 77 percent of participants confirm. With increasingly distributed data & analytics utilization across business units, a more federated approach is advisable.
Is data mesh understood and appreciated?
Despite the relevance of data mesh in general, not all organizations seem to consistently adhere to all four data mesh principles. Only a few of the companies that are already implementing a data mesh also want to shift responsibility for data assets to domain teams in business. On the other hand, most companies clearly see the connection between applying product thinking to data and data mesh. In terms of platform design, no clear preference can be discerned among the data mesh implementers. However, most of our study participants are convinced that data & analytics should be organized in a much more distributed manner in the future.
Independent of the discussion around data mesh, the concept addresses some highly relevant areas. The principles are currently being adopted by many organizations, regardless of whether they are called data mesh or not. Be inspired by this and evaluate their applicability to the current challenges of your data & analytics landscape and organization. Check which measures are currently realistic for you, because not all of them can be implemented equally well and equally quickly. Monitor the selected steps so you can show their benefits. Investments in data & analytics should pay off, and this should be transparent.
Data intelligence: the key to success
80 percent of study participants are planning a data intelligence initiative or are already implementing one. Data intelligence goes beyond the mere cataloging of data assets. It leverages metadata to ensure efficient and effective collaboration between data product owners and data consumers. Platforms that support data intelligence provide the source for the federated computational governance propagated in the data mesh concept.
The democratization of data will only succeed if access to and understanding of data assets is simplified. Data citizens need to be able to search for assets and find information about their origin and nature. Enable them to share experiences with other data workers. A rating system will help to make the benefits of data measurable. Consider implementing a data intelligence platform that supports data provisioning and use, and enables streamlined monitoring and governance.