From governance to mesh

Data is consolidated as the core of organizations: data-driven models are revolutionizing the way decisions are made in companies and are advancing hand in hand with the acceleration of digital transformation, the maturity of analytical solutions supported by artificial intelligence and the increasing number of use cases related to big data.

This approach makes concepts such as data governance more important: a system of decisions and responsibilities that aims to promote the management and quality of data in aspects such as availability (accessible whenever they are needed and available to whoever requires them), integrity (consistency throughout the different systems of the entire organization), usability (ease of extracting value from them) and security (only authorized people can access them).

When does the existence of data governance become relevant in an organization? When the size of the company, the complexity of the IT outlook, the criticality or the volume of data… make traditional data management impossible.

Plans and procedures

A data governance program includes something called a «governance council», i.e. a group of leaders who carry out the strategy and who will have different responsibilities in terms of monitoring, controlling and updating the data.

It also includes a set of procedures and an implementation plan that defines how data will be stored, archived, backed up and, of course, how data will be protected against incidents (accidental or as a result of attacks), as well as disaster recovery strategies.

At the same level, procedures are usually established regarding who can access each set of data, how, when, by what methods and what kind of use they can make of it. Controls and audits are also defined to ensure compliance with all the rules specified in this governance plan.

New architectural paradigms

The new challenges posed by data have led to the emergence of new architectures to support data-driven strategies. This is the case of data mesh: a self-service model that allows the use of resources and tools on demand to process, prepare and analyze the data necessary to fulfill a task at the level of each area, team or department.

The main objective of this model is to move towards a democratization of data access: avoiding gaps between those who manage data and those who use it, eliminating intermediaries and offering a unique 360º visualization of its assets.

Among the guiding principles of data mesh architecture is the need to establish federated data governance: a balance between general governance policies and controls, to ensure quality, privacy and compliance with policies and regulations, and flexibility on the user side, to provide agility and scalability.

Data fabric versus data mesh

Another architecture aimed at easing data management in distributed, diverse and complex environments is data fabric. Like data mesh, it provides management capabilities that enable single visibility of the entire data set regardless of whether it is located in the cloud, on premise or at the edge of the network.

It stands out for its ability to connect storage spaces, types and sources of data with the appropriate methods to access them in an accurate and timely manner, regardless of the process that needs them, the environment from which they are required or the use to which they will be put.

The difference between the two architectures is based on how they approach the problem: data fabric creates a single virtual management layer, while data mesh allows different teams to manage data according to their specific needs, while respecting certain common governance elements.

The paths chosen may vary, have certain details or differ according to the type of organization, but the destination is always the same: organizations must find the mechanisms to move towards data-driven models.