01
Data governance in data assets
As data resources are raised to the level of data assets, data governance becomes a set of processes and technologies to ensure effective management and utilization of data assets, while the data asset catalog is a repository that contains holographic description information of enterprise data assets and serves as an effective management A (logical) single source of truth for data assets. Analysts and data scientists in the organization effectively use the data asset catalog to answer business questions, and data governance specialists implement data governance policies through the data asset catalog and promote the correct use of data.
Data assets released through the asset catalog achieve asset certification through the following data governance capabilities:
- Data quality assurance: Data assets are valid data under effective data quality monitoring. Data governance measurement rules are established through business rule inheritance, business usage requirements, etc. to ensure that when users use data or when data pipelines transfer data, the system Quality issues are promptly alerted to data analysts so that data quality can be assessed for its usability for data analysis.
- Authoritative source certification: The data asset catalog helps us identify which data sets are the authoritative source of data, publish assets through certification, and track changes in data ownership and certification over time.
- Data classification and grading: Data security governance requires data security classification and grading based on data sensitivity, PII and other key metadata. Data security level is the basis for how data assets are shared and circulated, and is an essential attribute of data assets.
- Data lineage: Before using a data set, analysts must first understand the source of the underlying data. The data lineage diagram is a visual display of data sources. It establishes a complete data processing flow description for data integration and can help users determine whether the data has the correct information to help answer specific business questions.
- Metrics and Standards: If an organization does not have a consistent set of definitions of key enterprise metrics and business attributes, then over time different analysts will invariably measure the same metric using a different set of rules. This inconsistency leaves businesses with a conflicting set of analytics and leads to a lack of trust in the data.
- Other important information: Usage statistics are collected from the underlying BI tool and presented to users in the data governance tool. These statistics identify the extent to which each data set is used by business users, and it is up to business users to determine which data sets are being used by the user base and which data sets have yet to find business applications.
02
Data assets and enterprise-level BI
The rich business metadata provided by a data asset catalog is invaluable to data analysts and data scientists as they can understand more contextual information about the data and decide which existing assets to use in their analyses. However, this tool alone cannot fully meet an organization’s complete governance needs because they cannot support the needs of all data consumers in the enterprise. Typical business users do not use data catalog tools as part of their daily work. BI and analytics tools on the market often do not integrate effectively with data assets, and users do not benefit from the vast amount of information contained within them. As a result, many organizations struggle to realize business value from the substantial ongoing investment required to maintain governance data in these tools.
On the other hand, with the development of data management, enterprise-level BI has become a requirement for enterprise data management. More and more enterprises require data analysis to be carried out in a context where the data is safe, reliable, manageable and controllable:
- Data Security and Compliance: Managed BI solutions include strong data security measures to ensure the protection of sensitive data. This helps ensure that corporate data is not accessed by unauthorized persons. In addition, it also helps ensure that enterprises comply with regulatory and compliance requirements, such as data security laws, personal protection laws, etc.
- Data integration and quality control: Managed enterprise-wide BI solutions typically include data integration and quality control capabilities to ensure data consistency and accuracy. This helps reduce data errors and redundancy and improves data trustworthiness.
- User permissions and access controls: Managed enterprise-wide BI solutions allow enterprise administrators to configure user permissions and access controls to ensure that only authorized personnel have access to specific data and reports. This helps protect the security of your data.
To sum up, data asset governance and BI visual analysis are complementary to each other. Combining the two can make BI’s self-service capabilities stronger and benefit more business users. At the same time, it also allows data governance to be targeted and data assets are available on the ground, giving full play to the explicit value of data governance.
Let me share my exploration of Datablau below.
03
Data analysis and governance integrated solution
A Data & Analytics (D&A) governance program is an internal framework and policy used to ensure that data and analytics activities are effectively managed, protected and leveraged within the organization. A robust D&A governance program helps ensure the quality, compliance, security and availability of data to support decision making and business operations.
The entire solution involves products and tools, and mainly reaches the following points:
3.1 Unified data view
The unified cataloging of data assets can classify data according to business architectural relationships or analysis topics, making it very convenient for users to find useful data. The metadata usually collected from the database in BI tools is technical metadata without a business perspective. Business users need to classify and complete the data with the help of technical personnel. When this occurs at an enterprise level, it is very important. Enterprise-wide analysis creates great obstacles and is not conducive to data-driven data analysis.
In our products, through the BI interface, we write the business semantics of metadata and other information into the BI data set, and synchronize the data directory and data permission information to the user’s perspective. This is a very good experience for the end user. , which is also the way data governance organizations should be empowered.
(Take FineBI as an example)
3.2 Unified data permissions
Data security and compliance are key requirements for enterprise-level data management. In the definition of data assets, the information of stakeholders such as data owners, technical managers, and participants is improved. The security categories and levels of data are also defined. Finally, we need to define data access policies and authorization systems between data and organizations, which enables standardized circulation and sharing of data while being audited and monitored by the security system.
Traditional BI applications all use topic marts, which are a distributed department-based data usage model. Under this model, data authorization and copying are difficult to trace.
Most of the current enterprise data authorization systems are based on the electronic flow of permissions. This can still work when the data is relatively small, but once there is too much data to be implemented by the authorization department, we may be forced to relax or even let go of the data. Management of permissions. This has happened to many companies in the past.
Based on these pain points, Datablau released a solution for unified data access based on the enterprise’s job rights system.
(Data authorization and access based on position rights system)
In this solution, an individual’s access to data is entirely determined by their position. The data permission granularity is down to the row level and column level, and RBAC granular permissions are bound according to the authorization of the position. Finally, data access is completely controlled by the data gateway.
The advantage of this solution is that it is simple to manage and integrated into the job system. The end user is unaware, and permission constraints are completed by the data gateway.
(data gateway technology architecture)
3.3 Establish usable data asset development process
Availability of data assets is an important indicator to maintain the vitality of data assets. The industry has carried out a lot of data asset inventory work in the past and integrated definitions of data business entities (see Huawei L3-L4 entity definitions), which is important for promoting business understanding and management of data and business-oriented connection of data. Great effect. However, this is still not enough for the integrated operation of data assets and enterprise BI in this article.
The main problem with this work is that the data assets inventoried are preliminary products, and a lot of work is still needed before they can be delivered.
In our practice, the logical layer inventory of data assets and data delivery are unified to ensure that the data assets released to BI are applicable data and are specially managed.
By classifying data assets, we divide data assets into physical state, logical state, and deliverable state. By integrating data assets and BI data delivery into one system, users are better served. This is also how we practice the concept of active data governance and unleash the value of data governance.
04
Summarize
BI tools are the most important data analysis tools for our business department. Through this integration solution, data assets are empowered, which is more conducive to improving the data analysis capabilities of the business department. At the same time, this is also a very good opportunity for data governance. The active governance method that integrates governance with application makes the value of data governance explicit and improves the business connectivity of the organization.
Datablau’s product matrix and solutions provide support for the above solutions. They have been verified in several cases and achieved good results. I hope it can be of reference to you.
picture