Holistic Data Management
Encyclopedia
Holistic Data Management (HDM) framework is AHISDATA indigenous standard for implementing software implementations within an organization network. This framework extends the existing data management solutions such as data quality
, data governance
, data integration
, data processing
, master data management
and data validation
solutions.
The HDM framework specifies that:
Data mapping, data validation, data integration
,data processing
The data network scope (DNS) is the logical boundary that a software application database system of record (SOR) exists within an enterprise network. There can be multiple DNS within an enterprise network.
The data network domain (DND) is the logical boundary representing a collection of multiple data network scope (DNS). There can be multiple DND within an enterprise network.
A system of record applies to the master or principal database system that a parent data objects resides on.
There can only be one SOR within a data network scope.
A parent data object (PDO) is the system of record schema object name. Only one unique parent data object must exist within a data network scope.
A child data object (CDO) is a schema object name that derives its data from one or more parent data object(s).
A data-mapping link (DML) is the data requirement specification applied to the relationship between multiple database schema objects where one data object derives its data from one or more data objects. DML is only applicable for a data-mapping data management module.
The data–object relationship (DOR) is the data requirement, business rule, program function that applies to one or multiple data objects. DOR can be applied on data-mapping links for each data management modules. Only one DOR can exist on a DML within a data management module.
Data management modules are the common user interface (UI) programs that defines and manage the data object relationship(s) within a data network scope.
There are four data management modules:
Data mapping – This is the base data management user interface module. The data-mapping module provide the functionalities for managing data-mapping links and data object relationships for all database schemas within a data network scope. A data network scope must have at least one data-mapping design defined.
Data validation – This user interface module provides the functionalities for defining and managing validation events on data object relationships. Validation events include auditing, reporting, scheduler, logger, triggers and DNS health check. Data validation events requires a data-mapping design defined within a data network scope.
Data integration
– This user interface module provides the functionalities for defining and managing interface configurations on data object relationships. The interface configurations include scheduler, transmission mode, listener, interface API and reporting. The interface APIs would allow third-party systems to transfer data using the data object relationship defined within a data network scope. Data Integration interface configuration requires a Data Mapping design defined within a data network scope.
Data processing
– This user interface module provides the functionalities for defining and managing interface configurations and batch runtime engines on data object relationships. The interface configurations include scheduler, transmission mode, multi-batch transmission, user-defined DOR API and reporting. Data Processing interface configuration requires a data-mapping design defined within a data network scope.
The HDM framework should be implemented as a major organization project that is supervised by the project management office. This would require a project charter developed and a project manager assigned for managing the implementation process. There are several phases involve in implementing the HDM framework:
Data quality
Data are of high quality "if they are fit for their intended uses in operations, decision making and planning" . Alternatively, the data are deemed of high quality if they correctly represent the real-world construct to which they refer...
, data governance
Data governance
Data governance is an emerging discipline with an evolving definition. The discipline embodies a convergence of data quality, data management, data policies, business process management, and risk management surrounding the handling of data in an organization...
, data integration
Data integration
Data integration involves combining data residing in different sources and providing users with a unified view of these data.This process becomes significant in a variety of situations, which include both commercial and scientific domains...
, data processing
Data processing
Computer data processing is any process that a computer program does to enter data and summarise, analyse or otherwise convert data into usable information. The process may be automated and run on a computer. It involves recording, analysing, sorting, summarising, calculating, disseminating and...
, master data management
Master Data Management
In computing, master data management comprises a set of processes and tools that consistently defines and manages the non-transactional data entities of an organization...
and data validation
Data validation
In computer science, data validation is the process of ensuring that a program operates on clean, correct and useful data. It uses routines, often called "validation rules" or "check routines", that check for correctness, meaningfulness, and security of data that are input to the system...
solutions.
The HDM framework specifies that:
- All data objects must exist as a child data object or a parent data object.
- Only one unique parent data object must exist within a data network scope (DNS).
- All child data objects must have a data-mapping link defined within a data network scope.
- A data object relationship must exist at least in one of the following four data management modules:
Data mapping, data validation, data integration
Data integration
Data integration involves combining data residing in different sources and providing users with a unified view of these data.This process becomes significant in a variety of situations, which include both commercial and scientific domains...
,data processing
Data processing
Computer data processing is any process that a computer program does to enter data and summarise, analyse or otherwise convert data into usable information. The process may be automated and run on a computer. It involves recording, analysing, sorting, summarising, calculating, disseminating and...
HDM framework
The following entities are specified in the HDM framework.- Data network scope (DNS)
The data network scope (DNS) is the logical boundary that a software application database system of record (SOR) exists within an enterprise network. There can be multiple DNS within an enterprise network.
- Data network domain (DND)
The data network domain (DND) is the logical boundary representing a collection of multiple data network scope (DNS). There can be multiple DND within an enterprise network.
- System of record (SOR)
A system of record applies to the master or principal database system that a parent data objects resides on.
There can only be one SOR within a data network scope.
- Parent data object (PDO)
A parent data object (PDO) is the system of record schema object name. Only one unique parent data object must exist within a data network scope.
- Child data object (CDO)
A child data object (CDO) is a schema object name that derives its data from one or more parent data object(s).
- Data-mapping link (DML)
A data-mapping link (DML) is the data requirement specification applied to the relationship between multiple database schema objects where one data object derives its data from one or more data objects. DML is only applicable for a data-mapping data management module.
- Data–object relationship (DOR)
The data–object relationship (DOR) is the data requirement, business rule, program function that applies to one or multiple data objects. DOR can be applied on data-mapping links for each data management modules. Only one DOR can exist on a DML within a data management module.
- Data management modules (DMM)
Data management modules are the common user interface (UI) programs that defines and manage the data object relationship(s) within a data network scope.
There are four data management modules:
Data mapping – This is the base data management user interface module. The data-mapping module provide the functionalities for managing data-mapping links and data object relationships for all database schemas within a data network scope. A data network scope must have at least one data-mapping design defined.
Data validation – This user interface module provides the functionalities for defining and managing validation events on data object relationships. Validation events include auditing, reporting, scheduler, logger, triggers and DNS health check. Data validation events requires a data-mapping design defined within a data network scope.
Data integration
Data integration
Data integration involves combining data residing in different sources and providing users with a unified view of these data.This process becomes significant in a variety of situations, which include both commercial and scientific domains...
– This user interface module provides the functionalities for defining and managing interface configurations on data object relationships. The interface configurations include scheduler, transmission mode, listener, interface API and reporting. The interface APIs would allow third-party systems to transfer data using the data object relationship defined within a data network scope. Data Integration interface configuration requires a Data Mapping design defined within a data network scope.
Data processing
Data processing
Computer data processing is any process that a computer program does to enter data and summarise, analyse or otherwise convert data into usable information. The process may be automated and run on a computer. It involves recording, analysing, sorting, summarising, calculating, disseminating and...
– This user interface module provides the functionalities for defining and managing interface configurations and batch runtime engines on data object relationships. The interface configurations include scheduler, transmission mode, multi-batch transmission, user-defined DOR API and reporting. Data Processing interface configuration requires a data-mapping design defined within a data network scope.
Implementing the HDM framework
The HDM framework presents a standard for software implementations within an organization. The objective is to shed visibility, increase efficiency and centralized management of all other software implementations within an organization.The HDM framework should be implemented as a major organization project that is supervised by the project management office. This would require a project charter developed and a project manager assigned for managing the implementation process. There are several phases involve in implementing the HDM framework:
- Choose a data management module (DMM) – This exercise requires the acquisition of a data management module software application to be used for implementing the rest of the HDM framework. AHISDATA iNTEGRITY software is an integrated solution that provides DMM functionalities.
- Scrub (inventory of existing applications and data sources) – This exercise identifies all applications within an organization and the data sources that they are connected to.
- Formation (applications and data schema relation) – This exercise is to align all applications in relation to the data schemas within the data sources. The applications are grouped in the order of the data schemas that they access.
- First axe (applications eligible for decommission) – This exercise is to identify all applications that are rogue, obsolete and completely redundant. These applications are eligible for removal.
- Second axe (application eligible for consolidation) – This exercise is to identify all applications that have some functional similarities and some uniqueness in the data requirement. These applications are eligible for consolidation. The functionalities that are similar are left intact on one application and turned off or disabled on the other(s).
- Define data network domain (DND) – This exercise is to define the data network domain for all the approved applications within the enterprise network.
- Define Data Network Scope (DNS) – This exercise is to define the data network scope(s) required for each DND.
- Define system of records (SOR) – This exercise is to define the SOR for each DNS.
- Define parent data objects (PDO) – This exercise is to define all PDOs in each DNS.
- Define child data objects (CDO) – This exercise is to define all CDOs in each DNS.
- Define data mapping links (DML) – This exercise is to define all data-mapping links and object relationship in all DNS.
- Define data object relationships (DOR) – This exercise is to define the DOR requirement for each data management module implemented.
See also
- Reference dataReference dataReference data are data describing a physical or virtual object and its properties. Reference data are usually described with nouns.Reference data is used in data management to define characteristics of an identifier that are used within other data centric processes...
- Master dataMaster dataMaster data, which may include reference data, is information that is key to the operation of business and is the primary focus of the Information Technology discipline of Master Data Management . This key business information may include data about customers, products, employees, materials,...
- Customer data integrationCustomer Data IntegrationIn data processing, customer data integration combines the technology, processes and services needed to set up and maintain an accurate, timely, complete and comprehensive representation of a customer across multiple channels, business-lines, and enterprises — typically from multiple sources of...
- Product information managementProduct Information ManagementProduct information management or PIM refers to processes and technologies focused on centrally managing information about products, with a focus on the data required to market and sell the products through one or more distribution channels...
- Identity resolutionIdentity resolutionIdentity resolution is an operational intelligence process, typically powered by an identity resolution engine or middleware stack, whereby organizations can connect disparate data sources with a view to understanding possible identity matches and non-obvious relationships across multiple data silos...