Data architecture
Encyclopedia
Data Architecture in enterprise architecture
is the design of data for use in defining the target state and the subsequent planning needed to achieve the target state. It is usually one of several architecture domain
s that form the pillars of an enterprise architecture or solution architecture
.
s used by a business and/or its applications. There are descriptions of data in storage and data in motion; descriptions of data stores, data groups and data items; and mappings of those data artifacts to data qualities, applications, locations etc.
Essential to realizing the target state, Data Architecture describes how data is processed, stored, and utilized in a given system. It provides criteria for data processing operations that make it possible to design data flows and also control the flow of data in the system.
The Data Architect is typically responsible for defining the target state, aligning during development and then following up to ensure enhancements are done in the spirit of the original blueprint.
During the definition of the target state, the Data Architecture breaks a subject down to the atomic level and then builds it back up to the desired form. The Data Architect breaks the subject down by going through 3 traditional architectural processes:
The "data" column of the Zachman Framework
for enterprise architecture –
In this second, broader sense, data architecture includes a complete analysis of the relationships between an organization's functions, available technologies, and data type
s.
Data architecture should be defined in the planning phase of the design of a new data processing and storage system. The major types and sources of data necessary to support an enterprise should be identified in a manner that is complete, consistent, and understandable. The primary requirement at this stage is to define all of the relevant data entities, not to specify computer
hardware
items. A data entity is any real or abstracted thing about which an organization or individual wishes to store data.
. Physical data architecture encompasses database architecture. Database architecture is a schema of the actual database technology that will support the designed data architecture.
that will support common data operations (i.e. emergency procedures, data imports, data backups, external transfers of data).
Without the guidance of a properly implemented data architecture design, common data operations might be implemented in different ways, rendering it difficult to understand and control the flow of data within such systems. This sort of fragmentation is highly undesirable due to the potential increased cost, and the data disconnects involved. These sorts of difficulties may be encountered with rapidly growing enterprises and also enterprises that service different lines of business
(e.g. insurance
products
).
Properly executed, the data architecture phase of information system planning forces an organization to specify and delineate both internal and external information flows. These are patterns that the organization may not have previously taken the time to conceptualize. It is therefore possible at this stage to identify costly information shortfalls, disconnects between departments, and disconnects between organizational systems that may not have been evident before the data architecture analysis.
Enterprise requirements: These will generally include such elements as economical and effective system expansion, acceptable performance levels (especially system access speed), transaction
reliability, and transparent management of data. In addition, the conversion
of raw data such as transaction records
and image
files
into more useful information
forms through such features as data warehouse
s is also a common organizational requirement
, since this enables managerial decision making and other organizational processes. One of the architecture techniques is the split between managing transaction data
and (master) reference data
. Another one is splitting data capture systems from data retrieval systems (as done in a data warehouse).
Technology drivers: These are usually suggested by the completed data architecture and database architecture designs. In addition, some technology drivers will derive from existing organizational integration frameworks and standards, organizational economics, and existing site resources (e.g. previously purchased software licensing).
Economics: These are also important factors that must be considered during the data architecture phase. It is possible that some solutions, while optimal in principle, may not be potential candidates due to their cost. External factors such as the business cycle
, interest rates, market conditions, and legal considerations could all have an effect on decisions relevant to data architecture.
Business policies: Business policies that also drive data architecture design include internal organizational policies, rules of regulatory bodies, professional standards, and applicable governmental laws that can vary by applicable agency
. These policies and rules will help describe the manner in which enterprise wishes to process their data.
Data processing needs: These include accurate and reproducible transactions performed in high volumes, data warehousing for the support of management information systems (and potential data mining
), repetitive periodic reporting, ad hoc reporting, and support of various organizational initiatives as required (i.e. annual budgets, new product
development).
Enterprise architecture
An enterprise architecture is a rigorous description of the structure of an enterprise, which comprises enterprise components , the externally visible properties of those components, and the relationships between them...
is the design of data for use in defining the target state and the subsequent planning needed to achieve the target state. It is usually one of several architecture domain
Architecture domain
An architecture domain in enterprise architecture is a broad view of an enterprise or system. It is a partial representation of a whole system that addresses several concerns of several stakeholders...
s that form the pillars of an enterprise architecture or solution architecture
Solution architecture
Solution architecture in enterprise architecture is a kind of architecture domain, that aims to address specific problems and requirements, usually through the design of specific information systems or applications.Solution architecture is either:...
.
Overview
A data architecture describes the architecture of the data structureData structure
In computer science, a data structure is a particular way of storing and organizing data in a computer so that it can be used efficiently.Different kinds of data structures are suited to different kinds of applications, and some are highly specialized to specific tasks...
s used by a business and/or its applications. There are descriptions of data in storage and data in motion; descriptions of data stores, data groups and data items; and mappings of those data artifacts to data qualities, applications, locations etc.
Essential to realizing the target state, Data Architecture describes how data is processed, stored, and utilized in a given system. It provides criteria for data processing operations that make it possible to design data flows and also control the flow of data in the system.
The Data Architect is typically responsible for defining the target state, aligning during development and then following up to ensure enhancements are done in the spirit of the original blueprint.
During the definition of the target state, the Data Architecture breaks a subject down to the atomic level and then builds it back up to the desired form. The Data Architect breaks the subject down by going through 3 traditional architectural processes:
- Conceptual - represents all business entities.
- Logical - represents the logic of how entities are related.
- Physical - the realization of the data mechanisms for a specific type of functionality.
The "data" column of the Zachman Framework
Zachman framework
The Zachman Framework is an Enterprise Architecture framework for enterprise architecture, which provides a formal and highly structured way of viewing and defining an enterprise...
for enterprise architecture –
| Layer | View | Data (What) | Stakeholder | |
| 1 | Scope/Contextual | List of things important to the business (subject areas) | Planner | |
| 2 | Business Model/Conceptual | Semantic model or Conceptual Entity-relationship model In software engineering, an entity-relationship model is an abstract and conceptual representation of data. Entity-relationship modeling is a database modeling method, used to produce a type of conceptual schema or semantic data model of a system, often a relational database, and its requirements... /Enterprise Data Model |
Owner | |
| 3 | System Model/Logical | Enterprise/Logical Data Model Logical data model A logical data model in systems engineering is a representation of an organization's data, organized in terms of entities and relationships and is independent of any particular data management technology.- Overview :... |
Designer | |
| 4 | Technology Model/Physical | Physical Data Model Physical data model A physical data model is a representation of a data design which takes into account the facilities and constraints of a given database management system. In the lifecycle of a project it is typically derived from a logical data model, though it may be reverse-engineered from a given database... |
Builder | |
| 5 | Detailed Representations/ out-of-context | Data Definition | Subcontractor |
In this second, broader sense, data architecture includes a complete analysis of the relationships between an organization's functions, available technologies, and data type
Data type
In computer programming, a data type is a classification identifying one of various types of data, such as floating-point, integer, or Boolean, that determines the possible values for that type; the operations that can be done on values of that type; the meaning of the data; and the way values of...
s.
Data architecture should be defined in the planning phase of the design of a new data processing and storage system. The major types and sources of data necessary to support an enterprise should be identified in a manner that is complete, consistent, and understandable. The primary requirement at this stage is to define all of the relevant data entities, not to specify computer
Computer
A computer is a programmable machine designed to sequentially and automatically carry out a sequence of arithmetic or logical operations. The particular sequence of operations can be changed readily, allowing the computer to solve more than one kind of problem...
hardware
Hardware
Hardware is a general term for equipment such as keys, locks, hinges, latches, handles, wire, chains, plumbing supplies, tools, utensils, cutlery and machine parts. Household hardware is typically sold in hardware stores....
items. A data entity is any real or abstracted thing about which an organization or individual wishes to store data.
Physical data architecture
Physical data architecture of an information system is part of a technology plan. As its name implies, the technology plan is focused on the actual tangible elements to be used in the implementation of the data architecture designDesign
Design as a noun informally refers to a plan or convention for the construction of an object or a system while “to design” refers to making this plan...
. Physical data architecture encompasses database architecture. Database architecture is a schema of the actual database technology that will support the designed data architecture.
Elements of data architecture
There are certain elements that must be defined as the data architecture schema of an organization is designed. For example, the administrative structure that will be established in order to manage the data resources must be described. Also, the methodologies that will be employed to store the data must be defined. In addition, a description of the database technology to be employed must be generated, as well as a description of the processes that will manipulate the data. It is also important to design interfaces to the data by other systems, as well as a design for the infrastructureInfrastructure
Infrastructure is basic physical and organizational structures needed for the operation of a society or enterprise, or the services and facilities necessary for an economy to function...
that will support common data operations (i.e. emergency procedures, data imports, data backups, external transfers of data).
Without the guidance of a properly implemented data architecture design, common data operations might be implemented in different ways, rendering it difficult to understand and control the flow of data within such systems. This sort of fragmentation is highly undesirable due to the potential increased cost, and the data disconnects involved. These sorts of difficulties may be encountered with rapidly growing enterprises and also enterprises that service different lines of business
Business
A business is an organization engaged in the trade of goods, services, or both to consumers. Businesses are predominant in capitalist economies, where most of them are privately owned and administered to earn profit to increase the wealth of their owners. Businesses may also be not-for-profit...
(e.g. insurance
Insurance
In law and economics, insurance is a form of risk management primarily used to hedge against the risk of a contingent, uncertain loss. Insurance is defined as the equitable transfer of the risk of a loss, from one entity to another, in exchange for payment. An insurer is a company selling the...
products
Product (business)
In general, the product is defined as a "thing produced by labor or effort" or the "result of an act or a process", and stems from the verb produce, from the Latin prōdūce ' lead or bring forth'. Since 1575, the word "product" has referred to anything produced...
).
Properly executed, the data architecture phase of information system planning forces an organization to specify and delineate both internal and external information flows. These are patterns that the organization may not have previously taken the time to conceptualize. It is therefore possible at this stage to identify costly information shortfalls, disconnects between departments, and disconnects between organizational systems that may not have been evident before the data architecture analysis.
Constraints and influences
Various constraints and influences will have an effect on data architecture design. These include enterprise requirements, technology drivers, economics, business policies and data processing needs.Enterprise requirements: These will generally include such elements as economical and effective system expansion, acceptable performance levels (especially system access speed), transaction
Financial transaction
A financial transaction is an event or condition under the contract between a buyer and a seller to exchange an asset for payment. It involves a change in the status of the finances of two or more businesses or individuals.-History:...
reliability, and transparent management of data. In addition, the conversion
Data conversion
Data conversion is the conversion of computer data from one format to another. Throughout a computer environment, data is encoded in a variety of ways. For example, computer hardware is built on the basis of certain standards, which requires that data contains, for example, parity bit checks....
of raw data such as transaction records
Record (computer science)
In computer science, a record is an instance of a product of primitive data types called a tuple. In C it is the compound data in a struct. Records are among the simplest data structures. A record is a value that contains other values, typically in fixed number and sequence and typically indexed...
and image
Image
An image is an artifact, for example a two-dimensional picture, that has a similar appearance to some subject—usually a physical object or a person.-Characteristics:...
files
Computer file
A computer file is a block of arbitrary information, or resource for storing information, which is available to a computer program and is usually based on some kind of durable storage. A file is durable in the sense that it remains available for programs to use after the current program has finished...
into more useful information
Information
Information in its most restricted technical sense is a message or collection of messages that consists of an ordered sequence of symbols, or it is the meaning that can be interpreted from such a message or collection of messages. Information can be recorded or transmitted. It can be recorded as...
forms through such features as data warehouse
Data warehouse
In computing, a data warehouse is a database used for reporting and analysis. The data stored in the warehouse is uploaded from the operational systems. The data may pass through an operational data store for additional operations before it is used in the DW for reporting.A data warehouse...
s is also a common organizational requirement
Requirement
In engineering, a requirement is a singular documented physical and functional need that a particular product or service must be or perform. It is most commonly used in a formal sense in systems engineering, software engineering, or enterprise engineering...
, since this enables managerial decision making and other organizational processes. One of the architecture techniques is the split between managing transaction data
Transaction data
Transaction data are data describing an event and is usually described with verbs. Transaction data always has a time dimension, a numerical value and refers to one or more objects Transaction data are data describing an event (the change as a result of a transaction) and is usually described with...
and (master) reference data
Reference data
Reference data are data describing a physical or virtual object and its properties. Reference data are usually described with nouns.Reference data is used in data management to define characteristics of an identifier that are used within other data centric processes...
. Another one is splitting data capture systems from data retrieval systems (as done in a data warehouse).
Technology drivers: These are usually suggested by the completed data architecture and database architecture designs. In addition, some technology drivers will derive from existing organizational integration frameworks and standards, organizational economics, and existing site resources (e.g. previously purchased software licensing).
Economics: These are also important factors that must be considered during the data architecture phase. It is possible that some solutions, while optimal in principle, may not be potential candidates due to their cost. External factors such as the business cycle
Business cycle
The term business cycle refers to economy-wide fluctuations in production or economic activity over several months or years...
, interest rates, market conditions, and legal considerations could all have an effect on decisions relevant to data architecture.
Business policies: Business policies that also drive data architecture design include internal organizational policies, rules of regulatory bodies, professional standards, and applicable governmental laws that can vary by applicable agency
Government agency
A government or state agency is a permanent or semi-permanent organization in the machinery of government that is responsible for the oversight and administration of specific functions, such as an intelligence agency. There is a notable variety of agency types...
. These policies and rules will help describe the manner in which enterprise wishes to process their data.
Data processing needs: These include accurate and reproducible transactions performed in high volumes, data warehousing for the support of management information systems (and potential data mining
Mining
Mining is the extraction of valuable minerals or other geological materials from the earth, from an ore body, vein or seam. The term also includes the removal of soil. Materials recovered by mining include base metals, precious metals, iron, uranium, coal, diamonds, limestone, oil shale, rock...
), repetitive periodic reporting, ad hoc reporting, and support of various organizational initiatives as required (i.e. annual budgets, new product
Product (business)
In general, the product is defined as a "thing produced by labor or effort" or the "result of an act or a process", and stems from the verb produce, from the Latin prōdūce ' lead or bring forth'. Since 1575, the word "product" has referred to anything produced...
development).
See also
- Enterprise Information Security ArchitectureEnterprise Information Security ArchitectureEnterprise information security architecture is a part of enterprise architecture focusing on information security throughout the enterprise.- Overview :...
- (EISA) positions data security in the enterprise information framework. - FDIC Enterprise Architecture FrameworkFDIC Enterprise Architecture FrameworkFDIC Enterprise Architecture Framework is the Enterprise Architecture framework of the United States Federal Deposit Insurance Corporation...
- Controlled vocabularyControlled vocabularyControlled vocabularies provide a way to organize knowledge for subsequent retrieval. They are used in subject indexing schemes, subject headings, thesauri, taxonomies and other form of knowledge organization systems...
Further reading
- Bass, L.; John, B.; & Kates, J. (2001). Achieving Usability Through Software Architecture, Carnegie Mellon University.
- Lewis, G.; Comella-Dorda, S.; Place, P.; Plakosh, D.; & Seacord, R., (2001). Enterprise Information System Data Architecture Guide Carnegie Mellon University.
- Adleman, S.; Moss, L.; Abai, M. (2005). Data Strategy Addison-Wesley Professional.
External links
- Achieving Usability Through Software Architecture, sei.cmu.edu 2001
- The Logical Data Architecture, by Nirmal Baid