IDMS
Encyclopedia
IDMS is primarily a network
(CODASYL
) database management system
for mainframes. It was first developed at B.F. Goodrich
and later marketed by Cullinane Database Systems (renamed Cullinet
in 1983). Since 1989 the product has been owned by Computer Associates (now CA Technologies), who renamed it Advantage CA-IDMS and later simply to CA IDMS.
's IDS (Integrated Data Store
), an early database engine developed at General Electric
.
In the early 1960s IDS was taken from its original form, by the Computer Group of the B.F. Goodrich Chemical Division, and re-written in a language called ISL (Intermediate System Language). ISL was designed as a portable system programming language
able to produce code for a variety of target machines. Since ISL was actually written in its own language (ISL) it was able to be ported to other machine architectures with relative ease, and then to produce code that would execute on them.
The Chemical Division computer group had given some thought to selling copies of IDMS to other companies, but was told by management that they were not in the software products business. Eventually a deal was struck with John Cullinane to buy the rights and market the product.
Because Cullinane was required to remit royalties back to B.F. Goodrich, all add-on products were listed and billed as separate products - even if they were mandatory for the core IDMS product to work. This sometimes confused customers.
The original platforms were the GE 235 computer and GE Datanet 30 message switching computer: later the product was ported to IBM mainframes and to DEC
and ICL hardware.
The IBM-ported version runs on IBM mainframe
systems (System/360
, System/370
, System/390, zSeries
, System z9
). In the mid-1980s, it was claimed that some 2,500 IDMS licenses had been sold. Users included the Strategic Air Command, Ford of Canada, Ford of Europe, Jaguar Cars, Clarks Shoes UK, AXA
/PPP, MAPFRE
, Royal Insurance, Manulife, Hudson's Bay Company, Cleveland Clinic, Bank of Canada and BT
in the UK.
A version for use on the DECSYSTEM series of computers was sold to DEC and was marketed as DBMS10 and later DBMS20.
In 1976 the source code was sold to ICL, who ported the software to run on their 2900 series
mainframes, and subsequently also on the older 1900 range
. ICL continued development of the software independently of Cullinane, selling the original ported product under the original name IDMS and an enhanced version as IDMSX. In this form it was used by many large UK users, an example being the Pay-As-You-Earn system operated by Inland Revenue. Many of these systems are still running in 2005.
In the early to mid 1980s, relational database management system
s started to become more popular, encouraged by increasing hardware power and the move to minicomputer
s and client–server architecture. Relational databases offered improved development productivity over CODASYL
systems, and the traditional objections based on poor performance were steadily diminishing.
Cullinet
attempted to compete against IBM
's DB2
and other relational databases by developing a relational front-end and a range of productivity tools. These included Automatic System Facility (ASF), which made use of a pre-existing IDMS feature called LRF (Logical Record Facility). ASF was a fill-in-the-blanks database generator that would also develop a mini-application to maintain the tables.
It is difficult to judge whether such features may have been successful in extending the selling life of the product, but they made little impact in the long term. Those users who stayed with IDMS were primarily interested in its high performance, not in its relational capabilities. It was widely recognized (helped by a high-profile campaign by E. F. Codd, the father of the relational model
) that there was a significant difference between a relational database and a network database with a relational veneer.
In 1989 Computer Associates continued after Cullinet acquisition with the development and released Release 12.0 with full SQL
in 1992-93.
Nowadays, CA Technologies actively markets and supports the CA IDMS and enhanced IDMS in subsequent releases by TCP/IP support, Two-Phase commit support, XML
publishing, zIIP
specialty processor support, Web-enabled access in combination with CA IDMS Server and GUI database administration via CA IDMS Visual DBA tool.
IDMS legacy systems are still being run today. Few customers have migrated to Cullinet's other database offering IDMS/R.
(IDD). The IDD was primarily developed to maintain database definitions. It was itself an IDMS database.
DBAs (database administrators) and other users interfaced with the IDD using a language called Data Dictionary Definition Language (DDDL).
IDD was also used to store definitions and code for other products in the IDMS family such as ADS/Online and IDMS-DC.
IDD's power was that it was extensible and could be used to create definitions of just about anything. Some companies used it to develop in-house documentation.
network model. The main structuring concepts in this model are records and sets. Records essentially follow the COBOL
pattern, consisting of fields of different types: this allows complex internal structure such as repeating items and repeating groups.
The most distinctive structuring concept in the Codasyl model is the set. Not to be confused with a mathematical set, a Codasyl set represents a one-to-many relationship between records: one owner, many members. The fact that a record can be a member in many different sets is the key factor that distinguishes the network model from the earlier hierarchical model
. As with records, each set belongs to a named set type (different set types model different logical relationships). Sets are in fact ordered, and the sequence of records in a set can be used to convey information. A record can participate as an owner and member of any number of sets.
Records have identity, the identity being represented by a value known as a database key. In IDMS, as in most other Codasyl implementations, the database key is directly related to the physical address of the record on disk. Database keys are also used as pointers to implement sets in the form of linked lists and trees. This close correspondence between the logical model and the physical implementation (which is not a strictly necessary part of the Codasyl model, but was a characteristic of all successful implementations) is responsible for the efficiency of database retrieval, but also makes operations such as database loading and restructuring rather expensive.
Records can be accessed directly by database key, by following set relationships, or by direct access using key values. Initially the only direct access was through hashing, a mechanism known in the Codasyl model as CALC access. In IDMS, CALC access is implemented through an internal set, linking all records that share the same hash value to an owner record that occupies the first few bytes of every disk page.
In subsequent years, some versions of IDMS added the ability to access records using BTree-like indexes.
The DBA allocates a fixed number of pages in a file for each area. The DBA then defines which records are to be stored in each area, and details of how they are to be stored.
IDMS intersperses special space-allocation pages throughout the database. These pages are used to keep track of the free space available in each page in the database. To reduce I/O requirements, the free space is only tracked for all pages when the free space for the area falls below 30%.
Four methods are available for storing records in an IDMS database: Direct, Sequential, CALC, and VIA.
In direct mode the target is specified by the user and is stored as close as possible to that page.
Sequential placement (not to be confused with indexed sequential), simply places each new record at the end of the area. This option is rarely used.
CALC uses a hashing algorithm to decide where to place the record; the hash key then provides efficient retrieval of the record. The entire CALC area is preformatted each with a header consisting of a special CALC "owner" record. The hashing algorithm determines a page number (from which the physical disk address can be determined), and the record is then stored on this page, or as near as possible to it, and is linked to the header record on that page using the CALC set. The CALC records are linked to the page's CALC Owner record using a single link-list (pointers). The CALC Owner located in the page header thus owns the set of all records which target to its particular page (whether the records are stored on that page or, in the case of an overflow, on another page ).
CALC provides extremely efficient storage and retrieval: IDMS can retrieve a CALC record in 1.1 I/O operations. However, the method does not cope well with changes to the value of the primary key, and expensive reorganization is needed if the number of pages needs to be expanded.
VIA placement attempts to store a record near its owner in a particular set. Usually the records are clustered on the same physical page as the owner. This leads to efficient navigation when the record is accessed by following that set relationship. (VIA allows records to be stored in a different IDMS area so that they can be store separately from the owner, yet remain clustered together for efficiency.)
Sets are generally maintained as linked lists, using the database key as a pointer. Every record includes a forwards link to the next record; the database designer can choose whether to include owner pointers and prior pointers (if not provided, navigation in those directions will be slower).
Some versions of IDMS subsequently included the ability to define indexes: either record indexes, allowing records to be located from knowledge of a secondary key, or set indexes, allowing the members of a set to be retrieved by key value.
Network model
The network model is a database model conceived as a flexible way of representing objects and their relationships. Its distinguishing feature is that the schema, viewed as a graph in which object types are nodes and relationship types are arcs, is not restricted to being a hierarchy or lattice.The...
(CODASYL
CODASYL
CODASYL is an acronym for "Conference on Data Systems Languages". This was a consortium formed in 1959 to guide the development of a standard programming language that could be used on many computers...
) database management system
Database management system
A database management system is a software package with computer programs that control the creation, maintenance, and use of a database. It allows organizations to conveniently develop databases for various applications by database administrators and other specialists. A database is an integrated...
for mainframes. It was first developed at B.F. Goodrich
Goodrich Corporation
The Goodrich Corporation , formerly the B.F. Goodrich Company, is an American aerospace manufacturing company based in Charlotte, North Carolina. Founded in Akron, Ohio in 1870 as Goodrich, Tew & Co. by Dr. Benjamin Franklin Goodrich. The company name was changed to the "B.F...
and later marketed by Cullinane Database Systems (renamed Cullinet
Cullinet
Cullinet was a software company whose products included the database management system IDMS and the integrated software package Goldengate. In 1989, the company was bought by Computer Associates...
in 1983). Since 1989 the product has been owned by Computer Associates (now CA Technologies), who renamed it Advantage CA-IDMS and later simply to CA IDMS.
History
The roots of IDMS go back to Dr. Charles BachmanCharles Bachman
Charles William "Charlie" Bachman is an American computer scientist, who spent his entire career as an industrial researcher rather than in academia...
's IDS (Integrated Data Store
Integrated Data Store
Integrated Data Store is a network database largely used by industry for its performance.IDS was designed by Charles Bachman at General Electric in the 1960s. It was not known to be easy to use or implement applications with, because it was designed to maximize performance using the hardware...
), an early database engine developed at General Electric
General Electric
General Electric Company , or GE, is an American multinational conglomerate corporation incorporated in Schenectady, New York and headquartered in Fairfield, Connecticut, United States...
.
In the early 1960s IDS was taken from its original form, by the Computer Group of the B.F. Goodrich Chemical Division, and re-written in a language called ISL (Intermediate System Language). ISL was designed as a portable system programming language
System programming language
System programming languages are programming languages that are statically typed, allow arbitrarily complex data structures, are compiled, and are meant to operate largely independently of other programs. Prototypical system programming languages are C and Modula-2...
able to produce code for a variety of target machines. Since ISL was actually written in its own language (ISL) it was able to be ported to other machine architectures with relative ease, and then to produce code that would execute on them.
The Chemical Division computer group had given some thought to selling copies of IDMS to other companies, but was told by management that they were not in the software products business. Eventually a deal was struck with John Cullinane to buy the rights and market the product.
Because Cullinane was required to remit royalties back to B.F. Goodrich, all add-on products were listed and billed as separate products - even if they were mandatory for the core IDMS product to work. This sometimes confused customers.
The original platforms were the GE 235 computer and GE Datanet 30 message switching computer: later the product was ported to IBM mainframes and to DEC
Digital Equipment Corporation
Digital Equipment Corporation was a major American company in the computer industry and a leading vendor of computer systems, software and peripherals from the 1960s to the 1990s...
and ICL hardware.
The IBM-ported version runs on IBM mainframe
IBM mainframe
IBM mainframes are large computer systems produced by IBM from 1952 to the present. During the 1960s and 1970s, the term mainframe computer was almost synonymous with IBM products due to their marketshare...
systems (System/360
System/360
The IBM System/360 was a mainframe computer system family first announced by IBM on April 7, 1964, and sold between 1964 and 1978. It was the first family of computers designed to cover the complete range of applications, from small to large, both commercial and scientific...
, System/370
System/370
The IBM System/370 was a model range of IBM mainframes announced on June 30, 1970 as the successors to the System/360 family. The series maintained backward compatibility with the S/360, allowing an easy migration path for customers; this, plus improved performance, were the dominant themes of the...
, System/390, zSeries
ZSeries
IBM System z, or earlier IBM eServer zSeries, is a brand name designated by IBM to all its mainframe computers.In 2000, IBM rebranded the existing System/390 to IBM eServer zSeries with the e depicted in IBM's red trademarked symbol, but because no specific machine names were changed for...
, System z9
System z9
IBM System z9 is a line of IBM mainframe. It was announced on July 25, 2005 and the first models were available on September 16, 2005. The System z9 also marks the end of the previously used eServer zSeries naming convention, and it is the last z/Architecture 1 machine.- Background :System z9 is a...
). In the mid-1980s, it was claimed that some 2,500 IDMS licenses had been sold. Users included the Strategic Air Command, Ford of Canada, Ford of Europe, Jaguar Cars, Clarks Shoes UK, AXA
AXA
AXA S.A. is a French global insurance group headquartered in the 8th arrondissement of Paris. AXA is a conglomerate of independently run businesses, operated according to the laws and regulations of many different countries. The AXA group of companies engage in life, health and other forms of...
/PPP, MAPFRE
Mapfre
Mapfre is a Spanish insurance company, based in Majadahonda, Madrid. The name comes from the old mutual origin of the company , but the company now only refers to itself as Mapfre...
, Royal Insurance, Manulife, Hudson's Bay Company, Cleveland Clinic, Bank of Canada and BT
BT Group
BT Group plc is a global telecommunications services company headquartered in London, United Kingdom. It is one of the largest telecommunications services companies in the world and has operations in more than 170 countries. Through its BT Global Services division it is a major supplier of...
in the UK.
A version for use on the DECSYSTEM series of computers was sold to DEC and was marketed as DBMS10 and later DBMS20.
In 1976 the source code was sold to ICL, who ported the software to run on their 2900 series
ICL 2900 Series
The ICL 2900 Series was a range of mainframe computer systems announced by the UK manufacturer ICL on 9 October 1974. The company had started development, under the name "New Range" immediately on its formation in 1968...
mainframes, and subsequently also on the older 1900 range
ICT 1900 series
ICT 1900 was the name given to a series of mainframe computers released by International Computers and Tabulators and later International Computers Limited during the 1960s and '70s...
. ICL continued development of the software independently of Cullinane, selling the original ported product under the original name IDMS and an enhanced version as IDMSX. In this form it was used by many large UK users, an example being the Pay-As-You-Earn system operated by Inland Revenue. Many of these systems are still running in 2005.
In the early to mid 1980s, relational database management system
Relational database management system
A relational database management system is a database management system that is based on the relational model as introduced by E. F. Codd. Most popular databases currently in use are based on the relational database model....
s started to become more popular, encouraged by increasing hardware power and the move to minicomputer
Minicomputer
A minicomputer is a class of multi-user computers that lies in the middle range of the computing spectrum, in between the largest multi-user systems and the smallest single-user systems...
s and client–server architecture. Relational databases offered improved development productivity over CODASYL
CODASYL
CODASYL is an acronym for "Conference on Data Systems Languages". This was a consortium formed in 1959 to guide the development of a standard programming language that could be used on many computers...
systems, and the traditional objections based on poor performance were steadily diminishing.
Cullinet
Cullinet
Cullinet was a software company whose products included the database management system IDMS and the integrated software package Goldengate. In 1989, the company was bought by Computer Associates...
attempted to compete against IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
's DB2
IBM DB2
The IBM DB2 Enterprise Server Edition is a relational model database server developed by IBM. It primarily runs on Unix , Linux, IBM i , z/OS and Windows servers. DB2 also powers the different IBM InfoSphere Warehouse editions...
and other relational databases by developing a relational front-end and a range of productivity tools. These included Automatic System Facility (ASF), which made use of a pre-existing IDMS feature called LRF (Logical Record Facility). ASF was a fill-in-the-blanks database generator that would also develop a mini-application to maintain the tables.
It is difficult to judge whether such features may have been successful in extending the selling life of the product, but they made little impact in the long term. Those users who stayed with IDMS were primarily interested in its high performance, not in its relational capabilities. It was widely recognized (helped by a high-profile campaign by E. F. Codd, the father of the relational model
Relational model
The relational model for database management is a database model based on first-order predicate logic, first formulated and proposed in 1969 by Edgar F...
) that there was a significant difference between a relational database and a network database with a relational veneer.
In 1989 Computer Associates continued after Cullinet acquisition with the development and released Release 12.0 with full SQL
SQL
SQL is a programming language designed for managing data in relational database management systems ....
in 1992-93.
Nowadays, CA Technologies actively markets and supports the CA IDMS and enhanced IDMS in subsequent releases by TCP/IP support, Two-Phase commit support, XML
XML
Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....
publishing, zIIP
ZIIP
In IBM System z9 mainframes, the System z Integrated Information Processor is a special purpose processor. It was initially introduced to relieve the general mainframe central processors of specific DB2 processing loads, but currently is used to offload other z/OS workloads as described below...
specialty processor support, Web-enabled access in combination with CA IDMS Server and GUI database administration via CA IDMS Visual DBA tool.
IDMS legacy systems are still being run today. Few customers have migrated to Cullinet's other database offering IDMS/R.
Integrated Data Dictionary
One of the sophisticated features of IDMS was its built-in Integrated Data DictionaryData dictionary
A data dictionary, or metadata repository, as defined in the IBM Dictionary of Computing, is a "centralized repository of information about data such as meaning, relationships to other data, origin, usage, and format." The term may have one of several closely related meanings pertaining to...
(IDD). The IDD was primarily developed to maintain database definitions. It was itself an IDMS database.
DBAs (database administrators) and other users interfaced with the IDD using a language called Data Dictionary Definition Language (DDDL).
IDD was also used to store definitions and code for other products in the IDMS family such as ADS/Online and IDMS-DC.
IDD's power was that it was extensible and could be used to create definitions of just about anything. Some companies used it to develop in-house documentation.
Logical Data Model
The data model offered to users is the CODASYLCODASYL
CODASYL is an acronym for "Conference on Data Systems Languages". This was a consortium formed in 1959 to guide the development of a standard programming language that could be used on many computers...
network model. The main structuring concepts in this model are records and sets. Records essentially follow the COBOL
COBOL
COBOL is one of the oldest programming languages. Its name is an acronym for COmmon Business-Oriented Language, defining its primary domain in business, finance, and administrative systems for companies and governments....
pattern, consisting of fields of different types: this allows complex internal structure such as repeating items and repeating groups.
The most distinctive structuring concept in the Codasyl model is the set. Not to be confused with a mathematical set, a Codasyl set represents a one-to-many relationship between records: one owner, many members. The fact that a record can be a member in many different sets is the key factor that distinguishes the network model from the earlier hierarchical model
Hierarchical model
A hierarchical database model is a data model in which the data is organized into a tree-like structure. The structure allows representing information using parent/child relationships: each parent can have many children, but each child has only one parent...
. As with records, each set belongs to a named set type (different set types model different logical relationships). Sets are in fact ordered, and the sequence of records in a set can be used to convey information. A record can participate as an owner and member of any number of sets.
Records have identity, the identity being represented by a value known as a database key. In IDMS, as in most other Codasyl implementations, the database key is directly related to the physical address of the record on disk. Database keys are also used as pointers to implement sets in the form of linked lists and trees. This close correspondence between the logical model and the physical implementation (which is not a strictly necessary part of the Codasyl model, but was a characteristic of all successful implementations) is responsible for the efficiency of database retrieval, but also makes operations such as database loading and restructuring rather expensive.
Records can be accessed directly by database key, by following set relationships, or by direct access using key values. Initially the only direct access was through hashing, a mechanism known in the Codasyl model as CALC access. In IDMS, CALC access is implemented through an internal set, linking all records that share the same hash value to an owner record that occupies the first few bytes of every disk page.
In subsequent years, some versions of IDMS added the ability to access records using BTree-like indexes.
Storage
IDMS organizes its databases as a series of files. These files are mapped and pre-formatted into so-called areas. The areas are subdivided into pages which correspond to physical blocks on the disk. The database records are stored within these blocks.The DBA allocates a fixed number of pages in a file for each area. The DBA then defines which records are to be stored in each area, and details of how they are to be stored.
IDMS intersperses special space-allocation pages throughout the database. These pages are used to keep track of the free space available in each page in the database. To reduce I/O requirements, the free space is only tracked for all pages when the free space for the area falls below 30%.
Four methods are available for storing records in an IDMS database: Direct, Sequential, CALC, and VIA.
In direct mode the target is specified by the user and is stored as close as possible to that page.
Sequential placement (not to be confused with indexed sequential), simply places each new record at the end of the area. This option is rarely used.
CALC uses a hashing algorithm to decide where to place the record; the hash key then provides efficient retrieval of the record. The entire CALC area is preformatted each with a header consisting of a special CALC "owner" record. The hashing algorithm determines a page number (from which the physical disk address can be determined), and the record is then stored on this page, or as near as possible to it, and is linked to the header record on that page using the CALC set. The CALC records are linked to the page's CALC Owner record using a single link-list (pointers). The CALC Owner located in the page header thus owns the set of all records which target to its particular page (whether the records are stored on that page or, in the case of an overflow, on another page ).
CALC provides extremely efficient storage and retrieval: IDMS can retrieve a CALC record in 1.1 I/O operations. However, the method does not cope well with changes to the value of the primary key, and expensive reorganization is needed if the number of pages needs to be expanded.
VIA placement attempts to store a record near its owner in a particular set. Usually the records are clustered on the same physical page as the owner. This leads to efficient navigation when the record is accessed by following that set relationship. (VIA allows records to be stored in a different IDMS area so that they can be store separately from the owner, yet remain clustered together for efficiency.)
Sets are generally maintained as linked lists, using the database key as a pointer. Every record includes a forwards link to the next record; the database designer can choose whether to include owner pointers and prior pointers (if not provided, navigation in those directions will be slower).
Some versions of IDMS subsequently included the ability to define indexes: either record indexes, allowing records to be located from knowledge of a secondary key, or set indexes, allowing the members of a set to be retrieved by key value.
Release history (Computer Associates/CA era)
| Release version | Release date | Notable features |
|---|---|---|
| R12 | 1992 | 24-hour processing, deadlock Deadlock A deadlock is a situation where in two or more competing actions are each waiting for the other to finish, and thus neither ever does. It is often seen in a paradox like the "chicken or the egg"... management, centralized security facility, SQL SQL SQL is a programming language designed for managing data in relational database management systems .... |
| R14 | 1999-01-09 | Parallel sysplex IBM Parallel Sysplex In computing, a Parallel Sysplex is a cluster of IBM mainframes acting together as a single system image with z/OS. Used for disaster recovery, Parallel Sysplex combines data sharing and parallel computing to allow a cluster of up to 32 systems to share a workload for high performance and high... exploitation, multitasking |
| R15 | 2001-04-25 | Improved performance, data sharing |
| R16 | 2004-04-13 | Two-Phase commit, TCP/IP, Parallel Access Volume exploitation, XML XML Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards.... publishing |
| R17 | 2008-10-30 | Improved performance, zIIP ZIIP In IBM System z9 mainframes, the System z Integrated Information Processor is a special purpose processor. It was initially introduced to relieve the general mainframe central processors of specific DB2 processing loads, but currently is used to offload other z/OS workloads as described below... support, improved auto-recovery features |
| V18 | 2011-06-02 | Enhanced zIIP support, automatic system tuning, improved performance, simplified installation and maintenance |
User groups
IDMS has many non-profit user associations across the globe composed of information technology professionals who use and/or support CA IDMS or related products. They include:- CA IDMS (IUA EIUA) Global User Community
- AID - Anwendervereinigung IDMS Deutschland
- Australian IDMS User Group
- BeNeLux IDMS User Association
- Chicago IDMS Users Group
- Committee Quebec Users of IDMS
- Dallas CA-IDMS Users Forum
- European IDMS User Group
- Finnish IDMS User Group
- France IDMS User Association
- Rocky Mountain IDMS Users Group
- UK IDMS User Association
See also
- CA Datacom/DBDATACOM/DBDatacom/DB is a relational database management system for the mainframes. Originally developed by Insyte Datacom, later acquired by Applied Data Research, it is now owned by CA Technologies which renamed it to CA-Datacom/DB and later to CA Datacom/DB.- External links :****...
- IBM Information Management SystemInformation Management SystemIBM Information Management System is a joint hierarchical database and information management system with extensive transaction processing capabilities.- History :...
- IBM DB2IBM DB2The IBM DB2 Enterprise Server Edition is a relational model database server developed by IBM. It primarily runs on Unix , Linux, IBM i , z/OS and Windows servers. DB2 also powers the different IBM InfoSphere Warehouse editions...
- ICL IDMS(X)