SDXF is a data serialization format defined by RFC 3072.
It allows arbitrary structured data of different types to be assembled together for exchanging between computers of different architectures.
The ability to arbitrarily structure data and serialize it into a self-describing format is reminiscent of XML
XML
Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....
, but SDXF is not a text format (as XML) — one cannot manipulate an SDXF structure with a text editor.
The maximal length of a datum (composite as well as elementary) encoded using SDXF is 16777215 bytes (one less than 16 MiB).
Technical structure format
SDXF data can be structured in arbitrary level of deepness. The particular data elements are self-described, that means the information about the type of data (numeric, character string or structure) are encoded into the data elements. The design of this format is very simple, but nevertheless transparent: computer programs accesses SDXF data with the help of well defined functions, it is not necessary for the programmer to care about the precise layout of the data.
The word "exchange" in the labelling SDXF reflects another kind of transparency: the SDXF functions care for a computer architecture independent conversion of the data, the composed structure can be exchanged to other computers (via direct network, file transfer or CD) without further measures. The SDXF functions on the receiving side cares for the correct adaptation.
What exactly means "structured data"? In fact it is difficult to find an example for non-structured data: Also a common text is structured in lines, paragraphs, etc. But in this context structures have to be recognized and processed by the related programs.
Here an example out off the commercial area: Two companies wants to exchange their issuing of invoices on an electronic base. An invoice is therefore an electronic document with following hierarchical nested structure:
INVOICE
│
├─ INVOICE_NO
├─ DATE
├─ ADDRESS_SENDER
│ ├─ NAME
│ ├─ NAME
│ ├─ STREET
│ ├─ ZIP
│ ├─ CITY
│ └─ COUNTRY
├─ ADDRESS_RECIPIENT
│ ├─ NAME
│ ├─ NAME
│ ├─ STREET
│ ├─ ZIP
│ ├─ CITY
│ └─ COUNTRY
├─ INVOICE_SUM
├─ SINGLE_ITEMS
│ ├─ SINGLE_ITEM
│ │ ├─ QUANTITY
│ │ ├─ ITEM_NUMBER
│ │ ├─ ITEM_TEXT
│ │ ├─ CHARGE
│ │ └─ SUM
│ └─ ...
├─ CONDITIONS
...
Similar and more complex structures has to be covered with SDXF.
How is an SDXF structure build up?
The basic construction element is a "Chunk". The whole SDXF structure is a chunk and a chunk can consist out off a set of chunks.
A chunk itself has a very simple construction: It is composed of a header prefix of six bytes, followed by the actual data. The header contains an identification of this chunk as a 2-byte binary number (Chunk_ID), the length and a specification about the type of the following data. It may also contains additional information about compression, encryption and others.
The specification about the type of the data informs us if the data consists of text (a string of characters), or if it presents a binary number (can be an integer or a floating point representation) or if the chunk is build up out off a sequence of other chunks (giving a structured chunk).
Especially the availability of structured chunks enables the programmer to pack such hierarchically constructions as the INVOICE above into an SDXF structure in a simple way:
In a first step every named term (INVOICE, INVOICE_NO, DATE, ADDRESS_SENDER, etc.) has to be associated a unique number out off the range 1 to 65535 (2 byte binary integer without sign).
At next a first chunk is to be constructed with the ID INVOICE (that means with the associated numerical chunk_ID) as a structured chunk on level 1.
This INVOICE chunk will be filled step by step with a sequence of other chunks on level 2: INVOICE_NO, DATE, ADDRESS_SENDER, ADDRESS_RECIPIENT, INVOICE_SUM, SINGLE_ITEMS, CONDITIONS. Some of these chunks on level 2 are structured in turn as for the two addresses and SINGLE_ITEMS. For the date one has the possibility to specify it in text format e.g. in the form YYYY-MM-DD (as standardized in ISO 8601
ISO 8601
ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times is an international standard covering the exchange of date and time-related data. It was issued by the International Organization for Standardization and was first published in 1988...
) or alternatively as a structure consisting of the three numerical chunks YEAR, MONTH, DAY.
For a precise description see page 2 of the RFC or alternatively (http://pinpi.com/en/SDXF_2.htm)
The concept for SDXF determines also that the programmer has to work on SDXF structures with a well defined set of functions.
There are only few of them:
To read Chunks, following functions has to be used:
init
To initialize the parameter structure and linking to the existing Chunk.
enter
To step into a structured Chunk, the 1st Chunk of this structure is ready to process.
leave
To leave the current structure. This structure is already current.
next
Goes to next Chunk if exists (otherwise it leaves the current structure).
extract
To transfer (and adapt) data from the current Chunk into a program variable.
select
To search the next Chunk with a given Chunk ID and make it current.
To build Chunks, following functions has to be used:
init
To initialize the parameter structure and linking to an empty output buffer for to create a new Chunk.
create
Create a new Chunk and append it to the current existing structure (if exists).
append
Append a complete Chunk to an SDXF-Structure.
leave
To leave the current structure. This structure is already current.
With the invoice example above a creation program may be looks like following code:
init (sdx, buffersize=1000); // initialize the SDXF parameter structure sdx
create (sdx, ID=INVOICE, datatype=STRUCTURED); // start of the main structure
create (sdx, ID=INVOICE_NO, datatype=NUMERIC, value=123456); // create an elementary Chunk
create (sdx, ID=DATE, datatype=CHAR, value="2005-06-17"); // once more
create (sdx, ID=ADDRESS_SENDER, datatype=STRUCTURED); // Substructure
create (sdx, ID=NAME, datatype=CHAR, value="Peter Somebody"); // element. Chunk inside this substructure
...
create (sdx, ID= COUNTRY, datatype=CHAR, value="France"); // the last one inside this substructure
leave; // closing the substructure ADDRESS_SENDER
...
leave; // closing the substructure INVOICE
The Syntax of this code is only fictive to simplify matters, for a complete example in the programming language "C" look here.
A code to readout the INVOICE structure looks like:
init (sdx, container=pointer to an SDXF-structure); // initialize the SDXF parameter structure sdx
enter (sdx); // join into the INVOICE structure.
//
while (sdx.rc SDX_RC_ok)
{
switch (sdx. Chunk_ID)
{
case INVOICE_NO:
extract (sdx);
invno = sdx.value; // the extract function put integer values into the parameter field 'value'
break;
//
case DATE:
extract (sdx);
strcpy (invdate, sdx.data); // sdx.data is a pointer to the extracted character string
break;
//
case ADDRESS_SENDER:
enter (sdx); // we use 'enter' because ADDRESS is a structured Chunk
do while (sdx.rc SDX_RC_ok) // inner loop
...
break;
...
}
}
As the example shows the individual SDXF Chunks will be constructed step by step. At a first glance this appears somehow cumbersome, but:
This reflects exactly programmer's everyday life: The individual elements are distributed in various program variables or data base fields, in case of external data these elements are not all accessible at the same time.
With the processing step by step it is warranted that in case of a structure which consists of mixed data type these data will be transformed to a normalized form, so that the resulting structure can transferred from one computer to another without further adaptation. Even if these computer differs in their architecture. The problematic of different presentation of data is outlined here (presentation of characters) and there (byte swapping problem). With the use of the SDXF functions the application programmer is completely freed from these problems.
Encryption (e.g. AES) and compressing (e.g. ZIP) will be maintained by the SDXF functions too. It is also possible to encrypt or compress only parts of a structure. While extracting Chunks the decompressing and decryption will be automatically done (if the encryption key matches).
SDXF is not designed for readability by humans or that it can be modified by text editors. If someone wish to do so a special format is available which reflects an SDXF structure one to one, named SDEF.
External Data Representation is a standard data serialization format, for uses such as computer network protocols. It allows data to be transferred between different kinds of computer systems. Converting from the local representation to XDR is called encoding. Converting from XDR to the local...
Protocol Buffers are a serialization format with an interface description language developed by Google. The original Google implementation for C++, Java and Python is available under a free software, open source license....
Distinguished Encoding Rules , is a message transfer syntax specified by the ITU in X.690. The Distinguished Encoding Rules of ASN.1 is an International Standard drawn from the constraints placed on basic encoding rules encodings by X.509. DER encodings are valid BER encodings...
Facebook is a social networking service and website launched in February 2004, operated and privately owned by Facebook, Inc. , Facebook has more than 800 million active users. Users must register before using the site, after which they may create a personal profile, add other users as...
Thrift is an interface definition language that is used to define and create services for numerous languages. It is used as a remote procedure call framework and was developed at Facebook for "scalable cross-language services development"...
Cisco Systems, Inc. is an American multinational corporation headquartered in San Jose, California, United States, that designs and sells consumer electronics, networking, voice, and communications technology and services. Cisco has more than 70,000 employees and annual revenue of US$...
Etch is an open source, cross-platform framework for building network services, first announced in May 2008 by Cisco Systems. Etch encompasses a service description language, a compiler, and a number of language bindings...
ZeroC, Inc. is a company based in Palm Beach Gardens, Florida, U.S., revolving around the development and licensing of the Internet Communications Engine, or ICE, an object middleware system considered an alternative to CORBA and SOAP...
The Internet Communications Engine, or Ice, is an object-oriented middleware that provides object-oriented Remote Procedure Call, grid computing and Publish/subscribe functionality developed by ZeroC and dual-licensed under the GNU GPL and a proprietary license...
This is a comparison of data serialization formats, different ways to convert complex objects to sequences of bits. It does not include markup languages used exclusively as document file formats.-Overview:*a. The current default format is binary....
