Common Log File System
Encyclopedia
Common Log File System is a general-purpose logging subsystem that is accessible to both kernel-mode as well as user-mode applications for building high-performance transaction logs. It was introduced with Windows Server 2003
R2 and included in later Windows
OSs. CLFS can be used for both data logging
as well as for event logging. CLFS is used by TxF
and TxR to store transactional state changes before they commit a transaction.
logs records to in-memory buffers and then writes them to log-files on secondary storage (stable media in CLFS terminology) for permanent persistence. When the data will be flushed to stable media is controlled by built-in policies, but a CLFS client application can override that and force a flush. CLFS allows for customizable log formats, expansion and truncation of logs according to defined policies, as well as simultaneous use by multiple client applications. CLFS is able to store log files anywhere on the file system.
CLFS defines a device driver
interface (DDI), via which physical storage system specific drivers plug in to the CLFS API. The CLFS driver implements the ARIES
recovery algorithm; other algorithms can be supported by using custom drivers.
CLFS supports both dedicated logs, as well as multiplexed logs. A dedicated log contains a single stream of log records whereas multiplexed log contain multiple streams, each stream for a different application. Even though a multiplexed log has multiple streams, logs are flushed to the streams sequentially, in a single batch. CLFS can allocate space for a set of log records ahead-of-time (before the logs are actually generated) to make sure the operation does not fail due to lack of storage space.
A log record in a CLFS stream is first placed to Log I/O Block in a buffer in system memory. Periodically blocks are flushed to stable storage devices. On the storage device, a log contains a set of Containers, which are allocated contiguously, each containing multiple Log I/O Blocks. New log records are appended to the present set. Each record is identified by a Log Sequence Number (LSN), an increasing 32-bit sequence number. The LSN and other metadata are stored in the record header. The LSN encodes the identifier of the container, the offset to the record and the identifier of the record - this information is used to access the log record subsequently. However, the container identifiers are logical identifiers, they must be mapped to physical containers. The mapping is done by CLFS itself.
Windows Server 2003
Windows Server 2003 is a server operating system produced by Microsoft, introduced on 24 April 2003. An updated version, Windows Server 2003 R2, was released to manufacturing on 6 December 2005...
R2 and included in later Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...
OSs. CLFS can be used for both data logging
Computer data logging
Computer data logging is the process of recording events, with an automated computer program, in a certain scope in order to provide an audit trail that can be used to understand the activity of the system and to diagnose problems....
as well as for event logging. CLFS is used by TxF
TXF
TXF can refer to:* The X-Files, the American science fiction television series* The X Factor , the television music talent show franchise* Transactional NTFS, a component of Windows Vista and later operating systems...
and TxR to store transactional state changes before they commit a transaction.
Overview
The job of CLFS, like any other transactional logging system, is to record a series of steps required for some action so that they can be either played back accurately in the future to commit the transaction to secondary storage or undone if required. CLFS first marshalsMarshalling (computer science)
In computer science, marshalling is the process of transforming the memory representation of an object to a data format suitable for storage or transmission...
logs records to in-memory buffers and then writes them to log-files on secondary storage (stable media in CLFS terminology) for permanent persistence. When the data will be flushed to stable media is controlled by built-in policies, but a CLFS client application can override that and force a flush. CLFS allows for customizable log formats, expansion and truncation of logs according to defined policies, as well as simultaneous use by multiple client applications. CLFS is able to store log files anywhere on the file system.
CLFS defines a device driver
Device driver
In computing, a device driver or software driver is a computer program allowing higher-level computer programs to interact with a hardware device....
interface (DDI), via which physical storage system specific drivers plug in to the CLFS API. The CLFS driver implements the ARIES
Algorithms for Recovery and Isolation Exploiting Semantics
In computer science, Algorithms for Recovery and Isolation Exploiting Semantics, or ARIES is a recovery algorithm designed to work with a no-force, steal database approach; it is used by IBM DB2, Microsoft SQL Server and many other database systems....
recovery algorithm; other algorithms can be supported by using custom drivers.
CLFS supports both dedicated logs, as well as multiplexed logs. A dedicated log contains a single stream of log records whereas multiplexed log contain multiple streams, each stream for a different application. Even though a multiplexed log has multiple streams, logs are flushed to the streams sequentially, in a single batch. CLFS can allocate space for a set of log records ahead-of-time (before the logs are actually generated) to make sure the operation does not fail due to lack of storage space.
A log record in a CLFS stream is first placed to Log I/O Block in a buffer in system memory. Periodically blocks are flushed to stable storage devices. On the storage device, a log contains a set of Containers, which are allocated contiguously, each containing multiple Log I/O Blocks. New log records are appended to the present set. Each record is identified by a Log Sequence Number (LSN), an increasing 32-bit sequence number. The LSN and other metadata are stored in the record header. The LSN encodes the identifier of the container, the offset to the record and the identifier of the record - this information is used to access the log record subsequently. However, the container identifiers are logical identifiers, they must be mapped to physical containers. The mapping is done by CLFS itself.