Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
In computing, a system call is how a program requests a service from an operating system's kernel. This may include hardware related services , creating and executing new processes, and communicating with integral kernel services...
In computer programming, a file descriptor is an abstract indicator for accessing a file. The term is generally used in POSIX operating systems...
and a pipe without a round trip to user space. The related system call vmsplice moves or copies data between a pipe and user space. Ideally, splice and vmsplice work by remapping pages and do not actually copy any data, which may improve I/O
I/O
I/O may refer to:* Input/output, a system of communication for information processing systems* Input-output model, an economic model of flow prediction between sectors...
performance. As linear addresses do not necessarily correspond to contiguous physical addresses, this may not be possible in all cases and on all hardware combinations.
Workings
With splice, one can move data from one file descriptor to another without incurring any copies from user space into kernel space, which is usually required to enforce system security and also to keep a simple interface for processes to read and write to files. splice works by using the pipe buffer
Pipeline (Unix)
In Unix-like computer operating systems , a pipeline is the original software pipeline: a set of processes chained by their standard streams, so that the output of each process feeds directly as input to the next one. Each connection is implemented by an anonymous pipe...
. A pipe buffer is an in-kernel memory buffer that is opaque to the user space process. A user process can splice the contents of a source file into this pipe buffer, then splice the pipe buffer into the destination file, all without moving any data through userspace.
Linus Benedict Torvalds is a Finnish software engineer and hacker, best known for having initiated the development of the open source Linux kernel. He later became the chief architect of the Linux kernel, and now acts as the project's coordinator...
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
splice implementation borrows some ideas from an original proposal by Larry McVoy
Larry McVoy
Larry McVoy is the CEO of BitMover, the company that makes BitKeeper, a version control system that was used from February 2002 to early 2005 to manage the source code of the Linux kernel....
in 1998. The splice system calls first appeared in the Linux
Linux
Linux is a Unix-like computer operating system assembled under the model of free and open source software development and distribution. The defining component of any Linux system is the Linux kernel, an operating system kernel first released October 5, 1991 by Linus Torvalds...
kernel version 2.6.17 and was authored by Jens Axboe
Jens Axboe
Jens Axboe is a Linux kernel hacker. He is the current Linux kernel maintainer of the block layer and other block devices, along with contributing the CFQ I/O scheduler, Noop scheduler, Deadline scheduler and splice IO architecture. Jens is also the author of the blktrace utility and kernel parts,...
.
Prototype
ssize_t splice(int fd_in, loff_t *off_in, int fd_out, loff_t *off_out, size_t len, unsigned int flags);
Some constants that are of interest are:
/* Splice flags (not laid down in stone yet). */
#ifndef SPLICE_F_MOVE
#define SPLICE_F_MOVE 0x01
#endif
#ifndef SPLICE_F_NONBLOCK
#define SPLICE_F_NONBLOCK 0x02
#endif
#ifndef SPLICE_F_MORE
#define SPLICE_F_MORE 0x04
#endif
#ifndef SPLICE_F_GIFT
#define SPLICE_F_GIFT 0x08
#endif
Example
This is an example of splice in action:
/* This example is wrong. A pipe is a kernel buffer of fixed size,
* so if you try to splice a big enough file direcly into it,
* you're gonna either be blocked indefinitely or get an error.
* You should splice, in a loop, a chunk of data which does not excede
* the dimension of the pipe buffer in each iteration.
*/
/* Transfer from disk to a log. */
int log_blocks (struct log_handle * handle, int fd, loff_t offset, size_t size)
{
int filedes [2];
int ret;
size_t to_write = size;
ret = pipe (filedes);
if (ret < 0)
goto out;
/* splice the file into the pipe (data in kernel memory). */
while (to_write > 0) {
ret = splice (fd, &offset, filedes [1], NULL, to_write,
SPLICE_F_MORE | SPLICE_F_MOVE);
if (ret < 0)
goto pipe;
else
to_write -= ret;
}
to_write = size;
/* splice the data in the pipe (in kernel memory) into the file. */
while (to_write > 0) {
ret = splice (filedes [0], NULL, handle->fd,
&(handle->fd_offset), to_write,
SPLICE_F_MORE | SPLICE_F_MOVE);
if (ret < 0)
goto pipe;
else
to_write -= ret;
}
pipe:
close (filedes [0]);
close (filedes [1]);
out:
if (ret < 0)
return -errno;
return 0;
}
Complementary system calls
splice is one of three system calls that complete the splice architecture. vmsplice can map an application data area into a pipe (or vice versa), thus allowing transfers between pipes and user memory where sys_splice transfers between a file descriptor and a pipe. tee is the last part of the trilogy. It duplicates one pipe to another, enabling forks in the way applications are connected with pipes.
Requirements
When using splice with sockets, the network controller (NIC) must support DMA.
When the NIC does not support DMA then splice will not deliver any performance improvement. The reason for this is that each page of the pipe will just fill up to frame size (1460 bytes of the available 4096 bytes per page).
