HTTP pipelining
Encyclopedia
HTTP pipelining is a technique in which multiple HTTP requests are sent on a single HTTP connection without waiting for the corresponding responses.
The pipelining of requests results in a dramatic improvement in the loading times of HTML pages, especially over high latency
connections such as satellite Internet connections.
Since it is usually possible to fit several HTTP requests in the same TCP
packet, HTTP pipelining allows fewer TCP packets to be sent over the network, reducing network load.
Non-idempotent methods like POST
should not be pipelined. Sequences of GET and HEAD requests can be always pipelined. A sequence of other idempotent requests like GET, HEAD, PUT and DELETE can be pipelined or not depending on whether requests in the sequence depend on the effect of others.
HTTP pipelining requires both the client and the server to support it. HTTP/1.1 conforming servers are required to support pipelining. This does not mean that servers are required to pipeline responses, but that they are required not to fail if a client chooses to pipeline requests.
s is a relatively simple matter of making sure that network buffers are not discarded between requests. For that reason, most modern web servers handle pipelining without any problem.
has a fully working implementation that is enabled by default. All other browsers HTTP pipelining is disabled or not implemented.
Some versions of the Squid web proxy will pipeline up to two outgoing requests. This functionality has been disabled by default and needs to be manually enabled for "bandwidth management and access logging reasons." Squid supports multiple requests from clients.
The Polipo
proxy pipelines outgoing requests.
library made by the World Wide Web Consortium
, supports pipelining since version 5.1 released at 18 February 1997.
Other application development libraries that support HTTP pipelining include:
Some other applications currently exploiting pipelining are:
Multipart XHR is implementation of pipelining (without any browser or web server support) done purely in Javascript in combination with server side scripting.
Testing tools which support HTTP pipelining include:
The pipelining of requests results in a dramatic improvement in the loading times of HTML pages, especially over high latency
Lag
Lag is a common word meaning to fail to keep up or to fall behind. In real-time applications, the term is used when the application fails to respond in a timely fashion to inputs...
connections such as satellite Internet connections.
Since it is usually possible to fit several HTTP requests in the same TCP
Transmission Control Protocol
The Transmission Control Protocol is one of the core protocols of the Internet Protocol Suite. TCP is one of the two original components of the suite, complementing the Internet Protocol , and therefore the entire suite is commonly referred to as TCP/IP...
packet, HTTP pipelining allows fewer TCP packets to be sent over the network, reducing network load.
Non-idempotent methods like POST
POST (HTTP)
In computing, POST is one of many request methods supported by the HTTP protocol used by the World Wide Web. The POST request method is used when the client needs to send data to the server as part of the request, such as when uploading a file or submitting a completed form.In contrast to the GET...
should not be pipelined. Sequences of GET and HEAD requests can be always pipelined. A sequence of other idempotent requests like GET, HEAD, PUT and DELETE can be pipelined or not depending on whether requests in the sequence depend on the effect of others.
HTTP pipelining requires both the client and the server to support it. HTTP/1.1 conforming servers are required to support pipelining. This does not mean that servers are required to pipeline responses, but that they are required not to fail if a client chooses to pipeline requests.
Implementation in web servers
Implementing pipelining in web serverWeb server
Web server can refer to either the hardware or the software that helps to deliver content that can be accessed through the Internet....
s is a relatively simple matter of making sure that network buffers are not discarded between requests. For that reason, most modern web servers handle pipelining without any problem.
Implementation in web browsers
Out of all the major browsers, only OperaOpera (web browser)
Opera is a web browser and Internet suite developed by Opera Software with over 200 million users worldwide. The browser handles common Internet-related tasks such as displaying web sites, sending and receiving e-mail messages, managing contacts, chatting on IRC, downloading files via BitTorrent,...
has a fully working implementation that is enabled by default. All other browsers HTTP pipelining is disabled or not implemented.
- Internet Explorer 8Internet Explorer 8Windows Internet Explorer 8 is a web browser developed by Microsoft in the Internet Explorer browser series. The browser was released on March 19, 2009 for Windows XP, Windows Server 2003, Windows Vista, Windows Server 2008, and Windows 7. Both 32-bit and 64-bit builds are available...
does not pipeline requests, due to concerns regarding buggy proxies and head-of-line blockingHead-of-line blockingHead-of-line blocking is a performance-limiting phenomenon that occurs in buffered telecommunication network switches.-Description:...
. - Mozilla browsers (such as Mozilla FirefoxMozilla FirefoxMozilla Firefox is a free and open source web browser descended from the Mozilla Application Suite and managed by Mozilla Corporation. , Firefox is the second most widely used browser, with approximately 25% of worldwide usage share of web browsers...
, SeaMonkeySeaMonkeySeaMonkey is a free and open source cross-platform Internet suite. It is the continuation of the former Mozilla Application Suite, based on the same source code...
and CaminoCaminoCamino is a free, open source, GUI-based Web browser based on Mozilla's Gecko layout engine and specifically designed for the Mac OS X operating system...
) support pipelining, however it is disabled by default. Pipelining is disabled by default to avoid issues with misbehaving servers. When pipelining is enabled, Mozilla browsers use some heuristics, especially to turn pipelining off for older IISInternet Information ServicesInternet Information Services – formerly called Internet Information Server – is a web server application and set of feature extension modules created by Microsoft for use with Microsoft Windows. It is the most used web server after Apache HTTP Server. IIS 7.5 supports HTTP, HTTPS,...
servers. - KonquerorKonquerorNot to be confused with the Conqueror web browser.Konqueror is a web browser and file manager that provides file-viewer functionality for file systems such as local files, files on a remote ftp server and files in a disk image. It is a core part of the KDE desktop environment...
2.0 supports pipelining, but it's disabled by default. - Google ChromeGoogle ChromeGoogle Chrome is a web browser developed by Google that uses the WebKit layout engine. It was first released as a beta version for Microsoft Windows on September 2, 2008, and the public stable release was on December 11, 2008. The name is derived from the graphical user interface frame, or...
does not support pipelining in the stable release but it is currently in unstable builds and Chromium as a launch parameter.
Implementation in web proxies
Most HTTP proxies do not pipeline outgoing requests.Some versions of the Squid web proxy will pipeline up to two outgoing requests. This functionality has been disabled by default and needs to be manually enabled for "bandwidth management and access logging reasons." Squid supports multiple requests from clients.
The Polipo
Polipo
Polipo is a fast and lightweight, forwarding and caching proxy server and computer software daemon.By virtue of being a compliant HTTP 1.1 proxy, Polipo has all the uses of traditional Web proxies. It features HTTP 1.1, IPv4 & IPv6, traffic filtering and privacy-enhancement. Polipo supports HTTP...
proxy pipelines outgoing requests.
Other implementations
The libwwwLibwww
libwww is a highly-modular client-side web API for Unix and Windows, and is also the name of the reference implementation of this API....
library made by the World Wide Web Consortium
World Wide Web Consortium
The World Wide Web Consortium is the main international standards organization for the World Wide Web .Founded and headed by Tim Berners-Lee, the consortium is made up of member organizations which maintain full-time staff for the purpose of working together in the development of standards for the...
, supports pipelining since version 5.1 released at 18 February 1997.
Other application development libraries that support HTTP pipelining include:
- Perl modules providing client support for HTTP pipelining are HTTP::Async and the LWPng (libwww-perl New Generation) library.
- Apache Foundation project HttpComponents provides pipelining support in the HttpCore NIO extensions.
- The Microsoft .Net Framework.NET FrameworkThe .NET Framework is a software framework that runs primarily on Microsoft Windows. It includes a large library and supports several programming languages which allows language interoperability...
3.5 supports HTTP pipelining in the moduleSystem.Net.HttpWebRequest. - Qt class
QNetworkRequest, introduced in 4.4, supports HTTP Pipelining.
Some other applications currently exploiting pipelining are:
- phttpget from FreeBSDFreeBSDFreeBSD is a free Unix-like operating system descended from AT&T UNIX via BSD UNIX. Although for legal reasons FreeBSD cannot be called “UNIX”, as the direct descendant of BSD UNIX , FreeBSD’s internals and system APIs are UNIX-compliant...
(a minimalist pipelined HTTP client) - libcurl has limited support for pipelining using the CURLMOPT_PIPELINING option.
- portsnapPortsnapPortsnap is a system written by Colin Percival for secure distribution of compressed, digitally signed snapshots of the FreeBSD ports tree. The distribution follows the client–server model and uses the transport protocol HTTP ....
(a FreeBSDFreeBSDFreeBSD is a free Unix-like operating system descended from AT&T UNIX via BSD UNIX. Although for legal reasons FreeBSD cannot be called “UNIX”, as the direct descendant of BSD UNIX , FreeBSD’s internals and system APIs are UNIX-compliant...
ports treeFreeBSD PortsThe FreeBSD Ports collection is a package management system for the FreeBSD operating system, providing an easy and consistent way of installing software packages. As of October 2011, there are over 22,700 ports available in the collection...
distribution system) - Advanced Packaging ToolAdvanced Packaging ToolThe Advanced Packaging Tool, or APT, is a free user interface that works with core libraries to handle the installation and removal of software on the Debian GNU/Linux distribution and its variants...
support pipelining. - SubversionSubversionApache Subversion is a software versioning and a revision control system distributed under a free license. Developers use Subversion to maintain current and historical versions of files such as source code, web pages, and documentation...
has optional support for HTTP pipelining with the serf WebDAV access module (the default module, neonNeon (library)neon is a library for accessing HTTP and WebDAV servers for the C programming language. It is free software and is licensed under LGPL.neon relies on either OpenSSL or GnuTLS for secure [] connections and either libxml or expat for parsing WebDAV XML responses.Among others, neon is used by the...
does not have pipelining support). - Microsoft Message QueuingMicrosoft Message QueuingMicrosoft Message Queuing or MSMQ is a Message Queue implementation developed by Microsoft and deployed in its Windows Server operating systems since Windows NT 4 and Windows 95. The latest Windows 7 also includes this component...
on Windows Server 2003Windows Server 2003Windows Server 2003 is a server operating system produced by Microsoft, introduced on 24 April 2003. An updated version, Windows Server 2003 R2, was released to manufacturing on 6 December 2005...
utilises pipelining on HTTP by default, and can be configured to use it on HTTPS. IBM CICSCICSCustomer Information Control System is a transaction server that runs primarily on IBM mainframe systems under z/OS and z/VSE.CICS is a transaction manager designed for rapid, high-volume online processing. This processing is mostly interactive , but background transactions are possible...
3.1 supports HTTP pipelining within its client. - Pipelined::HttpClient is a C++ client library based on pipelining.
Multipart XHR is implementation of pipelining (without any browser or web server support) done purely in Javascript in combination with server side scripting.
Testing tools which support HTTP pipelining include:
- httperfHttperfhttperf is a testing tool to measure the performance of web servers. It was originally developed by David Mosberger and other staff at Hewlett-Packard Research Laboratories.httperf is able to test HTTP pipelining workloads.- References :* , ITmedia...