Comet (programming)
Encyclopedia
Comet is a web application
model in which a long-held HTTP request allows a web server
to push
data to a browser
, without the browser explicitly requesting it. Comet is an umbrella term
, encompassing multiple techniques for achieving this interaction. All these methods rely on features included by default in browsers, such as JavaScript
, rather than on non-default plugins. The Comet approach differs from the original model of the web, in which a browser requests a complete web page at a time.
The use of Comet techniques in web development
predates the use of the word Comet as a neologism for the collective techniques. Comet is known by several other names, including
Ajax Push,
Reverse Ajax, Two-way-web, HTTP Streaming, and
HTTP server push
among others.
by offering real-time interaction, using a persistent or long-lasting HTTP connection between the server and the client.
Since browsers and proxies are not designed with server events in mind, several techniques to achieve this have been developed, each with different benefits and drawbacks. The biggest hurdle is the HTTP 1.1 specification, which states that a browser should not have more than two simultaneous connections with a web server. However, holding one connection open for real-time events has a negative impact on browser usability: the browser may be blocked from sending a new request while waiting for the results of a previous request, e.g., a series of images. This can be worked around by creating a distinct hostname
for real-time information, which is an alias for the same physical server.
Specific methods of implementing Comet fall into two major categories: streaming and long polling.
to the server for all Comet events
. These events are incrementally handled and interpreted on the client side every time the server sends a new event, with neither side closing the connection.
Specific techniques for accomplishing streaming Comet include the following:
HTML element (an inline frame, which allows a website to embed one HTML document inside another). This invisible iframe is sent as a chunked
block, which implicitly declares it as infinitely long (sometimes called “forever frame”). As events occur, the iframe is gradually filled with
One benefit of the iframe method is that it works in every common browser. Two downsides of this technique are the lack of a reliable error handling method, and the impossibility of tracking the state of the request calling process.
(XHR) object, the main tool used by Ajax applications for browser–server communication, can also be pressed into service for server–browser Comet messaging, in a few different ways.
In 1995, Netscape Navigator
added a feature called “server push”, which allowed servers to send new versions of an image or HTML page to that browser, as part of a multipart HTTP response (see History section, below), using the content type
-based browsers such as Firefox
accept multipart responses to XHR, which can therefore be used as a streaming Comet transport. On the server side, each message is encoded as a separate portion of the multipart response, and on the client, the callback function
provided to the XHR
.
Instead of creating a multipart response, and depending on the browser to transparently parse each event, it is also possible to generate a custom data format for an XHR response, and parse out each event using browser-side JavaScript, relying only on the browser firing the
Specific technologies for accomplishing long-polling include the following:
or JSON
) or Javascript to be executed by the client. At the end of the processing of the response, the browser creates and sends another XHR, to await the next event. Thus the browser always keeps a request outstanding with the server, to be answered as each event occurs.
s (SLDs), due to browser security policies designed to prevent cross-site scripting
attacks. That is, if the main web page is served from one SLD, and the Comet server is located at another SLD, Comet events cannot be used to modify the HTML and DOM of the main page, using those transports. This problem can be side-stepped by creating a proxy server
in front of one or both sources, making them appear to originate from the same domain. However, this is often undesirable for complexity or performance reasons.
Unlike iframes or XMLHttpRequest objects,
, and JavaScript code in the response will be executed in the current HTML document. This creates a potential security risk for both servers involved, though the risk to the data provider (in our case, the Comet server) can be avoided using JSONP
.
A long-polling Comet transport can be created by dynamically creating
s into browsers (starting with Netscape 2.0 in March 1996) made real-time communications possible, using a raw TCP
socket to communicate between the browser and the server. This socket can remain open as long as the browser is at the document hosting the applet. Event notifications can be sent in any format — text or binary — and decoded by the applet.
Alex Russell coined the term Comet in a post on his personal blog. The new term was a play on Ajax
(Ajax and Comet
both being household cleaners common in the USA).
In 2006, while not really using the term Comet, some applications exposed those techniques to a wider audience: Meebo
’s multi-protocol web-based chat application enables users to connect to AOL
, Yahoo
, and Microsoft chat platforms through the browser; Google added web-based chat to Gmail
; JotSpot, a startup since acquired by Google, built Comet-based real-time collaborative document editing. New Comet companies and enterprise solutions were created, such as the Java-based ICEfaces
JSF
framework (although they prefer the term "Ajax Push"). Others that had previously used Java-applet based transports switched instead to pure-JavaScript implementations.
Web application
A web application is an application that is accessed over a network such as the Internet or an intranet. The term may also mean a computer software application that is coded in a browser-supported language and reliant on a common web browser to render the application executable.Web applications are...
model in which a long-held HTTP request allows a web server
Web server
Web server can refer to either the hardware or the software that helps to deliver content that can be accessed through the Internet....
to push
Push technology
Push technology, or server push, describes a style of Internet-based communication where the request for a given transaction is initiated by the publisher or central server...
data to a browser
Web browser
A web browser is a software application for retrieving, presenting, and traversing information resources on the World Wide Web. An information resource is identified by a Uniform Resource Identifier and may be a web page, image, video, or other piece of content...
, without the browser explicitly requesting it. Comet is an umbrella term
Umbrella term
An umbrella term is a word that provides a superset or grouping of concepts that all fall under a single common category. Umbrella term is also called a hypernym. For example, cryptology is an umbrella term that encompasses cryptography and cryptanalysis, among other fields...
, encompassing multiple techniques for achieving this interaction. All these methods rely on features included by default in browsers, such as JavaScript
JavaScript
JavaScript is a prototype-based scripting language that is dynamic, weakly typed and has first-class functions. It is a multi-paradigm language, supporting object-oriented, imperative, and functional programming styles....
, rather than on non-default plugins. The Comet approach differs from the original model of the web, in which a browser requests a complete web page at a time.
The use of Comet techniques in web development
Web development
Web development is a broad term for the work involved in developing a web site for the Internet or an intranet . This can include web design, web content development, client liaison, client-side/server-side scripting, web server and network security configuration, and e-commerce development...
predates the use of the word Comet as a neologism for the collective techniques. Comet is known by several other names, including
Ajax Push,
Reverse Ajax, Two-way-web, HTTP Streaming, and
HTTP server push
among others.
Implementations
Comet applications attempt to eliminate the limitations of the page-by-page web model and traditional pollingPolling (computer science)
Polling, or polled operation, in computer science, refers to actively sampling the status of an external device by a client program as a synchronous activity. Polling is most often used in terms of input/output , and is also referred to as polled or software driven .Polling is sometimes used...
by offering real-time interaction, using a persistent or long-lasting HTTP connection between the server and the client.
Since browsers and proxies are not designed with server events in mind, several techniques to achieve this have been developed, each with different benefits and drawbacks. The biggest hurdle is the HTTP 1.1 specification, which states that a browser should not have more than two simultaneous connections with a web server. However, holding one connection open for real-time events has a negative impact on browser usability: the browser may be blocked from sending a new request while waiting for the results of a previous request, e.g., a series of images. This can be worked around by creating a distinct hostname
Hostname
A hostname is a label that is assigned to a device connected to a computer network and that is used to identify the device in various forms of electronic communication such as the World Wide Web, e-mail or Usenet...
for real-time information, which is an alias for the same physical server.
Specific methods of implementing Comet fall into two major categories: streaming and long polling.
Streaming
An application using streaming Comet opens a single persistent connection from the client browserWeb browser
A web browser is a software application for retrieving, presenting, and traversing information resources on the World Wide Web. An information resource is identified by a Uniform Resource Identifier and may be a web page, image, video, or other piece of content...
to the server for all Comet events
Event (computing)
In computing an event is an action that is usually initiated outside the scope of a program and that is handled by a piece of code inside the program. Typically events are handled synchronous with the program flow, that is, the program has one or more dedicated places where events are handled...
. These events are incrementally handled and interpreted on the client side every time the server sends a new event, with neither side closing the connection.
Specific techniques for accomplishing streaming Comet include the following:
Hidden iframe
A basic technique for dynamic web application is to use a hidden iframeIFrame
iFrame can be:* I-frames, in video compression; see video compression picture types* iFrame * The HTML iframe element....
HTML element (an inline frame, which allows a website to embed one HTML document inside another). This invisible iframe is sent as a chunked
Chunked transfer encoding
Chunked transfer encoding is a data transfer mechanism in version 1.1 of the Hypertext Transfer Protocol in which a web server serves content in a series of chunks. It uses the Transfer-Encoding HTTP response header in place of the Content-Length header, which the protocol would otherwise require...
block, which implicitly declares it as infinitely long (sometimes called “forever frame”). As events occur, the iframe is gradually filled with
script tags, containing JavaScript to be executed in the browser. Because browsers render HTML pages incrementally, each script tag is executed as it is received.One benefit of the iframe method is that it works in every common browser. Two downsides of this technique are the lack of a reliable error handling method, and the impossibility of tracking the state of the request calling process.
XMLHttpRequest
The XMLHttpRequestXMLHttpRequest
XMLHttpRequest is an API available in web browser scripting languages such as JavaScript. It is used to send HTTP or HTTPS requests directly to a web server and load the server response data directly back into the script. The data might be received from the server as XML text or as plain text...
(XHR) object, the main tool used by Ajax applications for browser–server communication, can also be pressed into service for server–browser Comet messaging, in a few different ways.
In 1995, Netscape Navigator
Netscape Navigator
Netscape Navigator was a proprietary web browser that was popular in the 1990s. It was the flagship product of the Netscape Communications Corporation and the dominant web browser in terms of usage share, although by 2002 its usage had almost disappeared...
added a feature called “server push”, which allowed servers to send new versions of an image or HTML page to that browser, as part of a multipart HTTP response (see History section, below), using the content type
multipart/x-mixed-replace. Since 2004, GeckoGecko (layout engine)
Gecko is a free and open source layout engine used in many applications developed by Mozilla Foundation and the Mozilla Corporation , as well as in many other open source software projects....
-based browsers such as Firefox
Mozilla Firefox
Mozilla Firefox is a free and open source web browser descended from the Mozilla Application Suite and managed by Mozilla Corporation. , Firefox is the second most widely used browser, with approximately 25% of worldwide usage share of web browsers...
accept multipart responses to XHR, which can therefore be used as a streaming Comet transport. On the server side, each message is encoded as a separate portion of the multipart response, and on the client, the callback function
Callback (computer science)
In computer programming, a callback is a reference to executable code, or a piece of executable code, that is passed as an argument to other code. This allows a lower-level software layer to call a subroutine defined in a higher-level layer....
provided to the XHR
onreadystatechange function will be called as each message arrives. This functionality is only included in Gecko-based browsers, though there is discussion of adding it to WebKitWebKit
WebKit is a layout engine designed to allow web browsers to render web pages. WebKit powers Google Chrome and Apple Safari and by October 2011 held over 33% of the browser market share between them. It is also used as the basis for the experimental browser included with the Amazon Kindle ebook...
.
Instead of creating a multipart response, and depending on the browser to transparently parse each event, it is also possible to generate a custom data format for an XHR response, and parse out each event using browser-side JavaScript, relying only on the browser firing the
onreadystatechange callback each time it receives new data.Ajax with long polling
None of the above streaming transports work across all modern browsers without negative side-effects. This forces Comet developers to implement several complex streaming transports, switching between them depending on the browser. Consequently many Comet applications use long polling, which is easier to implement on the browser side, and works, at minimum, in every browser that supports XHR. As the name suggests, long polling requires the client to poll the server for an event (or set of events). The browser makes an Ajax-style request to the server, which is kept open until the server has new data to send to the browser, which is sent to the browser in a complete response. The browser initiates a new long polling request in order to obtain subsequent events.Specific technologies for accomplishing long-polling include the following:
XMLHttpRequest long polling
For the most part, XMLHttpRequest long polling works like any standard use of XHR. The browser makes an asynchronous request of the server, which may wait for data to be available before responding. The response can contain encoded data (typically XMLXML
Extensible Markup Language is a set of rules for encoding documents in machine-readable form. It is defined in the XML 1.0 Specification produced by the W3C, and several other related specifications, all gratis open standards....
or JSON
JSON
JSON , or JavaScript Object Notation, is a lightweight text-based open standard designed for human-readable data interchange. It is derived from the JavaScript scripting language for representing simple data structures and associative arrays, called objects...
) or Javascript to be executed by the client. At the end of the processing of the response, the browser creates and sends another XHR, to await the next event. Thus the browser always keeps a request outstanding with the server, to be answered as each event occurs.
Script tag long polling
While any Comet transport can be made to work across subdomains, none of the above transports can be used across different second-level domainSecond-level domain
In the Domain Name System hierarchy, a second-level domain is a domain that is directly below a top-level domain . For example, in example.com, example is the second-level domain of the .com TLD....
s (SLDs), due to browser security policies designed to prevent cross-site scripting
Cross-site scripting
Cross-site scripting is a type of computer security vulnerability typically found in Web applications that enables attackers to inject client-side script into Web pages viewed by other users. A cross-site scripting vulnerability may be used by attackers to bypass access controls such as the same...
attacks. That is, if the main web page is served from one SLD, and the Comet server is located at another SLD, Comet events cannot be used to modify the HTML and DOM of the main page, using those transports. This problem can be side-stepped by creating a proxy server
Proxy server
In computer networks, a proxy server is a server that acts as an intermediary for requests from clients seeking resources from other servers. A client connects to the proxy server, requesting some service, such as a file, connection, web page, or other resource available from a different server...
in front of one or both sources, making them appear to originate from the same domain. However, this is often undesirable for complexity or performance reasons.
Unlike iframes or XMLHttpRequest objects,
script tags can be pointed at any URIUniform Resource Identifier
In computing, a uniform resource identifier is a string of characters used to identify a name or a resource on the Internet. Such identification enables interaction with representations of the resource over a network using specific protocols...
, and JavaScript code in the response will be executed in the current HTML document. This creates a potential security risk for both servers involved, though the risk to the data provider (in our case, the Comet server) can be avoided using JSONP
JSONP
JSONP or "JSON with padding" is a complement to the base JavaScript Object Notation JSON data format, a pattern of usage allowing a page to request data from a server in a different domain...
.
A long-polling Comet transport can be created by dynamically creating
script elements, and setting their source to the location of the Comet server, which then sends back JavaScript (or JSONP) with some event as its payload. Each time the script request is completed, the browser opens a new one, just as in the XHR long polling case. This method has the advantage of being cross-browser while still allowing cross-domain implementations.Early Java applets
The ability to embed Java appletJava applet
A Java applet is an applet delivered to users in the form of Java bytecode. Java applets can run in a Web browser using a Java Virtual Machine , or in Sun's AppletViewer, a stand-alone tool for testing applets...
s into browsers (starting with Netscape 2.0 in March 1996) made real-time communications possible, using a raw TCP
Transmission Control Protocol
The Transmission Control Protocol is one of the core protocols of the Internet Protocol Suite. TCP is one of the two original components of the suite, complementing the Internet Protocol , and therefore the entire suite is commonly referred to as TCP/IP...
socket to communicate between the browser and the server. This socket can remain open as long as the browser is at the document hosting the applet. Event notifications can be sent in any format — text or binary — and decoded by the applet.
First Comet applications
In March 2006, software engineerSoftware engineer
A software engineer is an engineer who applies the principles of software engineering to the design, development, testing, and evaluation of the software and systems that make computers or anything containing software, such as computer chips, work.- Overview :...
Alex Russell coined the term Comet in a post on his personal blog. The new term was a play on Ajax
Ajax (programming)
Ajax is a group of interrelated web development methods used on the client-side to create asynchronous web applications...
(Ajax and Comet
Comet (cleanser)
Comet is a powdered cleaning product sold in North America and distributed in the United States by Prestige Brands. Scratch Free Comet with Bleach Disinfectant Cleanser contains 1.2% sodium dichloro-s-triazinetrione dihydrate and 98.8% "other" ingredients...
both being household cleaners common in the USA).
In 2006, while not really using the term Comet, some applications exposed those techniques to a wider audience: Meebo
Meebo
Meebo is a social platform connecting users with their friends across the web. It began in 2005 as a browser based instant messaging program which supported multiple IM services, including Yahoo! Messenger, Windows Live Messenger, AIM, ICQ, MySpaceIM, Facebook Chat, Google Talk, CafeMom and...
’s multi-protocol web-based chat application enables users to connect to AOL
AOL Instant Messenger
AOL Instant Messenger is an instant messaging and presence computer program which uses the proprietary OSCAR instant messaging protocol and the TOC protocol to allow registered users to communicate in real time. It was released by AOL in May 1997...
, Yahoo
Yahoo! Messenger
Yahoo! Messenger is an advertisement-supported instant messaging client and associated protocol provided by Yahoo!...
, and Microsoft chat platforms through the browser; Google added web-based chat to Gmail
Gmail
Gmail is a free, advertising-supported email service provided by Google. Users may access Gmail as secure webmail, as well via POP3 or IMAP protocols. Gmail was launched as an invitation-only beta release on April 1, 2004 and it became available to the general public on February 7, 2007, though...
; JotSpot, a startup since acquired by Google, built Comet-based real-time collaborative document editing. New Comet companies and enterprise solutions were created, such as the Java-based ICEfaces
ICEfaces
ICEfaces is an open source Ajax framework that enables Java EE application developers to create and deploy server-based rich Internet application using the Java language....
JSF
JavaServer Faces
JavaServer Faces is a Java-based Web application framework intended to simplify development integration of web-based user interfaces....
framework (although they prefer the term "Ajax Push"). Others that had previously used Java-applet based transports switched instead to pure-JavaScript implementations.
Alternatives
Browser-native technologies are inherent in the term Comet. Attempts to improve non-polling HTTP communication have come from multiple sides:- The HTML 5HTML 5HTML5 is a language for structuring and presenting content for the World Wide Web, and is a core technology of the Internet originally proposed by Opera Software. It is the fifth revision of the HTML standard and is still under development...
draft specification produced by the Web Hypertext Application Technology Working GroupWeb Hypertext Application Technology Working GroupThe Web Hypertext Application Technology Working Group is a community of people interested in evolving HTML and related technologies. The WHATWG was founded by individuals from Apple, the Mozilla Foundation and Opera Software in 2004. Since then, the editor of the WHATWG specifications, Ian...
(WHATWG) specifies so called server-sent eventsServer-sent eventsServer-sent events is a technology for providing push notifications from a server to a browser client in the form of DOM events. The Server-Sent Events EventSource API is now being standardized as part of HTML5 by the W3C.-History:...
, it offers a new HTML element,event-sourceand a new data format called DOM event stream. Experimental implementation of this feature was introduced in OperaOpera (web browser)Opera is a web browser and Internet suite developed by Opera Software with over 200 million users worldwide. The browser handles common Internet-related tasks such as displaying web sites, sending and receiving e-mail messages, managing contacts, chatting on IRC, downloading files via BitTorrent,...
9. - The HTML 5HTML 5HTML5 is a language for structuring and presenting content for the World Wide Web, and is a core technology of the Internet originally proposed by Opera Software. It is the fifth revision of the HTML standard and is still under development...
WebSocket API working draft specifies a method for creating a persistent connection with a server and receiving messages via anonmessagecallback. - The Bayeux protocol by the Dojo Foundation. It leaves browser-specific transports in place, and defines a higher-level protocol for communication between browser and server, with the aim of allowing re-use of client-side JavaScriptClient-side JavaScriptClient-side JavaScript is JavaScript that runs on the client-side. While JavaScript was originally created to run this way, the term was coined because the language is no longer limited to just client-side, since server-side JavaScript is now available.-Environment:The most common Internet media...
code with multiple Comet servers, and allowing the same Comet server to communicate with multiple client-side JavaScript implementations. Bayeux is based on a publish/subscribe model, so servers supporting Bayeux have publish/subscribe built-in. - The BOSHBOSHBidirectional-streams Over Synchronous HTTP is a transport protocol that emulates a bidirectional stream between two entities by using multiple synchronous HTTP request/response pairs without requiring the use of polling or asynchronous chunking.It is a draft standard of the XMPP Standards...
protocol by the XMPP standards foundation. It emulates a bidirectional stream between browser and server by using two synchronous HTTP connections. - The JSONJSONJSON , or JavaScript Object Notation, is a lightweight text-based open standard designed for human-readable data interchange. It is derived from the JavaScript scripting language for representing simple data structures and associative arrays, called objects...
Request object, proposed by Douglas Crockford, would be an alternative to the XHR object. - Use of plugins, such as Java appletJava appletA Java applet is an applet delivered to users in the form of Java bytecode. Java applets can run in a Web browser using a Java Virtual Machine , or in Sun's AppletViewer, a stand-alone tool for testing applets...
s or the proprietary Adobe FlashAdobe FlashAdobe Flash is a multimedia platform used to add animation, video, and interactivity to web pages. Flash is frequently used for advertisements, games and flash animations for broadcast...
(Java BlazeDSBlazeDSBlazeDS is a server-based Java remoting and web messaging technology that allows you to connect to back-end distributed data and push data to Adobe Flex and Adobe Integrated Runtime Rich Internet applications...
is a server plugin which streams events to Flash applications). These have the advantage of working identically across all browsers with the appropriate plugin installed and need not rely on HTTP connections, but the disadvantage of requiring the plugin to be installed - GoogleGoogleGoogle Inc. is an American multinational public corporation invested in Internet search, cloud computing, and advertising technologies. Google hosts and develops a number of Internet-based services and products, and generates profit primarily from advertising through its AdWords program...
announced a new Channel API for Google App EngineGoogle App EngineGoogle App Engine is a platform as a service cloud computing platform for developing and hosting web applications in Google-managed data centers. It virtualizes applications across multiple servers,...
, implementing a Comet-like API with the help of a client Javascript library on the browser. It could be later replaced by HTML5 WebSocketsWebSocketsWebSocket is a technology providing for bi-directional, full-duplex communications channels, over a single Transmission Control Protocol socket. It is designed to be implemented in web browsers and web servers, but it can be used by any client or server application...
, however.
External links
- Comet Daily – a website dedicated to articles on Comet techniques
- HTTP Streaming at Ajax Patterns