Overlay network
Encyclopedia
An overlay network is a computer network
which is built on the top of another network. Node
s in the overlay can be thought of as being connected by virtual or logical links, each of which corresponds to a path, perhaps through many physical links, in the underlying network. For example, distributed systems such as cloud computing
, peer-to-peer
networks, and client-server
applications are overlay networks because their nodes run on top of the Internet
. The Internet was built as an overlay upon the telephone network
.
because of the availability of digital circuit switching
equipments and optical fiber
. Telecommunication transport networks and IP networks (that combined make up the broader Internet) are all overlaid with at least an optical layer, a transport layer and an IP or circuit layers (in the case of the PSTN).
Enterprise private network
s were first overlaid on telecommunication networks such as frame relay
and Asynchronous Transfer Mode
packet switching
infrastructures but migration from these (now legacy) infrastructures to IP based MPLS
networks and virtual private network
s started (2001~2002).
From a physical standpoint overlay networks are quite complex (see Figure 1) as they combine various logical layers that are operated and built by various entities (businesses, universities, government etc) but they allow separation of concerns (and healthy business competition) that over time permitted the build up of a broad set of services that could not have been proposed by a single telecommunication operator (ranging from broadband Internet access
, voice over IP or IPTV
, competitive telecom operators etc).
of messages to destinations not specified by an IP address
. For example, distributed hash table
s can be used to route messages to a node having a specific logical address
, whose IP address is not known in advance.
Overlay networks have also been proposed as a way to improve Internet routing
, such as through quality of service
guarantees to achieve higher-quality streaming media
. Previous proposals such as IntServ, DiffServ, and IP multicast
have not seen wide acceptance largely because they require modification of all routers in the network. On the other hand, an overlay network can be incrementally deployed on end-hosts running the overlay protocol software, without cooperation from ISPs
. The overlay has no control over how packets are routed in the underlying network between two overlay nodes, but it can control, for example, the sequence of overlay nodes a message traverses before reaching its destination.
For example, Akamai Technologies
manages an overlay network which provides reliable, efficient content delivery (a kind of multicast
). Academic research includes End System Multicast and Overcast for multicast; RON
(Resilient Overlay Network) for resilient routing; and OverQoS for quality of service guarantees, among others.
For example, Virtela Technology Services provides an overlay network in 90+ countries on top of 500+ different underlying telecom providers.
Overlay network protocols based on UDP/IP include:
Computer network
A computer network, often simply referred to as a network, is a collection of hardware components and computers interconnected by communication channels that allow sharing of resources and information....
which is built on the top of another network. Node
Node (networking)
In communication networks, a node is a connection point, either a redistribution point or a communication endpoint . The definition of a node depends on the network and protocol layer referred to...
s in the overlay can be thought of as being connected by virtual or logical links, each of which corresponds to a path, perhaps through many physical links, in the underlying network. For example, distributed systems such as cloud computing
Cloud computing
Cloud computing is the delivery of computing as a service rather than a product, whereby shared resources, software, and information are provided to computers and other devices as a utility over a network ....
, peer-to-peer
Peer-to-peer
Peer-to-peer computing or networking is a distributed application architecture that partitions tasks or workloads among peers. Peers are equally privileged, equipotent participants in the application...
networks, and client-server
Client-server
The client–server model of computing is a distributed application that partitions tasks or workloads between the providers of a resource or service, called servers, and service requesters, called clients. Often clients and servers communicate over a computer network on separate hardware, but both...
applications are overlay networks because their nodes run on top of the Internet
Internet
The Internet is a global system of interconnected computer networks that use the standard Internet protocol suite to serve billions of users worldwide...
. The Internet was built as an overlay upon the telephone network
Telephone network
A telephone network is a telecommunications network used for telephone calls between two or more parties.There are a number of different types of telephone network:...
.
In telecommunication
Overlay networks are used in telecommunicationTelecommunication
Telecommunication is the transmission of information over significant distances to communicate. In earlier times, telecommunications involved the use of visual signals, such as beacons, smoke signals, semaphore telegraphs, signal flags, and optical heliographs, or audio messages via coded...
because of the availability of digital circuit switching
Circuit switching
Circuit switching is a methodology of implementing a telecommunications network in which two network nodes establish a dedicated communications channel through the network before the nodes may communicate. The circuit guarantees the full bandwidth of the channel and remains connected for the...
equipments and optical fiber
Optical fiber
An optical fiber is a flexible, transparent fiber made of a pure glass not much wider than a human hair. It functions as a waveguide, or "light pipe", to transmit light between the two ends of the fiber. The field of applied science and engineering concerned with the design and application of...
. Telecommunication transport networks and IP networks (that combined make up the broader Internet) are all overlaid with at least an optical layer, a transport layer and an IP or circuit layers (in the case of the PSTN).
Enterprise private network
Enterprise Private Network
An enterprise private network is a computer network built by a business to interconnect its various company sites in order to share computer resources.-History:...
s were first overlaid on telecommunication networks such as frame relay
Frame relay
Frame Relay is a standardized wide area network technology that specifies the physical and logical link layers of digital telecommunications channels using a packet switching methodology...
and Asynchronous Transfer Mode
Asynchronous Transfer Mode
Asynchronous Transfer Mode is a standard switching technique designed to unify telecommunication and computer networks. It uses asynchronous time-division multiplexing, and it encodes data into small, fixed-sized cells. This differs from approaches such as the Internet Protocol or Ethernet that...
packet switching
Packet switching
Packet switching is a digital networking communications method that groups all transmitted data – regardless of content, type, or structure – into suitably sized blocks, called packets. Packet switching features delivery of variable-bit-rate data streams over a shared network...
infrastructures but migration from these (now legacy) infrastructures to IP based MPLS
Multiprotocol Label Switching
Multiprotocol Label Switching is a mechanism in high-performance telecommunications networks that directs data from one network node to the next based on short path labels rather than long network addresses, avoiding complex lookups in a routing table. The labels identify virtual links between...
networks and virtual private network
Virtual private network
A virtual private network is a network that uses primarily public telecommunication infrastructure, such as the Internet, to provide remote offices or traveling users access to a central organizational network....
s started (2001~2002).
From a physical standpoint overlay networks are quite complex (see Figure 1) as they combine various logical layers that are operated and built by various entities (businesses, universities, government etc) but they allow separation of concerns (and healthy business competition) that over time permitted the build up of a broad set of services that could not have been proposed by a single telecommunication operator (ranging from broadband Internet access
Broadband Internet access
Broadband Internet access, often shortened to just "broadband", is a high data rate, low-latency connection to the Internet— typically contrasted with dial-up access using a 56 kbit/s modem or satellite Internet with inherently high latency....
, voice over IP or IPTV
IPTV
Internet Protocol television is a system through which television services are delivered using the Internet protocol suite over a packet-switched network such as the Internet, instead of being delivered through traditional terrestrial, satellite signal, and cable television formats.IPTV services...
, competitive telecom operators etc).
Over the Internet
Nowadays the Internet is the basis for more overlaid networks that can be constructed in order to permit routingRouting
Routing is the process of selecting paths in a network along which to send network traffic. Routing is performed for many kinds of networks, including the telephone network , electronic data networks , and transportation networks...
of messages to destinations not specified by an IP address
IP address
An Internet Protocol address is a numerical label assigned to each device participating in a computer network that uses the Internet Protocol for communication. An IP address serves two principal functions: host or network interface identification and location addressing...
. For example, distributed hash table
Distributed hash table
A distributed hash table is a class of a decentralized distributed system that provides a lookup service similar to a hash table; pairs are stored in a DHT, and any participating node can efficiently retrieve the value associated with a given key...
s can be used to route messages to a node having a specific logical address
Logical address
In computing, a logical address is the address at which an item appears to reside from the perspective of an executing application program....
, whose IP address is not known in advance.
Overlay networks have also been proposed as a way to improve Internet routing
Routing
Routing is the process of selecting paths in a network along which to send network traffic. Routing is performed for many kinds of networks, including the telephone network , electronic data networks , and transportation networks...
, such as through quality of service
Quality of service
The quality of service refers to several related aspects of telephony and computer networks that allow the transport of traffic with special requirements...
guarantees to achieve higher-quality streaming media
Streaming media
Streaming media is multimedia that is constantly received by and presented to an end-user while being delivered by a streaming provider.The term "presented" is used in this article in a general sense that includes audio or video playback. The name refers to the delivery method of the medium rather...
. Previous proposals such as IntServ, DiffServ, and IP multicast
IP Multicast
IP multicast is a method of sending Internet Protocol datagrams to a group of interested receivers in a single transmission. It is often employed for streaming media applications on the Internet and private networks. The method is the IP-specific version of the general concept of multicast...
have not seen wide acceptance largely because they require modification of all routers in the network. On the other hand, an overlay network can be incrementally deployed on end-hosts running the overlay protocol software, without cooperation from ISPs
Internet service provider
An Internet service provider is a company that provides access to the Internet. Access ISPs directly connect customers to the Internet using copper wires, wireless or fiber-optic connections. Hosting ISPs lease server space for smaller businesses and host other people servers...
. The overlay has no control over how packets are routed in the underlying network between two overlay nodes, but it can control, for example, the sequence of overlay nodes a message traverses before reaching its destination.
For example, Akamai Technologies
Akamai Technologies
Akamai Technologies, Inc. is an Internet content delivery network headquartered in Cambridge, Massachusetts, US.The company was founded in 1998 by then-MIT graduate student Daniel M. Lewin, and MIT Applied Mathematics professor Tom Leighton...
manages an overlay network which provides reliable, efficient content delivery (a kind of multicast
Multicast
In computer networking, multicast is the delivery of a message or information to a group of destination computers simultaneously in a single transmission from the source creating copies automatically in other network elements, such as routers, only when the topology of the network requires...
). Academic research includes End System Multicast and Overcast for multicast; RON
RON
RON may refer to:* Polish–Lithuanian Commonwealth , a dualist monarchy of Poland and Lithuania* Re-open Nominations, a British ballot option similar to "none of the above"...
(Resilient Overlay Network) for resilient routing; and OverQoS for quality of service guarantees, among others.
For example, Virtela Technology Services provides an overlay network in 90+ countries on top of 500+ different underlying telecom providers.
List of overlay network protocols
Overlay network protocols based on TCP/IP include:- Distributed hash tableDistributed hash tableA distributed hash table is a class of a decentralized distributed system that provides a lookup service similar to a hash table; pairs are stored in a DHT, and any participating node can efficiently retrieve the value associated with a given key...
s (DHTs), such as KADKad NetworkThe Kad network is a peer-to-peer network which implements the Kademlia P2P overlay protocol. The majority of users on the Kad Network are also connected to servers on the eDonkey network, and Kad Network clients typically query known nodes on the eDonkey network in order to find an initial node...
and other protocols based on the KademliaKademliaKademlia is a distributed hash table for decentralized peer-to-peer computer networks designed by Petar Maymounkov and David Mazières in 2002. It specifies the structure of the network and the exchange of information through node lookups. Kademlia nodes communicate among themselves using UDP. A...
algorithm, for example. - JXTAJXTAJXTA is an open source peer-to-peer protocol specification begun by Sun Microsystems in 2001. The JXTA protocols are defined as a set of XML messages which allow any device connected to a network to exchange messages and collaborate independently of the underlying network topology.As JXTA is based...
- Many peer-to-peerPeer-to-peerPeer-to-peer computing or networking is a distributed application architecture that partitions tasks or workloads among peers. Peers are equally privileged, equipotent participants in the application...
protocols including GnutellaGnutellaGnutella is a large peer-to-peer network which, at the time of its creation, was the first decentralized peer-to-peer network of its kind, leading to other, later networks adopting the model...
, Gnutella2Gnutella2Gnutella2, often referred to as G2, is a peer-to-peer protocol developed mainly by Michael Stokes and released in 2002. While inspired by the gnutella protocol, G2 shares little of its design with the exception of its connection handshake and download mechanics. It adopts an extensible binary...
, FreenetFreenetFreenet is a decentralized, censorship-resistant distributed data store originally designed by Ian Clarke. According to Clarke, Freenet aims to provide freedom of speech through a peer-to-peer network with strong protection of anonymity; as part of supporting its users' freedom, Freenet is free and...
and I2PI2PI2P is a mixed-license, free and open source project building an anonymous network .The network is a simple layer that applications can use to anonymously and securely send...
. (Examples: LimewireLimeWireLimeWire is a free peer-to-peer file sharing client program that runs on Windows, Mac OS X, Linux, and other operating systems supported by the Java software platform. LimeWire uses the gnutella network as well as the BitTorrent protocol. A free software version and a purchasable "enhanced"...
, ShareazaShareazaShareaza is a peer-to-peer file sharing client running under Microsoft Windows which supports the gnutella, Gnutella2 , eDonkey, BitTorrent, FTP, HTTP and HTTPS network protocols and handles magnet links, ed2k links, and the now deprecated gnutella and Piolet links...
, utorrent, etc.) - PUCCP2P Universal Computing ConsortiumP2P Universal Computing Consortium is promoting research and development of an open P2P/Overlay network service platform that connects multi-types of devices users use, and conducts the standardization efforts. PUCC is a cross-industry consortium for open P2P/Overlay network standards...
- SolipsisSolipsisSolipsis is a free and open source system for a massively multi-participant shared virtual world designed by Joaquin Keller and Gwendal Simon at France Télécom Research and Development Labs. It aims to provide the infrastructure for a metaverse-like public virtual territory...
: a France TélécomFrance TélécomFrance Telecom S.A. is the main telecommunications company in France, the third-largest in Europe and one of the largest in the world. It currently employs about 180,000 people and has 192.7 million customers worldwide . In 2010 the group had revenue of €45.5 billion...
system for massively shared virtual world
Overlay network protocols based on UDP/IP include:
- Real Time Media Flow ProtocolReal Time Media Flow ProtocolReal Time Media Flow Protocol is a proprietary protocol developed by Adobe Systems. RTMFPenables low latency UDP streamings and direct peer-to-peer communication between multiple Adobe Flash Players...
- Adobe FlashAdobe FlashAdobe Flash is a multimedia platform used to add animation, video, and interactivity to web pages. Flash is frequently used for advertisements, games and flash animations for broadcast...