100 Gigabit Ethernet
Encyclopedia
40 Gigabit Ethernet, or 40GbE, and 100 Gigabit Ethernet, or 100GbE, are high-speed computer network
Computer network
A computer network, often simply referred to as a network, is a collection of hardware components and computers interconnected by communication channels that allow sharing of resources and information....

 standards developed by the Institute of Electrical and Electronics Engineers
Institute of Electrical and Electronics Engineers
The Institute of Electrical and Electronics Engineers is a non-profit professional association headquartered in New York City that is dedicated to advancing technological innovation and excellence...

 (IEEE). They support sending Ethernet
Ethernet
Ethernet is a family of computer networking technologies for local area networks commercially introduced in 1980. Standardized in IEEE 802.3, Ethernet has largely replaced competing wired LAN technologies....

 frames at 40 and 100 gigabits per second over multiple 10 Gbit/s or 25 Gbit/s lanes. Previously, the fastest published Ethernet
Ethernet
Ethernet is a family of computer networking technologies for local area networks commercially introduced in 1980. Standardized in IEEE 802.3, Ethernet has largely replaced competing wired LAN technologies....

 standard was 10 Gigabit Ethernet
10 Gigabit Ethernet
The 10 gigabit Ethernet computer networking standard was first published in 2002. It defines a version of Ethernet with a nominal data rate of 10 Gbit/s , ten times faster than gigabit Ethernet.10 gigabit Ethernet defines only full duplex point to point links which are generally connected by...

.
They were first studied in November 2007, proposed as IEEE 802.3ba in 2008, and ratified in June 2010. Another variant was added in March 2011.

History

In June 2007 a trade group called "Road to 100G" was formed after the NXTcomm
Nxtcomm
NXTcomm is a company created after the joining of the Telecommunications Industry Associate and the United States Telecom Association in 2006.- McCormick Place :...

 trade show in Chicago
Chicago
Chicago is the largest city in the US state of Illinois. With nearly 2.7 million residents, it is the most populous city in the Midwestern United States and the third most populous in the US, after New York City and Los Angeles...

.
Official standards work was started by IEEE 802.3 Higher Speed Study Group. The P802.3ba Ethernet Task Force commenced on December 5, 2007 with the following project authorization request:

The purpose of this project is to extend the 802.3
protocol to operating speeds of 40 Gb/s and 100
Gb/s in order to provide a significant increase in
bandwidth while maintaining maximum
compatibility with the installed base of 802.3
interfaces, previous investment in research and
development, and principles of network
operation and management. The project is to
provide for the interconnection of equipment
satisfying the distance requirements of the
intended applications.

Physical standards

The 40/100 Gigabit Ethernet standards encompass a number of different Ethernet physical layer
Ethernet physical layer
The Ethernet physical layer is the physical layer component of the Ethernet family of computer network standards.The Ethernet physical layer evolved over a considerable time span and encompasses quite a few physical media interfaces and several magnitudes of speed...

 (PHY) specifications. A networking device may support different PHY types by means of pluggable modules. Optical modules are not standardized by any official standards body but are in multi-source agreements (MSAs). One agreement that supports 40 and 100 Gigabit Ethernet is the C Form-factor Pluggable
C Form-factor Pluggable
The C form-factor pluggable is a multi-source agreement to produce a common form-factor for the transmission high-speed digital signals...

 (CFP) MSA which was adopted for distances of 100+ meters. QSFP and CXP
CXP (connector)
In the field of computer networking, CXP is a copper connector system with twelve 10 Gbit/s links suitable for 100 Gigabit Ethernet, three 40 Gigabit Ethernet channels, or twelve 10 Gigabit Ethernet channels. or a single Infiniband 12× QDR link. The C is the Roman numeral for 100 as a memory...

 connector modules support shorter distances.

The standard supported only full-duplex operation. Other electrical objectives include:
  • Preserve the 802.3 / Ethernet frame format utilizing the 802.3 MAC
  • Preserve minimum and maximum FrameSize of current 802.3 standard
  • Support a bit error ratio
    Bit error ratio
    In digital transmission, the number of bit errors is the number of received bits of a data stream over a communication channel that have been altered due to noise, interference, distortion or bit synchronization errors....

     (BER) better than or equal to at the MAC/PLS service interface
  • Provide appropriate support for OTN
    Optical Transport Network
    ITU-T defines an Optical Transport Network as a set of Optical Network Elements connected by optical fibre links, able to provide functionality of transport, multiplexing, switching, management, supervision and survivability of optical channels carrying client signals...

  • Support MAC
    Media Access Control
    The media access control data communication protocol sub-layer, also known as the medium access control, is a sublayer of the data link layer specified in the seven-layer OSI model , and in the four-layer TCP/IP model...

     data rates of 40 and 100 Gbit/s
  • Provide Physical Layer specifications (PHY) for operation over single-mode optical fiber
    Single-mode optical fiber
    In fiber-optic communication, a single-mode optical fiber is an optical fiber designed to carry only a single ray of light . Modes are the possible solutions of the Helmholtz equation for waves, which is obtained by combining Maxwell's equations and the boundary conditions...

     (SMF), laser optimized multi-mode optical fiber
    Multi-mode optical fiber
    Multi-mode optical fiber is a type of optical fiber mostly used for communication over short distances, such as within a building or on a campus...

      (MMF) OM3 and OM4, copper cable assembly, and backplane
    Backplane
    A backplane is a group of connectors connected in parallel with each other, so that each pin of each connector is linked to the same relative pin of all the other connectors forming a computer bus. It is used as a backbone to connect several printed circuit boards together to make up a complete...

    .


The following nomenclature was used for the physical layers:
Physical layer 40 Gigabit Ethernet 100 Gigabit Ethernet
at least 1 m over a backplane 40GBASE-KR4
approximately 7 m over copper cable 40GBASE-CR4 100GBASE-CR10
at least 100 m over OM3 MMF 40GBASE-SR4 100GBASE-SR10
at least 125 m over OM4 MMF
at least 10 km over SMF 40GBASE-LR4 100GBASE-LR4
at least 40 km over SMF 100GBASE-ER4
serial SMF over 2 km 40GBASE-FR


The 100 m laser optimized multi-mode fiber (OM3) objective was met by parallel ribbon cable with 850 nm wavelength 10GBASE-SR like optics (40GBASE-SR4 and 100GBASE-SR10). The 1 m backplane objective with 4 lanes of 10GBASE-KR type PHYs (40GBASE-KR4). The 10 m copper cable objective is met with 4 or 10 differential lanes using SFF-8642 and SFF-8436 connectors. The 10 and 40 km 100G objectives with four wavelengths (around 1310 nm) of 25G optics (100GBASE-LR4 and 100GBASE-ER4) and the 10 km 40G objective with four wavelengths (around 1310 nm) of 10G optics (40GBASE-LR4).

In January 2010 another IEEE project authorization started a task force to define a 40 gigabit per second serial single-mode optical fiber standard (40GBASE-FR). This was approved as standard 802.3bg in March 2011. It used 1550 nm optics, had a reach of 2 km and was capable of receiving 1550 nm and 1310 nm wavelengths of light. The capability to receive 1310 nm light allows it to inter-operate with a longer reach 1310 nm PHY should one ever be developed. 1550 nm was chosen as the wavelength for 802.3bg transmission to make it compatible with existing test equipment and infrastructure.

In December 2010, a 10x10 Multi Source Agreement (10x10 MSA) began to define an optical Physical Medium Dependent (PMD) sublayer and establish compatible sources of low-cost, low-power, pluggable optical transceivers based on 10 optical lanes at 10 gigabits/second each. The 10x10 MSA was intended as an lower cost alternative to 100GBASE-LR4 for applications which do not require a link length longer than 2 km. It was intended for use with standard single mode G.652.C/D type low water peak cable with ten wavelengths ranging from 1523 to 1595 nm. The founding members were Google
Google
Google Inc. is an American multinational public corporation invested in Internet search, cloud computing, and advertising technologies. Google hosts and develops a number of Internet-based services and products, and generates profit primarily from advertising through its AdWords program...

, Brocade Communications, JDSU and Santur.
Other member companies of the 10x10 MSA included MRV, Enablence, Cyoptics, AFOP, OPLINK, Hitachi Cable America, AMS-IX, EXFO, Huawei
Huawei
Huawei is a Chinese multinational networking and telecommunications equipment and services company headquartered in Shenzhen, Guangdong, China...

, Kotura, Facebook
Facebook
Facebook is a social networking service and website launched in February 2004, operated and privately owned by Facebook, Inc. , Facebook has more than 800 million active users. Users must register before using the site, after which they may create a personal profile, add other users as...

 and Effdon when the 2 km specification was announced in March 2011.
The 10X10 MSA modules were intended to be the same size as the C Form-factor Pluggable
C Form-factor Pluggable
The C form-factor pluggable is a multi-source agreement to produce a common form-factor for the transmission high-speed digital signals...

 specifications.

Backplane

NetLogic Microsystems announced backplane modules in October 2010. This industry trend is important because standards-based 100GE interconnects may allow building optical backplanes at a fraction of price currently required by VCSEL
VCSEL
The vertical-cavity surface-emitting laser, or VCSEL , is a type of semiconductor laser diode with laser beam emission perpendicular from the top surface, contrary to conventional edge-emitting semiconductor lasers which emit from surfaces formed by cleaving the individual chip out of a...

 based implementations - such as those in found in multichassis systems from Cisco (CRS) and Juniper Networks (T-series).

Multimode fiber

In 2009, Mellanox and Reflex Photonics announced modules based on the CFP agreement.

Single mode fiber

Finisar
Finisar
Finisar Corporation is a global technology leader in optical communication components and subsystems. These products enable high-speed voice, video and data communications for networking, storage, wireless, and cable TV applications....

, Sumitomo Electric Industries
Sumitomo Electric Industries
is a major manufacturer of electric wire and optical fiber cables. It holds the lead in market share for electric cables within Japan. Its headquarters are in Chūō-ku, Osaka, Japan. The company's shares are listed in the first section of the Tokyo, Osaka, and Nagoya Stock Exchanges, and the Fukuoka...

, and OpNext all demonstrated singlemode 40 or 100 Gigabit Ethernet modules based on the C Form-factor Pluggable
C Form-factor Pluggable
The C form-factor pluggable is a multi-source agreement to produce a common form-factor for the transmission high-speed digital signals...

 agreement at the European Conference and Exhibition on Optical Communication in 2009.

Compatibility

Optical domain IEEE 802.3ba implementations were not compatible with the numerous 40G and 100G line rate transport systems which feature different optical layer and modulation formats. In particular, existing 40 Gigabit transport solutions that used dense wavelength-division multiplexing to pack four 10 Gigabit signals into one optical medium were not compatible with the IEEE 802.3ba standard, which used either coarse WDM in 1310 nm wavelength region with four 25 Gigabit or four 10 Gigabit channels, or parallel optics with four or ten optical fibers per direction.

Test and Measurement

Ixia developed Physical Coding Sublayer Lanes and
announced test equipment in 2009.

JDS Uniphase introduced test and measurement products for 40 and 100 Gigabit Ethernet in 2009.
Discovery Semiconductors introduced optoelectronics
Optoelectronics
Optoelectronics is the study and application of electronic devices that source, detect and control light, usually considered a sub-field of photonics. In this context, light often includes invisible forms of radiation such as gamma rays, X-rays, ultraviolet and infrared, in addition to visible light...

 converters for 100 gigabit testing of the 10 km and 40 km Ethernet standards.

Spirent Communications
Spirent
Spirent Communications plc is a multinational telecommunications testing company headquartered in Crawley, United Kingdom.It is listed on the London Stock Exchange and is a constituent of the FTSE 250 Index.-History:...

  introduced test and measurement products in 2009 and 2010.
Xena Networks demonstrated test equipment at the Technical University of Denmark
Technical University of Denmark
The Technical University of Denmark , often simply referred to as DTU, is a university just north of Copenhagen, Denmark. It was founded in 1829 at the initiative of Hans Christian Ørsted as Denmark's first polytechnic, and is today ranked among Europe's leading engineering institutions, and the...

 in January 2011.
EXFO
EXFO
EXFO is a provider of test and service assurance solutions for wireless and wireline network operators and equipment manufacturers in the global telecommunications industry...

 demonstrated interoperability in January 2010.

These products verify Ethernet protocol implementation but do not test physical layer compliance to IEEE PMD specifications.

First commercial 100GE systems

Unlike the "race to 10Gbps" that was driven by the imminent needs to address growth pains of Internet in late 1990s, customer interest to 100Gbit/s technologies was mostly driven by economy factors. Among those, the commonly reasons to adopt 100GE were:
  • Reduction in number of lambdas, ability to stopgap proliferation of lit fiber
  • Better bandwidth utilization relative to 10Gbit/s link aggregates
  • Cheaper wholesale, internet peering and datacenter interconnect connectivity
  • Desire to "skip" the relatively expensive 40Gbit/s technology and move directly from 10Gbit/s to 100Gbit/s


Considering that 100GE technology is natively compatible with OTN
Optical Transport Network
ITU-T defines an Optical Transport Network as a set of Optical Network Elements connected by optical fibre links, able to provide functionality of transport, multiplexing, switching, management, supervision and survivability of optical channels carrying client signals...

 hierarchy and there is no separate adaptation for SONET/SDH and Ethernet networks, it was widely believed that 100GE technology adoption will be driven by products in all network layers, from transport systems to edge routers and datacenter switches. Nevertheless, in 2011 components for 100GE networks were not a commodity and most vendors entering this market relied on both internal R&D projects and extensive cooperation with other companies.

Optical Transport Systems

Solving the challenges of optical signal transmission over a nonlinear medium is principally an analog design problem. As such, it has evolved at a slower rate relative to digital circuit lithography advances (which have generally progressed in step with Moore's law
Moore's Law
Moore's law describes a long-term trend in the history of computing hardware: the number of transistors that can be placed inexpensively on an integrated circuit doubles approximately every two years....

.) This explains why 10Gbit/s transport systems have existed since the mid-1990s, while the first forays into 100Gbit/s transmission happened about 15 years later - a 10x speed increase over 15 years is far slower than the 2x speed per 1.5 years typically cited for Moore's law tracking technologies. Nevertheless, as of Aug 2011 at least four firms (Ciena, Alcatel-Lucent, MRV, ADVA Optical and Huawei) have made customer announcements for 100Gbit/s transport systems - although with varying degrees of capabilities. Although most vendors claim that 100Gbit/s lightpaths can utilize existing analog optical infrastructure, in practice deployment of new, high-speed lambdas remains tightly controlled and extensive interoperability tests are required before moving new capacity into service.

Routers and switches with 100GE interfaces

Design of router or switch with support for 100Gbit/s interfaces is not an easy feat for multiple reasons. One of them is the need to process a 100Gbit/s stream of packets at line rate without reordering within IP/MPLS microflows. As of 2011, most components in the 100Gbit/s packet processing path (PHY chips, NPUs, memories) were not readily available off the shelf or require extensive qualification and co-design. Another problem is related to the low-output production of 100Gbit/s optical components, which were also not easily available - especially in pluggable, long-reach or tunable laser flavors. Therefore, in the early days of 100GE, vendors considered this market to be a technology showcase and were not shy to advertise their technological prowess.

In the below historical breakdown of 100GE routing and switching milestones, we keep separate track for the dates of product announcements, trials and revenue shipments (where known).

Alcatel-Lucent
Alcatel-Lucent
Alcatel-Lucent is a global telecommunications corporation, headquartered in the 7th arrondissement of Paris, France. It provides telecommunications solutions to service providers, enterprises, and governments around the world, enabling these customers to deliver voice, data, and video services...



Alcatel-Lucent first announced 100GbE interfaces for their 7450 ESS/7750 SR platform in June 2009, with field trials following in June–September 2010. However, in April 2011 presentation, James Watt (ALU optical division president) still mentioned 100GE technology as "demo" staged for T-Systems and Portugal Telecom. Later, in a June 2011 press-release with Verizon, the company again referenced 100GE as "trial" Thus, despite of being able to bundle the self-developed optical and routing system, Alcatel apparently missed the chance to book early revenue with 100GE deployments.

In a separate press release from June 2011, Alcatel-Lucent announced the new generation of packet processing silicon dubbed FP3, which may hint towards company's strategy and timeline on commercial shipments of 100GE products.

Brocade Communications Systems
Brocade Communications Systems
Brocade Communications Systems, Inc. , based in Silicon Valley , is a vendor of storage area network hardware and software. The company also designs, manufactures, and sells networking products and management applications for local, metro, and wide area networks...



In September 2010, Brocade announced their first 100GbE solution to be based on the former Foundry Networks hardware (MLXe). Quite impressively, in June 2011 (less than a year from initial press statement), the new product went live at AMS-IX traffic exchange point in Amsterdam, bringing first-ever 100GE revenue for Brocade. This feat is even more impressive considering that Brocade commonly uses 3rd party network processors and optics. Rumored to be priced around $100K per port, the 2x 100GE linecard for MLXe appears geared for aggressive competition, although it is still unknown whether this product is capable to perform beyond IP peering applications or support long-haul / tunable optics.

Cisco Systems
Cisco Systems
Cisco Systems, Inc. is an American multinational corporation headquartered in San Jose, California, United States, that designs and sells consumer electronics, networking, voice, and communications technology and services. Cisco has more than 70,000 employees and annual revenue of US$...



The joint Cisco-Comcast press release on first-ever 100GE trials went out back in 2008, however it is doubtful this transmission could approach 100Gbit/s speeds when using a 40Gbit/s/per slot CRS-1 platform for packet processing. The need to wait for the next generation of routing hardware can explain the fact that the following milestone for Cisco 100GE program did not happen until March 2010, when field trial in AT&T network added color to launch of a new CRS-3 router. The first 100GE deployments at AT&T
AT&T
AT&T Inc. is an American multinational telecommunications corporation headquartered in Whitacre Tower, Dallas, Texas, United States. It is the largest provider of mobile telephony and fixed telephony in the United States, and is also a provider of broadband and subscription television services...

 and Comcast
Comcast
Comcast Corporation is the largest cable operator, home Internet service provider, and fourth largest home telephone service provider in the United States, providing cable television, broadband Internet, and telephone service to both residential and commercial customers in 39 states and the...

 happened 12 months later, in April 2011. In addition, later in the same year, Cisco have tested the 100GE interface between CRS-3 and the next generation of their ASR9K edge router, although offering no information on hardware availability for the latter.

Huawei
Huawei
Huawei is a Chinese multinational networking and telecommunications equipment and services company headquartered in Shenzhen, Guangdong, China...



In October 2008, the Chinese vendor presented the "industry's first" 100GE interface for their flagship router, NE5000e. Almost a year later, in September 2009, Huawei also presented an end-to-end 100G solution consisting of OSN6800/8800 optical transport and 100GE ports on NE5000e. This time, it was also mentioned that Huawei's solution had the new self-developed NPU "Solar 2.0 PFE2A" onboard and was using pluggable optics in CFP
C Form-factor Pluggable
The C form-factor pluggable is a multi-source agreement to produce a common form-factor for the transmission high-speed digital signals...

 form-factor. In a mid-2010 solution brief, the new NE5000e linecards were given commercial name (LPUF-100) and were credited with using two Solar-2.0 NPUs per 100GE port in opposite (ingress/egress) configuration. Nevertheless, in October 2010, the company referenced shipments of NE5000e to Russian cell operator "Megafon" as "40Gbps/slot" solution, with "scalability up to" 100Gbit/s.

April 2011 brought a new 100GE announcement from Huawei - now the NE5000e platform was updated to carry 2x100GE interfaces per slot using LPU-200 linecards. In a related solution brief, Huawei reported 120 thousand 20G/40G Solar 1.0 chips as shipped to customers, but no Solar 2.0 numbers were given. Also, following the August 2011 100G trial in Russia, Huawei reported paying 100G DWDM customers, but no 100GE shipments on NE5000e.

Juniper Networks
Juniper Networks
Juniper Networks is an information technology and computer networking products multinational company, founded in 1996. It is head quartered in Sunnyvale, California, USA. The company designs and sells high-performance Internet Protocol network products and services...



Juniper first announced the 100GE to come to its T-series routers in June 2009. By this time, the latest incarnation of T-series, known as T1600 has been shipping for almost two years and supported the 100Gbit linecards in 10x10GE configuration. The 1x100GE option followed in Nov 2010, when a joint press release with academic backbone network Internet2 marked the first production 100GE interfaces going live in real network. Later in the same year, Juniper demoed 100GE operation between core (T-series) and edge (MX 3D) routers. Juniper confirmed it's grip of the market again in March 2011, stealing thunder from Cisco by announcing first shipments of 100GE interfaces to a major North American service provider (Verizon). In the meanwhile, the company was apparently busy selling 100GE cards to a host of smaller operators (such as UK's JANET).

Standardization time line

IEEE standardization project history:
  • Call for interest at IEEE 802.3 plenary meeting in San Diego — July 18, 2006
  • First HSSG study group meeting — September 2006
  • Last study group meeting — November 2007
  • Task Force formally approved as P802.3ba by IEEE LMSC — December 5, 2007
  • First P802.3ba task force meeting — January 2008
  • IEEE 802.3 working group ballot — March 2009
  • IEEE LMSC sponsor ballot — November 2009
  • First 40 Gbit/s Ethernet Single-mode Fibre PMD study group meeting — January 2010.
  • P802.3bg task force approved for 40 Gbit/s serial SMF PMD— March 25, 2010
  • IEEE 802.3ba standard approved — June 17, 2010
  • IEEE 802.3bg standard approved — March 2011
  • IEEE 802.3bj 100 Gbit/s Backplane and Copper Cable Task Force PAR approval due — September 2011


P802.3ba Task Force draft release dates:
  • Draft 1.0 — October 1, 2008
  • Draft 1.1 — December 9, 2008
  • Draft 1.2 — February 10, 2009
  • Draft 2.0 — March 12, 2009 (for working group ballot)
  • Draft 2.1 — May 29, 2009
  • Draft 2.2 — August 15, 2009
  • Draft 2.3 — October 14, 2009
  • Draft 3.0 — November 18, 2009 (for sponsor group ballot)
  • Draft 3.1 — February 10, 2010
  • Draft 3.2 — March 24, 2010
  • Final — June 17, 2010

See also

  • CFP MSA module
    C Form-factor Pluggable
    The C form-factor pluggable is a multi-source agreement to produce a common form-factor for the transmission high-speed digital signals...

  • Road to 100G
    Road to 100G
    The Road to 100G was a nonprofit corporation formed by its members for the purpose of providing seamless interoperability among the disparate, standards-based components required in building high capacity network elements...

  • Energy Efficient Ethernet
    Energy Efficient Ethernet
    Energy-Efficient Ethernet is a set of enhancements to the twisted-pair and backplane Ethernet family of computer networking standards that will allow for less power consumption during periods of low data activity. The intention was to reduce power consumption by 50% or more, while retaining full...

  • Optical interconnect
    Optical interconnect
    Optical interconnect is a way of communication by optical cables. Compared to traditional cables, optical wires are capable of a much higher bandwidth, from 10 Gb/s up to 100 Gb/s....

  • Interconnect bottleneck
    Interconnect bottleneck
    The interconnect bottleneck, the point at which integrated circuits reach their capacity, is expected sometime around 2010.Improved performance of computer systems has been achieved, in large part, by downscaling the IC minimum feature size. This allows the basic IC building block, the transistor,...

  • Optical fiber cable
    Optical fiber cable
    An optical fiber cable is a cable containing one or more optical fibers. The optical fiber elements are typically individually coated with plastic layers and contained in a protective tube suitable for the environment where the cable will be deployed....

  • Optical communication
    Optical communication
    Optical communication is any form of telecommunication that uses light as the transmission medium.An optical communication system consists of a transmitter, which encodes a message into an optical signal, a channel, which carries the signal to its destination, and a receiver, which reproduces the...

  • Parallel optical interface
    Parallel optical interface
    A parallel optical interface is a form of fiber optic technology aimed primarily at communications and networking over relatively short distances , and at high bandwidths....



Further reading



External links

The source of this article is wikipedia, the free encyclopedia.  The text of this article is licensed under the GFDL.
 
x
OK