Myrinet
Encyclopedia
Myrinet, ANSI/VITA 26-1998, is a high-speed local area network
ing system designed by Myricom to be used as an interconnect between multiple machines to form computer clusters. Myrinet has much lower protocol overhead than standards such as Ethernet
, and therefore provides better throughput
, less interference, and lower latency
while using the host CPU. Although it can be used as a traditional networking system, Myrinet is often used directly by programs that "know" about it, thereby bypassing a call into the operating system
.
Myrinet physically consists of two fibre optic cables, upstream and downstream, connected to the host computers with a single connector. Machines are connected via low-overhead routers and switch
es, as opposed to connecting one machine directly to another. Myrinet includes a number of fault-tolerance features, mostly backed by the switches. These include flow control, error control, and "heartbeat" monitoring on every link. The newest, "fourth-generation" Myrinet, called Myri-10G, supports a 10 Gbit/s data rate and is interoperable with 10 Gigabit Ethernet
on PHY
, the physical layer (cables, connectors, distances, signaling). Myri-10G started shipping at the end of 2005.
, a high-performance parallel system tends to be bottlenecked by its slowest sequential process, which in all but the most embarrassingly parallel
supercomputer workloads is often the latency of message transmission across the network.
supercomputers used Myrinet technology. In the November 2005 TOP500, the number of supercomputers using Myrinet was down to 101 computers, or 20.2%, in November 2006, 79 (15.8%), and by November 2007, 18 (3.6%), a long way behind Gigabit Ethernet
at 54% and Infiniband
at 24.2%.
In the latest TOP500 list (June 2011), the number of supercomputers using Myrinet interconnect is 4 (0.8%).
Local area network
A local area network is a computer network that interconnects computers in a limited area such as a home, school, computer laboratory, or office building...
ing system designed by Myricom to be used as an interconnect between multiple machines to form computer clusters. Myrinet has much lower protocol overhead than standards such as Ethernet
Ethernet
Ethernet is a family of computer networking technologies for local area networks commercially introduced in 1980. Standardized in IEEE 802.3, Ethernet has largely replaced competing wired LAN technologies....
, and therefore provides better throughput
Throughput
In communication networks, such as Ethernet or packet radio, throughput or network throughput is the average rate of successful message delivery over a communication channel. This data may be delivered over a physical or logical link, or pass through a certain network node...
, less interference, and lower latency
Lag
Lag is a common word meaning to fail to keep up or to fall behind. In real-time applications, the term is used when the application fails to respond in a timely fashion to inputs...
while using the host CPU. Although it can be used as a traditional networking system, Myrinet is often used directly by programs that "know" about it, thereby bypassing a call into the operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
.
Myrinet physically consists of two fibre optic cables, upstream and downstream, connected to the host computers with a single connector. Machines are connected via low-overhead routers and switch
Network switch
A network switch or switching hub is a computer networking device that connects network segments.The term commonly refers to a multi-port network bridge that processes and routes data at the data link layer of the OSI model...
es, as opposed to connecting one machine directly to another. Myrinet includes a number of fault-tolerance features, mostly backed by the switches. These include flow control, error control, and "heartbeat" monitoring on every link. The newest, "fourth-generation" Myrinet, called Myri-10G, supports a 10 Gbit/s data rate and is interoperable with 10 Gigabit Ethernet
10 Gigabit Ethernet
The 10 gigabit Ethernet computer networking standard was first published in 2002. It defines a version of Ethernet with a nominal data rate of 10 Gbit/s , ten times faster than gigabit Ethernet.10 gigabit Ethernet defines only full duplex point to point links which are generally connected by...
on PHY
PHY
PHY is an abbreviation for the physical layer of the OSI model.An instantiation of PHY connects a link layer device to a physical medium such as an optical fiber or copper cable. A PHY device typically includes a Physical Coding Sublayer and a Physical Medium Dependent layer. The PCS encodes and...
, the physical layer (cables, connectors, distances, signaling). Myri-10G started shipping at the end of 2005.
Performance
Myrinet is a lightweight protocol with little overhead that allows it to operate with throughput close to the basic signaling speed of the physical layer. On the latest 2.0 Gbit/s links, Myrinet often runs at 1.98 Gbit/s of sustained throughput, considerably better than what Ethernet offers, which varies from 0.6 to 1.9 Gbit/s, depending on load. However, for supercomputing, the low latency of Myrinet is even more important than its throughput performance, since, according to Amdahl's lawAmdahl's law
Amdahl's law, also known as Amdahl's argument, is named after computer architect Gene Amdahl, and is used to find the maximum expected improvement to an overall system when only part of the system is improved...
, a high-performance parallel system tends to be bottlenecked by its slowest sequential process, which in all but the most embarrassingly parallel
Embarrassingly parallel
In parallel computing, an embarrassingly parallel workload is one for which little or no effort is required to separate the problem into a number of parallel tasks...
supercomputer workloads is often the latency of message transmission across the network.
Deployment
According to Myricom, 141 (28.2%) of the June 2005 TOP500TOP500
The TOP500 project ranks and details the 500 most powerful known computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year...
supercomputers used Myrinet technology. In the November 2005 TOP500, the number of supercomputers using Myrinet was down to 101 computers, or 20.2%, in November 2006, 79 (15.8%), and by November 2007, 18 (3.6%), a long way behind Gigabit Ethernet
Gigabit Ethernet
Gigabit Ethernet is a term describing various technologies for transmitting Ethernet frames at a rate of a gigabit per second , as defined by the IEEE 802.3-2008 standard. It came into use beginning in 1999, gradually supplanting Fast Ethernet in wired local networks where it performed...
at 54% and Infiniband
InfiniBand
InfiniBand is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable...
at 24.2%.
In the latest TOP500 list (June 2011), the number of supercomputers using Myrinet interconnect is 4 (0.8%).
See also
- List of device bandwidths
- NUMAlinkNUMAlinkNUMAlink is a system interconnect developed by SGI for use in its distributed shared memory ccNUMA computer systems. NUMAlink was originally developed by SGI for their Origin 2000 and Onyx2 systems...
- HIPPIHIPPIHIPPI is a computer bus for the attachment of high speed storage devices to supercomputers. It was popular in the late 1980s and into the mid-to-late 1990s, but has since been replaced by ever-faster standard interfaces like SCSI and Fibre Channel.The first HIPPI standard defined a 50-wire...
- InfinibandInfiniBandInfiniBand is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable...
- QuadricsQuadricsQuadrics was a supercomputer company formed in 1996 as a joint venture between Alenia Spazio and the technical team from Meiko Scientific. They produced hardware and software for clustering commodity computer systems into massively parallel systems. Their highpoint was in June 2003 when six out of...
- Scalable Coherent InterconnectScalable Coherent InterconnectSCI, for Scalable Coherent Interface, is a high-speed interconnect standard for shared memory multiprocessing and message passing. The IEEE Std 1596-1992, IEEE Standard for Scalable Coherent Interface was approved by the IEEE standards board on March 19, 1992...
(SCI) - Gigabit EthernetGigabit EthernetGigabit Ethernet is a term describing various technologies for transmitting Ethernet frames at a rate of a gigabit per second , as defined by the IEEE 802.3-2008 standard. It came into use beginning in 1999, gradually supplanting Fast Ethernet in wired local networks where it performed...
- 10 Gigabit Ethernet10 Gigabit EthernetThe 10 gigabit Ethernet computer networking standard was first published in 2002. It defines a version of Ethernet with a nominal data rate of 10 Gbit/s , ten times faster than gigabit Ethernet.10 gigabit Ethernet defines only full duplex point to point links which are generally connected by...