Magerit
Encyclopedia
Magerit is the name of the one of the most powerful supercomputers of Spain
. It also reached the second best Spanish
position in the TOP500
list of supercomputer
s.
This computer is installed in CeSViMa, a research center of the Technical University of Madrid
.
Magerit was first installed in 2006 and reached the 9th fastest in Europe
and the 34th in the world, the second best position of a Spanish supercomputer in the list. It also reached the 275th position in the first Green500 list published.
The second version, installed in 2011 reached the 1st position of Spain, 44th of Europe and 136th fastest of the world. It also reached the 18th position in the Green500 list.
Magerit (for *Materit or *Mageterit) is the most ancient recorded name of the current city of Madrid
. The name comes from the name of a fortress built on the Manzanares River in the 9th century AD
, and means "Place of abundant water".
and IBM
. The computer is housed in the newly created CeSViMa. This first version had only 124 nodes and was housed temporaly in the Computer Science School of Madrid.
The founds for this supercomputer was provided by the Spanish Ministry of Education and Science and the Autonomous Region of Madrid.
(Red Española de Supercomputación or RES in Spanish) and the supercomputer was upgraded. The new configuration has 1204 nodes reaching a speed of 14 TFLOPS. This is considered the first version due to its inclusion in the TOP500
list in the position 34th, the second best position of a Spanish supercomputer in the list.
In 2007 the first users from the access committee of Spanish Supercomputing Network
(the agreement makes that the Network can schedule the use of the 68% of the resources) and users managed at local (CeSViMa) access committee (using the other 32%).
In May 2008, CeSViMa and Magerit supercomputer migrated to a new building in the same campus (only 500 meters from previous location at Computer Science School).
The computer was upgraded: change of communication switch, storage subsystem and replacement of some blades with a new version. This upgrade increase the power of the supercomputer near 2 TFLOPS reaching 15.95 TFLOPS. This upgrade does not avoid the fall from the TOP500
list in november 2008
In this configuration the 59.7% of the supercomputer CPU time is assigned via RES access committee and 40.3% is assigned via CeSViMa policies.
One year later, in 2009, the operating system and other system software was upgraded (migrating to SLES10).
During 2010, CeSViMa acquire a new massive storage system with 1 PetaByte of capacity in parallel with the own storage of Magerit.
The new distribution of use is 80% managed by CeSViMa-UPM access committee and 20% managed by Spanish Supercomputing Network. Despite of the fact that the RES managed percent is lower the resources doned to the network increased 4-5 times.
The upgrade does not include the storage subsystem (maintain the storage upgraded in 2008). There is a small upgrade in next years to adapt the storage system to the new requirements.
This setup reach the second best position in the TOP500 list (34th, november 2006). When this version enters in produciton it reach the 2nd of Spain, 9th of Europe y 34º of the world in the TOP500 list and the 275th position in the first Green500 list
The final version setup (reached after the upgrade of 2008) is a cluster of 1204 nodes eServer
BladeCenter (1036 JS20 and 168 JS21, both PowerPC
64 bits) under a SLES9 Operating System.
The system has a distributed storage system with a capacity of 190 TB
under GPFS. The access to this shared storage is provided by a high bandwitdh switch that allows peaks of 1 Tbit/s.
All the nodes are interconnected with a low latency (2.6 - 3.2 μs
) and high bandwidth network called Myrinet
. This network is used only for MPI
messages of users' tasks.
Finally, an auxiliary Ethernet
network is deployed for administration tasks.
The system maintains the cluster architecture with 245 PS702 nodes, each one with 16 cores in two 64-bit processors POWER7
(eight cores each) 3.0 GHz
, 32 GB
of RAM
and 300 GB
of local hard disk
. Each core provides 18.38 Gflops.
The interconnection was replaced with an Infiniband
network, a high-bandwidth (40 Tbps) and low latency (0.3 μs
). The system maintains two independent Gigabit Ethernet for auxiliary tasks: deployment of images and access to storage subsystem.
The storage system remains the same (192 TB
under GPFS) with a bandwidth near 1 Tbps.
The upgrade includes an update of the software: operating system (SLES11SP1), deployment system (xCAT, eXtreme Cluster Administration Toolkit
) and all software and libraries used in the system.
(Spanish participation in Blue Brain Project).
These jobs are organized by a queue manager. Due to the characteristic of the jobs (runs in hundred of CPUs a few days) its impossible to use more conventional access to the resources. The supercomputer must be running jobs without interrupts all the year.
The use of a queue manager of batch jobs allows a global scheduling of the resources increasing the use of the resources and a fair play between users.
Spain
Spain , officially the Kingdom of Spain languages]] under the European Charter for Regional or Minority Languages. In each of these, Spain's official name is as follows:;;;;;;), is a country and member state of the European Union located in southwestern Europe on the Iberian Peninsula...
. It also reached the second best Spanish
Spain
Spain , officially the Kingdom of Spain languages]] under the European Charter for Regional or Minority Languages. In each of these, Spain's official name is as follows:;;;;;;), is a country and member state of the European Union located in southwestern Europe on the Iberian Peninsula...
position in the TOP500
TOP500
The TOP500 project ranks and details the 500 most powerful known computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year...
list of supercomputer
Supercomputer
A supercomputer is a computer at the frontline of current processing capacity, particularly speed of calculation.Supercomputers are used for highly calculation-intensive tasks such as problems including quantum physics, weather forecasting, climate research, molecular modeling A supercomputer is a...
s.
This computer is installed in CeSViMa, a research center of the Technical University of Madrid
Technical University of Madrid
The Technical University of Madrid or sometimes called Polytechnic University of Madrid is a Spanish University, located in Madrid. It was founded in 1971 as the result of merging different Technical Schools of Engineering and Architecture, originated mainly in the 18th century...
.
Magerit was first installed in 2006 and reached the 9th fastest in Europe
Europe
Europe is, by convention, one of the world's seven continents. Comprising the westernmost peninsula of Eurasia, Europe is generally 'divided' from Asia to its east by the watershed divides of the Ural and Caucasus Mountains, the Ural River, the Caspian and Black Seas, and the waterways connecting...
and the 34th in the world, the second best position of a Spanish supercomputer in the list. It also reached the 275th position in the first Green500 list published.
The second version, installed in 2011 reached the 1st position of Spain, 44th of Europe and 136th fastest of the world. It also reached the 18th position in the Green500 list.
Magerit (for *Materit or *Mageterit) is the most ancient recorded name of the current city of Madrid
Madrid
Madrid is the capital and largest city of Spain. The population of the city is roughly 3.3 million and the entire population of the Madrid metropolitan area is calculated to be 6.271 million. It is the third largest city in the European Union, after London and Berlin, and its metropolitan...
. The name comes from the name of a fortress built on the Manzanares River in the 9th century AD
Anno Domini
and Before Christ are designations used to label or number years used with the Julian and Gregorian calendars....
, and means "Place of abundant water".
History
First steps (2005)
Magerit was created as a collaboration between Technical University of MadridTechnical University of Madrid
The Technical University of Madrid or sometimes called Polytechnic University of Madrid is a Spanish University, located in Madrid. It was founded in 1971 as the result of merging different Technical Schools of Engineering and Architecture, originated mainly in the 18th century...
and IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
. The computer is housed in the newly created CeSViMa. This first version had only 124 nodes and was housed temporaly in the Computer Science School of Madrid.
The founds for this supercomputer was provided by the Spanish Ministry of Education and Science and the Autonomous Region of Madrid.
Joining the Spanish Supercomputer Network (2006-2007)
Late 2006 CeSViMa joins Spanish Supercomputing NetworkSpanish Supercomputing Network
The Spanish Supercomputing Network is a scientific network created by the Spanish Education Ministry. The network is composed by several supercomputers distributed over Spain...
(Red Española de Supercomputación or RES in Spanish) and the supercomputer was upgraded. The new configuration has 1204 nodes reaching a speed of 14 TFLOPS. This is considered the first version due to its inclusion in the TOP500
TOP500
The TOP500 project ranks and details the 500 most powerful known computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year...
list in the position 34th, the second best position of a Spanish supercomputer in the list.
In 2007 the first users from the access committee of Spanish Supercomputing Network
Spanish Supercomputing Network
The Spanish Supercomputing Network is a scientific network created by the Spanish Education Ministry. The network is composed by several supercomputers distributed over Spain...
(the agreement makes that the Network can schedule the use of the 68% of the resources) and users managed at local (CeSViMa) access committee (using the other 32%).
Migration and small upgrades (2008-2010)
The computer was upgraded: change of communication switch, storage subsystem and replacement of some blades with a new version. This upgrade increase the power of the supercomputer near 2 TFLOPS reaching 15.95 TFLOPS. This upgrade does not avoid the fall from the TOP500
TOP500
The TOP500 project ranks and details the 500 most powerful known computer systems in the world. The project was started in 1993 and publishes an updated list of the supercomputers twice a year...
list in november 2008
November 2008
November 2008 was the 11th month of the leap year. It began on a Saturday and ended 30 days later on a Sunday.-Portal:Current Events:This is an archived version of Wikipedia's Current events Portal from November 2008....
In this configuration the 59.7% of the supercomputer CPU time is assigned via RES access committee and 40.3% is assigned via CeSViMa policies.
One year later, in 2009, the operating system and other system software was upgraded (migrating to SLES10).
During 2010, CeSViMa acquire a new massive storage system with 1 PetaByte of capacity in parallel with the own storage of Magerit.
Upgrade (2011)
In the first half of 2011, the supercomputer was fully upgrades replacing all compute nodes and interconexion network with the latest technologies in only one month (a record time). This second configuration reach the position 136 in the TOP500 list becaming the most powerful supercomputer of Spain.. The Barcelona Supercomputing Center has announced a new computer that outperforms Magerit.The new distribution of use is 80% managed by CeSViMa-UPM access committee and 20% managed by Spanish Supercomputing Network. Despite of the fact that the RES managed percent is lower the resources doned to the network increased 4-5 times.
The upgrade does not include the storage subsystem (maintain the storage upgraded in 2008). There is a small upgrade in next years to adapt the storage system to the new requirements.
Architecture
We can consider two versions of the supercomputer:- The original 2006 (the 124 nodes of the agreement of 2005 was included in this configuration) with a small upgrade in 2008.
- The full upgrade in 2011 that makes magerit the 1st supercomputer of Spain..
First version (2005-2010)
The final version setup (reached after the upgrade of 2008) is a cluster of 1204 nodes eServer
EServer
IBM eServer was a family of computer servers from IBM Corporation.Announced in the year 2000, it combined the various IBM server brands under one brand.The various sub-brands were at the same time rebranded from:...
BladeCenter (1036 JS20 and 168 JS21, both PowerPC
PowerPC
PowerPC is a RISC architecture created by the 1991 Apple–IBM–Motorola alliance, known as AIM...
64 bits) under a SLES9 Operating System.
- Each JS20 node has two processors IBMIBMInternational Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
PowerPCPowerPCPowerPC is a RISC architecture created by the 1991 Apple–IBM–Motorola alliance, known as AIM...
single-core 970FX (two cores) with 2.2 GHzGHZGHZ or GHz may refer to:# Gigahertz .# Greenberger-Horne-Zeilinger state — a quantum entanglement of three particles.# Galactic Habitable Zone — the region of a galaxy that is favorable to the formation of life....
, 4 GBGigabyteThe gigabyte is a multiple of the unit byte for digital information storage. The prefix giga means 109 in the International System of Units , therefore 1 gigabyte is...
of RAMRam-Animals:*Ram, an uncastrated male sheep*Ram cichlid, a species of freshwater fish endemic to Colombia and Venezuela-Military:*Battering ram*Ramming, a military tactic in which one vehicle runs into another...
and 40 GB of local hard diskHard diskA hard disk drive is a non-volatile, random access digital magnetic data storage device. It features rotating rigid platters on a motor-driven spindle within a protective enclosure. Data is magnetically read from and written to the platter by read/write heads that float on a film of air above the...
. - Each JS21 node has two processors IBMIBMInternational Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
PowerPCPowerPCPowerPC is a RISC architecture created by the 1991 Apple–IBM–Motorola alliance, known as AIM...
dual-core 970FX (four cores) with 2.2 GHzGHZGHZ or GHz may refer to:# Gigahertz .# Greenberger-Horne-Zeilinger state — a quantum entanglement of three particles.# Galactic Habitable Zone — the region of a galaxy that is favorable to the formation of life....
, 8 GBGigabyteThe gigabyte is a multiple of the unit byte for digital information storage. The prefix giga means 109 in the International System of Units , therefore 1 gigabyte is...
of RAMRam-Animals:*Ram, an uncastrated male sheep*Ram cichlid, a species of freshwater fish endemic to Colombia and Venezuela-Military:*Battering ram*Ramming, a military tactic in which one vehicle runs into another...
and 80 GB of local hard diskHard diskA hard disk drive is a non-volatile, random access digital magnetic data storage device. It features rotating rigid platters on a motor-driven spindle within a protective enclosure. Data is magnetically read from and written to the platter by read/write heads that float on a film of air above the...
.
The system has a distributed storage system with a capacity of 190 TB
Terabyte
The terabyte is a multiple of the unit byte for digital information. The prefix tera means 1012 in the International System of Units , and therefore 1 terabyte is , or 1 trillion bytes, or 1000 gigabytes. 1 terabyte in binary prefixes is 0.9095 tebibytes, or 931.32 gibibytes...
under GPFS. The access to this shared storage is provided by a high bandwitdh switch that allows peaks of 1 Tbit/s.
All the nodes are interconnected with a low latency (2.6 - 3.2 μs
Microsecond
A microsecond is an SI unit of time equal to one millionth of a second. Its symbol is µs.A microsecond is equal to 1000 nanoseconds or 1/1000 millisecond...
) and high bandwidth network called Myrinet
Myrinet
Myrinet, ANSI/VITA 26-1998, is a high-speed local area networking system designed by Myricom to be used as an interconnect between multiple machines to form computer clusters. Myrinet has much lower protocol overhead than standards such as Ethernet, and therefore provides better throughput, less...
. This network is used only for MPI
Message Passing Interface
Message Passing Interface is a standardized and portable message-passing system designed by a group of researchers from academia and industry to function on a wide variety of parallel computers...
messages of users' tasks.
Finally, an auxiliary Ethernet
Ethernet
Ethernet is a family of computer networking technologies for local area networks commercially introduced in 1980. Standardized in IEEE 802.3, Ethernet has largely replaced competing wired LAN technologies....
network is deployed for administration tasks.
Second version (2011)
This setup converts Magerit into the most powerful supercomputer of Spain. When this setup enters in production stage in 2011, it reach the first position of Spain, 44th of Europe and 136th of the world.The system maintains the cluster architecture with 245 PS702 nodes, each one with 16 cores in two 64-bit processors POWER7
IBM POWER
POWER is a reduced instruction set computer instruction set architecture developed by IBM. The name is an acronym for Performance Optimization With Enhanced RISC....
(eight cores each) 3.0 GHz
GHZ
GHZ or GHz may refer to:# Gigahertz .# Greenberger-Horne-Zeilinger state — a quantum entanglement of three particles.# Galactic Habitable Zone — the region of a galaxy that is favorable to the formation of life....
, 32 GB
Gigabyte
The gigabyte is a multiple of the unit byte for digital information storage. The prefix giga means 109 in the International System of Units , therefore 1 gigabyte is...
of RAM
Ram
-Animals:*Ram, an uncastrated male sheep*Ram cichlid, a species of freshwater fish endemic to Colombia and Venezuela-Military:*Battering ram*Ramming, a military tactic in which one vehicle runs into another...
and 300 GB
Gigabyte
The gigabyte is a multiple of the unit byte for digital information storage. The prefix giga means 109 in the International System of Units , therefore 1 gigabyte is...
of local hard disk
Hard disk
A hard disk drive is a non-volatile, random access digital magnetic data storage device. It features rotating rigid platters on a motor-driven spindle within a protective enclosure. Data is magnetically read from and written to the platter by read/write heads that float on a film of air above the...
. Each core provides 18.38 Gflops.
The interconnection was replaced with an Infiniband
InfiniBand
InfiniBand is a switched fabric communications link used in high-performance computing and enterprise data centers. Its features include high throughput, low latency, quality of service and failover, and it is designed to be scalable...
network, a high-bandwidth (40 Tbps) and low latency (0.3 μs
Microsecond
A microsecond is an SI unit of time equal to one millionth of a second. Its symbol is µs.A microsecond is equal to 1000 nanoseconds or 1/1000 millisecond...
). The system maintains two independent Gigabit Ethernet for auxiliary tasks: deployment of images and access to storage subsystem.
The storage system remains the same (192 TB
Terabyte
The terabyte is a multiple of the unit byte for digital information. The prefix tera means 1012 in the International System of Units , and therefore 1 terabyte is , or 1 trillion bytes, or 1000 gigabytes. 1 terabyte in binary prefixes is 0.9095 tebibytes, or 931.32 gibibytes...
under GPFS) with a bandwidth near 1 Tbps.
The upgrade includes an update of the software: operating system (SLES11SP1), deployment system (xCAT, eXtreme Cluster Administration Toolkit
XCAT
xCAT is open-source distributed computing management software used for the deployment and administration of clusters...
) and all software and libraries used in the system.
Use
Magerit processes batch jobs with large processing requirements, such as models of the universe, simulations of materials and climate models. An example of project is the project Cajal Blue BrainBlue Brain
The Blue Brain Project is an attempt to create a synthetic brain by reverse-engineering the mammalian brain down to the molecular level.The aim of the project, founded in May 2005 by the Brain and Mind Institute of the École Polytechnique Fédérale de Lausanne is to study the brain's architectural...
(Spanish participation in Blue Brain Project).
These jobs are organized by a queue manager. Due to the characteristic of the jobs (runs in hundred of CPUs a few days) its impossible to use more conventional access to the resources. The supercomputer must be running jobs without interrupts all the year.
The use of a queue manager of batch jobs allows a global scheduling of the resources increasing the use of the resources and a fair play between users.
Access to resources
The system is available to any person, institution or company that request access via:- Directly CeSViMa, filling the request for access forms on CeSViMa web page.
- As an collaboration agreement with CeSViMa
- Via Spanish Supercomputing NetworkSpanish Supercomputing NetworkThe Spanish Supercomputing Network is a scientific network created by the Spanish Education Ministry. The network is composed by several supercomputers distributed over Spain...
. This is a competitive process. The access committee evaluates all the projects and can assign resources on any other supercomputer of the network so it can be scheduled in the 20% of Magerit resources managed by RES.
External links
- CeSViMa - Centro de Supercomputación y Visualización de Madrid where Magerit is installed
- Estado del supercomputador Magerit in the same site
- Timelapse of the building of second version of Magerit