SPECint
Encyclopedia
SPECint is a computer benchmark
specification for CPU's integer
processing power. It is maintained by the Standard Performance Evaluation Corporation
(SPEC). SPECint is the integer performance testing component of the SPEC test suite. The first SPEC test suite, CPU92, was announced in 1992. It was followed by CPU95, CPU2000, and CPU2006. The latest standard of SPECint is CINT2006 (aka SPECint2006).
computer system. It is split into 2 components, the first being CINT2006, the other being CFP2006 (SPECfp
), for floating point testing.
SPEC defines a base runtime for each of the 12 benchmark programs. For SPECint2006, that number ranges from 1000–3000 seconds. The timed test is run on the system, and the time of the test system is compared to the reference time, and a ratio is computed. That ratio becomes the SPECint score for that test. (This differs from the rating in SPECINT2000, which multiplies the ratio by 100.)
As an example for SPECint2006, consider a processor which can run 400.perlbench in 2000 seconds. The time it takes the reference machine to run the benchmark is 9770 seconds. Thus the ratio is 4.885. Each ratio is computed, and then the geometric mean
of those ratios is computed to produce an overall value.
files to users wanting to test their systems. These files are written in a standard programming language
, which is then compiled for each particular CPU architecture and operating system
. Thus, the performance measured is that of the CPU, RAM, and compiler
, and does not test I/O
, networking
, or graphics
.
Two metrics are reported for a particular benchmark, "base" and "peak". Compiler options account for the difference between the two numbers. As the SPEC benchmarks are distributed as source code
, it is up to the party performing the test to compile this code. There is agreement that the benchmarks should be compiled in the same way as a user would compile a program, but there is no consistent method for user compilation, it varies system by system. SPEC, in this case, defines two reference points, "base" and "peak". Base has a more strict set of compilation rules than peak. Less optimization can be done, the compiler flags must be the same for each benchmark, in the same order, and there must be a limited number of flags. Base, then, is closest to how a user would compile a program with standard flags. The 'peak' metric can be performed with maximum compiler optimization, even to the extent of different optimizations for each benchmark. This number represents maximum system performance, achieved by full compiler optimization.
SPECint tests are carried out on a wide range of hardware, with results typically published for the full range of system-level implementations employing the latest CPUs. For SPECint2006, the CPUs include Intel and AMD x86 & x86-64
processors, Sun
SPARC
CPUs, IBM
POWER
CPUs, and IA-64 CPUs. This range of capabilities, specifically in this case the number of CPUs, means that the SPECint benchmark is usually run on only a single CPU, even if the system has many CPUs. If a single CPU has multiple cores, only a single core is used; hyper-threading
is also typically disabled,
A more complete system-level benchmark that allows all CPUs to be used is known as SPECint_rate2006, also called "CINT2006 Rate".
The benchmarks are:
Benchmark (computing)
In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it...
specification for CPU's integer
Integer
The integers are formed by the natural numbers together with the negatives of the non-zero natural numbers .They are known as Positive and Negative Integers respectively...
processing power. It is maintained by the Standard Performance Evaluation Corporation
Standard Performance Evaluation Corporation
The Standard Performance Evaluation Corporation is a non-profit organization that aims to "produce, establish, maintain and endorse a standardized set" of performance benchmarks for computers....
(SPEC). SPECint is the integer performance testing component of the SPEC test suite. The first SPEC test suite, CPU92, was announced in 1992. It was followed by CPU95, CPU2000, and CPU2006. The latest standard of SPECint is CINT2006 (aka SPECint2006).
SPECint 2006
CPU2006 is a set of benchmarks designed to test the CPU performance of a modern serverServer (computing)
In the context of client-server architecture, a server is a computer program running to serve the requests of other programs, the "clients". Thus, the "server" performs some computational task on behalf of "clients"...
computer system. It is split into 2 components, the first being CINT2006, the other being CFP2006 (SPECfp
SPECfp
SPECfp is a computer benchmark designed to test the floating point performance of a computer. It is managed by the Standard Performance Evaluation Corporation. SPECfp is the floating point performance testing component of the SPEC CPU testing suit. The first stander SPECfp was released in 1989 as...
), for floating point testing.
SPEC defines a base runtime for each of the 12 benchmark programs. For SPECint2006, that number ranges from 1000–3000 seconds. The timed test is run on the system, and the time of the test system is compared to the reference time, and a ratio is computed. That ratio becomes the SPECint score for that test. (This differs from the rating in SPECINT2000, which multiplies the ratio by 100.)
As an example for SPECint2006, consider a processor which can run 400.perlbench in 2000 seconds. The time it takes the reference machine to run the benchmark is 9770 seconds. Thus the ratio is 4.885. Each ratio is computed, and then the geometric mean
Geometric mean
The geometric mean, in mathematics, is a type of mean or average, which indicates the central tendency or typical value of a set of numbers. It is similar to the arithmetic mean, except that the numbers are multiplied and then the nth root of the resulting product is taken.For instance, the...
of those ratios is computed to produce an overall value.
Background
For a fee, SPEC distributes source codeSource code
In computer science, source code is text written using the format and syntax of the programming language that it is being written in. Such a language is specially designed to facilitate the work of computer programmers, who specify the actions to be performed by a computer mostly by writing source...
files to users wanting to test their systems. These files are written in a standard programming language
Programming language
A programming language is an artificial language designed to communicate instructions to a machine, particularly a computer. Programming languages can be used to create programs that control the behavior of a machine and/or to express algorithms precisely....
, which is then compiled for each particular CPU architecture and operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
. Thus, the performance measured is that of the CPU, RAM, and compiler
Compiler
A compiler is a computer program that transforms source code written in a programming language into another computer language...
, and does not test I/O
Input/output
In computing, input/output, or I/O, refers to the communication between an information processing system , and the outside world, possibly a human, or another information processing system. Inputs are the signals or data received by the system, and outputs are the signals or data sent from it...
, networking
Computer network
A computer network, often simply referred to as a network, is a collection of hardware components and computers interconnected by communication channels that allow sharing of resources and information....
, or graphics
Graphics
Graphics are visual presentations on some surface, such as a wall, canvas, computer screen, paper, or stone to brand, inform, illustrate, or entertain. Examples are photographs, drawings, Line Art, graphs, diagrams, typography, numbers, symbols, geometric designs, maps, engineering drawings,or...
.
Two metrics are reported for a particular benchmark, "base" and "peak". Compiler options account for the difference between the two numbers. As the SPEC benchmarks are distributed as source code
Source code
In computer science, source code is text written using the format and syntax of the programming language that it is being written in. Such a language is specially designed to facilitate the work of computer programmers, who specify the actions to be performed by a computer mostly by writing source...
, it is up to the party performing the test to compile this code. There is agreement that the benchmarks should be compiled in the same way as a user would compile a program, but there is no consistent method for user compilation, it varies system by system. SPEC, in this case, defines two reference points, "base" and "peak". Base has a more strict set of compilation rules than peak. Less optimization can be done, the compiler flags must be the same for each benchmark, in the same order, and there must be a limited number of flags. Base, then, is closest to how a user would compile a program with standard flags. The 'peak' metric can be performed with maximum compiler optimization, even to the extent of different optimizations for each benchmark. This number represents maximum system performance, achieved by full compiler optimization.
SPECint tests are carried out on a wide range of hardware, with results typically published for the full range of system-level implementations employing the latest CPUs. For SPECint2006, the CPUs include Intel and AMD x86 & x86-64
X86-64
x86-64 is an extension of the x86 instruction set. It supports vastly larger virtual and physical address spaces than are possible on x86, thereby allowing programmers to conveniently work with much larger data sets. x86-64 also provides 64-bit general purpose registers and numerous other...
processors, Sun
Sun Microsystems
Sun Microsystems, Inc. was a company that sold :computers, computer components, :computer software, and :information technology services. Sun was founded on February 24, 1982...
SPARC
SPARC
SPARC is a RISC instruction set architecture developed by Sun Microsystems and introduced in mid-1987....
CPUs, IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
POWER
IBM POWER
POWER is a reduced instruction set computer instruction set architecture developed by IBM. The name is an acronym for Performance Optimization With Enhanced RISC....
CPUs, and IA-64 CPUs. This range of capabilities, specifically in this case the number of CPUs, means that the SPECint benchmark is usually run on only a single CPU, even if the system has many CPUs. If a single CPU has multiple cores, only a single core is used; hyper-threading
Hyper-threading
Hyper-threading is Intel's term for its simultaneous multithreading implementation in its Atom, Intel Core i3/i5/i7, Itanium, Pentium 4 and Xeon CPUs....
is also typically disabled,
A more complete system-level benchmark that allows all CPUs to be used is known as SPECint_rate2006, also called "CINT2006 Rate".
Benchmarks
The SPECint2006 test suite consists of 12 benchmark programs, designed to test exclusively the integer performance of the system.The benchmarks are:
| Benchmark | Language Programming language A programming language is an artificial language designed to communicate instructions to a machine, particularly a computer. Programming languages can be used to create programs that control the behavior of a machine and/or to express algorithms precisely.... |
Category |
|---|---|---|
| 400.perlbench | C | Programming Language |
| 401.bzip2 | C | Compression |
| 403.gcc | C | C Compiler |
| 429.mcf | C | Combinatorial Optimization |
| 445.gobmk | C | Artificial Intelligence |
| 456.hmmer | C | Search Gene Sequence |
| 458.sjeng | C | Artificial Intelligence |
| 462.libquantum | C | Physics / Quantum Computing |
| 464.h264ref | C | Video Compression |
| 471.omnetpp | C++ | Discrete Event Simulation |
| 473.astar | C++ | Path-finding Algorithms |
| 483.xalancbmk | C++ | XML Processing |
External links
- SPEC Website
- CINT2006 (Integer Component of SPEC CPU2006)
- Published results of SPECint2006
- Description of each benchmark