VMware ESX Server
Encyclopedia
VMware ESX is an enterprise-level computer virtualization
product offered by VMware, Inc.
ESX is a component of VMware's larger offering, VMware Infrastructure
, and adds management and reliability services to the core server product. The original ESX is being replaced by ESXi.
VMware ESX and VMware ESXi are bare-metal embedded hypervisor
s that are VMware's enterprise software
hypervisor
s for servers that run directly on server
hardware without requiring an additional underlying operating system
.
The basic server requires some form of persistent storage (typically an array of hard disk drives) for storing the hypervisor and support files. A smaller footprint variant, ESXi, does away with the first requirement by permitting placement of the hypervisor onto a dedicated compact storage device. Both variants support the services offered by VMware Infrastructure.
used by VMware ESX as "vmkernel".
is started first, and is used to load a variety of specialized virtualization components, including VMware's vmkernel component. This previously-booted Linux kernel then becomes the first running virtual machine and is called the service console. Thus, at normal run-time, the vmkernel is running on the bare computer and the Linux-based service console runs as the first virtual machine.
The vmkernel itself, which VMware says is a microkernel
, has three interfaces to the outside world:
and the SRAT (system resource allocation table) to track allocated memory.
Access to other hardware (such as network or storage devices) takes place using modules. At least some of the modules derive from modules used in the Linux kernel
. To access these modules, an additional module called
The vmkernel uses the device drivers:
These drivers mostly equate to those described in VMware's hardware compatibility list
. All these modules fall under the GPL
. Programmers have adapted them to run with the vmkernel: VMware Inc has changed the module-loading and some other minor things.
The Service Console, for all intents and purposes, is the operating system used to interact with VMware ESX and the virtual machines that run on the server.
to load additional code: often referred to by VMware, Inc. as the "vmkernel". The dependencies between the "vmkernel" and the Linux part of the ESX server have changed drastically over different major versions of the software. The VMware FAQ states: "ESX Server also incorporates a service console based on a Linux 2.4 kernel that is used to boot the ESX Server virtualization layer". The Linux kernel runs before any other software on an ESX host.
On ESX versions 1 and 2, no VMkernel processes run on the system during the boot process.
After the Linux kernel has loaded, the S90vmware script loads the vmkernel.
VMware Inc states that vmkernel does not derive from Linux, but acknowledges that it has adapted certain device-drivers from Linux device drivers. The Linux kernel continues running, under the control of the vmkernel, providing functions including the proc file system used by the ESX and an environment to run support applications.
ESX version 3 loads the VMkernel from the Linux initrd
, thus much earlier in the boot-sequence than in previous ESX versions.
In traditional systems, a given operating system runs a single kernel. The VMware FAQ mentions that ESX has both a Linux 2.4 kernel and vmkernel — hence confusion over whether ESX has a Linux base. An ESX system starts a Linux kernel first, but it loads vmkernel (also described by VMware as a kernel), which according to VMware 'wraps around' the linux kernel, and which (according to VMware Inc) does not derive from Linux.
The ESX userspace environment, known as the "Service Console" (or as "COS" or as "vmnix"), derives from a modified version of Red Hat Linux
, (Red Hat 7.2 for ESX 2.x and Red Hat Enterprise Linux
3 for ESX 3.x). In general, this Service Console provides management interfaces (CLI, webpage MUI, Remote Console
). This VMware ESX hypervisor virtualization
approach provides lower overhead and better control and granularity for allocating resources (CPU-time, disk-bandwidth, network-bandwidth, memory-utilization) to virtual machines, compared to so-called "hosted" virtualization, where a base OS handles the physical resources. It also increases security.
As a further detail which differentiates the ESX from other VMware virtualization products: ESX supports the VMware proprietary cluster file system VMFS. VMFS enables multiple hosts to access the same SAN
LUNs simultaneously, while file-level locking provides simple protection to file-system integrity.
).
Upon displaying a PSOD, the vmkernel writes debug information to the core dump partition. This information, together with the error codes displayed on the PSOD can be used by VMware support to determine the cause of the problem.
VMware ESX 1.5
VMware ESX 2.0 (July 21, 2003)
VMware ESX 2.5 (December 14, 2004)
VMware Infrastructure 3.0 (VI3) (June 5, 2006)
VMware vSphere 4.0 (May 20, 2009)
July 18, 2010
vSphere 4.1 and its subsequent update and patch releases are the last releases to include both ESX and ESXi hypervisor architectures. Future major releases of VMware vSphere will include only the VMware ESXi architecture. For this reason, VMware recommends that deployments of vSphere 4.x utilize the ESXi hypervisor architecture.
VMware ESXi was originally a compact version of VMware ESX that allowed for a smaller 32 MB disk footprint on the Host. With a simple configuration console for mostly network configuration and remote based VMware Infrastructure Client Interface, this allows for more resources to be dedicated to the Guest environments.
There are two variations of ESXi, VMware ESXi Installable and VMware ESXi Embedded Edition. It has the ability to upgrade to VMware Infrastructure
3 or VMware vSphere
4.0 ESXi.
Originally named VMware ESX Server ESXi edition, through several revisions, finally becoming VMware ESXi 3. New editions then followed ESXi 3.5, ESXi 4 and now ESXi 5.
Version release history:
Paravirtualization
or other virtualization techniques may help with these issues. VMware invented the Virtual Machine Interface
for this purpose, and selected operating systems support this. A comparison between full virtualization
and paravirtualization
for the ESX Server shows that in some cases paravirtualization is much faster.
Hardware virtualization
Computer hardware virtualization is the virtualization of computers or operating systems. It hides the physical characteristics of a computing platform from users, instead showing another abstract computing platform...
product offered by VMware, Inc.
VMware
VMware, Inc. is a company providing virtualization software founded in 1998 and based in Palo Alto, California, USA. The company was acquired by EMC Corporation in 2004, and operates as a separate software subsidiary ....
ESX is a component of VMware's larger offering, VMware Infrastructure
VMware Infrastructure
VMware Infrastructure 3 is a suite of computer hardware virtualization products from VMware, Inc. . The suite includes:* VMware ESX Server version 3* VMware ESXi version 3.x...
, and adds management and reliability services to the core server product. The original ESX is being replaced by ESXi.
VMware ESX and VMware ESXi are bare-metal embedded hypervisor
Embedded Hypervisor
Software virtualization has been a major topic in the enterprise space for quite some time, but has become an important technology for embedded systems only in the last few years. The application of the technology to the enterprise space has been well described elsewhere by companies such as...
s that are VMware's enterprise software
Enterprise software
Enterprise software, also known as enterprise application software , is software used in organizations, such as in a business or government, contrary to software chosen by individuals...
hypervisor
Hypervisor
In computing, a hypervisor, also called virtual machine manager , is one of many hardware virtualization techniques that allow multiple operating systems, termed guests, to run concurrently on a host computer. It is so named because it is conceptually one level higher than a supervisory program...
s for servers that run directly on server
Server (computing)
In the context of client-server architecture, a server is a computer program running to serve the requests of other programs, the "clients". Thus, the "server" performs some computational task on behalf of "clients"...
hardware without requiring an additional underlying operating system
Operating system
An operating system is a set of programs that manage computer hardware resources and provide common services for application software. The operating system is the most important type of system software in a computer system...
.
The basic server requires some form of persistent storage (typically an array of hard disk drives) for storing the hypervisor and support files. A smaller footprint variant, ESXi, does away with the first requirement by permitting placement of the hypervisor onto a dedicated compact storage device. Both variants support the services offered by VMware Infrastructure.
Technical description
VMware, Inc. refers to the hypervisorHypervisor
In computing, a hypervisor, also called virtual machine manager , is one of many hardware virtualization techniques that allow multiple operating systems, termed guests, to run concurrently on a host computer. It is so named because it is conceptually one level higher than a supervisory program...
used by VMware ESX as "vmkernel".
Architecture
VMware states that the ESX product runs on "bare metal". In contrast to other VMware products, it does not run atop a third-party operating system, but instead includes its own kernel. Up through the current ESX version 4.1, a Linux kernelLinux kernel
The Linux kernel is an operating system kernel used by the Linux family of Unix-like operating systems. It is one of the most prominent examples of free and open source software....
is started first, and is used to load a variety of specialized virtualization components, including VMware's vmkernel component. This previously-booted Linux kernel then becomes the first running virtual machine and is called the service console. Thus, at normal run-time, the vmkernel is running on the bare computer and the Linux-based service console runs as the first virtual machine.
The vmkernel itself, which VMware says is a microkernel
Microkernel
In computer science, a microkernel is the near-minimum amount of software that can provide the mechanisms needed to implement an operating system . These mechanisms include low-level address space management, thread management, and inter-process communication...
, has three interfaces to the outside world:
- hardware
- guest systems
- service console (Console OS)
Interface to hardware
The vmkernel handles CPU and memory directly, using scan-before-execution (SBE) to handle special or privileged CPU instructionsand the SRAT (system resource allocation table) to track allocated memory.
Access to other hardware (such as network or storage devices) takes place using modules. At least some of the modules derive from modules used in the Linux kernel
Linux kernel
The Linux kernel is an operating system kernel used by the Linux family of Unix-like operating systems. It is one of the most prominent examples of free and open source software....
. To access these modules, an additional module called
vmklinux implements the Linux module interface. According to the README file, "This module contains the Linux emulation layer used by the vmkernel."The vmkernel uses the device drivers:
- net/e100
- net/e1000
- net/e1000e
- net/bnx2
- net/tg3
- net/forcedeth
- net/pcnet32
- block/cciss
- scsi/adp94xx
- scsi/aic7xxx
- scsi/aic79xx
- scsi/ips
- scsi/lpfcdd-v732
- scsi/megaraid2
- scsi/mptscsi_2xx
- scsi/qla2200-v7.07
- scsi/megaraid_sas
- scsi/qla4010
- scsi/qla4022
- scsi/vmkiscsi
- scsi/aacraid_esx30
- scsi/lpfcdd-v7xx
- scsi/qla2200-v7xx
These drivers mostly equate to those described in VMware's hardware compatibility list
Hardware Compatibility List
A hardware compatibility list is a list of computer hardware that is compatible with a particular operating system or device management software. In today's world, there is a vast amount of computer hardware in circulation, and many operating systems too...
. All these modules fall under the GPL
GNU General Public License
The GNU General Public License is the most widely used free software license, originally written by Richard Stallman for the GNU Project....
. Programmers have adapted them to run with the vmkernel: VMware Inc has changed the module-loading and some other minor things.
Datacenter
Large companies might use datacenter for hosts and virtual machines as a primary container. Multiple datacenters can be used to represent organizational/business units......Service console
The Service Console is a vestigial general purpose operating system most significantly used as bootstrap for the VMware kernel, vmkernel, and secondarily used as a management interface. Both of these Console Operating System functions are being deprecated as VMware migrates to exclusively the 'embedded' ESX model, current version being ESXi.The Service Console, for all intents and purposes, is the operating system used to interact with VMware ESX and the virtual machines that run on the server.
Linux dependencies
ESX uses a Linux kernelLinux kernel
The Linux kernel is an operating system kernel used by the Linux family of Unix-like operating systems. It is one of the most prominent examples of free and open source software....
to load additional code: often referred to by VMware, Inc. as the "vmkernel". The dependencies between the "vmkernel" and the Linux part of the ESX server have changed drastically over different major versions of the software. The VMware FAQ states: "ESX Server also incorporates a service console based on a Linux 2.4 kernel that is used to boot the ESX Server virtualization layer". The Linux kernel runs before any other software on an ESX host.
On ESX versions 1 and 2, no VMkernel processes run on the system during the boot process.
After the Linux kernel has loaded, the S90vmware script loads the vmkernel.
VMware Inc states that vmkernel does not derive from Linux, but acknowledges that it has adapted certain device-drivers from Linux device drivers. The Linux kernel continues running, under the control of the vmkernel, providing functions including the proc file system used by the ESX and an environment to run support applications.
ESX version 3 loads the VMkernel from the Linux initrd
Initrd
In computing, initrd is a scheme for loading a temporary file system into memory in the boot process of the Linux kernel. initrd and initramfs refer to slightly different methods of achieving this...
, thus much earlier in the boot-sequence than in previous ESX versions.
In traditional systems, a given operating system runs a single kernel. The VMware FAQ mentions that ESX has both a Linux 2.4 kernel and vmkernel — hence confusion over whether ESX has a Linux base. An ESX system starts a Linux kernel first, but it loads vmkernel (also described by VMware as a kernel), which according to VMware 'wraps around' the linux kernel, and which (according to VMware Inc) does not derive from Linux.
The ESX userspace environment, known as the "Service Console" (or as "COS" or as "vmnix"), derives from a modified version of Red Hat Linux
Red Hat Linux
Red Hat Linux, assembled by the company Red Hat, was a popular Linux based operating system until its discontinuation in 2004.Red Hat Linux 1.0 was released on November 3, 1994...
, (Red Hat 7.2 for ESX 2.x and Red Hat Enterprise Linux
Red Hat Enterprise Linux
Red Hat Enterprise Linux is a Linux-based operating system developed by Red Hat and targeted toward the commercial market. Red Hat Enterprise Linux is released in server versions for x86, x86-64, Itanium, PowerPC and IBM System z, and desktop versions for x86 and x86-64...
3 for ESX 3.x). In general, this Service Console provides management interfaces (CLI, webpage MUI, Remote Console
Remote administration
Remote administration refers to any method of controlling a computer from a remote location.Software that allows remote administration is becoming increasingly common and is often used when it is difficult or impractical to be physically near a system in order to use it, or in order to access web...
). This VMware ESX hypervisor virtualization
Hypervisor
In computing, a hypervisor, also called virtual machine manager , is one of many hardware virtualization techniques that allow multiple operating systems, termed guests, to run concurrently on a host computer. It is so named because it is conceptually one level higher than a supervisory program...
approach provides lower overhead and better control and granularity for allocating resources (CPU-time, disk-bandwidth, network-bandwidth, memory-utilization) to virtual machines, compared to so-called "hosted" virtualization, where a base OS handles the physical resources. It also increases security.
As a further detail which differentiates the ESX from other VMware virtualization products: ESX supports the VMware proprietary cluster file system VMFS. VMFS enables multiple hosts to access the same SAN
Storage area network
A storage area network is a dedicated network that provides access to consolidated, block level data storage. SANs are primarily used to make storage devices, such as disk arrays, tape libraries, and optical jukeboxes, accessible to servers so that the devices appear like locally attached devices...
LUNs simultaneously, while file-level locking provides simple protection to file-system integrity.
Purple Screen of Death
In the event of a hardware error, the vmkernel can 'catch' a Machine Check Exception. This results in an error message displayed on a purple console screen. This is colloquially known as a PSOD, or Purple Screen of Death (cf. Blue Screen of DeathBlue Screen of Death
To forse a BSOD Open regedit.exe,Then search: HKLM\SYSTEM\CurrentControlSet\services\i8042prt\ParametersThen make a new DWORD called "CrashOnCtrlScroll" And set the value to 1....
).
Upon displaying a PSOD, the vmkernel writes debug information to the core dump partition. This information, together with the error codes displayed on the PSOD can be used by VMware support to determine the cause of the problem.
Related products
The following products operate in conjunction with ESX:- vCenter Server, enables monitoring and management of multiple ESX, ESXi and GSX servers. In addition, users must install it to run infrastructure services such as:
- VMotion (transferring virtual machines between servers on the fly, with zero downtime)
- SVMotion (transferring virtual machines between Shared Storage LUNs on the fly, with zero downtime)
- DRS (automated VMotion based on host/VM load requirements/demands)
- HA (restarting of Virtual Machine Guests in the event of a physical ESX Host failure)
- Converter, enables users to create VMware ESX Server- or Workstation-compatible virtual machines from either physical machines or from virtual machines made by other virtualization products. Converter replaces the VMware "P2V Assistant" and "Importer" products — P2V Assistant allowed users to convert physical machines into virtual machines; and Importer allowed the import of virtual machines from other products into VMware Workstation.
- vSphere Client (formerly VMware Infrastructure Client), enables monitoring and management of a single instance of ESX or ESXi server. After ESX 4.1, vSphere Client was no longer available from the ESX/ESXi server, but must be downloaded from the VMware web site.
VMware ESX
Version release history:- VMware ESX Server 1.0 First release was in 2001
- VMware ESX Server 1.1 (January 7, 2002)
VMware ESX 1.5
- VMware ESX Server 1.5 (May 13, 2002)
VMware ESX 2.0 (July 21, 2003)
- VMware ESX Server 2.1 Build 22983 (4/13/06)
- VMware ESX Server 2.0.2 Build 23922 (5/4/06)
VMware ESX 2.5 (December 14, 2004)
- VMware ESX Server 2.5.0 Build 11343 (11/29/2004)
- VMware ESX Server 2.5.1 Build 13057 (05/20/2005)
- VMware ESX Server 2.5.1 Build 14182 (06/20/2005)
- VMware ESX Server 2.5.2 Build 16390 (09/15/2005)
- VMware ESX Server 2.5.3 Build 22981 (04/13/2006)
- VMware ESX Server 2.5.4 Build 32233 (10/05/2006)
- VMware ESX Server 2.5.5 Build 57619 (10/08/2007)
VMware Infrastructure 3.0 (VI3) (June 5, 2006)
- VMware ESX Server 3.0 Build 27701 (06/13/06)
- VMware ESX Server 3.0.1 Build 32039 (09/25/06)
- VMware ESX Server 3.0.2 Build 52542 (07/31/07)
- VMware ESX Server 3.0.3 Build 104629 (August 8, 2008)
- VMware ESX Server 3.0.3 Update 1 Build 231127 (08 Mar 2010)
- VMware ESX Server 3.5 (December 10, 2007)
- VMware ESX Server 3.5 Build 64607 (20 Feb 2008)
- VMware ESX Server 3.5 Update 1 Build 82663 (10 Apr 2008)
- VMware ESX Server 3.5 Update 2 Build 110268 (13 Aug 2008)
- VMware ESX Server 3.5 Update 3 Build 123630 (6 Nov 2008)
- VMware ESX Server 3.5 Update 4 Build 153875 (30 Mar 2009)
- VMware ESX Server 3.5 Update 5 Build 207095 (03 Dec 2009)
VMware vSphere 4.0 (May 20, 2009)
- VMware ESX 4.0 Build 164009 (21 May 2009)
- VMware ESX 4.0 Update 1 Build 208167 (19 Nov 2009)
- VMware ESX 4.0 Update 2 Build 261974 (10 Jun 2010)
- VMware ESX 4.0 Update 3 Build 398348 (5 May 2011)
- VMware ESX 4.0 Update 4 Build 504850
- VMware ESX 4.1 Build 260247 (July 13, 2010)
- VMware ESX 4.1 Update 1 Build 348481 (Feb 10, 2011)
- VMware ESX 4.1 Update 2 Build 502767 (Oct 27, 2011)
July 18, 2010
vSphere 4.1 and its subsequent update and patch releases are the last releases to include both ESX and ESXi hypervisor architectures. Future major releases of VMware vSphere will include only the VMware ESXi architecture. For this reason, VMware recommends that deployments of vSphere 4.x utilize the ESXi hypervisor architecture.
VMware ESXi
VMware ESXi is a smaller footprint version of ESX that does not include ESX's Service Console. It is available as a free download from VMware though certain features are disabled without the purchase of a vCenter license.VMware ESXi was originally a compact version of VMware ESX that allowed for a smaller 32 MB disk footprint on the Host. With a simple configuration console for mostly network configuration and remote based VMware Infrastructure Client Interface, this allows for more resources to be dedicated to the Guest environments.
There are two variations of ESXi, VMware ESXi Installable and VMware ESXi Embedded Edition. It has the ability to upgrade to VMware Infrastructure
VMware Infrastructure
VMware Infrastructure 3 is a suite of computer hardware virtualization products from VMware, Inc. . The suite includes:* VMware ESX Server version 3* VMware ESXi version 3.x...
3 or VMware vSphere
VMware vSphere
VMware vSphere is VMware's cloud computing virtualization operating system.-History:While VMware Infrastructure 3.5 was in development, vSphere was conceived as an enhanced suite of tools with cloud computing utilizing VMware ESX/ESXi 4.The cloud computing-enabled tool suite was spun off as VMware...
4.0 ESXi.
Originally named VMware ESX Server ESXi edition, through several revisions, finally becoming VMware ESXi 3. New editions then followed ESXi 3.5, ESXi 4 and now ESXi 5.
Version release history:
- VMware ESX 3 Server ESXi edition
- -- unknown --
- VMware ESXi 3.5 First Public Release (Build 67921)
- VMware ESXi 3.5 Initial Release (Build 70348)
- VMware ESXi 3.5 Update 1 (Build 82664)
- VMware ESXi 3.5 Update 2 (Build 110271)
- VMware ESXi 3.5 Update 3 (Build 123629)
- VMware ESXi 3.5 Update 4 (Build 153875)
- VMware ESXi 3.5 Update 5 (Build 207095)
- VMware ESXi 4.0 (Build 164009)
- VMware ESXi 4.0 Update 1 (Build 208167)
- VMware ESXi 4.0 Update 2 (Build 261974)
- VMware ESXi 4.0 Update 3 Build 398348
- VMware ESXi 4.0 Update 4 Build 504850
- VMware ESXi 4.1 (Build 260247)
- VMware ESXi 4.1 Update 1 (Build 348481)
- VMware ESXi 4.1 Update 2 (Build 502767)
- VMware ESXi 5.0 (Build 456551)
Infrastructure limitations
Some limitations in ESX Server 4 may constrain the design of data centers:- Guest system maximum RAM: 255 GB
- Host system maximum RAM: 1 TB
- Number of hosts in a high availabilityHigh availabilityHigh availability is a system design approach and associated service implementation that ensures a prearranged level of operational performance will be met during a contractual measurement period....
cluster: 32 - Number of Primary Nodes in ESX Cluster high availability: 5
- Number of hosts in a Distributed Resource Scheduler cluster: 32
- Maximum number of processors per virtual machine: 8
- Maximum number of processors per host: 160
- Maximum number of cores per processor: 12
- Maximum number of virtual machines per host: 320
- VMFS-3 limits files to 262,144 (218) blocks, which translates to 256 GB for 1 MB block sizes (the default) or up to 2 TB for 8 MB block sizes. However you should be aware that on a VMFS Boot drive, it is very difficult to use anything other than 1 MB Block size.
Performance limitations
In terms of performance, virtualization imposes a cost in the additional work the CPU has to perform to virtualize the underlying hardware. Instructions that perform this extra work, and other activities that require virtualization, tend to lay in operating system calls. In an unmodified operating system, OS calls introduce the greatest portion of virtualization "overhead".Paravirtualization
Paravirtualization
In computing, paravirtualization is a virtualization technique that presents a software interface to virtual machines that is similar but not identical to that of the underlying hardware....
or other virtualization techniques may help with these issues. VMware invented the Virtual Machine Interface
Virtual Machine Interface
Virtual machine interface may refer to a communication protocol for running parallel programs on a distributed memory system.Virtual Machine Interface is also the name given by VMware to the proposed open standard protocol that guest operating systems can use to communicate with the hypervisor of...
for this purpose, and selected operating systems support this. A comparison between full virtualization
Full virtualization
In computer science, full virtualization is a virtualization technique used to provide a certain kind of virtual machine environment, namely, one that is a complete simulation of the underlying hardware...
and paravirtualization
Paravirtualization
In computing, paravirtualization is a virtualization technique that presents a software interface to virtual machines that is similar but not identical to that of the underlying hardware....
for the ESX Server shows that in some cases paravirtualization is much faster.
See also
- Comparison of platform virtual machines
- List of VMware software
- Virtual applianceVirtual applianceA virtual appliance is a virtual machine image designed to run on a virtualization platform ....
- Virtual machineVirtual machineA virtual machine is a "completely isolated guest operating system installation within a normal host operating system". Modern virtual machines are implemented with either software emulation or hardware virtualization or both together.-VM Definitions:A virtual machine is a software...
- Virtual disk imageVirtual disk imageA virtual disk image is a file on a physical disk, which has a well-defined, published or proprietary, format and is interpreted by a Virtual Machine Monitor as a hard disk. IT administrators and software developers administer them through offline operations using built-in or third-party tools...
- VMware VMFSVMware VMFSVMware VMFS is VMware, Inc.'s cluster file system. It is used by VMware ESX Server and the company's flagship server virtualization suite, VMware Infrastructure. It was developed and is used to store virtual machine disk images, including snapshots...
- x86 virtualizationX86 virtualizationIn computing, x86 virtualization is the facility that allows multiple operating systems to simultaneously share x86 processor resources in a safe and efficient manner, a facility generically known as hardware virtualization...